Improved error handling in sceKernelAllocateDirectMemory

[StevenMiller123]

Some games, most notably the Call of Duty series of games, intentionally make sceKernelAllocateDirectMemory calls that fail. To match hardware behavior, we should return ORBIS_KERNEL_ERROR_EAGAIN instead of throwing an assert.

This should fix games that still hit Unable to find free direct memory area asserts after #1896.

Credits to @red-prig and @raphaelthegreat for helping with the error cases.

This needs testing, as it could hide deeper issues in some games.

[Missake212]

Pretty sure we already tested this on Discord but since these commits are more recent I thought I'd give them a try and send the logs here.

NBA2K15
shad_log.txt

NBA2K17
shad_log.txt

[squidbus]

What is the current status of this PR?

[StevenMiller123]

This does resolve the issues I saw in Call of Duty games, but leads to hangs or crashes in other games that hit the Unable to find free direct memory area asserts, like Hatsune Miku Project Diva X and the NBA games Missake212 tested.
I'd like to see if I can figure out why those NBA games are behaving weirdly before marking this PR as ready, since the issues in those games seem to be related to sceKernelAllocateDirectMemory.

[Missake212]

This does resolve the issues I saw in Call of Duty games, but leads to hangs or crashes in other games that hit the Unable to find free direct memory area asserts, like Hatsune Miku Project Diva X and the NBA games Missake212 tested. I'd like to see if I can figure out why those NBA games are behaving weirdly before marking this PR as ready, since the issues in those games seem to be related to sceKernelAllocateDirectMemory.

If you want me to I could try to merge this PR locally and check through VS why it's hanging

[Missake212]

Not sure if I did this properly but here's what I got, if you want to we can try continuing the debugging on Discord. First screen is 2k15 second one is 2k17