The direct buffer GC problem

Some background: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4879883



The problem is that direct buffers are not garbage collected in the same way as regular (heap) objects. The heap representation of a direct buffer is lightweight and as long as the heap is not filled up, the buffers won’t be finalized. Even if the direct memory available is scarce. I have read about the same problem regarding file handles, the JVM doesn’t know that the native resource is limited. There seems to be no way of manually destroying the direct buffers either.



This means that you can get an OutOfMemory for the direct memory even though most of it is garbage. This is a problem if you create and discard a lot of direct buffers.



So, I ran into this and changed -XX:MaxDirectMemorySize to 1G . No more out of memory problems. Until I reach 1G (out of memory) or the physical RAM limit (swap to drive).



What to do? How do you handle this? Reuse of buffers seems like the C++ thing to do, and is a bit messy with different buffer sizes.



I use Sun JVM 1.6.0.03 both ubuntu and windows xp, but this will show up in any JVM I reckon.

I'm having big problems with this. Maybe someone here can come up with a nice solution.



For my scrolling backgrounds, I have different images for different resolutions. sky1_1024, sky1_768 and so on. When the user changes display mode from within the game, I need to load the right image for that display mode. This means that after a few display mode changes the jvm starts to swap to drive and everything grinds to a halt, since the direct buffers of the previous background images are not garbage collected even though they are garbage.



Any ideas? Has anyone else run into problems with this? Am I being thick?

We already reuser buffers for "common" operations (because if you don't do this, the GC will bring your application to a stutter). Other "temporary" structures (such as collision trees) are Java objects and not native.



Good to see you post this link though, cause some people did report OutofMemory exception without being out of Memory. A "pool" of direct buffers for your scene object could help, if you (re)create a lot of them, but the problem of course is that you can't resize them. We could make some changes in jME that you can use buffers bigger than what you actually need, but it's still a weird solution.



The C++ way of doing things would be to just delocate them when you don't need them, which unfortunatly Java can not do.

You may need to tell the GPU to remove the image.



look at TextureManager.doTextureCleanup(), this may mean that you also need to keep track of what textures you dont want.

That bug link is for Java 1.4.2-BETA. I would say you are linking to a very outdated bug in a beta version, and that bug is not relevant to the current state of affairs with Java.



I have made a test will allocating a lot of direct buffers and then making them garbage collectible by nullifying the pointer to the buffer. It works like a charm. I have also tested direct buffer allocation / deallocation with TextureManager and textures loaded into the video card memory with Java happily deallocating unneeded buffers.



Your problem might be that you are not releasing those buffers by keeping a pointer to them inside the JVM.

The actual bug is not relevant but the information therein.



I can't seem to reproduce this every time, it currently only occurs 1 time out of 10. Most of the time the JVM happily eats up all my memory but keeps playing nicely with others and itself :slight_smile: . Once in a while the swapping to disk gets jammed up really quick.



I might be on the wrong track. I'll do some additional profiling.