Another OoME

Methius · June 16, 2008, 10:39pm

Hi,

I've been working on a MoleculeViewer application, and it's necessary to load a lot of different (large; 3000+ nodes) models.

The problem is that I get a OoME exception after I load a particular large molecule for a third time:

System Setup:

Q6600 2.4Ghz

Vista X64 Ultimate

JDK1.6

Ati 2900 XT

4gb RAM

12gb Page File

java.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorjava.lang.OutOfMemoryErrorException in thread "Thread-10" java.lang.OutOfMemoryError at sun.misc.Unsafe.allocateMemory(Native Method) at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:99) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288) at com.jme.util.geom.BufferUtils.createFloatBuffer(Unknown Source) at com.jme.util.geom.BufferUtils.createVector2Buffer(Unknown Source) at com.jme.scene.shape.Box.setTextureData(Unknown Source) at com.jme.scene.shape.Box.setData(Unknown Source) at com.jme.scene.shape.Box.<init>(Unknown Source) at View3D.Binding.<init>(Binding.java:33) at moleculeviewer.MoleculeImport.importMol2H(MoleculeImport.java:92) at View3D.RenderGamestate.importMolecule(RenderGamestate.java:226) at moleculeviewer.GUI$11.run(GUI.java:555) # # An unexpected error has been detected by Java Runtime Environment: # # java.lang.OutOfMemoryError: requested 32756 bytes for ChunkPool::allocate. Out of swap space? # # Internal Error (allocation.cpp:120), pid=9292, tid=15828 # Error: ChunkPool::allocate # # Java VM: Java HotSpot(TM) Client VM (10.0-b22 mixed mode windows-x86) # An error report file with more information is saved as: # D:ProjectsMoleculeViewerhs_err_pid9292.log # java.lang.OutOfMemoryError # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp # The crash happened outside the Java Virtual Machine in native code. # See problematic frame for where to report the bug. # Java Result: 1 BUILD SUCCESSFUL (total time: 44 seconds)

To make things even more interesting, the Heap size is only around 90Mb at the point of the crash.
(with -Xms256m -Xmx1024m -XX:MaxDirectMemorySize=512m)
I also made sure that all the nodes are dereferenced, deParented, cleared from RenderStates, and the TextureManager is given the clear command before every molecule load.

Heap stack search gives me no remaining objects which could trigger the OoME. (Using Netbeans, HeapWalker)

The only thing I can think of is that all the atoms are locked() and that that GL data is not being released?
(Although I unlock() them in the destroy method, before the new Molecule is loaded)

To recap:
- Created Heap Dump
- Profiled Application
- Located Memory Leak
- Still OoME @ 90mb Heap Size Usage

Any help would be greatly appreciated!

Methius · June 16, 2008, 11:25pm

Hmm, I just tried to import a model of around 3500 Atoms, 3500 Bonds.

This to a blank StandardGame, and it crashed at only 71mB Heap Space usage.

Exception in thread "Thread-10" java.lang.OutOfMemoryError at sun.misc.Unsafe.allocateMemory(Native Method) at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:99) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288) at com.jme.util.geom.BufferUtils.createFloatBuffer(Unknown Source) at com.jme.util.geom.BufferUtils.createVector2Buffer(Unknown Source) at com.jme.scene.shape.Cylinder.allocateVertices(Unknown Source) at com.jme.scene.shape.Cylinder.setRadius(Unknown Source) at com.jme.scene.shape.Cylinder.<init>(Unknown Source) at com.jme.scene.shape.Cylinder.<init>(Unknown Source) at View3D.Binding.createNormalBinding(Binding.java:131) at View3D.Binding.createBinding(Binding.java:99) at View3D.Binding.<init>(Binding.java:75) at moleculeviewer.MoleculeImport.importMol2H(MoleculeImport.java:110) at View3D.RenderGamestate.importMolecule(RenderGamestate.java:228) at moleculeviewer.GUI$11.run(GUI.java:560)

Profiler within NetBeans must be set wrong somehow. Even though it is giving the correct amount of objects created, the RAM usage (Task Manager) is much higher. And reaches 771Mb before failing.

Strangely, increasing the -XMX, makes the program fail earlier... 1500Mb instead of 1024Mb -XMX, gives a fail at 300Mb.

MrCoder · June 17, 2008, 2:14am

That's because it's direct memory, which is natively allocated and not heap memory. You have to use this type of memory for stuff to be sent to the graphics card for example… You need to find ways of optimizing your memory usage like using shared meshes and cleaning up properly etc

Methius · June 17, 2008, 2:25am

Thanks for clearing that up. Again, it seems that I haven't searched well enough on this.

Already knew there was a difference, but the strange thing was that the OoME takes longer to occur when Heap AND Direct are defined lower in the VM parameters. (-XMX 128Mb, -XX:MaxDirectMemorySize=256Mb)

It let's me import the large molecule more then 11 times. (which is what threw up all the questions for me)

I'll search for myself how this can be measured.

But, I do need some help on the issue of further cleaning up a Node…

The objects are no longer in the heap stack, don't have any parents/childs, every Renderstate is removed, TextureManager is used to clean texture allocation. What further do I need to do to make GL free up the resources?

MrCoder · June 17, 2008, 3:38am

We've had lots of problems with direct memory in the past, and it's an area where java has been lacking in control. We have high hopes in the upcoming java 7 which will have better control and querying possibilities with these kinds of buffers.

In jME 2.0, renanse has at least added a way of approximating the direct memory usage (if you are using BufferUtils for your allocations). Set the system property "jme.trackDirect" and then you can get some data through BufferUtils.printCurrentDirectMemory