[SOLVED] Memory issue when converting in j3o

VonDriguen · February 15, 2023, 8:32am

Dear all,

When I try to convert an object from .glb to .j3o with a right click → “convert to j3o Binary”, the JME JDK fails. And the object is only 18Mo.

I tried everything I found to solve the issue:

First I increased the memory of the project by changing the VM options to -Xmx2048m, but it does not help.

Then I increase the config file by adding some memory: default_options=“–branding jmonkeyplatform -J-Xms512m -J-Xmx1024m -J-XX:PermSize=1024m -J-XX:MaxDirectMemorySize=2048m”. The error changed a little bit. I had first this one:

 java.lang.OutOfMemoryError: Direct buffer memory
     at java.base/java.nio.Bits.reserveMemory(Bits.java:175)
     at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118)
     at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
     ...

I have now this one:

java.lang.OutOfMemoryError
	at java.base/jdk.internal.misc.Unsafe.allocateMemory(Unsafe.java:616)
	at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:122)
	at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
	...

And if I try to increase again the config file like -J-Xms1024m -J-Xmx2048m -J-XX:PermSize=2048m -J-XX:MaxDirectMemorySize=2048m, the JDK does not open anymore.

Is it my configuration which is broken or do I need to use another tool for the conversion? Or else?

Thanks in advance for your support.

tonihele · February 15, 2023, 11:06am

You can try to use the model directly in your project. Eventually a conversion to j3o should be perhaps done. j3o is optimized format for jME. Eventually meaning when your project is production ready. But nothing prevents you from using other supported formats really (.glb is supported, isn’t that a binary GLFT?). It might be even easier for you during the development to use models directly like this.

VonDriguen · February 15, 2023, 3:32pm

Hello,

Well, I just tried with the original model and, with JME JDK, I have got the following error:

java.lang.OutOfMemoryError
	at java.base/jdk.internal.misc.Unsafe.allocateMemory(Unsafe.java:616)
	at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:122)
	at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
	at com.jme3.util.ReflectionAllocator.allocate(ReflectionAllocator.java:178)
	at com.jme3.util.BufferUtils.createByteBuffer(BufferUtils.java:982)
	at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:136)
	at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:183)
	at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:192)
	...

I tried with my usual eclipse configured long time ago and it works… This means I did not understand how to correctly setup this editor. But I really like to use it to be able to use the SceneComposer.
So does anybody know how can I correctly set it up? Or is there another way to use the SceneComposer and the j3o converter?

Thanks.

sgold · February 15, 2023, 5:38pm

This sounds like either a corrupt model or a bug in JME’s glTF importer.

Please provide a more complete stack trace. Knowing where allocateDirect() is called from might provide a clue as to the root cause.
Are you able to load the GLB into glTF tools such as https://gltf-viewer.donmccurdy.com ?
Would it be feasible to obtain a copy of the failing .glb file for us to test?

tonihele · February 15, 2023, 5:54pm

VonDriguen:

Hello,

Well, I just tried with the original model and, with JME JDK, I have got the following error:

java.lang.OutOfMemoryError
	at java.base/jdk.internal.misc.Unsafe.allocateMemory(Unsafe.java:616)
	at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:122)
	at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
	at com.jme3.util.ReflectionAllocator.allocate(ReflectionAllocator.java:178)
	at com.jme3.util.BufferUtils.createByteBuffer(BufferUtils.java:982)
	at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:136)
	at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:183)
	at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:192)
	...

I understand this now that you tried this as a code? As in run it as a java application that loaded your model through jME? That has nothing to do with the SDK or IDE. Then it is the memory configuration of your app. If not a bug in GLB import (in jME or the SDK, they both use the same code here).

You could now then try to tweak the memory params of your own application to get it loaded.

This all is just a way to circumvent it for now. But what sgold said above is better so that we could see if it is a bug. Or tell you straight a good memory configuration for the SDK.

VonDriguen · February 15, 2023, 6:59pm

Are you able to load the GLB into glTF tools such as https://gltf-viewer.donmccurdy.com

Yes, I can.

Please provide a more complete stack trace. Knowing where allocateDirect() is called from might provide a clue as to the root cause

Sure, this is the complete stacktrace when I tried to load the .glb via the following code:

sceneNode = (Node)assetManager.loadModel("Scenes/classroom_edit.glb");

java.lang.OutOfMemoryError
	at java.base/jdk.internal.misc.Unsafe.allocateMemory(Unsafe.java:616)
	at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:122)
	at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
	at com.jme3.util.ReflectionAllocator.allocate(ReflectionAllocator.java:178)
	at com.jme3.util.BufferUtils.createByteBuffer(BufferUtils.java:982)
	at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:136)
	at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:183)
	at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:192)
	at com.jme3.asset.DesktopAssetManager.loadLocatedAsset(DesktopAssetManager.java:260)
	at com.jme3.asset.DesktopAssetManager.loadAssetFromStream(DesktopAssetManager.java:342)
	at com.jme3.scene.plugins.gltf.GltfLoader.readImage(GltfLoader.java:724)
	at com.jme3.scene.plugins.gltf.GltfLoader.readTexture(GltfLoader.java:695)
	at com.jme3.scene.plugins.gltf.GltfLoader.readTexture(GltfLoader.java:679)
	at com.jme3.scene.plugins.gltf.GltfLoader.readMaterial(GltfLoader.java:610)
	at com.jme3.scene.plugins.gltf.GltfLoader.readMeshPrimitives(GltfLoader.java:439)
	at com.jme3.scene.plugins.gltf.GltfLoader.readNode(GltfLoader.java:217)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:268)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readScenes(GltfLoader.java:183)
	at com.jme3.scene.plugins.gltf.GltfLoader.loadFromStream(GltfLoader.java:127)
	at com.jme3.scene.plugins.gltf.GlbLoader.load(GlbLoader.java:47)
	at com.jme3.asset.DesktopAssetManager.loadLocatedAsset(DesktopAssetManager.java:260)
	at com.jme3.asset.DesktopAssetManager.loadAsset(DesktopAssetManager.java:374)
	at com.jme3.asset.DesktopAssetManager.loadModel(DesktopAssetManager.java:417)
	at com.jme3.asset.DesktopAssetManager.loadModel(DesktopAssetManager.java:421)
	at model.scene.Scene.initialization(Scene.java:27)
	at model.state.ExplorationState.loadScene(ExplorationState.java:57)
	at model.state.ExplorationState.initState(ExplorationState.java:40)
	at core.state.GameState.singleInitialization(GameState.java:117)
	at core.state.GameState.initialize(GameState.java:101)
	at model.state.LoadingState.updateState(LoadingState.java:44)
	at core.state.GameState.update(GameState.java:132)
	at com.jme3.app.state.AppStateManager.update(AppStateManager.java:356)
	at com.jme3.app.SimpleApplication.update(SimpleApplication.java:255)
	at core.application.SimpleApplication.update(SimpleApplication.java:41)
	at com.jme3.system.lwjgl.LwjglAbstractDisplay.runLoop(LwjglAbstractDisplay.java:151)
	at com.jme3.system.lwjgl.LwjglDisplay.runLoop(LwjglDisplay.java:197)
	at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:232)
	at java.base/java.lang.Thread.run(Thread.java:834)

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Metaspace
	at java.base/java.lang.ClassLoader.findBootstrapClass(Native Method)
	at java.base/java.lang.ClassLoader.findBootstrapClassOrNull(ClassLoader.java:1257)
	at java.base/java.lang.System$2.findBootstrapClassOrNull(System.java:2129)
	at java.base/jdk.internal.loader.ClassLoaders$BootClassLoader.loadClassOrNull(ClassLoaders.java:118)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:609)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	at java.base/java.lang.ClassLoader.defineClass1(Native Method)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1016)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
	at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
	at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:698)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
	at com.jme3.system.JmeDesktopSystem$1.run(JmeDesktopSystem.java:124)
	at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:313)
	at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:770)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
	at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:715)
	at java.base/java.security.AccessController.doPrivileged(Native Method)
	at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
	at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:740)
	at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
	at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
	at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)
java.lang.OutOfMemoryError

Would it be feasible to obtain a copy of the failing .glb file for us to test?

I found it on sketchfab. But I do not found back the exact link. I uploaded it here.

VonDriguen · February 15, 2023, 7:07pm

Well, I tried to load it directly in glb format directly: sceneNode = (Node)assetManager.loadModel("Scenes/classroom_edit.glb");

But when I try to convert it using the JDK, I have an “exception box”.
memory-issue_2

And if I click on it, I got the following stacktrace:

java.lang.OutOfMemoryError
	at java.base/jdk.internal.misc.Unsafe.allocateMemory(Unsafe.java:616)
	at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:122)
	at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
	at com.jme3.util.ReflectionAllocator.allocate(ReflectionAllocator.java:178)
	at com.jme3.util.BufferUtils.createByteBuffer(BufferUtils.java:982)
	at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:128)
	at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:183)
	at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:192)
	at com.jme3.asset.DesktopAssetManager.loadLocatedAsset(DesktopAssetManager.java:260)
	at com.jme3.asset.DesktopAssetManager.loadAssetFromStream(DesktopAssetManager.java:342)
	at com.jme3.scene.plugins.gltf.GltfLoader.readImage(GltfLoader.java:724)
	at com.jme3.scene.plugins.gltf.GltfLoader.readTexture(GltfLoader.java:695)
	at com.jme3.scene.plugins.gltf.GltfLoader.readTexture(GltfLoader.java:679)
	at com.jme3.scene.plugins.gltf.GltfLoader.readMaterial(GltfLoader.java:609)
	at com.jme3.scene.plugins.gltf.GltfLoader.readMeshPrimitives(GltfLoader.java:439)
	at com.jme3.scene.plugins.gltf.GltfLoader.readNode(GltfLoader.java:217)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:268)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:276)
	at com.jme3.scene.plugins.gltf.GltfLoader.readScenes(GltfLoader.java:183)
	at com.jme3.scene.plugins.gltf.GltfLoader.loadFromStream(GltfLoader.java:127)
	at com.jme3.scene.plugins.gltf.GlbLoader.load(GlbLoader.java:47)
	at com.jme3.asset.DesktopAssetManager.loadLocatedAsset(DesktopAssetManager.java:260)
	at com.jme3.asset.DesktopAssetManager.loadAsset(DesktopAssetManager.java:374)
	at com.jme3.asset.DesktopAssetManager.loadModel(DesktopAssetManager.java:417)
	at com.jme3.gde.core.assets.SpatialAssetDataObject.loadAsset(SpatialAssetDataObject.java:94)
	at com.jme3.gde.core.assets.actions.ConvertModel$1.run(ConvertModel.java:65)
	at java.base/java.lang.Thread.run(Thread.java:834)

I hope my 2 messages clarifies

tonihele · February 15, 2023, 7:09pm

Yes, thank you. I tried it and it worked out of the box. I can already see from the icons that your SDK release is quite old. I tried with the latest, I haven’t touched any memory flags, just plain install. Maybe you encountered a old bug. Is it possible for you to upgrade your SDK to the latest version?

VonDriguen · February 15, 2023, 7:54pm

I tried. I have uninstalled the old version and installed the 3.5.2. But it still fails (right click → convert):

java.lang.OutOfMemoryError: Direct buffer memory
    at java.base/java.nio.Bits.reserveMemory(Bits.java:175)
    at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:118)
    at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
    at com.jme3.util.ReflectionAllocator.allocate(ReflectionAllocator.java:171)
    at com.jme3.util.BufferUtils.createByteBuffer(BufferUtils.java:995)
    at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:122)
    at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:184)
    at com.jme3.texture.plugins.AWTLoader.load(AWTLoader.java:193)
    at com.jme3.asset.DesktopAssetManager.loadLocatedAsset(DesktopAssetManager.java:272)
    at com.jme3.asset.DesktopAssetManager.loadAssetFromStream(DesktopAssetManager.java:355)
    at com.jme3.scene.plugins.gltf.GltfLoader.readImage(GltfLoader.java:756)
    at com.jme3.scene.plugins.gltf.GltfLoader.readTexture(GltfLoader.java:727)
    at com.jme3.scene.plugins.gltf.GltfLoader.readTexture(GltfLoader.java:712)
    at com.jme3.scene.plugins.gltf.GltfLoader.readMaterial(GltfLoader.java:643)
    at com.jme3.scene.plugins.gltf.GltfLoader.readMeshPrimitives(GltfLoader.java:475)
    at com.jme3.scene.plugins.gltf.GltfLoader.readNode(GltfLoader.java:252)
    at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:303)
    at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:311)
    at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:311)
    at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:311)
    at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:311)
    at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:311)
    at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:311)
    at com.jme3.scene.plugins.gltf.GltfLoader.readChild(GltfLoader.java:311)
    at com.jme3.scene.plugins.gltf.GltfLoader.readScenes(GltfLoader.java:218)
    at com.jme3.scene.plugins.gltf.GltfLoader.loadFromStream(GltfLoader.java:155)
    at com.jme3.scene.plugins.gltf.GlbLoader.load(GlbLoader.java:76)
    at com.jme3.asset.DesktopAssetManager.loadLocatedAsset(DesktopAssetManager.java:272)
    at com.jme3.asset.DesktopAssetManager.loadAsset(DesktopAssetManager.java:388)
    at com.jme3.asset.DesktopAssetManager.loadModel(DesktopAssetManager.java:439)
    at com.jme3.gde.core.assets.SpatialAssetDataObject.loadAsset(SpatialAssetDataObject.java:94)
    at com.jme3.gde.core.assets.actions.ConvertModel$1.run(ConvertModel.java:65)
    at java.base/java.lang.Thread.run(Thread.java:829)

But:

I was surprised that I did not have to setup the workspace and all of the projects were already present in the projects tab. I assume the “old” configuration was stored somewhere and does not change.
In your screenshot, I see you have 2.5Gb of space. Mine has only 269Mb… I assume this is the problem.

Before doing anything stupid, how do you recommend me to increase the memory buffer?
FYI, the VM options of the project is already “-Xmx1048m”. But I am convinced it is the memory of the JDK I need to increase…

Thanks.

tonihele · February 15, 2023, 8:04pm

It has something to do with memory yes. But I have never played with those settings. Do you have a 64-bit OS? Windows?

I guess the conversion used quite a bit memory because the memory jumped to that 2,4Gb. Since when I started the IDE it just says 600Mb. So I just don’t have any restrictions. My SDK is using the bundled JDK (JDK 11).

edit: I have 64-bit Windows and 32Gb of RAM. Maybe default Java configuration allows it to reserve a % of this memory? Like so: How is the default max Java heap size determined? - Stack Overflow

VonDriguen · February 16, 2023, 7:45am

I have 64-bit Windows and only 8Gb of RAM. And when I looked on the link you provide, my max heap size is 2Gb. And during the process of conversion, you jumped to 2,4Gb…

Is there any other solution for me to convert the file into j3o without having to change my computer?

tonihele · February 16, 2023, 8:16am

So it is close but no cigar… Those in the link are defaults if I understood correctly. So by tweaking the memory options, one could go higher? You already did this for the SDK and eventually it didn’t start anymore. Maybe it would be easier then, as you already did, just open this with code if the memory flags on the app itself are more flexible?

I can also try to take a look that why opening this model consumes so much memory. From your stacks I saw that it finally crashed allocating memory for a texture. Lets see that jME is not clinging to unneeded buffers at least.

VonDriguen · February 16, 2023, 2:18pm

OK, I continue to try a lot of things to increase the memory and it all fail. I will use eclipse because I can load the .gtl and use it in my game.

Is there an option to, in my code, create a .j3o and record it somewhere? I read in a book that, at the end, only j3o are kept when the executable is created. Also, I really would like to be able to use the SceneComposer.

Thanks.

tonihele · February 16, 2023, 4:44pm

The way you build your application is really up to you. You can include models other than .j3o. You can even do that on a production level application.

You can create .j3os also yourself by code. Just load the model and export it straight as .j3o. But that is how the SDK also does it, so the same memory constraints apply.

There is also this, it does the same. But maybe since it doesn’t do anything but that, the memory is just enough to run it:

VonDriguen · February 18, 2023, 7:08pm

Many thanks for your help. I use now eclipse and I do as you proposed.

I consider now this topic as closed. My memory problem is not solved but the issue related to the conversion in j3o models has now a workaroud

Thanks again for the time you spent on this request.