java.lang.OutOfMemoryError: Direct buffer memory with material or scene loading


I’m having the issue below when I load a custom material (containing a shader) or a scene from the jMonkeyEditor containing that material. It loads well the first time, but crashes the second time (after I close the window and reload it again - for example if I change the shader code). My current workaround is to restart jME every time.

I have seen several posts containing a similar error message (but most happen in-game) and I’ve looked at what I could apply, including the advice from @normen here : Java.lang.OutOfMemoryError: Direct buffer memory - #3 by Momoko_Fan

But the jmonkey*.conf in etc/ already has quite a high MaxDirectMemorySize parameter

default_options="–branding jmonkeyplatform -J-Xms24m -J-Dsun.java2d.dpiaware=true -J-Dapple.laf.useScreenMenuBar=true -J-Dawt.useSystemAAFontSettings=lcd -J-Dswing.aatext=true -J-Xmx512m -J-XX:MaxDirectMemorySize=2048m -J-Dsun.java2d.noddraw=true"

In-game (where I use the shader on thousands of different objects) I never saw the error

I’m not sure where to look, do you think it could be an issue in the shader itself (essentially I just declare a matrix 3x3 and a few floats, I assume they should get cleared from memory after usage, shouldn’t they?)


Actually I couldn’t (or didn’t want to) believe it but that parameter was actually too low, raising it to 8192m solves the issue (temporarily)

Which leads me to two other questions:

  • my scene only contains two (simple) meshes, one of them using the shader (which is a billboard shader), and I see that just loading it takes 4G. Is it expected to have such an overhead just because of a shader on a single mesh?

  • more importantly, if I look at the memory usage, I see that if I reload multiple time the scene, I always end up reaching the memory cap and trigger that error again. Is it possible that the issue is in the SDK itself? I tried the same thing with a scene that has no shader, and the issue doesn’t happen (when you reload the scene multiple times, the memory always reaches the same level)

I have the exact same issue!

But you don’t open the Shaders themselves in the Shader Editor then?
Anyway it looks like the SDK might have some kind of “memory leak”, however it’s tricky because technically we do nothing more than run the engine inside, so it’s just like assetManager.loadModel, we don’t care about the shaders.

What is true is that the SDK doesn’t seem to clear the AssetCache so it could be that the memory rises because of that, but that should also happen without shaders and should not affect reloading the scene…

What are the shader counts in the stats? Do they continually rise as the game is running?

I offer you to use SpaceShift Editor :wink:

1 Like

@Darkchaos Yes I also try to open the shaders in the shader editor. The crash happens, but later on (like I would say I can open the shader editor 7-10 times before memory cap is reached, vs 3-4 times for the scene editor. Do you want me to send you the shader I have so that you can try to reproduce?

@jayfella when I run an actually SimpleApplication on that scene, the shader count doesn’t increase with time (it goes up or down depending on what the camera sees)

@javasabr what is it? any link?

Yeah, the shader would help however given that this seems to happen for anything it seems like a rather central bug. On the other hand you say it doesn’t happen without a shader? mhh

What jayfella meant is clicking the small “i” in the SceneComposer and looking at the Shader Count there

Count is similar (= not growing with time)

Shader is here