Out Of Memory Exception when loading Spatials from another Thread

Hello guys, I’m loading stuff from another thread, so as to not slow down the main thread (slowing animations and other stuff down in say a loading gui) and after loading something once or twice I get this error.

Exception in thread "AdvancedLoadingAppState" java.lang.OutOfMemoryError: Direct buffer memory
	at java.nio.Bits.reserveMemory(Bits.java:693)
	at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
	at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
	at com.jme3.util.BufferUtils.createIntBuffer(BufferUtils.java:934)
	at com.jme3.scene.mesh.IndexBuffer.createIndexBuffer(IndexBuffer.java:73)
	at com.jme3.terrain.geomipmap.LODGeomap.writeIndexArrayLodDiff(LODGeomap.java:165)
	at com.jme3.terrain.geomipmap.LODGeomap.createMesh(LODGeomap.java:86)
	at com.jme3.terrain.geomipmap.LODGeomap.createMesh(LODGeomap.java:78)
	at com.jme3.terrain.geomipmap.TerrainPatch.cloneFields(TerrainPatch.java:972)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:255)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:160)
	at com.jme3.util.clone.ListCloneFunction.cloneFields(ListCloneFunction.java:66)
	at com.jme3.util.clone.ListCloneFunction.cloneFields(ListCloneFunction.java:43)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:228)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:160)
	at com.jme3.scene.Node.cloneFields(Node.java:725)
	at com.jme3.terrain.geomipmap.TerrainQuad.cloneFields(TerrainQuad.java:1801)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:255)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:160)
	at com.jme3.util.clone.ListCloneFunction.cloneFields(ListCloneFunction.java:66)
	at com.jme3.util.clone.ListCloneFunction.cloneFields(ListCloneFunction.java:43)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:228)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:160)
	at com.jme3.scene.Node.cloneFields(Node.java:725)
	at com.jme3.terrain.geomipmap.TerrainQuad.cloneFields(TerrainQuad.java:1801)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:255)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:160)
	at com.jme3.util.clone.ListCloneFunction.cloneFields(ListCloneFunction.java:66)
	at com.jme3.util.clone.ListCloneFunction.cloneFields(ListCloneFunction.java:43)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:228)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:160)
	at com.jme3.scene.Node.cloneFields(Node.java:725)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:255)
	at com.jme3.util.clone.Cloner.clone(Cloner.java:160)
	at com.jme3.scene.Spatial.clone(Spatial.java:1360)
	at com.jme3.scene.Node.clone(Node.java:682)
	at com.jme3.scene.Node.clone(Node.java:62)
	at com.jme3.scene.Spatial.clone(Spatial.java:1448)
	at com.jme3.scene.Spatial.clone(Spatial.java:70)
	at com.jme3.asset.CloneableAssetProcessor.createClone(CloneableAssetProcessor.java:48)
	at com.jme3.asset.DesktopAssetManager.registerAndCloneSmartAsset(DesktopAssetManager.java:317)
	at com.jme3.asset.DesktopAssetManager.loadAsset(DesktopAssetManager.java:379)
	at com.jme3.asset.DesktopAssetManager.loadModel(DesktopAssetManager.java:416)
	at com.jme3.asset.DesktopAssetManager.loadModel(DesktopAssetManager.java:420)
	at com.epicest.reusables.PixelApplication.openLoad(PixelApplication.java:270)
	at com.epicest.reusables.loading.AdvancedLoadingAppState.update(AdvancedLoadingAppState.java:53)
	at com.jme3.app.state.AppStateManager.update(AppStateManager.java:287)
	at com.epicest.reusables.loading.AdvancedLoadingAppState.run(AdvancedLoadingAppState.java:60)
	at java.lang.Thread.run(Thread.java:745)

I’ve tried doing something like System.gc(); after running my loader but nothing happens. my conflicting code:

 public void openLoad() {
  try {
   final Spatial scene = assetManager.loadModel(load);
   this.enqueue(new Callable<Boolean>() {
    @Override
    public Boolean call() {
     initScene(scene);
     return true;
    }
   });
  } catch (Exception e) {
   loaderAppState.markForLoad();
  }
 }

Where load is a String specifying the path to my j3o scene and initScene(); attach’s my scene to the rootNode and prepares other stuff (such as Character Controls, App States etc.)

If it’s failing from a separate thread then it will fail similarly from the main thread. There is no special magic here, really.

So the answer is that you are loading more things than will fit in memory, I guess.

Well dang, I just tested this, and your right, so I guess my question should really be how to better organize my variables to remove unwanted junk.

Or increase the direct memory size… though you really should look into the others also.

We have no idea because we literally know nothing about your game, scene, files except that they are too big and use terrain.

I’ve already increased the memory size, to 4 Gig, really I think the biggest thing that my scene holds is the terrain, and otherwise it’s just a bunch of links to other j3o scene files I’m using as models, would those links I have be the problem? I doubt it, but I don’t really know. the scene file’s tree is small enough to fit on my screen, so I doubt it’s too big, but idk, I’m finding out I can’t move my terrain to somewhere more… manageable so I’ll probably just recreate my terrain with existing height map files, maybe that will fix something, probably not.

:chimpanzee_woot: Yeah… that’s a problem.

Well, those links are going to be sucked in and displayed, though, right? So they will take up memory, too.

But yeah, it sounds like you are loading too much terrain at once.

Here are some examples I wrote years ago regarding paging terrain.

I can’t remember which one is the newest, but it demonstrates how to add and remove things as you move around.

https://github.com/jayfella/TerrainWorld

https://github.com/jayfella/World

1 Like

Oh hey, I was just writing that I don’t really know how to do that, but it would probably help a lot. :smiley:

Oh wow, while trying to regenerate a terrain (haven’t been through that process for a while) the error actually happened to the SDK

Could it be that I’m using the 3.1 beta2 snapshot? I never had this problem generating terrain on 3.0

We can’t know, because you aren’t providing us with any information. All I can presume personally is that your scene is way too big, and thus you have run out of allocated memory.

Well I could upload the source of the projects I’m using doing this, assets and all, because I really don’t know any other info that might help this, but I doubt anyone here wants to shuffle through my code, trying to find info on this.

The first port of call is what you are loading, and when this occurs. Does it occur when you load a specific object - or after a period of time? If it’s the former - the object is way too big and needs to be sectioned off into chunks. If it’s the latter, there are too many objects in the scene and you need to again chunk it all out into sections.

I have a feeling you are loading objects into the scene with no regard for unloading them. So as you move around, more and more objects are being added, but nothing is being removed. Is that correct?

I load the scene into memory to put into the rootNode, remove everything from the rootNode and then add the scene I loaded. I realize I have no regard for loading in and loading out nicely… there are a couple ways I can optimize this code (such as detaching everything from the rootNode before loading stuff, and I realize im removing view port procc3essors when im basically re-adding them, a couple other things to that extent)

 /**
  * Loads a scene.
  */
 public void initScene(Spatial newScene) {
  //reset scene
  curScene = newScene;
  rootNode.detachAllChildren();
  viewPort.clearProcessors();
  //init scene
  assetsInit();
  displayInit();
  physicsInit();
  scriptingInit();
  otherInit();
  //finish
  //niftyState.gotoScreen("hud");
  loadState = LOAD_FINISHED;
  isInWorld = true;
 }

 /**
  * Initializes assets Honestly I'm not even sure I need this.
  */
 protected abstract void assetsInit();

 /**
  * Initializes any graphical needs Such as putting the scene you are loading on
  * the screen, and any filters.
  */
 protected void displayInit() {
  //Display
  rootNode.attachChild(curScene);
  // Drop shadows
  DirectionalLight sun = new DirectionalLight();
  sun.setDirection(new Vector3f(-0.5f, -0.5f, -0.5f));
  //[1.0, 0.9, 0.7, 1.0]
  final int SHADOWMAP_SIZE = 1024;
  DirectionalLightShadowRenderer dlsr = new DirectionalLightShadowRenderer(assetManager, SHADOWMAP_SIZE, 3);
  dlsr.setLight(sun); //<==assuming a directional light
  //viewPort.addProcessor(dlsr);
  //Post Processor Filters */
  viewPort.addProcessor(assetManager.loadFilter("Shaders/Default.j3f"));
 }

 /**
  * Initializes physics.
  */
 protected void physicsInit() {
  //Initiallize AppState
  stateManager.detach(stateManager.getState(BulletAppState.class));
  bulletAS = new BulletAppState();
  stateManager.attach(bulletAS);
  bulletAS.getPhysicsSpace().setGravity(normalGravity);
  inputManager.setCursorVisible(false);
  //bulletAS.getPhysicsSpace().enableDebug(assetManager);
  //SceneCollisions
  CollisionShape sceneShape = CollisionShapeFactory.createMeshShape(curScene);
  landscape = new RigidBodyControl(sceneShape, 0);
  curScene.addControl(landscape);
  bulletAS.getPhysicsSpace().add(landscape);
  //Player
  playerInit(spawn);
  //TODO: Invisible Walls
 }

 /**
  * Initializes the player physics object at a spawnpoint.
  *
  * @param spawnNode the name of a Node for the player to spawn at.
  */
 protected void playerInit(String spawnNode) {
  stateManager.detach(stateManager.getState(FlyCamAppState.class));
  stateManager.detach(stateManager.getState(CharacterInputAppState.class));
  stateManager.detach(stateManager.getState(InteractablesAppState.class));
  //Player
  Node playerNode = new Node("Player");
  playerNode.setLocalTranslation(rootNode.getChild(spawnNode).getWorldTranslation());
  //FOJItems itemset = new FOJItems();
  if (player == null) {
   player = new GameCharacterControl(1.1f, 5f, 7f, playerNode.getWorldTranslation());
   player.setGravity(normalGravity);
   player.setCamera(cam);
   player.setTrait(113, "a");
  }
  player.setSpatial(playerNode);
  playerNode.addControl(player);
  bulletAS.getPhysicsSpace().add(player);

  charInAS = new CharacterInputAppState();
  charInAS.setCharacter(player);
  stateManager.attach(charInAS);
  interactAS = new InteractablesAppState();
  stateManager.attach(interactAS);
  rootNode.attachChild(playerNode);
 }

 /**
  * Initialises ScriptAppState.
  */
 private void scriptingInit() {
  scriptAS = new ScriptAppState();
  stateManager.attach(scriptAS);
  //Initialize any scripts applicable in the scene
 }

 /**
  * Anything else needed to initialize during scene loading.
  */
 protected abstract void otherInit();

I figured out a temporary solution, listed below, that keeps this from happening in the IDE (and most likely games too,) but for now I’m just staying away from large terrains.

Temporary Solution: https://jmonkeyengine.github.io/wiki/sdk/increasing_heap_memory.html

o_O

Not the heap memory itself, wich i was increasing, but the direct memory

You can see my confusion. :slight_smile:

oh. didn’t see that sorry :stuck_out_tongue: