Help on OOM issue loading terrains from file

Me again,
I tried to put AssetManager back in as a loader for the tiles but again memory leaks.
Does anyone know how to disable AssetManager from caching or how to clear it out???

These are my two versions of loaders:

BinaryImporter version: (no leaks but needs absolute paths - Maybe I need to look for an alternate to File…I get the feeling I shouldn’t be using it…)
[java] // BinaryImporter verson:
BinaryImporter bImp = BinaryImporter.getInstance();
bImp.setAssetManager(app.getAssetManager());
File file = new File(tileDirectory+tileName+".j3o");
TerrainQuad tq = (TerrainQuad) bImp.load(file);
[/java]

And the AssetManager version: (Works for tiles packed into the assets folder but is leaking memory…)
[java] // AssetManager loader version (one line but leaks memory!):
//TerrainQuad tq = (TerrainQuad) app.getAssetManager().loadModel(tileName+".j3o");
[/java]

For now I’m staying with the BinaryImporter version as it works, but since it’s not nice with the whole asset management I’m not sure if this would damper the chances of this being a viable plugin to the SDK.
Right now the code grabs the users home dir and puts/loads the tiles from a set directory in there but I feel this would have issues for mobile platforms…

For now I’ll go back to reading through the source code for the AssetManager and compare to the binaryImporter to see if I can figure it out ut if anyone has ideas I’m all ears.

Thanks.

http://hub.jmonkeyengine.org/javadoc/com/jme3/asset/DesktopAssetManager.html#clearCache()
http://hub.jmonkeyengine.org/javadoc/com/jme3/asset/DesktopAssetManager.html#deleteFromCache(com.jme3.asset.AssetKey)

You are using the assetManager wrong anyway if you use BinaryImporter directly, no wonder it can’t cache properly. ClearCache also won’t work as you don’t have those keys to identify the assets (the paths are something you pass with the file).
The content of this post is meant to be read as a straight information or question without an implicit dismissive stance or interest in having the other party feel offended unless there’s emotes that hint otherwise or there’s an increased use of exclamation marks and all-capital words.

@pspeed: I’ll try the general clearcache() function and see if it helps, thanks. As normen says, can’t clear keys I don’t have unless I can get them…

@normen: Sorry if I’m not understanding but I’m only using binaryImporter as an alternative at the moment, when I use the assetManager.loadModel I only use the one line, all the binaryImporter stuff gets commented out. Likewise when I’m using BinaryImported the AssetManager line is commented out.
When I use BinaryImporter on its own I get no memory leaks… that’s the only reason I’m using it.

Thanks.

@radanz said: @pspeed: I'll try the general clearcache() function and see if it helps, thanks. As normen says, can't clear keys I don't have unless I can get them...

Here is what loadModel(String) does:
[java]public Spatial loadModel(String name){
return loadModel(new ModelKey(name));
}
[/java]

Hopefully you can figure out how to construct the proper asset key from that. Or do that from the beginning using loadModel(ModelKey) and just keep the keys around or whatever.

If you clear the whole cache it clears everything, materials, textures, etc… all of which will have to be reloaded the next time you load one.

@radanz said:Sorry if I'm not understanding but I'm only using binaryImporter as an alternative at the moment, when I use the assetManager.loadModel I only use the one line, all the binaryImporter stuff gets commented out. Likewise when I'm using BinaryImported the AssetManager line is commented out.
Yeah, sorry, I was all tl;dr there but you got the sensible core of what I wanted to say so thanks ;) As the asset caching stuff is new and there seemed to be issues with the terrain there could have been other issues but with this. But you have just disabled caching etc. completely by going to "system level" to grab your data basically. So "normally" just use the asset manager and it should "normally" also only bail with oom's when its really due (as in you want to *hold* more data in memory than possible). The content of this post is meant to be read as a straight information or question without an implicit dismissive stance or interest in having the other party feel offended unless there's emotes that hint otherwise or there's an increased use of exclamation marks and all-capital words.

Guys, really sorry, seems I was’t waiting long enough for the cache to clear itself… just spent the last 5 mins wandering all over my test terrain on both cases and although the AssetManager version of the loader seems to take a bit longer to clear it is actually clearing.
So I’ll apologize and go back to testing and coding…
(might need a break first :wink: )

(edit) A touch more info, my current test map is using small tiles so they’re not using much memory, direct mem goes up to around 100Mb uses then clears down to about 50Mb. then builds again,
I’ll re-test with some larger terrains to double check.

@radanz said: Guys, really sorry, seems I was't waiting long enough for the cache to clear itself.... just spent the last 5 mins wandering all over my test terrain on both cases and although the AssetManager version of the loader seems to take a bit longer to clear it is actually clearing. So I'll apologize and go back to testing and coding.. (might need a break first ;) )

(edit) A touch more info, my current test map is using small tiles so they’re not using much memory, direct mem goes up to around 100Mb uses then clears down to about 50Mb. then builds again,
I’ll re-test with some larger terrains to double check.

Curious, how are you determining direct memory usage?

Added “BufferUtils.setTrackDirectMemoryEnabled(true);” in the main app startup
Then hit the M key every now an then as I wander over the tiles.
Gives me something like this:

Existing buffers: 4631 (b: 63 f: 2303 i: 9 s: 2256 d: 0) Total heap memory held: 69367kb Total direct memory held: 183347kb (b: 154653kb f: 26687kb i: 0kb s: 2005kb d: 0kb)

Just tried with larger tiles (128x128)
BinaryImporter method: no issues: Peaked at about 230Mb (limit is 256mb) then dropped to around 50Mb, then up to 220Mb, then dropped again. would not oom

AssetManager method: Built steadily through the range with a few small drops until around 250Mb then only grew slow like it was trying to clear up but eventually hit and went oom

I’ll try playing with memory configs and go over it again after a break and see if I can nail it down. There’s still a difference in the two methods…

If you can set up a test case for this then I’m sure @Momoko_Fan will be interested.
The content of this post is meant to be read as a straight information or question without an implicit dismissive stance or interest in having the other party feel offended unless there’s emotes that hint otherwise or there’s an increased use of exclamation marks and all-capital words.

What memory settings are you using? ie: do you specify any non-default max heap or max direct memory settings on the command line or in the VM options?

@normen: Just added some functions and pushed the code up, it’s usable on its own for testing if it will do: TerrainTiler Source, just grab and run.
On first run it will create a TempTerrainTest directory in the user’s home dir and then generate 32x32 tiles. they’re scaled up and a bit ugly but they serve the purpose.
In the Main.java simpleapp just after the TerrainTiler construction is a switch line: terrainTiler.setAssetLoader(false); (fairly explanatory)
set to false it uses Binary importer and I can go anywhere and no OOM, set to true so it uses AssetManager it will go OOM for me before I reach the far corner of the map.
I’ve also added a line to print the buffer sizes after each tile load. just for debugging…

If you need a smaller simpleapp only testcase then I’ll try to put something together… (for ref: BufferUtils.printCurrentDirectMemory(null):wink:

@pspeed: Currently using default settings, which on this PC means I go OOM at 256Mb in direct Buffer memory. I have set it higher like 512Mb but it still eventually OOMs

The only difference in my testing is how the TerrainQuads are loading from file… I don’t know if it is the assetmanager cache or something else, but there is a difference.
The other oddity is when I remove all LOD uses it doesn’t go OOM on either case, so to my mind it says something is creating a reference issue between (maybe) the cache and the LOD buffers… something like a cyclic reference maybe, but thats just me speculating without knowledge.
Bizarre problems like this always make for fun debugging… :frowning:

Thanks.

If your app runs out of direct heap faster than regular heap then you will get OOM errors because the GC is never run. Direct memory does not cause GC even if it’s almost full. Try setting direct memory even higher… like 1024m.

Or if you are worried that it’s a leak, add a debug key to force GC to run and see if that helps.

I just now found a way to monitor direct memory usage. I will be posting that in a sec.

http://hub.jmonkeyengine.org/forum/topic/monitor-direct-memory-usage-in-your-app/

@pspeed: Read it, cool class, just what I wanted before: stats on the HUD, I’ll integrate it later. Make life easier.

On the memory settings I have indeed tried higher settings, up and up again until I ran OOM on system memory (this pc runs win7 32-bit so only really 3.2Gb available.
Also I’ve run the netbeans profiler on the app and I can see the GC getting run, plus I have tried forcing System.gc() and still to no avail.

Thanks tho.

@radanz said: Also I'm a little worried about a comment in the code for TerrainTestTile.java that says the order in which the terrains are added to MultiTerrrainLODControl seems to matter.
Oh, that can be removed. Ignore it. I always comment when I code and forgot to remove that one.

i reduced a test case for the OOM problem vs the two methods for loading tiles. Hope its not too big
Set the boolean as the start of simpleInitApp to set which loader to use.
Note: NO LOD is used, just basic terrainQuads.

Let me know if its acceptable or not…
Thanks

Part 1 cause its eating it
[java]
package mygame;

import com.jme3.app.SimpleApplication;
import com.jme3.export.binary.BinaryExporter;
import com.jme3.export.binary.BinaryImporter;
import com.jme3.font.BitmapText;
import com.jme3.material.Material;
import com.jme3.math.Vector3f;
import com.jme3.renderer.RenderManager;
import com.jme3.terrain.geomipmap.TerrainQuad;
import com.jme3.terrain.heightmap.HillHeightMap;
import com.jme3.util.MemoryUtils;
import java.io.File;
import java.io.IOException;
import java.util.HashMap;
import java.util.logging.Level;
import java.util.logging.Logger;

/**

  • test

  • @author radan
    */
    public class Main extends SimpleApplication {

    public static void main(String[] args) {
    Main app = new Main();
    app.start();
    }
    private int loadit;
    private int unloadit;
    private HashMap tiles;
    private boolean AssetLoader;
    private float tick;
    private BitmapText helloText;

    @Override
    public void simpleInitApp() {
    // Choose false for BinaryImporter
    // or true for AssetManager.loadModel()
    AssetLoader = false;
    //
    if (!new File(“assets/QUAD-000.j3o”).isFile()) {
    // create 100 terrainQuads and save them for the test
    for (int i = 0; i < 100; i++) {
    HillHeightMap hMap;
    try {
    hMap = new HillHeightMap(257, 16, 64, 128);
    } catch (Exception ex) {
    Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
    return;
    }
    String name = String.format(“QUAD-%03d”, i);
    TerrainQuad tq = new TerrainQuad(name, 65, 257, hMap.getHeightMap());
    Material mat = new Material(assetManager, “Common/MatDefs/Misc/ShowNormals.j3md”);
    tq.setMaterial(mat);
    BinaryExporter bExp = BinaryExporter.getInstance();
    System.out.println("Saving tile: " + name + “.j3o\n”);
    File file = new File(“assets/”+name + “.j3o”);
    try {
    bExp.save(tq, file);
    } catch (IOException ex) {
    Logger.getLogger(Main.class.getName()).log(Level.SEVERE,
    "Error: Failed to save Tile: " + name, ex);
    }
    }
    }
    loadit = 0;
    unloadit=0;
    tiles = new HashMap();
    // Load the 1st 4 tiles
    for (int i = 0; i < 4; i++) {
    try {
    tiles.put(i, LoadTile(loadit));
    } catch (IOException ex) {
    Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
    }
    rootNode.attachChild(tiles.get(i));
    loadit++;
    }
    tick = 0;
    /**
    * Write text on the screen (HUD)
    */
    guiNode.detachAllChildren();
    guiFont = assetManager.loadFont(“Interface/Fonts/Default.fnt”);
    helloText = new BitmapText(guiFont, false);
    helloText.setSize(guiFont.getCharSet().getRenderedSize());
    float hmem = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory();
    hmem /= 1048576;
    float dmem = MemoryUtils.getDirectMemoryUsage();
    dmem /= 1048576;
    String text = String.format(“Heap: %03.3fmb, Direct: %03.3fmb”, hmem, dmem);
    helloText.setText(text);
    helloText.setLocalTranslation(300, helloText.getLineHeight(), 0);
    guiNode.attachChild(helloText);
    cam.setLocation(new Vector3f(-50f, 50f, -50f));
    cam.lookAt(new Vector3f(100f,0f,100f), Vector3f.UNIT_Y);
    }
    [/java]

Gah, see what others mean by the bbcode eating code
Heres the part 2
[java]

@Override
public void simpleUpdate(float tpf) {
    tick += tpf;
    if (tick &gt; 1f) {
        tick = 0;
        rootNode.detachChild(tiles.get(unloadit));
        tiles.get(unloadit).detachAllChildren();
        tiles.remove(unloadit);
        unloadit++;
        unloadit %= 100;
        try {
            tiles.put(loadit, LoadTile(loadit));
        } catch (IOException ex) {
            Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
        }
        rootNode.attachChild(tiles.get(loadit));
        loadit++;
        loadit %= 100;
    }
    float hmem = Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory();
    hmem /= 1048576;
    float dmem = MemoryUtils.getDirectMemoryUsage();
    dmem /= 1048576;
    String text = String.format("Heap: %03.3fmb, Direct: %03.3fmb", hmem, dmem);
    helloText.setText(text);
}

@Override
public void simpleRender(RenderManager rm) {
    //TODO: add render code
}

private TerrainQuad LoadTile(int num) throws IOException {
    String tileName = String.format("QUAD-%03d", num);
    System.out.println("Loading Tile: " + tileName + ".j3o");
    TerrainQuad tq;
    if (!AssetLoader) {
        // BinaryImporter verson:
        BinaryImporter bImp = BinaryImporter.getInstance();
        bImp.setAssetManager(assetManager);
        File file = new File("assets/"+tileName + ".j3o");
        tq = (TerrainQuad) bImp.load(file);
    } else {
        // AssetManager loader version:
        tq = (TerrainQuad) assetManager.loadModel(tileName + ".j3o");
    }
    tq.setLocalScale(0.5f);
    float x = num % 2; // 0 or 1
    float z = (num % 8) / 2;  // 0,1,2,3
    tq.setLocalTranslation(x * 128, 0, z * 128);
    if (tq == null) {
        System.out.println("Returning null");
    }
    return tq;
}

}
[/java]

1 Like

or failing that above download from here :smiley:

Source Code Main.java

Thanks.