Memory issue while loading models with shared texture across multiple materials

There is an issue when you load an model that has multiple materials sharing the same texture. For each new material it reserves enough space to load the texture into memory. This may be unnoticeable with smaller texture files but it causes huge lag and crashes when loading a big texture such a 36mb texture atlas.

If you have 50 objects sharing the one atlas it will take 1,800mb of memory to load the file. Eventually the memory will be reclaimed once the garbage collector comes around. On low end systems this will cause the app to freeze up or crash. I tested it on a computer with 2g ram and it crashed two of five times. The garbage collector simply can’t do it’s job when the system is using 100% of ram.

I test the same model and memory usage using Unreal , Unity and LumberYard using multiple computers. The memory on each of those was around 200mb on load which is normal. It’s definitely not an issue with the model. Something in JME is causing extra memory to be reserved when it shouldn’t be.

The same thing happens when loading the model in the SDK scene explorer.

How are they shared across multiple materials? Are these JME materials or?

…because if the texture was a file on disk then this is kind of impossible. Since the texture itself wouldn’t be loaded once.

I have the same problem as you do. I use several models with several materials which have a applied one diffuse and sometimes even one normal map.

In the SDK I can open my scene only once! When I try to open it the second time, I get a DirectBufferMemory exception. I also think there is something wrong in the SDK.

Ingame there is no problem loading the scene.

Blender materials , I use modular snap to grid to pieces for my level. Technically the scene is hundreds of pieces but once I’m done I combine the pieces into cells. During the joining process blender creates new materials for the new object. I’ll end up with 20 - 40 new materials. Most of them sharing only 1 or 2 textures.

I tried creating a JME material and assigning it to all of my objects but I get the same problem when loading the scene into my game. I’m using the standard assetManager.load for all of my loading. Rather it’s JME or default materials the problem still persist.

An 8 mb .j3o with a 24mb and 36mb texture atlas should not take 2gb for memory to load. I had to change the max direct memory in the SDK to ridiculous amounts to get it to import the models as well.

You have to up the direct memory settings in the SDK config file. By default it is 2048. I set mine at 4096. It shouldn’t have to go that high but it’s a work around for now.

…but the texture is still not a file on disk?

I mean, if the texture is a file on disk then it will only be loaded once. Else, run the debugger and trace through the asset manager to find where it’s 100% totally f-ing busted.

It’s on my hard drive.

Where else would it be?

I mean, is the texture image embedded right in the material or is it loaded via the asset manager and set on the material? ie: does it have an asset key associated with it?

assetManager.loadTexture(“Textures/myTexture.png”);
assetManager.loadTexture(“Textures/myTexture.png”);

assetManager.loadTexture(“Textures/myTexture.png”);

assetManager.loadTexture(“Textures/myTexture.png”);

assetManager.loadTexture(“Textures/myTexture.png”);

assetManager.loadTexture(“Textures/myTexture.png”);

assetManager.loadTexture(“Textures/myTexture.png”);

assetManager.loadTexture(“Textures/myTexture.png”);

assetManager.loadTexture(“Textures/myTexture.png”);

assetManager.loadTexture(“Textures/myTexture.png”);

assetManager.loadTexture(“Textures/myTexture.png”);

assetManager.loadTexture(“Textures/myTexture.png”);

should only load the texture data once.

It’s embedded in the material. I never us the load texture function directly to load my model textures.

I just did some experimenting. I imported my level file which has 36mb and 67mb tga into the sdk. It crashed ,direct buffer memory error , as expected. I shrunk both those files down to practically nothing and it imported almost instantly.Same goes with the scene explorer and game loading.

In particular importing assets with large texture files is a pain in the arse. I’m not sure what is going on during the import process but it takes massive amounts of memory and processing power when handling large textures.

Even if I reassign the new correct materials there is still a pretty big chunk of memory coming out when loading.

I think this issue has flown under the radar for so long because not too many people use 60mb + texture atlases.

About SDK: As temporary solution you can just allow it to allocate more memory… It helped me to load into sdk raw 230 mb scene.

It looks like your texture is NOT in the assets folder and/or is NOT referenced with a relative path in the blender file. In that case the importer has no way of referencing the correct image file, embeds the texture in the j3o file and consequently makes them completely separate “entities” for the jME asset system.

After I import the model and create a new material then assign it a texture that is indeed in the assets path then put said material on my objects I still get the same problem.

That’s a lot of memory for 6mb j3o file with 2 textures totaling 103mb.

I’m not sure if assigning a texture via a j3m will remove the contained texture. Is the j3o actually getting smaller?

It remains the same size.

Here is the basic structure

3 materials
land mat / land mat alpha share a 67mb atlas for the objects
terrain mat share a 36mb atlas for terrain objects

I have 3 materials sharing two textures.

Via the scene explorer I manually assigned the materials to each geometry. My entire scene only has 3 materials and 2 textures.