I’ve been working on trying to optimize and create the first map for my game and I’m running into trouble pinpointing why my memory usage is going up in some cases. Each level in my game is going to be built by attaching Nodes created in the scene/terrain editor, and I’m finding that anytime I re-open and sculpt or paint the Terrain in one of my scenes, the memory goes up slightly, and if I reverse the changes the memory never goes back down.
I noticed this at first this when I went to reduce the size of my textures- after reducing 2 of my textures by a significant amount I reloaded the scene and the memory usage had gone up.
To test this I created a new scene and terrain with the same size, a drastically larger alpha-blend amount (I’m not sure if this affects DM or not), and all the same 5 textures with some hills. I also took the scene that was getting too big and I removed all 5 textures and flattened it, so it looks like it was just created but it has a much higher memory usage than the fresh scene with hills and textures.
Here’s a screenshot with the high memory usage from the scene I’ve frequently edited and entirely flattened (811mb - higher with a flat scene)
And this one is the new scene with 5 textures and a big hill. (544mb - lower with a more detailed scene)
The simple workaround for me for now is just to stick to 3-4 tiles and only sculpt and paint their terrains once, and then after that let the terrain go without further edits until i figure this out. It doesn’t seem to give me any problems if i place and remove objects in the scene composer, but unless I’m missing something else here, I’ve found that whenever I edit any terrain in the terrain editor, the j3o file’s size always increases regardless of whether I add or remove detail.
If i remember correctly the LOD control(?) creates its own thread for each control and you must either dispose of it correctly or manage the executor yourself by specifying one.
A threadpool is probably the easiest solution for most instances.
That seemed to do the trick for some part of the clean up that’s giving me trouble. I was able to cleanup the physics correctly but noticed if I keep replaying my game while the LODs were attached, the memory would slowly rise every time I reloaded my map.
I’ve been working on a way to page a small map of nodes and only load a small grid at a time, but I’ve stopped working on that for now because I noticed that even when I detach and cleanup all the tiles when the player dies I’m still getting nearly the same memory usage as when the map is loaded and attached. Since I fixed the LOD cleanup I’m getting closer, now I can lose and play again unlimited times without getting any memory errors, but I’m still having issues cleaning up the references to my nodes i think? If I play again after losing , the main menu sits around 900mb rather than 20mb like it does on the initial launch. The map sits at 1000-1100 max when its loaded, so it looks like I’m doing some cleanup correctly wit the physics and LODs now, but I can’t seem to get rid of the dm for my nodes. I tried calling assetManager.clearCache() when the player dies and gets sent back to the main menu, but that ends up giving me a higher memory usage when I restart.
And I also don’t know if I explained the other problem I’m having with the SDK’s terrain editor well, but every time I go back into the terrain editor to edit one of my tiles and save my changes, the direct memory usage and overall file size of that j3o seems to go up no matter what I do, so my very first tile has the highest DM despite being the simplest, only because I’ve loaded and saved it many times. I tested this again when I was fixing the LODs: I went into my scenes and all I did was rename the Terrain to “terrain” so I could reference them all in a loop to get the LOD control, and my file went up by 17kb in my assets folder just from renaming the terrain.
For the longest time I thought that that Terrain just takes an unrealistic amount of memory if you sculpt lots of mountains and use more textures, but it seems as though I can make an extremely rigid, mountainous terrain with 6-7 textures and it will stay at a very low DM if I sculpt it in one sitting and never reload and change that terrain again. I’m not sure how much the actual file size of a j3o correlates to its direct memory usage, but I’m finding that once you add memory to the terrain in a j3o, there’s no way to reduce its memory usage even if you flatten it and remove all the textures.
I’m really not sure if this is something I’m doing wrong that’s causing my j3o scenes to do this or if its a glitch, but something is definitely off here with my memory. In my last two screenshots, the top one contains the scenes “startZone.j3o” and “waterFallTop.j3o” and the bottom one contains “startZone.j3o” and “testPlane.j3o” so you can see that I flattened out my scene that was previously a waterfall and removed all of its textures, yet it’s memory actually went up higher than it already was and is higher than the scene on the bottom right that i sculpted in one sitting. It would make sense if the file size gets bigger since there’s a history section for j3o scenes in the editor, but I assume the direct memory usage of the loaded model is not supposed to work like this. IDK if i messed something up or not but I’m really getting myself confused here trying to get a few scenes loaded at the same time
Here’s the code, I based this off of the multi-threaded loading screen example in the tutorials, and as of now everything with the cleanup visibly works, I just need to fix this to clean out the memory and make room for new tiles when the old ones are unloaded.
private boolean load = false;
private ScheduledThreadPoolExecutor exec = new ScheduledThreadPoolExecutor(1);
private Future loadFuture = null;
public int tileWidth, totalX, totalZ;
public Tile[][] mapTiles;
private boolean[][] tileAttached;
private Node visibleTiles[][];
private Node lastTiles[][];
public int visibleWidth, diffToCenter;
public int xCenterIndex, zCenterIndex, lastXCenter, lastZCenter;
private Vector3f currentLoc, currentCenter;
private float xLoc, zLoc;
public Map(GameState gs, int xTot, int zTot, int vis){
gameState = gs;
app = gameState.getApp();
tileAttached = new boolean[xTot][zTot];
visibleTiles = new Node[vis][vis];
lastTiles= new Node[vis][vis];
visibleWidth = vis; //this SHOULD be odd for best centering results
totalX = xTot;
totalZ = zTot;
mapTiles = new Tile[totalX][totalZ];
mapTiles[3][5] = new Tile(assetPath + "testPlane.j3o"); // place each Tile obejct in its corect spot manually
mapTiles[3][4] = new Tile(assetPath + "waterfallBottom.j3o");
if(visibleWidth < 3){
diffToCenter = 0;
}
else if(visibleWidth < 5){
diffToCenter = 1;
}
else if(visibleWidth < 7){
diffToCenter = 2;
}
else if(visibleWidth < 9){
diffToCenter = 3;
}
}
public void update(float tpf){
//checkes to see if the player has moved to a new tile
currentLoc = gameState.getMainAgent().getGroundLoc();
xLoc = currentLoc.getX();
zLoc = currentLoc.getZ();
if(currentCenter != null){
float xDiff = currentCenter.getX() - xLoc;
float zDiff = currentCenter.getZ() - zLoc;
double allowedDist = tileWidth * .56;
if(xDiff < 0 ){
xDiff *= -1;
}
if(zDiff < 0 ){
zDiff *= -1;
}
if(zDiff > allowedDist || xDiff > allowedDist){
pageTiles();
}
}
//load new tiles on seperate thread
if(load){
if (loadFuture == null) {
loadFuture = exec.submit(loadTilesCallable);
}
if (loadFuture.isDone()) {
// these calls have to be done on the update loop thread,
// especially attaching the terrain to the rootNode
// after it is attached, it's managed by the update loop thread
// and may not be modified from any other thread anymore!
cleanUpTiles();
for(int x = 0; x < visibleWidth; x ++){
for(int z = 0; z < visibleWidth; z ++){
int xIndex = (x - diffToCenter + xCenterIndex);
int zIndex = (z - diffToCenter + zCenterIndex);
if(xIndex > -1 && zIndex > -1){
if(tileAttached[xIndex][zIndex]==false){
if(visibleTiles[x][z] != null){
app.getRootNode().attachChild(visibleTiles[x][z]);
initLOD(visibleTiles[x][z]);
tileAttached[xIndex][zIndex] = true;
}
}
}
}
}
load = false;
loadFuture = null;
System.gc();
}
}
}
//loads specified index as center. ensure your main agent is placed in this center tile somewhere to prevent immediate re-paging after the intial loadup
public void loadStartTiles(int x,int z){
xCenterIndex = x;
zCenterIndex = z;
float newX, newZ;
newX = x * tileWidth;
newZ = z * tileWidth;
currentCenter = new Vector3f(newX,0,newZ);
load = true;
}
public void pageTiles(){
if(!load){
float half = tileWidth /2;
lastXCenter = xCenterIndex;
lastZCenter = zCenterIndex;
for(int q = 0; q < totalX; q++){
if(xLoc < (tileWidth * q) + half){
xCenterIndex = q;
q = totalX+1; //break loop and keep this as the X index
}
}
for(int w = 0; w < totalZ; w++){
if(zLoc < (tileWidth * w) + half){
zCenterIndex = w;
w = totalZ+1; //break loop and keep this as the Z index
}
}
for(int x = 0; x < visibleWidth; x++){
for(int z = 0; z < visibleWidth; z++){
int xIndex = (x -diffToCenter + lastXCenter);
int zIndex = (z -diffToCenter + lastZCenter);
if(zIndex > -1 && xIndex > -1){
System.out.println( xIndex + " , " + zIndex);
}
}
}
currentCenter.set(xCenterIndex * tileWidth ,0, zCenterIndex * tileWidth);
load = true;
}
}
public void initQuadrant(Node node){
SafeArrayList list = (SafeArrayList)node.getChildren();
for(int q= 0; q< list.size(); q++){
Spatial item = (Spatial)list.get(q);
String string = (String)item.getUserData("physics");
if(string!=null){
int mass = Integer.parseInt(string);
RigidBodyControl rbc = new RigidBodyControl(mass);
item.addControl(rbc);
rbc.setSpatial(item);
gameState.getBAS().add(item);
}
String invis = (String) item.getUserData("invis");
if(invis != null){
Material mat = app.getAssetManager().loadMaterial("Materials/invis_mat.j3m");
item.setMaterial(mat);
}
String doorName = (String) item.getUserData("zoneDoor");
if(doorName != null){
zoneDoors.add(item);
RigidBodyControl rbc = new RigidBodyControl(0);
item.addControl(rbc);
rbc.setSpatial(item);
gameState.getBAS().add(item);
}
}
}
private void initLOD(Node node){
if(node!= null){
Spatial terrain = node.getChild("terrain");
if(terrain!= null){
TerrainLodControl control = terrain.getControl(TerrainLodControl.class);
control.setCamera(app.getCamera());
}
}
}
public void cleanUpQuadrant(Node node){
SafeArrayList list = (SafeArrayList)node.getChildren();
Spatial terrain = node.getChild("terrain");
if(terrain!= null){
TerrainLodControl control = terrain.getControl(TerrainLodControl.class);
control.detachAndCleanUpControl();
}
for(int q= 0; q< list.size(); q++){
Spatial item = (Spatial)list.get(q);
String string = (String)item.getUserData("physics");
if(string!=null){
item.removeControl(RigidBodyControl.class);
gameState.getBAS().remove(item);
}
String doorName = (String) item.getUserData("zoneDoor");
if(doorName != null){
zoneDoors.remove(item);
item.removeControl(RigidBodyControl.class);
gameState.getBAS().remove(item);
}
}
}
Callable<Void> loadTilesCallable = new Callable<Void>() {
@Override
public Void call() {
for(int x = 0; x < visibleWidth; x ++){
for(int z = 0; z < visibleWidth; z ++){
int xIndex = (x -diffToCenter + xCenterIndex);
int zIndex = (z -diffToCenter + zCenterIndex);
if(zIndex > -1 && xIndex > -1 && mapTiles[xIndex][zIndex] != null){
ModelKey mk = mapTiles[xIndex][zIndex].getTileString();
lastTiles[x][z] = visibleTiles[x][z];
visibleTiles[x][z] = (Node) app.getAssetManager().loadModel(mk);
Vector3f attachLoc = new Vector3f(tileWidth * xIndex,0,tileWidth * zIndex);
visibleTiles[x][z].setLocalTranslation(attachLoc);
initQuadrant(visibleTiles[x][z]);
}
}
}
return null;
}
};
private void cleanUpTiles(){
for(int x = 0; x < visibleWidth; x++){
for(int z = 0; z < visibleWidth; z++){
int xIndex = (x -diffToCenter + xCenterIndex);
int zIndex = (z -diffToCenter + zCenterIndex);
Node tile = lastTiles[x][z];
//removes the tile from the scene, removes its
if(tile != null){
gameState.getBAS().removeAll(tile);
tile.removeControl(RigidBodyControl.class);
cleanUpQuadrant(tile);
app.getRootNode().detachChild(tile);
if(xIndex > -1 && zIndex > -1){
tileAttached[xIndex][zIndex] = false;
}
}
}
}
for(int x = 0; x < visibleWidth; x++){
for(int z = 0; z < visibleWidth; z++){ //attempting to clean up references to the nodes in the lastTiles array once the last tiles have been removed
lastTiles[x][z] = null;
}
}
}
public Vector3f getPhysicalCollision(Collidable collidable){ //
Vector3f collisionLoc = null;
CollisionResults results = new CollisionResults();
CollisionResults tempResults;
for(int x = 0;x <3;x++){
for(int z = 0; z <3;z++){
tempResults = new CollisionResults();
Node node = visibleTiles[x][z];
if(node != null){
node.collideWith(collidable, tempResults);
for(int q = 0; q < tempResults.size();q++){
results.addCollision(tempResults.getCollision(q));
}
}
}
}
if(results.size()>0){
Spatial item = (Spatial)results.getClosestCollision().getGeometry();
String string = (String)item.getUserData("physicsItem");
int count = 0; //index for the second closest collision in case it is not flagged for physics ray interaction
while(string==null && count < results.size()-1){
count++;
item = (Spatial)results.getCollision(count).getGeometry();
string = (String)item.getUserData("physicsItem");
}
collisionLoc = results.getCollision(count).getContactPoint();
}
return collisionLoc;
}
and to fill my map with tiles I just fill the mapTiles[][] array with a Tile object, which is fairly simple and only holds the Model Key for it’s node. I’m wondering if this is the problem, but I get the same results if I take out a model key and just use strings.
public class Tile {
public final String modelKey;
public Tile(String ts){
modelKey = (ts);
}
public String getTileString(){
return modelKey;
}
}
I’ll try and update and see if that works, my SDK updater is broke so I’ll have to manually when I get a chance. Right now I’m running 3.1 stable.
I’m wondering if there’s also a way to extract the height map and the splat maps from a Terrain object once you create it in the terrain editor? Then if I have the data for the terrain on its own, I don’t need to worry how large my j3o file gets since I won’t be using a j3o to load in my terrain, I could still use j3o’s for normal objects and then assemble the terrain based on the height and splat maps through code for each tile
The only other solution is if I start sculpting my terrain in blender, but I’d much rather stick to using the terrain editor. I’ve gotten some really nice looking terrains, and I like how easy it is to go back into a j3o scene and perfect the terrain but right now that seems to be whats causing my j3o’s to soar in memory usage
