Training pills: procedural endless terrain, vegetation batching and LODs

Hello everyone,
I started this thread to talk with you about the terrains in JME.
As an exercise, I tried to write a solution to optimize the vegetation generation time by taking advantage of multithreading and a cache. I used the internal code of the jme3-terrain module to write a simple and clear example, but there are many other ways to do this.


  • TreeGridListener.java
/**
 * 
 * @author capdevon
 */
public class TreeGridListener implements TerrainGridListener {

    private static final Logger log = Logger.getLogger(TreeGridListener.class.getCanonicalName());

    public enum ThreadingType {
        Sequence, Parallel
    }

    private Application app;

    // cache needs to be 1 row (4 cells) larger than what we care is cached
    private LRUCache<Vector3f, Node> cache = new LRUCache<>(20);
    private Node gridNode = new Node("TreeGrid");
    
    private ThreadingType threadingType = ThreadingType.Parallel;
    private ExecutorService cacheExecutor;
    private PlantingAlgorithm pAlgorithm;

    public TreeGridListener(Application app) {
        this.app = app;
    }
    
    public Node getGridNode() {
        return gridNode;
    }

    private Vector3f toCellSpace(TerrainQuad quad) {
        int chunkSize = quad.getTotalSize() - 1;
        Vector3f scale = quad.getWorldScale();
        Vector3f tileCell = quad.getWorldTranslation().divide(scale.mult(chunkSize));
        tileCell = new Vector3f(Math.round(tileCell.x), tileCell.y, Math.round(tileCell.z));
        return tileCell;
    }

    @Override
    public void gridMoved(Vector3f newCenter) {
    }

    @Override
    public void tileAttached(Vector3f cell, TerrainQuad quad) {
        if (threadingType == ThreadingType.Sequence) {
//            Uncomment this code to use sequential mode.
//            The application may freeze a few seconds or slow down due to data processing time.
//            
//            Vector3f coord = toCellSpace(quad);
//            Node node = cache.get(coord);
//            if (node == null) {
//                node = pAlgorithm.generateData(quad);
//                cache.put(coord, node);
//                System.out.println("Loaded Node " + node.getName() + " from TerrainQuad");
//            }
//            gridNode.attachChild(node);

        } else {
            if (cacheExecutor == null) {
                cacheExecutor = createExecutorService();
            }
            System.out.println("Submit new UpdateTreeQuadCache for " + quad);
            cacheExecutor.submit(new UpdateTreeQuadCache(quad));
        }
    }

    @Override
    public void tileDetached(Vector3f cell, TerrainQuad quad) {
        Vector3f coord = toCellSpace(quad);
        Node node = cache.get(coord);
        if (node != null) {
            node.removeFromParent();
            System.out.println("Unloaded Node " + node.getName() + " from TerrainQuad");
        }
    }

    class UpdateTreeQuadCache implements Runnable {

        protected final TerrainQuad quad;

        protected UpdateTreeQuadCache(TerrainQuad quad) {
            this.quad = quad;
        }

        @Override
        public void run() {
            Vector3f coord = toCellSpace(quad);
            Node node = cache.get(coord);
            if (node == null) {
                node = pAlgorithm.generateData(quad);
                // Cache the results to optimize resources
                cache.put(coord, node);
                System.out.println("Loaded Node " + node.getName() + " from TerrainQuad");
            }

            final Node newNode = node;
            app.enqueue(() -> {
                // execute in the jME3 rendering thread.
                gridNode.attachChild(newNode);
            });
        }

    }

    /**
     * This will print out any exceptions from the thread
     */
    protected ExecutorService createExecutorService() {
        final ThreadFactory threadFactory = new ThreadFactory() {
            @Override
            public Thread newThread(Runnable r) {
                Thread th = new Thread(r);
                th.setName("jME TreeGrid Thread");
                th.setDaemon(true);
                return th;
            }
        };
        ThreadPoolExecutor pool = new ThreadPoolExecutor(1, 1, 0L, TimeUnit.MILLISECONDS,
                new LinkedBlockingQueue<Runnable>(), threadFactory) {
            @Override
            protected void afterExecute(Runnable r, Throwable t) {
                super.afterExecute(r, t);
                if (t == null && r instanceof Future<?>) {
                    try {
                        Future<?> future = (Future<?>) r;
                        if (future.isDone())
                            future.get();
                    } catch (CancellationException | ExecutionException ex) {
                        t = ex;
                    } catch (InterruptedException ie) {
                        Thread.currentThread().interrupt(); // ignore/reset
                    }
                }
                if (t != null) {
                    t.printStackTrace();
                }
            }
        };
        return pool;
    }

}
  • TreeGenerator.java
    This class implements a simple random tree planting algorithm.
/**
 * @author capdevon
 */
public class TreeGenerator implements PlantingAlgorithm {

    // range [1, mapSize]
    public int step = 4;
    // range [0, 1] high values, more trees
    public float density = 0.4f;
    public boolean optimize = true;
    public boolean useLods = true;
    public Spatial model;
    
    @Override
    public Node generateData(TerrainQuad quad) {
        Node batchNode = new Node("TreeNode-" + quad.getName());
        int mapSize = quad.getTerrainSize() - 1;
        Vector3f wp = quad.getWorldTranslation();
        
        int i = 0;
        for (int z = -mapSize; z < mapSize; z += step) {
            for (int x = -mapSize; x < mapSize; x += step) {

                if (FastMath.nextRandomFloat() > 1 - density) { 
                    // plant tree
                    float height = quad.getHeight(new Vector2f(x + wp.x, z + wp.z));
                    Vector3f location = new Vector3f(x + wp.x, height, z + wp.z);

                    boolean cloneMaterial = false; // share the same material for all models
                    Spatial clone = model.clone(cloneMaterial);
                    clone.setName(clone.getName() + "_" + i);
                    clone.setLocalTranslation(location);
                    // Randomize the scale and rotation of the model, if desired
                    batchNode.attachChild(clone);
                    i++;
                }
            }
        }

        if (optimize) {
            System.out.println("Batching... " + batchNode + ", useLods: " + useLods);
            batchNode = GeometryBatchFactory.optimize(batchNode, useLods);
        }
        
        if (useLods) {
            batchNode.depthFirstTraversal(new SceneGraphVisitorAdapter() {
                @Override
                public void visit(Geometry geom) {
                    if (geom.getMesh().getNumLodLevels() > 1) {
                        System.out.println("Add LodControl: " + geom);
                        geom.addControl(new LodControl());
                    }
                }
            });
        }
        
        return batchNode;
    }

}
  • Main.java
/**
 * @author capdevon
 */
public class Main extends SimpleApplication {

    private TerrainGrid terrain;
    private Node worldNode = new Node("World");

    @Override
    public void simpleInitApp() {

        ...

        Material matTerrain = createHeightBasedTerrain(assetManager);
        FilteredBasis ground = createFilteredBasis();

        int patchSize = 65;
        int tileSize = 128;
        int terrainSize = (tileSize * 2);
        float heightScale = 256f;
        terrain = new TerrainGrid("MyTerrain", patchSize, terrainSize + 1, new FractalTileLoader(ground, heightScale));
        TerrainLodControl control = new TerrainGridLodControl(terrain, cam);
        control.setLodCalculator(new DistanceLodCalculator(patchSize, 2.7f)); // patch size, and a multiplier
        terrain.addControl(control);
        terrain.setMaterial(matTerrain);
        terrain.setShadowMode(RenderQueue.ShadowMode.CastAndReceive);
        
        Spatial tree = assetManager.loadModel("Models/MyCoolTree.j3o");
        bakeLods(model, 0.5f);

        TreeGenerator treeGenerator = new TreeGenerator();
        treeGenerator.setModel(tree);

        TreeGridListener treeGridListener = new TreeGridListener(this);
        treeGridListener.setPlantingAlgorithm(treeGenerator);
        terrain.addListener(treeGridListener);
        
        worldNode.attachChild(terrain);
        worldNode.attachChild(treeGridListener.getGridNode());
        rootNode.attachChild(worldNode);
    }

    ...
}

Here are some screenshots:

The Scene Hierarchy at Runtime:

WIP:

  1. Batching & LODs
  2. Post Processing Filters…

I will do further testing on these points and update the topic with the results.

Let me know if you are interested in this topic or if you have other ideas. :wink:

11 Likes

Batch & LODs

Hello everyone,
Here is an example of how to use Batching + LODs. Let me know if and how you have used this technique in your games and if you find this kind of discussion interesting. If I made any mistakes, please let me know.

Source code:

/**
 * @author capdevon
 */
public class TestBatchLod extends SimpleApplication {

    /**
     * @param args
     */
    public static void main(String[] args) {
        TestBatchLod app = new TestBatchLod();
        AppSettings settings = new AppSettings(true);
        settings.setResolution(1280, 720);

        app.setSettings(settings);
        app.setShowSettings(false);
        app.setPauseOnLostFocus(false);
        app.start();
    }

    @Override
    public void simpleInitApp() {

        configureCamera();

        initLights();

        Node model = (Node) assetManager.loadModel("Models/MyCoolTree.j3o");
        bakeLods(model, 0.5f);

        System.out.println("Batch optimize... " + model);
        boolean useLods = true;
        Node batchNode = GeometryBatchFactory.optimize(model, useLods);

        batchNode.depthFirstTraversal(new SceneGraphVisitorAdapter() {
            @Override
            public void visit(Geometry geom) {
                if (geom.getMesh().getNumLodLevels() > 1) {
                    System.out.println("Add LodControl: " + geom);
                    geom.addControl(new LodControl());
                }
            }
        });

        rootNode.attachChild(batchNode);
    }
    
    private void configureCamera() {
        float aspect = (float) cam.getWidth() / cam.getHeight();
        cam.setFrustumPerspective(45, aspect, 0.01f, 1000f);
        
        flyCam.setMoveSpeed(50);
        flyCam.setDragToRotate(true);

        cam.setLocation(Vector3f.UNIT_XYZ.mult(5));
        cam.lookAt(Vector3f.ZERO, Vector3f.UNIT_Y);
    }

    private void initLights() {
        viewPort.setBackgroundColor(new ColorRGBA(0.7f, 0.8f, 1f, 1f));

        AmbientLight al = new AmbientLight();
        rootNode.addLight(al);

        DirectionalLight dl = new DirectionalLight();
        dl.setDirection(new Vector3f(-1, -1, -1).normalizeLocal());
        rootNode.addLight(dl);
    }

    /**
     * Computes the LODs and bakes them into the mesh. 
     * @param model
     * @param reductionValues Valid range is a number between 0.0 and 1.0
     */
    private void bakeLods(Node model, float... reductionValues) {
        model.depthFirstTraversal(new SceneGraphVisitorAdapter() {
            @Override
            public void visit(Geometry geom) {
                LodGenerator lodGenerator = new LodGenerator(geom);
                lodGenerator.bakeLods(LodGenerator.TriangleReductionMethod.PROPORTIONAL, reductionValues);
                System.out.println(geom + ", NumLodLevels: " + geom.getMesh().getNumLodLevels());
            }
        });
    }
}

Output:

bark (Geometry), NumLodLevels: 2
leaves (Geometry), NumLodLevels: 2
Batch optimize... Scene (Node)
Add LodControl: batch[0] (Geometry)
Add LodControl: batch[1] (Geometry)
  • Here’s the scene hierarchy before batching:

  • Here’s the scene hierarchy after batching (Num Lod Levels > 1 + LodControl)

Note:
I replaced the old tree model because the LogGenerator could not generate a LOD layer of the geometry. I still don’t understand in which situations the algorithm fails to generate the LOD of the mesh.

Updated the code examples in the previous message with the generation of LODs on 3D models.

5 Likes

Hi @capdevon,
Inspired by examples kindly provided by you, I have managed to greatly reduce the lag caused by the trees generation and their attachment process, which was happening during the Tile loading process.

Code refactoring also resulted in terrain unloading bug being gone so I can get rid of the abomination workaround TerrainGrid inside FractalTerrainGrid now.

6 Likes

Hi @Arifolth
I’m glad my examples inspired you to optimize the tree generation process! The sky color variation looks great. Did you use any free libraries?

To further enhance the scene, consider experimenting with different terrain generation parameters to create a more natural look. Additionally, exploring alternative grass placement algorithms could significantly improve realism. For example, in this video I only placed the green spheres (which should be replaced with a more detailed grass model) on a given ground texture using the alpha map. This approach could help achieve a more organic distribution. :wink:

1 Like

Did you use any free libraries?

Yes, I’m using slightly modified version of jmeDayNight for the Sky, please note it contains some GPL3 code.

To further enhance the scene, consider experimenting with different terrain generation parameters to create a more natural look.

Thanks for the suggestions @capdevon!
That is indeed very interesting topic, FractalTerrainGrid has a lot of settings, which could probably be changed dynamically at runtime. Once, I have bumped into the nice idea - to use a whole control plane consisting of different types of 3D points, each type mapped to certain terrain parameter, so as you move forward of backward/left or right in this control plane, terrain generation parameters changes appropriately. Would be nice to implement a mechanism like this.

Additionally, exploring alternative grass placement algorithms could significantly improve realism.

I think the vegetation could be improved much much further, if existing (transparent quad) grass placement engine would get combined with the shader based one


Need to work on GLSL stuff for this.

2 Likes