Well… the title says it all. I’m not sure of the best approach for implementing a simple paging system for procedural terrain, vegetation, etc. Any and all info, links to articles, even opinions of what you think is the right way to approach this would be ultra helpful. There is plenty of info on the web, however, since I’ve never done this, I don’t want to pick an approach that sucks not knowing it sucks.
Thanks in advance!!
Is that still for a learning purpose or is that you need it?
Because, if you need a paging system for vegetation and terrain, you have the Forester made by @androlo and the JME terrain system that is into the core.
If it’s for a learning purpose, i guess @androlo and @sploreg would have some insight.
One or more of the GPU Gems books has a few articles on this subject, I think. I know there is at least one article.
@nehon said:
Is that still for a learning purpose or is that you need it?
Because, if you need a paging system for vegetation and terrain, you have the Forester made by @androlo and the JME terrain system that is into the core.
If it's for a learning purpose, i guess @androlo and @sploreg would have some insight.
This is for learning purposes only... I'm working on that procedural terrain thing (which everyone and their brother/sister has done) in my spare time and the paging system was one of the reason I got interested in trying it to begin with. Although, I have some fairly specific needs for vegetation, so I might be working on that specifically for the game I'm working on. One of the things I would like to do while playing around with it is build out the trees/bushes utilizing some sort of... um... fractal based mesh generation (I'm guessing it would be)? I want to incorporate the GPU Animation thingy I wrote and a few other ideas. Not to step on any ones toes, but I'll probably post the outcome of that effort... maybe @androlo would be interested in incorporating the mesh generator and vegetation animation at some point.
Speaking of the terrain thingy... @pspeed Thanks a ton for the idea about the skirts. It worked great! Only thing I have left to change after adding this is a minor issue with texture coordinates. I'm sure it's an easy fix... it's just escaping my tiny brain atm.
I'll see if I can track down the articles in GPUGems.
Forester (as it was called) was based on the Paged Geometry plugin for Ogre. It’s a simple paging system with distance based lod built in. Seems they are planning on adding terrain streaming to jME, in which case I’ll most likely just use terrain grid listeners to page in trees and grass etc. In fact I got something like that set up already, using my own crappy terrain page loader as a placeholder (its like terrain grid but without LOD or stitching).
A few “musts” i noticed was to divide pages up into smaller pieces, and use a separate page type for grass and one for trees etc. The page size would be the terrain quad size, and each pageloader has their own resolution (number of pieces within a page) and their own far viewing range and lod settings. Otherwise you’re stuck with the same settings for grass and trees and terrain, and can only attach/detach entire pages-worth of tree/grass. In theory that may be better, but in practice it causes lag spikes and nobody likes that.
A secondary page cache I noticed was also a good idea. Like if you move camera to the right, and it causes the page loader to load new stuff, then quickly move back left, it’ll just start loading new tiles where the old ones were. If someone moves back and forth across two tiles it will trigger the pageloader a lot of times, creating lots of new stuff unneccesarily and triggering those nasty full GC’s that locks the entire app down.
I put in a timed secondary cache that catches discarded pages and store them for some time. Every time the page loader loads new pages it checks the secondary cache first to see if there are pages there whos indices match. In that case it just switches them in instead of creating new ones. The pages are also “bumped” every time they’re re-used, to store frequently used pages longer. It’s broken now tho, for some reason…
Normen talked about including the asset manager in the terrain streaming, maybe it’s for a similar reason.
I wouldn’t use forester/biomonkey code as a reference, making a lot of big changes atm. Paged Geometry itself is pretty clean, it could be used I guess. It’s a bit dated but apparently they are still using it. http://www.ogre3d.org/2012/09/02/ogre3d-powered-space-epic-salvation-prophecy-released
1 Like
Seeeeew… I decided to wing it, roll my own from what I guessed my be a decent approach and just see what happened. Since I’m just doing this for fun… I thought this way would be MORE fun 
It’s relatively simple thus far, but runs as it’s own thread in the background managing adding/removing terrain tiles (not from the scene… just the cache). It runs pretty smooth, actually… although, I did have a question.
Currently, I’m testing it by building out relatively small tiles (a radius of 8 tiles from the center… camera). This way as I move, it has to quickly add tiles whose position would enter the radius and removes the ones that exceed the radius. Every now and then, there is slight pause… say… every 8-15 second. It is almost unnoticeable & is completely gone when I set the tile size to something a bit more appropriate. But!! I’m interested in knowing what this is from. I know it has something to do with adding new objects to the scene… but what I am curious about is why it wouldn’t always happen then?
Anyways… if anyone knows… I’d love to as well.
If you are adding and removing items from the scene then it might be the garbage collector. Things build up until Java decides to get rid of them… and that’s when JME will purge the native objects, too. It used to do all of the reclaimed objects at once (which I saw take 20 seconds one time in Mythruna) but now limits it to 100 per frame.
Still, it’s possible that if you have a lot of objects to reclaim that you might notice a hitch… especially if it has to reclaim enough objects that it maxes it for two or three frames.
The garbage collection idea sounds pretty solid to me.
Also (and this theory won’t make much sense if the pause is very regular) it could be the JVM growing the heap size.
If you’re storing terrain pages in a cache, and the cache isn’t timing out entries or removing them in some other way, then memory consumption will be steadily growing. Once it reaches the current heap limit, the JVM will increase the heap size, if possible. This could well cause a momentary pause in your application.
One test would be to profile the JVM and monitor when the heap gets exhausted and grows - it should match up with the pauses in the game. The profiler might slow down the system too much to make this a useful test though, so in any case you could increase the minimum heap size to match the maximum heap size (by setting something like -Xms512m -Xmx512m in your JVM launch settings), which will avoid any heap resizing at runtime.
However, that’ll really just push back the problem until the heap is eventually exhausted and you get an OutOfMemoryException… So the real problem is that the cache is always growing.
If that is the problem, the solution would be to prevent your cache from growing beyond a certain size. You could either move out-of-sight pages out of memory on to the disk (which means you may have quite a lot of disk I/O going on at runtime), or dump them completely and regenerate when the player walks back that way again (though this means your terrain generation has to be fully deterministic - and if the terrain can be modified, you’ll want to at least save the changes).
Anyway, that’s my two cents - best of luck with it!