Making the terrain move instead is no problem really. Its the same backwards. OpenGL is 32bit, theres not much use in raising to 64bit for the whole scenegraph, the visual issues will appear anyway. I also don’t think that its needed for all types of games. Even with physics involved (which “bugs out” even before the visuals) you can easily make 50 square miles areas with the “traditional” approach. Unless you have procedurally generated terrain 50 square miles means a lot of work in terms of editing the map for an indie developer.
@normen I think 50 square miles will be enough for any monkey 
If we do moving the terrain rather than the player, how does that work in terms of mapping “human” map coordinates to the correct place on the terrain?
It could be good to have the terrain manager able to get and set a latitude and longitude position (+40.689060 -74.044636)? That would make it much easier to make maps and teleports and things (and you could theoretically just dump an image of the terrain into Google maps or similar)
There would be an intermediate translator that does all that. As far as the programmer should be aware, they just set a boolean or something to indicate “conveyor belt mode” and the rest should be done automatically from behind the scenes.
Anyway. I’m just finishing up the image-based section and ill push some code.
2 Likes
Ok, i’m not done with the scene-based world yet, but the noise-based and image-based worlds are done.
https://github.com/jayfella/TerrainWorld/
Points of note:
The camera position is monitored in the update loop, and when the camera moves out of a tile, a new thread is spawned that calculates the new tiles required and the old tiles to remove. New tiles are created in this thread. After that has been determined, a final chunkstate is created containing the results, and we push back to the GL thread so that we can add/remove from the rootNode safely. Before the tile is added or removed, the tileLoaded or tileUnloaded event is fired. You can use this event to cancel the load or unload of the tile, or manipulate the tile as you see fit.
The reason we have the option to cancel loading and unloading are quite interesting. Say for example we have a side-scrolling motorbike game. we know that we dont need want tiles above +3 or -1 on the Z plane, although they may be triggered while we move around:
This is a prime example of wanting to be able to cancel a TileLoad event.
A good example of cancelling a TileUnload event would be for something like keeping important areas loaded. Maybe you have some amazing physics object doing some work, or you just want to keep a town loaded, or whatever. These situations would be a prime example of wanting to cancel a TileUnload event.
I thought about adding another event that fires inside the thread after each tile creation, so you are able to manipulate the tile before the thread dies… Something like a preTileLoadEvent. Maybe you want to change the material, a height, or something… It would be useful to be able to to this inside a threaded environment. Im not sure, so I left it out for now.
You are able to set a viewDistance in the North, East, South and West direction according to the initial direction of Vector3f.UNIT_Z. So you could, for example, have a landing-strip type scene - again, I guess useful for certain type of games.
Regarding Vegetation: The way I have it in my game is by extending TerrainQuad and adding a couple of Nodes called “staticRigids” and “staticNonRigids”. On tile creation I pick random locations from within the tile and add to each item accordingly (a tree would go in staticRigids, some grass would go in staticNonRigids, for example). I then optimize the nodes via GeometryBatchFactory and attach these nodes to the terrainQuad. This means Tile loading and unloading also does the same for the vegetation. I could implement this too if we felt it was of genuine use.
2 Likes
@jayfella this is absolutely AWESOME! I am testing it right now and I love it!
I was finding the tiles loading way too slow at first and they were appearing after I got there, but after increasing the view distance and slowing the camera right down, it seems to work pretty well indeed!
Some tweaks I can suggest though:
-
Could we fade the alpha of the tiles in an out as they appear/disappear so it’s not so obvious?
-
When loading tiles, instead of doing:
[java]
+++
[/java]
wouldn't it be better to it prioritised steps so the most important tiles are pushed to the view before we waste time calculating the others?
[java]
+
===
===
[/java]
[java]
+=
===
===
[/java]
[java]
==+
===
===
[/java]
[java]
===
===
[/java]
- represents a tile being added
= represents a tile staying the same
~ represents a tile being removed
If you set a view distance to zero, it won’t load any in front of you. Set it to a minimum of 1 and it should be fine. Also of note, I found that a block size of 129 and tile size of 65 with a view distance of 2 yielded the best “middle ground” for me.
Personally, I think the time it would take to calculate the priority of the chunks it should load first would be longer than it would take to just load them. I guess the fog filter would be the best solution to mask world loading - at least it seems to be the way most games mask that behaviour…
It’s a little difficult for me to guage performance, i’m running an i7 @ 4.5GHz with an ati 7970, what are your framerates like? Try this: on line 53 of NoiseBasedWorld.java - try adjusting the float value of the LOD control to maybe 2.0 - see if that helps any.
I should probably add a method to change that value.
I should also note that these tiles are being generated on-the-fly. Significant performance gains can be made by loading them from a save file.
@jayfella said: Personally, I think the time it would take to calculate the priority of the chunks it should load first would be longer than it would take to just load them.
In general, this greatly depends on how much data you are loading and how fast the player can move. It’s pretty easy to come up with scenarios where local paging never catches up because for some reason the fringe was sorted first in the priority queue. For example, if you move north and start loading the northern border then immediately start heading west very fast, it’s possible that you are always loading that northern border or whatever. (This is considering the case of a 3x3 local view or a 5x5 local view… I think the 2x2 styles waste way too much space.)
This used to happen all the time in Mythruna and I actually did have a PriorityQueue. The issue is that the items in the queue weren’t getting resorted as new items were added (and the player moved to the point where the old priorities were bad). Now whenever the camera crosses a boundary, I empty and readd all items still in the priority queue. This is still faster than loading a tile.
Also, look into java.util.concurrent. The idea of “starting a thread whenever X…” is pretty ugly and there are much more elegant and performant solutions.
@pspeed I can setup a ScheduledThreadPoolExecutor instead of firing a new thread every time - It just seemed a little overkill for one single thread, however It is my intention to do so when I implement the cache, as I would need to check the lifetime of each item in the cache at a regular interval. I guess using a Deque too instead of a List when I add the tiles would save on some vital milliseconds. I’ll implement them now and get back on the results.
I am getting pretty good frame rates on my old Q6600 with GTX280… around 500fps. The problem for me is due to tile loading.
For example, in the github code, exactly as posted, the camera moves pretty fast. It seems to take about 3-5 seconds between the triggering of a new set of pages, and for those pages to actually appear. I can quite easily reach the end of the world in those 5 seconds at the speed of 300.
As @pspeed says, it gets much worse if you trigger a “false alarm” - for example, if I edge forward slowly, just enough to start the new pages loading, then I shoot forward at full speed, it will load 1 row of new pages in, but actually by the time I get there, it needs 2 or 3 rows. So it’s playing catch up and things start noticeably popping around.
Obviously, with bigger tiles and slower cameras the effect is much less noticable, and with the tiles loaded from saved data rather than generated, it’s going to be much better still. Certainly on my machine though, the load order would be worth doing I think. It’s only going to take a few ms to calculate vs several seconds to generate all the new tiles.
With a bit more testing, I’ve also found the LOD can cause issues where you can see through the terrain… maybe it needs to be restitched at the new LOD?
p.s. @jayfella I think the paging of trees and grass should definitely be in sync with the terrain so your system would be great to try. There is also “forester” and “biomonkey” to look at there. I think @ghoust has got something quite good working based on biomonkey if I recall correctly (I seem to remember reading something about that.
Ok, well this is what posting the code was all about. Let me toy around…
@monkeychops said: @Sploreg Surely non-square grids could be as simple as allowing the tiles to be nullable?Depends on the implementation really. It's just a feature that many people requested so I suggest it be a requirement for all future endless terrain systems.
I’ve been testing various implementations over the past couple of hours. I’ve added a cache and a tile queue to avoid potential blocking of repeated requests for new tiles. I’m not entirely satisfied yet to commit anything on github, but I believe I should have something tomorrow. I didn’t realize how different it would be for lesser powered computers. a Q6600 is still a beast of a chip - I had one myself. If that chip can’t sail through, something is definitely not as it should be.
On the topic of vegetation placement, it’s definitely got to be based on this concise article: http://www.gamasutra.com/view/feature/130071/random_scattering_creating_.php?print=1
@jayfella thanks for the hard work you’re clearly putting into this - I will keep a keen eye on the github repo for when there’s something new to try out
That vegetation placement method looks really cool. I think it would be good to have trees/grass/foliage painting tools like other engines in the end though. Maybe in the shorter term we could do something with assigning grass and tree types to terrain textures, so that it randomly places them but only on the textures specified in code. That would allow grass and trees to grow on grass textures but not rock textures. Together with the slope and height based painting that should give really realistic results!
Ok, I believe that I have come up with a solution that fixes your issue, @monkeychops
Link to save you finding it again: https://github.com/jayfella/TerrainWorld/
The problem was two-fold: Firstly, it just takes time to generate a chunk. C’est la vie. I can’t solve that. But secondly, the old method removed old chunks from the scene, generated all new chunks, and then placed them in the scene - all in one frame. This is what caused the visible delay. We magnified the fact that chunk gen was time consuming by a factor of however many tiles we needed, and threw salt in the wound by adding and removing items from the rootNode too, all in one frame. Not cool.
This new method does one thing every frame, meaning the game isn’t left stuck carrying out quite an intensive task all in one frame. It also has a queue and a cache. Tiles are flagged as needed and put into a queue to get generated in a callable. This queue is checked to see if any tile has generated every frame. If it has generated, it is added to the scene and removed from the queue. This means we aren’t stuck waiting for that tile to load, and also that each step is carried out in different frames. Tiles are cached when removed. After 5 seconds (the default cache time), the tile will be removed from the cache. You can set the cache time to something else if you wish by using “.setCacheTime(long millis);”. The cache aids for when players are moving around in close proximity.
I have not added a smart “load the tiles in front of me first” system, but if these changes are a success, I can move onto that.
@monkeychops please do let me know how this works for you.
1 Like
@jayfella initial results really good!
It’s massively faster… it runs about 3x the FPS now and the tiles are loading about 5 times faster. At the default settings you can clearly see them being drawn in front of you but it’s much much better.
I will experiment with some larger tiles and slower camera and add some fog, I have a feeling this will be quite usable
Main issue remaining seems to be noticable holes in the terrain, blue background showing through in more distant areas. My guess is the LOD is messing with the stitching.
@monkeychops said: Main issue remaining seems to be noticable holes in the terrain, blue background showing through in more distant areas. My guess is the LOD is messing with the stitching.A quick way to test that is to remove the LOD control and see if the holes are still there. If not, then check that the neighbourFinder is actually finding the correct neighbours.
@monkeychops - fantastic. I’ll do some more optimizations, but one that comes to mind already. on line 45 of World.java, change it to this:
[java]
ScheduledThreadPoolExecutor threadpool = new ScheduledThreadPoolExecutor(Runtime.getRuntime().availableProcessors() * 2);[/java]
That will help generate the tiles quicker.
It is definitely the LOD control, I can see what you are referring to myself, try changing line 57 of NoiseBasedWorld.java - change the float value to 3.0f
@jayfella both those tweaks seem to have made an improvement but both issues are still there.
I will try to make a video so you can see what I see
[video]https://vimeo.com/78760153[/video]
Here’s a video
The quality is low because I had issues with the screencam lagging it up otherwise, but the white lines and paging are visible. It’s pretty good, but maybe we can do better?
The seems are clearly visible at 16-20 seconds, the holes in the terrain seem to be mostly gone though.
I am finding there’s quite a noticable jolt in performance as it loads the tiles… it judders and the frameright drops right down for 500ms or so.
I wonder - would it be smoother if the tile loading thread, rather than wait for the next trigger, went ahead and pre-loaded the next set of tiles in the direction of travel right away, before they’re needed? That way, when we trigger the loading of new tiles, if it turns out we need the set we were expecting, we just show them, with no perceivable lag. If it turns out to be a different set that’s needed (because we changed direction) just throw them away and load the actual set we need.