If the frame rate is dropping then either:
a) you are on a single core CPU
b) there is something wrong
c) it has nothing to do with the loading (since it’s on a separate thread)… unless there is something wrong with that implementation.
It’s quite possible that it’s just the GPU stalling on the new data… in which case loading data differently won’t change anything.
@pspeed said: If the frame rate is dropping then either: a) you are on a single core CPU b) there is something wrong c) it has nothing to do with the loading (since it's on a separate thread)... unless there is something wrong with that implementation.It’s quite possible that it’s just the GPU stalling on the new data… in which case loading data differently won’t change anything.
a) it’s quadcore…
b) probably 
c) I guess some stuff must happen in the main thread?
It’s not a bad GPU - GTX280… it doesn’t stall like this in other engines/games, so it must be to do with our code base surely?
The update loop checks if the required tiles in the view distance are in the loaded tiles. If not, it requests the loaded tile in a callable and puts that callable in the que. In every update loop that que is iterated through, checking if future.isDone(), and if so, adds it to the rootNode and removes it from the que. This means that tile loading does not freeze the game. In fact, the way it is designed, the whole update loop should be consistent in speed no matter what you do.
“Guessing” the direction the player will would be a terrible waste of resources in my opinion. The terrain often dictates the path, and it is surprisingly easy to head east or west when your intention was to head forward.
Let me upload a video of what I see…
Ok. Obviously the framerate is pathetic as a result of FRAPS, and its kind of an unfair comparative, because i’m running an i7 @ 4.5GHz with an ati 7970 and an SSD, but what the video shows is the fluid movement throughout. This is using a view distance of 3 in all directions (49 chunks) and a camera speed of 300f.
Also note that the object count remains under 100 and the triangle count stays under 100,000 - both are way below these thresholds - this should be a walk in the park demonstration for most PC’s …
@jayfella hmm, true… I wonder what causes the lag as tiles are loaded. The more tiles I added, the more obvious the lag is.
I find with a view distance of 5 and fog distance of 1000 it’s pretty well hidden.
I have no suggestions as to why, just reporting what I am seeing 
It’s certainly better than any other JME terrain paging system I’ve tried though, and certainly usable. A lot of pro games have worse issues than this 
Try dropping the threadpool back down to 2 or 3 instead of double your CPU count. Maybe it’s just getting bogged down.
I’ve tested this just a few minutes ago and first of all, it’s really nice!
However, I do too notice both framedrops and the seems, and tried to debug some stuff as much as possible but I’ve got to go now so can’t do any more debugging.
A thing i noticed about the fps drops: when the world is loading worldtiles from the cache, there are no framedrops whatsoever. The weird thing is that they are loaded the same way as a non cached worldtile (with that I mean they’re both obtained in a Callable passed into the same thread executor…). So the loading can’t really be the problem, the detaching maybe?
Also, maybe it’s an idea to just attach worldtiles if they exists in the cache inside the loop in World#checkForNewChunks(), because creating a thread to get something out of a hashmap, that seems like quite an overhead :p?
While debugging I also tried putting the loops in checkForNewChunks() inside an else which seemed to result in alot smaller drops, I really don’t get why though because it just can’t be the loading of the tiles. Maybe I can look into it more tomorrow if you still need some help by then
[java]
if (worldTilesQue.isEmpty() == false)
{
…
} else {
for(;
{
for(; {
}
}
}
[/java]
instead of
[java]
if (worldTilesQue.isEmpty() == false)
{
…
}
for(; {
for(; {
}
}
[/java]
EDIT
I’m using an i7 930 with a GTX 460
@jayfella Wow, that’s way quicker than mine hehe. Looks a lot better at that speed!
Still, even on your machine the terrain is visibly loading as you go - can we use bigger tiles or something so that we get a large view distance without seeing the tiles appear?
@jayfella I’ve set my thread pool to equal CPU count, that seems a bit better - I will try @reveance 's change to see if it does anything for me too. I am glad for the backup so I am not going insane at least 
I dont have time right now to read the thread, but this looks very interesting. As of now, how many threads is it using? If more than maybe 2 threads are used at a time, it leaves less threads to be used by the game if the user wants to do some concurrent stuff for ai.
Also, I saw something about a cache, but is that a cache of recently used tiles? In most modern games, when a player moves out of a tile, the one taken away is not immediately destroyed, so if the player move back into that tile it doesnt have to load. Usually tiles are kept around for a certain amount of time.
@reveance I noticed my mistake too, calling cached tiles from the callable, which just gets them from the cache. I doubt its the problem, but yeah. that needs to go.
In actual fact, if I set the block size to 513 I get that stutter issue too. If set the tile sizes smaller it doesnt happen. I guess the problem is adding such a large object to the scene.
@jayfella yep - I can confirm everything @reveance has said, I get the exact same results here. Does run much smoother with the loops inside an else{}
@8Keep123 we experimented with “Runtime.getRuntime().availableProcessors() * 2” and just a couple of threads. Its negligible on the difference. It just determines how many tiles to load at the same time really. The cache is exactly as you describe. When a tile is removed from the scene it is put in the cache and kept alive for 5 seconds, though you can change that via .setCacheTime(long millis).
@jayfella said: The update loop checks if the required tiles in the view distance are in the loaded tiles. If not, it requests the loaded tile in a callable and puts that callable in the que. In every update loop that que is iterated through, checking if future.isDone(), and if so, adds it to the rootNode and removes it from the que. This means that tile loading does not freeze the game. In fact, the way it is designed, the whole update loop should be consistent in speed no matter what you do.“Guessing” the direction the player will would be a terrible waste of resources in my opinion. The terrain often dictates the path, and it is surprisingly easy to head east or west when your intention was to head forward.
Let me upload a video of what I see…
Hmmmm… I wouldn’t do it this way. Looping over the queue every frame is both not cheap and contentious with the other thread (leading it to be not cheap).
Better to have two queues… one for requests and one for results. Don’t bother with callables and isDone(), etc. as you are just doing lots of busy work.
You can still use callable, I suppose… add a Callable to the executor to load the tile which then adds a Callable to some results queue to add it. Then the update loop just pulls one result per update.
But you might do better to implement your own Runnable now so that you might use a PriorityQueue later. Add the runnable to your executor to build the tile and then it adds itself to the results queue when it’s done.
Update() then just pulls one tile from the result queue (poll()) and grabs the tile and adds it. No reason to have a Callable here.
I haven’t looked at the code… but what are you using for a cache?
yes… I see your vision… let me give that a try…
The cache is a hashmap <Co-ordinate, TerrainChunk> - so I can just call a quick .get() on the required co-ordinate. I believe this is the most performant collection and way to do so.
This is the class in question:
https://github.com/jayfella/TerrainWorld/blob/master/src/worldtest/world/World.java
I pushed the changes @reveance mentioned for World.java
> dont request from the cache using the callable/que
> if / else added in checkForNewChunks
@jayfella I’ve been testing with some different tile and block sizes… I think your settings give the best results, but there must be more we can do to improve the speed I think…
Hopefully @pspeed is onto something
p.s.
case 1024: return 10;
Don’t you want 1025?
I corrected that issue and a couple others, @monkeychops - pull the new World.java - that’s the only file that changed. I also did the requests as mentioned by reveance.
@jayfella said: I pushed the changes @reveance mentioned for World.java> dont request from the cache using the callable/que
> if / else added in checkForNewChunks
@jayfella well it’s zipping along like a bat out of hell compared to an hour ago
I bet we can do even better though.
The biggest issue I have right now (aside from the visible seams) is that the view distance is too low if I set it to 2, and the performance is too slow if I make the tiles bigger.
If I increase the view distance the load order becomes a problem because it starts loading tiles 2 squares away before the ones that are 1 square away, and it gets trippy 
Some ideas which might speed it up (I am not smart like @pspeed so just guessing):