I have some spare time on my hands again, and intend on finishing the code already submitted in this topic.
I’ve re-written parts of the code to get rid of some bloat, and added scaling support as previously requested, and am currently in the process of adding support for pre-created scenes.
I figured an official thread will make life easier instead of re-using the oddly-titled thread previously used. Any suggestions are welcome as always.
Once this is done I’ll be taking a stab at a character control since that has been something of a thorn too in certain situations. I’m just not knowledgeable enough to help with the lighting and shadow support you guys have been working on, but I figure there are still some areas I can help with.
You know… the more I think about this the more I’m convinced it’s not viable…
I’m working on the paging system for pre-created scenes. So essentially you have a model imported from blender or wherever. And the aim is to add and remove objects based on the distance from the camera to the object. If said object is (for example) 100f or more away from the camera, remove it from the scene, else add it. Simpru. But the way people design levels is not that simple. Often the roads are all one object, and even the floor. Large quantities of buildings are grouped together in one mesh. Take the world.zip file for example:
Is there something I’m missing here? @Sploreg and maybe @pspeed would appreciate your opinion.
Its hard to make a generic paging system thats much more than “this is the cam distance and the LOD percentage the user wishes, now tell me what you want to draw” and then having the whole implementation in the user code again. If you take mythruna, in that case the world even moves instead of the player so you don’t even have this “simple” kind of relation anymore.
Another way would be to make the generators separate and just make the framework a means of communication. For example let the terrain be generated by one system and it can publish information about the terrain to another system that places objects or vegetation on the terrain. But already at this point you have to make decisions about the end result, like are there cavities in the terrain or not etc.
So if you want to make a framework for it it will have to be exactly that, making some decisions beforehand and forcing the user to do things “its way”, as in the example of roads, rivers, sea etc. etc.
The initial implementation of TerrainGrid was extended to be able to simply load parts of the scenegraph via j3o files that would contain the neighbor terrain and could even contain AssetLinkNodes to load other objects or Controls to create vegetation. This way the communication could happen within the jME objects without any metadata, e.g. the terrain tiles would find each other and join seamlessly, vegetation controls could just initialize based on the spatials they were attached to etc. This seemed to be the most generic way that could be extended by users with “normal jME techniques” and we even had it work in SceneComposer/TerrainEditor prototypes, using the editing options that are already there instead of adding a whole new way of doing things.
That doesn’t mean you have to do it this way though, storing too much info just in the scenegraph probably isn’t too great either. Say you want to generate a map of the whole world or find a certain type of house all across the world, you’d have to load each j3o to search for them. But I guess this is also a general question of “generating the environment based on available data” or “parsing the data from a manually modeled environment to have it available in the code”. So of course your editor could generate the map and a list of all objects, too.
For roads as an example, the designer might have to paint a road underneath the actual road model. Or choose a road-like texture that can represent the road when viewed from far away and the LOD of the road removes it from view. A lot of map design is curved, or has valleys and hills to hide popping and hidden meshes. So a lot of it is up to the map designer.
As Normen said, having a one size fit all solution is really hard and never perfect. Giving the developer some basic implementations that they can take and modify on their own to fit their exact needs is best.
Hi, is the common TerrainLighting.j3md shader supposed to work with those “World” terrain classes? If not, are there any plans to adapt to the TerrainLighting.j3md or any other lighting-enabled material files to work with your terrain classes? I tried to set the Material from HeightBasedTerrain.j3md (unshaded) to TerrainLighting.j3md and (obviously) it gave me wrong input variable name errors, but they look like they’re in the same order and everything so I renamed them and I could come up with something, but it’s kind of buggy, I think it’s because normals are not what the shader expects them to be. For instance, if I pan near Z=1 i can see lighting almost correctly, but when I pan around, then everything goes dark and at some point everything in shadows become very bright. It’s pretty weird. Anyhow, you guys sure know better than me about this
Well, I managed to take apart 3 shaders here and there from the Common JME3 libs like Lighting.j3md, HeightBasedTerrain.j3md and TerrainLighting.j3md and I succeeded at making it work properly by mixing code from the 3. It took me 3 days, but hey, I’m no expert when it comes down to GLSL, so I am very happy to show you what I managed to do with the problem I posted 3 days ago, here it is:
I’m actually testing this, but I’m still on the JME3 SDK Stable release that everybody uses publicly, so therefore I’m not sure if what I see is a bug that will go away when I’ll upgrade to the nightly build, but I’m sharing it with you anyways. I have noticed that some tiles do not hide when I go away from them. Some do, some don’t. I have nothing attached to the terrain nodes yet, no other nodes and I’m using the vanilla physics too, so I have not modified anything related to your code. This is what I see from far away:
This is where my problem occurs: I could not use a raytracer on the terrain patch where the + marker is. On the right and left, I could ray trace and the little white boxes you see would follow the physics according to the terrain, so all is good, except the patches where I could place boxes would not unload ever, even if I go very very far.
Now if I go away from this point I get this:
You can see that everywhere I could not place a box (thus the ray tracer would not detect the terrain patch) the terrain would be correctly hidden. It would do exactly the same thing if I did not place any white boxes anywhere at all.
So therefore it’s pretty weird, whenever the physics work, the patch stays visible and does not “unload” if you will from the view. And then everywhere the physics would not work (ray tracer does not detect the terrain patch) it would correctly hide as I go away, which is what we expect from it on a display basis, but somehow not on a physics basis. Pretty weird no? Do you think it would have something to do with my SDK being the latest available stable version? I tried to reinstall it just in case, but it’s still the same. Next thing I would have to go on the nightly build to see if it does that too… maybe you have a clue? Maybe your load/unload thread code is not working properly with the current stable release of the SDK?
Let me know what you think! Thanks and keep it up, looks very very promising!
I have activated the Directional Light Shadows filter and on every terrain patch where the physics don’t work, the shadows also won’t work (and the terrain WILL unload when I step away from it compared to not unloading but having shadows/physics working fine). What’s your thought @jayfella ?
@.Ben. sorry for taking so long to reply. Between Starbound, the gym and my two little bundles of chaos I’ve had little time to breath. This project is unstable and probably isn’t that great to use right now, however it is a second iteration of some code I already wrote here:
The TerrainWorld project is stable and should work perfectly fine without issue, however only supports noise and image based worlds, and not pre-created scenes. It also requires nightly builds, because, as the thread I linked to describes, instead of creating one thread per LoD control, we instead use a threadpool. This ability was added in the nightly builds, which have not yet made their way to the stable release. I should also note that I use the nightly builds and I find them to be very stable and useable, and that when I find a bug or issue, the devs are usually very quick to respond and push an update to the nightlies, allowing you to carry on your merry way
For now, feel free to use that version until this one is complete. Again, my apologies for taking so long to respond.
Hi @jayfella - really cool to see you’ve done more on this!
I really think the editor is the key here. For reasons you and normen have already discussed it’s really not practical to just import scenes from blender. They need to be composed inside the editor with the paging system already running to give the full-sized world canvas to work on.
That’s how pretty much every engine I’ve seen that successfully uses paging does it.
To make this viable in the long run I think it needs to be integrated into the editor to replace the default terrain system so that we can easily create huge terrains and terraform them and place objects on them.
I’d really recommend looking at the esenthel engine editor for how he works with tiles in an intuitive and performant way.
Hi @jayfella, well thank you so much for replying. So you got 2 kids too! That’s awesome, so you share what I feel when they run around the house like hurricanes lol
Well, I have been subscribed and reading the older thread (TerrainWorld) but at some point I saw that “World” was more active on GitHub so I decided to go with the more recent, even tough now that you described the difference, it would not affect my situation because I do not use pre-created scenes at all in my procedural application. But I really want to stick with the newest, so I’ll install the nightly builds instead, also you mentionned they’re stable enough and maintained frequently, so it’s obvious I should start to use them and see what’s ahead with JME3! Thank you, I’ll post how it goes with the nightly builds later today.
EDIT: OK I installed the nightly build and now it won’t compile on LWJGL, when I do:
[java]world = new Example_NoiseHeightMap(this, patchSize, blockSize, worldHeight, worldScale);[/java]
Maybe LWJGL tightened the object modeling or something. It’s like if the construction was supposed to return a World object instead of a Example_NoiseHeightMap object even tough the class definition is: public class Example_NoiseHeightMap extends World
Any thoughts?
EDIT 2: Instead of declaring like “World world = new Example_NoiseHeightMap(…)” it would completely fix the issue if I declare like this instead: Example_NoiseHeightMap world = new Example_NoiseHeightMap(…)" not sure why, but I suspect it’s something from the newest SDK nightly build… hope it helps anyone having the same issue and/or fixing it for the upcoming 3.1 stable SDK release.
BUT…
That being said, the issue is STILL exactly the same. @jayfella, did you try to enable a simple Directional Light Shadow just to see if it does that on your setup too? BTW I notice a slight fps improvement with the nightly build! I’m very pleased with this (small) performance boost!
@jayfella said:
I should also note that I use the nightly builds and I find them to be very stable and useable, and that when I find a bug or issue, the devs are usually very quick to respond and push an update to the nightlies, allowing you to carry on your merry way :)
I’ll note here that we will break nightly at points, it will have feature freaks at times and we won’t fix it immediately while we are in a transition of engine functionality. Of course some bugs that might annoy you at the time might be fixed but unless you develop for the engine itself its never a good idea to use nightly.
@.Ben. you should be using the TerrainWorld github, not the World one. If you don’t want to use nightlies for the reasons @normen mentioned, you can remove the offending line (.setExecutor) and it will just create one thread per LoD control.
@monkeychops Yes I think you’re right, Editor integration would be a nice addition.
There has been no movement on this project because I’m still wondering about the best way to go about pre-created scenes, though I just stumbled on an idea… I had this problem when I wrote this plugin a while ago for minecraft.
The key here is that during load-time, you organize the array (the scene) into a HashMap.
So you create a class that has an integer pair. Lets call it IntLoc. Then another class called SceneCollection.
[java]Map<IntLoc, SceneCollection> organizedScene = new HashMap<IntLoc, SceneCollection>;[/java]
So during load-time you iterate over everything in the scene, bitshift the coords, and store them in the appropriate SceneCollection in the map. So now I can get the bitshifted co-ordinates of the camera and use that to query the hashmap for local scene objects.
Hi @normen, I’m totally fine with using the nightly builds and I understand it may break functionality at ANY moment, I’m totally fine with it, altough this is not the issue here. I have both the stable and edge nightly build sitting on my desktop so I can swap between the 2 to test and I wouldn’t release an app made upon the nightly build, I understand all of this.
@jayfella, I found what the whole issue was! I’m so happy to see it running perfectly now. It turns out it was my fault. By examining your World.java code under the checkForNewItems() function, I found out that you are attaching the TerrainQuad node to the rootNode on TWO different lines and a week ago, when I set it up for the first time, I had put the ShadowMode.CastAndReceive only on ONE of the two lines, same with my ray traceable node (for the physics to work) but you know, I would never have guessed you would have 2 attachment function calls in the same function. Why is that? Isn’t this not “normalized” if you will? Because it’s like repeating code that does the same thing, no? Anyhow thank you so much for your time. Sorry for the confusion, it’s all working fine for me now, even on the 3.0 stable SDK release. Yes I removed the thread pool executor calls on the 3.0 stable release last week, that’s how I figured I would make it work… but I prefer to use the nightly builds for now
When a tile is requested for addition to the scene:
Check if the tile is in the cache. If true, add it to the “pending addition” que. Remove from cache.
If not, check if the tile is already waiting to be added to the scene (“pending addition”).
If not, check if the tile is already being generated (previously requested).
We hit failure. Its nowhere to be seen, generate it.
So the two points of addition will be both from the cache and from the “pending addition” que. We have a “pending addition que” so that we can regulate how many items are added to the scene each frame. If we add too many it causes stutter (theorized to be as a result of the graphics card given too much information to parse in such a tiny amount of time) so we instead feed it baby spoons as to alleviate that side-effect.
Very interesting, thank you for sharing this knowledge. I have modified my copy of the World.java class today and achieved to make the new tiles fade in/out which looks pretty smooth, instead of having new tiles to just instantly show/hide. I personally think it was missing.
Feel free to share your improvements. I’ll be implementing my idea for scene-based worlds, and possibly the ability to add it as a control to a scene so you can use it in the editor.
…and then I pass a float to my shader to progressively blend alpha from 0f to 1f with a timer control, so that the terrain chunk appears from completely transparent to fully opaque. OK so this is working perfectly, except that the directional shadows casted by this new appearing terrain chunk are immediately 100% opaque. I would have thought that making a node transparent to a certain degree would have made that if it’s like let’s say 0.1f then the shadows are almost invisible and 0.5f are like half opaque shadows like light-gray and 1f would be full force shadows like near black, but to my surprise, a fully transparent node and a fully opaque node cast the exact same shadow opaqueness which is what I was globally by: dlsr.setShadowIntensity(0.8f);
