I have an issue that is not directly linked to jMonkey but I would like to know how you guys resolve it.
I have an infinite map divided in regions. Each region is a file on the hard drive, containing heighmap, texture atlas, assets positionning, etc.
I want to load these regions at demand, while the game is running and without lag. I identify three steps :
typical file deserialisation,
region drawing in a Node,
attachment of the node to the scene.
Step 1 and 2 should be run in kind of low priority thread, while step 3 must be done in JME thread. How would you set that up? which architecture? does jMonkey have something that may help? or can you point me to an exemple?
Hi Paul and thanks for the links. I understand your Builder is running the tasks you pass to it in itâs thread but I miss two things :
How do you pass the result to jMonkey? I was waiting to find something like :
app.enqueue(rootNode.attachChild(myResultingNode));
But I canât see it anywhere. Do you use jMonkey enqueue feature? events? runnables ? something else?
In the Builder class I canât find where you manage priority to prevent frame drop. Iâve seen the maxUpdate property but I donât really understand the use of it, since an update can take more time than a frame time.
If the main thread lags depends on the size of the data needed to upload to the gpu. Unfortunately there is no easy way of dooing that on a background thread. Especially with textures you can hit that limit easy.
Multiple shared contexts would be the only solution that comes into my mind.
The BuilderState.update method is where it calls Builder.applyChanges().
@Override
public void update( float tpf ) {
builder.applyUpdates(maxUpdates);
}
It does this once a frame to pull âmaxUpdatesâ results off of the âdoneâ queue to apply them. maxUpdates is usually a fairly small number like 2 to prevent frame drops.
The BuilderReference (the thing that the builder is running) is what has a getPriority().
Itâs probably best to look at using the whole PagedGrid, though, as it handles management of the requests and so on. Unless your world is not a grid and then you will have some other more complex way to manage things but you might still look at PagedGrid to see how it pauses/resumes the builder when itâs adding a bunch of new requests. The primary method to look at would be
protected boolean setCenterCell( int xNew, int zNew )
This is called when the center zone of the grid has changed thus some things on the edge will fall off and some new edges will be added (or an entire new set of cells if the player has been teleported). At any rate, it pauses the builder at the beginning of the method to prevent any new BuildReferences from being worked on. It can then add new tasks, cancel old ones, etc. with less overhead.
At the end, it calls resume which will let the threads start grabbing new references again but it also reprioritizes everything based on the latest getPriority().
PagedGrid also deals with child grids which is what makes those code look a little more complicated than one might think. Having parent/child relationships is good for when you want to make sure not to build trees/grass/etc until the terrain exists for it to sit on since a) that usually looks really silly, and b) sometimes you canât even place these things properly without knowing the terrain first.
I wasnât able to adapt the PagedGrid class to my need, thought I think it is very close to my need. But Iâve successfully used Builder and BuilderReference. This code doesnât offer more performance than my previous implementation (I will dig into it someday) but it makes lot of things clearer. And the parent/child mechanism seems very interesting and I will certainly use it.
I have a question : The release method in BuilderReference seems to be able to be called before the build() method was complete. Did I make something wrong, or is it an unmanaged case and I have to avoid releasing a reference that is not completly built?
Typically, I enqueue a BuilderReference, the Builder execute build() method in another thread, then the release method is called from the main thread during its execution and causes exception.
This shouldnât happen. If you call Builder.release() it only marks it for release. The code in Builder.PrioritizedRef.apply() should avoid freeing it until it has completed⌠though it may block in that case, I guess.
I suppose it could be an issue if the reference was being rebuilt. Maybe thereâs a bug.
Edit: it wonât block. I will have to relearn this code to answer properly.
Ok, reread the code and wrapped my brain around it.
Things only get released if the state of the PrioritizedRef is marked as such and the ref is put in the done queue.
If the BuilderReference is still being built when Builder.markForRelease(ref) is called then it is marked for release() but otherwise not added to the done queue.
When the processing has finished, markDone() is called and if it is in the Release state (as per above) then it is finally put in the done pile like it should be.
So if you are using Builder to manage your references and there are no bugs then BuilderReference.release() should definitely not be called while BuilderReference.build() is doing its thing.
Edit: but note that a BuilderReference CAN be marked for release before build() is ever called⌠but release() shouldnât be called in that case unless build() was called and completed sometime before.
Iâve made a few tests that tend to say the same thing than you are. Itâs probably on my side.
This said, it seems to me there is a missing state. The release() code with this architecture is supposed to be the same in all situations. Look at this :
void build(){
// Node building
}
void apply(){
rootNode.attachChild(node);
applied = true;
}
void release(){
// behavior in case of release depends on whether the reference has already been attached or not.
if(applied)
rootNode.detachChild(node);
}
Wouldnât be usefull that the interface manage this, by separating the release in two, like release()/cancel() or whatever?
Anyway, great piece of code as usual. Thanks for you work, your sharing, and the time spent to help !
Okay my issue is debugged. While loading many region fast, I get this LWJGL exception :
GRAVE: An OpenGL error has occured!
org.lwjgl.opengl.OpenGLException: Invalid operation (1282)
at org.lwjgl.opengl.Util.checkGLError(Util.java:59)
at com.jme3.system.lwjgl.LwjglOffscreenBuffer.checkGLError(LwjglOffscreenBuffer.java:98)
at com.jme3.system.lwjgl.LwjglOffscreenBuffer.runLoop(LwjglOffscreenBuffer.java:126)
at com.jme3.system.lwjgl.LwjglOffscreenBuffer.run(LwjglOffscreenBuffer.java:151)
at java.lang.Thread.run(Unknown Source)
It happens since Iâve refactor to use Builder. Any idea?
Well, usually, release is cheap. In this case, you should really be calling node.removeFromParent() as that will be a no-op.
itâs important for release() to be called even if you never attached because this is also where youâd do stuff like free buffers, etc. that would have been built. Itâs generally considered that the attach/detach part is âalmost instantâ. Itâs the other stuff that we need to worry about.
Unfortunatly I donât run 3.1. Is it available via gradle now?
The strange think with this error is that it happens on my personnal computer but not on my workâs, which are quite similar. The main difference is the GPU, AMD vs nVidia. Any chance that this error would come from AMD driver?
I will run more tests.
Anyway, the Builder works well ! I have performance issues but thatâs on me. I will track down the slow code to see if there is easy optimisation to implement.
I have put the 3.1 for testing purpose and I get these warnings when I attach geometries to the scene :
[JME3] OpenGL debug message
ID: 131185
Source: API
Type: OTHER
Severity: null
Message: Buffer detailed info: Buffer object 1 (bound to GL_ARRAY_BUFFER_ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffer object operations.
java.lang.Exception: Stack trace
at java.lang.Thread.dumpStack(Unknown Source)
at com.jme3.system.lwjgl.LwjglGLDebugOutputHandler.handleMessage(LwjglGLDebugOutputHandler.java:76)
[JME3] OpenGL debug message
ID: 131185
Source: API
Type: OTHER
Severity: null
Message: Buffer detailed info: Buffer object 3 (bound to GL_ELEMENT_ARRAY_BUFFER_ARB, usage hint is GL_STATIC_DRAW) will use VIDEO memory as the source for buffer object operations.
java.lang.Exception: Stack trace
at java.lang.Thread.dumpStack(Unknown Source)
at com.jme3.system.lwjgl.LwjglGLDebugOutputHandler.handleMessage(LwjglGLDebugOutputHandler.java:76)
And this one continuously :
[JME3] OpenGL debug message
ID: 131154
Source: API
Type: PERFORMANCE
Severity: MEDIUM
Message: Pixel-path performance warning: Pixel transfer is synchronized with 3D rendering.
java.lang.Exception: Stack trace
at java.lang.Thread.dumpStack(Unknown Source)
at com.jme3.system.lwjgl.LwjglGLDebugOutputHandler.handleMessage(LwjglGLDebugOutputHandler.java:76)
I donât know what it means. Does it tell something to you guys?
While Iâm here with the 3.1, it appeared that jBullet wonât work and I had to detach the BulletAppState to avoid exception. Is that normal?
Here is additionnal information at startup, just in case :
Itâs a little large but in subtstance, there is an heightmap, and a mesh built from it by this method.
For each terrainâs triangle, the mesh receives three vertices, three indices, three normals (smoothed if needed) and three texture coorcinates. Very classical I think.
All the parcel meshes of a regionâs terrain are attached to a Node. Then, in the apply() method running on the JME thread, this node is attached to the scene.
I have such GRAVE exception (pasted earlied on the thread), running 3.0. This occurs when the thread is very late. I think itâs something iâm doing wrong with Paulâs Builder, but whatâs strange is that I donât get any error in the same situation running JME 3.1.
So maybe I reach some limitations that the 3.1 version has pushed away?
Anyway, I should not have reached such limitations and I will try to fix this in my code. My problem is that the the error is not very explicit ^^.
I will give you feed back. Thanks to both of you for your time.
