Multithreaded JME Core

Troubleshooting General Help
41
tom said:

It may be possible to improve JME performance without resorting to multithreading. OpenGL is a pipeline. Things can (in theory) be done in parallel with the rendering.


Well put tom.

Think that a simplistic multithreading can be put around the update ( frustrum etc ), this will give more power to occlusion / portals etc.

To completely make the whole scenegraph thread safe will take years to debug alone, the simpler it is the easier it will be
42

I feel the need to say it again :stuck_out_tongue:



The only thing people would need to start doing multithreaded update/cull/etc. is a safe FastMath implementation (which vear kindly provided in the linked topic).



And to make my post not completely redundant, while this is hard to implement in some engines, jME's "traditional" scenegraph, and usage of Java, should make this pretty easy… this approach also has the most "predictable".



http://arstechnica.com/articles/paedia/cpu/valve-multicore.ars (Valve calls this "finegrained" multithreading). It's not explicitly mentioned in the article, but if you read it you can see it enables them to do near linear speedups for things like CPU based particle effects.

43

after reading all these, i still think culling is the best place to start and most of the culling procedures can be easily separated into independent tasks.



this should be fast to implement so easier to c results.

44

I believe that the more modest approach of minimizing the OpenGL calls to rendering could provide a lot of benefits with minimal risk.  This would allow the multithreading of operations that don't really need OpenGL access.  The following are some basic examples, but I hope they demonstrate the types of changes I'm proposing.


  1. Getting access to an OpenGL surface (window, canvas, etc.) without waiting for the surface to be fully initialized would simplify embedding games within larger applications, applets, etc.  Setting the background color is currently a standard part of game initialization.  The following snippet is from BaseSimpleGame.initSystem():

display.getRenderer().setBackgroundColor(ColorRGBA.black.clone());


Looking at LWJGLRenderer, we see that this results in an immediate call to OpenGL:

GL11.glClearColor(backgroundColor.r, backgroundColor.g, backgroundColor.b, backgroundColor.a);


However, we could defer this update until the rendering phase without any negative impact to the system, just as long as we make sure that the value is pushed before a call to glClear().

2. Loading models in a background thread would be quite useful.  If the OpenGL references within the Camera objects were minimized, then creating and manipulating the camera could be performed in any thread.  Looking at the current Collada importer, this can be found:

Renderer r = DisplaySystem.getDisplaySystem().getRenderer();
int width = r.getWidth();
int height = r.getHeight();
// FIXME: THIS LINE IS SUPPOSED TO ONLY BE DONE IN A GL THREAD.
Camera c = r.createCamera(width, height);


This comment exists because the camera construction pushes frustum, view port, and frame changes to the OpenGL pipeline.  If these changes could be safely deferred until rendering, then the FIXME comment above is no longer accurate.

3. Most of the states go through an initialization phase where they determine the types of extensions that are available.  However, these values are not needed until the rendering phase.  Picking BlendState as an example, the following code can be found as part of a onetime initialization within the constructor:


supportsConstantColor = supportsEq = GLContext.getCapabilities().GL_ARB_imaging;
supportsSeparateFunc = GLContext.getCapabilities().GL_EXT_blend_func_separate;
supportsSeparateEq = GLContext.getCapabilities().GL_EXT_blend_equation_separate;
supportsMinMax = GLContext.getCapabilities().GL_EXT_blend_minmax;
supportsSubtract = GLContext.getCapabilities().GL_EXT_blend_subtract;


These 5 values are only used within the codebase during the rendering phase.  There are public accessors for these values, but they are either unused, or only used by callers active during rendering.

Some of the changes may have cascading effects, such as the interaction between ShaderObjectsState and several of the Pass objects.  More analysis is needed, but I so far I'm not aware of anything that invalidates this concept.
45
  1. Sure, that one is easy as we could basically use the same logic as is done in states.


  2. Most models can be done in a thread already.  (At NCSoft we loaded all of our assets in a thread.)  We did not use Cameras though, so yes there are a few special cases that could be cleaned up.


  3. Most of the one time state initialization is not actually a big deal because the default states are created when the renderer is created, and as such is not a big deal if it must be done in the GL thread.
46

Separating the rendering from manipulation of the camera, model, geometry, etc. provides greater flexibility, and sets the stage for more complex multithreading work.  The changes themselves were intended to be safe and simple, although I think that some of the complexity of (3) was lost.



In the JOGL port I've been working on, I've noticed that many of the OpenGL calls only work if the GLContext is initialized.  As an example, the following code results in an infinite loop unless the underlying GLCanvas has been made visible.



// Make the new context the current context, waiting if necessary as the
// context is initializing.
while (contextKey.makeCurrent() == GLContext.CONTEXT_NOT_CURRENT) {
    try {
        logger.info("Waiting for the GLContext to initialize...");
        Thread.sleep(500);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
}



In the case of createWindow() this could be addressed by actually making the window visible, before attempting to make the context current.  However, implementing createCanvas() is problematic, as the Canvas is returned to the caller so that it can be added as a component outside of the DisplaySystem.  This creates a Catch-22 where operations that require the execution of makeCurrent() (e.g. initialization of the camera, constructing rendering states, constructing current state records, setting of the background color, etc.) can't occur because the canvas has not be initialized (i.e. made visible) and the canvas won't be made visible until the createCanvas() method returns.

I don't mean to sidetrack the discussion with this detail, but I think that it highlights how difficult multi-threaded development is with the current codebase.  Many seemingly simple methods actually are only safely performed by the single game thread.  The example game cases provided with the codebase avoid a lot of the threading issues by consisting of a single OpenGL surface with a single thread.
47

I've looked into makeCurrent() further, and this is definitely an issue for JOGL.  A call to makeCurrent() results in an attempt to lock the surface.  The lock checks to see if the surface has been realized, which only occurs when addNotify() is called.

48

I think these items would be a good starting point, most of which were mentioned by previous posts. Trying to convert JME core to a hyperthreaded architecture in one shot would probably make HEAD a useless mess for a while. JME's HEAD is impressively stable, it would be great to keep it that way.


  1. It would be nice to have a standard API built into JME core to know whether I'm in the OpenGL thread, and also to execute tasks in it (a la SwingUtilities.invokeLater, invokeAndWait, and isEventDispatchThread). The StandardGame class in jmex did something like this, but it would be nice to have a simple util class where its always available to JME apps.


  2. Knowing which JME classes can be accessed outside of the OpenGL thread by some standard JavaDoc marking or using java packages to signify what is opengl thread only


  3. Work on making as many JME core classes opengl-free as possible. When large chunks of JME are opengl-free, things like threaded culling become much much easier to add later.


  4. Use the assert keyword to test the thread context in larger critical areas, something like this:


assert SwingUtilities.isEventDispatchThread();



That way the check is only executed when I enable assertion checking. During development I leave assertions enabled, then turn them off when I want to look at performance and they're off by default for releases.


As a side note, I've also got a ThreadedGame architecture with thread-safe tasks, shared state, global state (singleton), event management, and a network API using MINA for remoting events, request/response, etc. After I get a few sample games going to sanity-check my approach, I'll post the code somewhere to see if you guys find it useful.
49
lazlohf said:

I think these items would be a good starting point, most of which were mentioned by previous posts. Trying to convert JME core to a hyperthreaded architecture in one shot would probably make HEAD a useless mess for a while. JME's HEAD is impressively stable, it would be great to keep it that way.

1. It would be nice to have a standard API built into JME core to know whether I'm in the OpenGL thread, and also to execute tasks in it (a la SwingUtilities.invokeLater, invokeAndWait, and isEventDispatchThread). The StandardGame class in jmex did something like this, but it would be nice to have a simple util class where its always available to JME apps.


The framework StandardGame uses is actually useable for the rest of jME too (GameTaskQueueManager or something like that (too lazy to look, yeah))


As a side note, I've also got a ThreadedGame architecture with thread-safe tasks, shared state, global state (singleton), event management, and a network API using MINA for remoting events, request/response, etc. After I get a few sample games going to sanity-check my approach, I'll post the code somewhere to see if you guys find it useful.


I'm sure any code with regards to threading is of intrestred to the people in this thread, if only to compare approaches.
50
llama said:


The framework StandardGame uses is actually useable for the rest of jME too (GameTaskQueueManager or something like that (too lazy to look, yeah))


I just think the methods I mentioned would be very useful in com.jme, which shouldn't refer to anything in com.jmex I think? Also you can't use GameTaskQueueManager unless you inherit from StandardGame, which I don't. I'm not sure if there are plans to merge com.jme.app and com.jmex.game.
51
lazlohf said:

Also you can't use GameTaskQueueManager unless you inherit from StandardGame, which I don't.

hmm thats not true, the GameTaskQueueManager is also set up in BaseSimpleGame.
You just need to call the update and render methods of the manager in the game loop, so you can use it in any game implementation.
52

We make use of the queue in the applet base classes as well.  What is missing though is a nice "am I the OpenGL thread".

53
Core-Dump said:

lazlohf said:

Also you can't use GameTaskQueueManager unless you inherit from StandardGame, which I don't.

hmm thats not true, the GameTaskQueueManager is also set up in BaseSimpleGame.
You just need to call the update and render methods of the manager in the game loop, so you can use it in any game implementation.


Oh wow, I missed that. I also thought com.jmex code was not referenced in com.jme. Its a bit unexpected given how core and extension libs are normally packaged, but now I know.

renanse said:

We make use of the queue in the applet base classes as well.  What is missing though is a nice "am I the OpenGL thread".


I'm not sure what the best place for it would be, but it would be a nice addition.
54

GameTaskQueueManager is in com.jme.util.  It was part of core before standard game. :slight_smile:

55
renanse said:

GameTaskQueueManager is in com.jme.util.  It was part of core before standard game. :)


Is it too late for me to plead temporary insanity?
56
lazlohf said:

renanse said:

GameTaskQueueManager is in com.jme.util.  It was part of core before standard game. :)


Is it too late for me to plead temporary insanity?


You probably did see it and took it out of your code as it requires a call to a synchronized method each update. It being synchronized is strange... It uses a hashmap which is unsynchronized anyway - so why synchronize it when you can lock the construction of it before it is used.




57

Edit Ouch, just re-read what I have written in a different way. I Dont mean to be rude per the last post or unconstructively critical, I just see technical issues as black or white and then express thoughts in that manner.

58
theprism said:

lazlohf said:

renanse said:

GameTaskQueueManager is in com.jme.util.  It was part of core before standard game. :)


Is it too late for me to plead temporary insanity?


You probably did see it and took it out of your code as it requires a call to a synchronized method each update. It being synchronized is strange... It uses a hashmap which is unsynchronized anyway - so why synchronize it when you can lock the construction of it before it is used.


Nah it was nothing that complex, I somehow got GameTaskQueueManager and GameStateManager confused in package placement.  :) I haven't look at the HashMap issue you mention, I'm currently fixing up the GameTask stuff, patch inc in a 5 minutes or so.
59

FYI



Here is how Sun did it for swing in java 6



http://java.sun.com/developer/technicalArticles/javase/6_desktop_features/index.html#Single-ThreadedRendering

60

If anyone has any implementations ready - Vear for example, that they want to put ofrward for JME 3.0, please go ahead