Multithreading Revisited

I know this has been raised and suppressed more than once, but now that 0.10 is out and 0.11 is beginning I'd like to make another push for multithreading support in jME.  There are many things that this would be of benefit to, but one of them that I'm having to deal with at the moment is loading progress.  I'm trying to create a clean method to show loading progress of game states and doing it in a single-threaded environment makes things incredibly difficult and incredibly ugly.  I have quite a bit of knowledge on the subject of threads and would be happy to assist or even lead this endeavor, I just wanted to make sure there's not a lot of resistance to the idea before I begin?



darkfrog

I second that, Next gen pc's are multiprocessor - multithreaded makes sense

You can have mutiple threads in your app. But you need to do all GL calls from one thread - I believe lwjgl demands this (and formally even GL itself). So if you want to change that start with lwjgl.

On the jME side we should take care that you can do most background loading without accessing GL. That esp. means changing texture caching - which is a todo already.

Irrisor, yes, that is what I was referring to.  Most of the aspects of jME don't require directl calls to GL so they can be multithreaded, but there are still quite a few that make direct calls and if we can resolve that then jME will be able to be multithreaded pretty easily.  Does anyone object to this being done?  I know mojo had some reservations at one point, but I don't know if it was principially or that it just wasn't a big concern to him.



darkfrog

May I plug this thread?

I did no extensive tests but since it’s influenced by existing (and tested) SWT code it should do. Exposing the API to the client would make it the client developers job of delegating the appropriate calls and not jME’s. Depending on how heavy the impact would be we could throw an exception if method using OpenGL isn’t called from the renderer thread.

Sure, Gaheris, most of the calls can be executed in a single thread (btw. You can already use InputHandlers for the same purpose (e.g. with JMEAction)), but if you want to have background loading this alone wont help.

I actually already have multithreading working here in our custom design tool using a GL queue to store and execute GL-only actions in the GL thread.  It wouldn't be all that hard to write a new app type that did something similar with updates.

There is an early version of my own task framework included in my terrain engine (in the org.llama.jmex.task package if I remember correctly).



http://www.jmonkeyengine.com/jmeforum/index.php?topic=3098.0



It's meant for prioritized scheduling and executing of tasks in variaty of ways (including inside the OpenGL thread), and making those tasks split themselves up if needed (for example inbetween different frames). Not quite documented though, since it's not really released on it's own yet.

I would like to have a load method on GameState that receives a Progress object.  During the execution of the load method it can call progress.setProgress(n) setting a percentage completed.  This would then be reflected in the game screen to display a progress bar of some sort.  If the load method could be called and executed entirely in its own thread then the graphical thread would never be locked during load and would provide a much nicer feel of game loading as well as a great means to see the progress.



That's the primary thing I am pursuing at this point.  We can always do Jobs and such, but that's a lot of headache to implement when you may or may not want to split it out into multiple objects to be called.



darkfrog

Then we should start by identifying which things you want to load from a different thread, and which don't work (and why).



Now that .10 I can implement this:



http://www.jmonkeyengine.com/jmeforum/index.php?topic=3004.0



Which means you could load textures in the background.


maybe add to the list the loading of models…



A new player entering an area for instance

I believe the biggest deal is model and texture loading.  Ironically, those are also the ones that take the longest to perform and are usually the basis of needing a loading screen. :slight_smile:



darkfrog

darkfrog said:

I believe the biggest deal is model and texture loading.  Ironically, those are also the ones that take the longest to perform and are usually the basis of needing a loading screen. :)

darkfrog

I will always maintain that having the texture loading in the same thread as the main rendering loop is a mistake. It's done to prevent a model loading without its texture but it causes momentary game stalls when loading a new model. Multiply that by a hundred new models with high res textures and it doesn't matter what computer rig you have, it's going to stall for a second or two when those models first come into view and are loaded.

There are plenty of ways to handle this and it's up to personal choice which way is used but we should not be forced to use one specific method. Currently, we don't have a choice because we can't separate the rendering and the loading into separate threads and only render what has loaded. Unless there's a way to do that in JME which I am overlooking. Perhaps all objects to be rendered could have a flag that is set to 1 by a separate loader thread once it's loaded and the renderer ignores things set to 0?

Well just to be clear on one thing, you can already load textures outside of the jME thread, you just can't do it using TextureManager. Anyway, as you see in the topic I gave the link from I've already commited myself to solving that.



Don't expect that to solve stuttering problems for all your games and users though. There's a reason most games load textures and models at the beginning of the game. Doing all kinds of IO while in the middle of rendering a scene doesn't quite speed things up for your game.

llama said:

Well just to be clear on one thing, you can already load textures outside of the jME thread, you just can't do it using TextureManager. Anyway, as you see in the topic I gave the link from I've already commited myself to solving that.

Don't expect that to solve stuttering problems for all your games and users though. There's a reason most games load textures and models at the beginning of the game. Doing all kinds of IO while in the middle of rendering a scene doesn't quite speed things up for your game.

Could you not implement a form of load balancing to maintain a minimum rendering rate, sacrificing loading times etc? There's a difference between the game slowing and the game stuttering.

Nyphur, I believe at least to some extent this can be accomplished by thread priorities in Java.  If you make the graphical thread a high priority thread and the loading thread a lower priority thread and make sure it miinimally blocks then you can hopefully decrease the stuttering at least to a minimum.  Correct me if I'm wrong here llama.



darkfrog

darkfrog said:

Nyphur, I believe at least to some extent this can be accomplished by thread priorities in Java.  If you make the graphical thread a high priority thread and the loading thread a lower priority thread and make sure it miinimally blocks then you can hopefully decrease the stuttering at least to a minimum.  Correct me if I'm wrong here llama.

darkfrog

Priorities would be a good idea if it works but it's not a perfect solution. A system that has a hard-coded minimum rate of rendering and input threads would theoretically prevent any stalling or stuttering from occuring by shunting the problem onto other time-heavy processes like loading models and textures. So instead of stuttering occasionally, your models wouldn't load for a few seconds. if you had the loader load models as they came within x distance of the camera, you'd have "popping" near distance x where models which took a while to load would appear.

You would simply set x to higher than the far point, enough higher that on a reasonable computer, popping would not occur. You could have models fade in after they are loaded rather than simply appearing and use a fog system to set the far point or something, there are plenty of ways around it but the only way around stuttering in games today is "get a better computer" and I am simply not happy with that.

I was thinking of ways to fix TextureManager so it can load textures from different threads.



The best Way I can come up with is making a linked list where each texture knows the previous and next Texture with the same art.



When one clone gets it's id the others will get the same one. This can be done in while rendering a texture for the first time.

Regards the caching of textures…



see http://www.jmonkeyengine.com/jmeforum/index.php?topic=2646.0



Badmi - can give you some code to play about with ( nothing special, just a cache ), the downside i found, which llama gave a workaround ( but havent had time to play with ) is :-

Suppose you have a brick wall texture - you have one box which is 3x3x3 and one box which is 10 x 10 x 10. You want the bricks in the texture to remain a consistent size, but the bricks will be larger in the 10x10x10 model



I may be wrong, but isn't it that you can request multiple id's from OpenGL at a time? So why not forward cache a bunch of them and use them up as textures load? I'm not fond of "on the fly" loading, if badly working, it can hick the players display in the worst moment. If it happens at regular intervals it gets annoying, if it constantly slows down the game, then whats the point in it? None of the FPS or RTS games i know of does on the fly loading, but RPG's need it i guess. The background loading could run when:

  1. The player is idle for some time
  2. Is presented with some text to read
  3. Is passing trough some well known place, like city gates. The scenegraph could have a special node: "Load area", which starts loading new content when the player enters it.
  4. Has finished some transaction on his HUD

    Should not run, when the player:
  5. Is working with his HUD. Unresponsive mouse is the worst thing.
  6. Is engaged in combat



    I think threading could be put to good use in the render loop like this:
  7. Thread running the update() for scene branches already drawn (or culled)
  8. Thread going before the rendering thread, doing the culling, filling up and sorting the renderqueues
  9. Rendering thread doing draw methods only



    Those three are the most distinct time consumers, and by threading them, the load would be balanced. AI, Phisics and networking could go into first, scenegraph traversal into second, while the third would spend all its time in OpenGL, pushing data onto the graph card. The scene would be traversed only once for each loop, and the update() would work from a queue.