Scene manager for jME?

Hello,

Some years ago jME3 did still use the same brute force scene management that jME2 used, which means just sending all bounding spheres to render that are inside the view frustum.

I just wanted to ask: Is this still the reality? Or does jME now come with bsp, kd, octree, cells and portals? I did not find anything on this topic so far, and the forum search is still broken.

In my opinion, a good scene management is an integral, core component of a good render engine that can save vast amounts of unnecessary draw calls and bounding volume checks.

:slight_smile:

You can use google to search the forums, just include site:jmonkeyengine.org in your query string. Not ideal but itā€™s a way to keep moving without a properly working forum search.

JME3 uses a scene graph and culls based on bounding volumes. So if your scene is well organized then culling is efficient. BSP trees, portals, etc. would subvert the scene graph structure and have their own (pretty severe in some cases) limitationsā€¦ so itā€™s up to the game to use them if it wants to.

Personally, BSP trees kind of strike me as something that was more important back when every polygon was expensive. The performance characteristics have changed a lot in 20 years.

Yes, the game coder should decide whether some scene is ā€˜indoorā€™ or ā€˜outdoorā€™ and which part of the scene is a ā€˜cellā€™ or a ā€˜portalā€™ for some techniques. But are they offered to the programmer? Most certainly not, so additional coding is necessary (also by beginners who will quickly be overextended by things like these). Maybe I will try to code things like these and make this very basic part of a game engine public as soon as I can. You should offer several scene managers, Ogre3D does it, and they are right to do it.

Octrees should at least be standard, or grid-based scene managers, maybe with spheres instead of boxes, for better speed.
They would work very similar to the raw scene graph that beginners would use, and do fancy stuff backstage, thus be transparent to the coder.
Maybe they could switch the ā€˜cullhintā€™ flag for parent Nodes automatically.

It is not only the polygon count, but also the number of cpu-based culling tests of the scene graph against the frustum. Imagine a vast outdoor or indoor scene with thousands of objects.

Still the polygon count does matter, because jME uses forward rendering and even transparent faces might be included and a user could have an old or weak computer like mobile devices or third world computing systems. And optimizations are always good, even if not really needed on beefy machines.

Itā€™s not the only thing thatā€™s been missing, also a clever LOD system, input configuration (game action mappings, joystick calibration, joystick interface is very poor in particular), an ingame console, sound management, out of the box soft particles, fur shaders, lighting management, ingame cube map generation, system diagnostics and auto-configuration, stringless ID mappings, a demo for entity-systems with jME, a better concept for multithreading, water shaders are flawed, some test demos crash always, files are missing in the SDK, an asset store, a framework for version update, articles on modding, articles on marketing (steam integration etc.), examples for server applications and other things.
I guess there will be a lot of work to be done until jME is a really good engine that needs only few individual additions by the coder.
Whenever I come to engine-coding (I would rather code games instead of engines), I will try to help.

What I cannot do, is coding the web presence, the forum in particular.
If you want to look professional and be taken serious, you should improve this first.
Even as Iā€™m typing this text, my cursor is sometimes stuck in this textbox, Iā€™m using the IE on a fresh windows 7 system.
Please consider using a different forum software, even if this means having a different lookā€™nā€™feel.

Well this is an open source engine, pass the code if you think it ā€œneedsā€ to be in.

I'm using the IE on a fresh windows 7 system. Please consider using a different forum software, even if this means having a different look'n'feel.

I will not let anyone with IE tell me what i should or shouldnā€™t do.

For the rest of your post ā€¦ well, a lot of things are missing in jme, but itā€™s because:
1 - itā€™s open source.
2 - they (we ?) try to keep a ā€œcleanā€ sdk, with a consistent approach for problems. I am myself writing a document of what i like and donā€™t like in jme (what i would love to see in jme4) but itā€™s not a ā€œmore contentā€ request/whine as your code is. I try to see what is fundamentally wrong in jme.

If you can provide something, itā€™s very nice and very kind. If you have a request about something that is a ā€œmust-haveā€, then we are happy to hear it. If you only ask for ā€œout of boxā€ things ā€¦ well, jme is a core engine, not a resources pack.

One thing we miss, for exemple, is morphing. But itā€™s not an easy problem at all.

You sure have a big mouth. Iā€™m really looking forward your next contribution.
But reallyā€¦after this post, it better be frigging perfect.

4 Likes

@Ogli: We are eagerly awaiting your contribution! Please do not disappoint us!

@bubuche said: I will not let anyone with IE tell me what i should or shouldn't do.
With Firefox I had experienced the same problems, sorry for the bad news.

About the other thing: :stuck_out_tongue_winking_eye: :wink: Let me see what I can do 4u. I really like jME.

You can always pay money in royalties, or large license fees for a ā€˜professionalā€™ engine. Imho JME is doing pretty well having been built with zero dollars spent and hundreds of hours of volunteer time.

If we could afford some full time developers the features that you list could be implemented. But as it turns out the engine is free, so itā€™s hard to find the hundreds of thousands of dollars per year needed for those developers.

So, when a feature is missing the community builds it and adds it for others to use. Thatā€™s how an open source project grows while remaining free. Usually these community members refrain from bashing the project they want to contribute to; thereā€™s just so many better ways of going about it.

There are some decent opensource bsp implementations out there that you could just use.
JME is after all a core engine, meaning that it is not tailord to a specific type of game (eg source engine mostly for shooters) but can be used for everything as it mostly only uses the most minimal common determinator.

This is really great for indy, hobby projects, as it allows you to do stuff outside the box, but for a more standart project it might not be the best engine.

@Ogli said:

Itā€™s not the only thing thatā€™s been missing, also a clever LOD system, input configuration (game action mappings, joystick calibration, joystick interface is very poor in particular), an ingame console, sound management, out of the box soft particles, fur shaders, lighting management, ingame cube map generation, system diagnostics and auto-configuration, stringless ID mappings, a demo for entity-systems with jME, a better concept for multithreading, water shaders are flawed, some test demos crash always, files are missing in the SDK, an asset store, a framework for version update, articles on modding, articles on marketing (steam integration etc.), examples for server applications and other things.
I guess there will be a lot of work to be done until jME is a really good engine that needs only few individual additions by the coder.
Whenever I come to engine-coding (I would rather code games instead of engines), I will try to help.

What I 'd like to do is to put some line break in this mess ā€¦ but is ok for now :slight_smile:

Anyway, did you know Java yet, dude?

  1. system diagnostics and auto-configuration : It call profiling and you should have JVM support to do this, to monitoring the assets and memory, JME3 actually did have this feature, you just didnā€™t do a proper search yet. And there is nothing like auto-configuration ever implemented in any game engine I knowā€¦ Is it the universal box for every purpose?
  2. Multithreading: You should check Guavaā€™s ListenableFuture or RxJava Observable. I will let you search for it because the two libs provide very generic concurrent framework for wide range of usecases, any Java applications can use them, including every JME3 games.
  3. an asset store, a framework for version update: You should be kidding me? There is dozen of Asset Store for assets (remember JME3 support a lot of model format) and there is UpdateCentral for contributed libraries and plugins, what you need more?
  4. About input, logic and entity, there is also some way to get the work done:

I wrote a framework to support those features for skilled game developer.
https://github.com/atomixnmc/atom-game-framework
Itā€™s not complete but you can see what you asked for there in shape. Iā€™m still very active in developing this framework, hope it will complete some missing pieces in our engine.

^^ you noticed the thread is kinda old :stuck_out_tongue:

@Empire Phoenix said: ^^ you noticed the thread is kinda old :P

One of the side-effects of lowering the spam wall is that we get more spamā€¦ which means we get more threads looking currentā€¦ which I think will lead to more necro-threads.

Old post, yes, I can only guess what I meant by ā€œauto-configurationā€, but I guess I meant platform-specific and system-power-specific auto-detection of grafix settings. I have ideas how to make this and itā€™s still on my list for later.

Nevertheless - I see @atomix 's post as a helpful hint. Will look into those things. I found Guava, for example, some days ago when I wanted a proper ListMultiMap implementation.

ā€œasset storeā€: Game-ready assets (recent game contest showed, that many assets are not jme-ready), Also I was referring to the great https://www.assetstore.unity3d.com/en/ asset store of Unity. You can also find helpful assets that consist of code and shader code and so on - this opens up a whole world of small business and support for that specific engine (ā€œsupportā€ meaning, people motivated by fame and money, who are not in the pay-list of the Unity makers, similar to modder-supportā€¦).

In the meantime, I must say:
@pspeed and others made a great job extending the base engine. I will soon follow with my contribs.
Also, there are different views of ā€œwhat an engine needsā€ and ā€œwhat would be a nice-to-have pluginā€.
I always wanted too much, even commercial engines donā€™t have all that I want :wink:

Will have a look into atom when Iā€™m done with SimAboreal, Lemur, TonegodUI, MonkeyZone, and all the other stuffā€¦ :slight_smile:

Have fun,

Iā€™ve just realize I 've answered the old thread right after I push the submit button. :slight_smile: Anyway, i still think there is people who asked for features built in for game engine, and forget that feature belong to Java and already very ready to use. The way JME was written allowed developers to very easily intergrate other technologies, itā€™s great because of that

Well, why not necro a little more, if we already started with necroing. :wink:

@pspeed said: JME3 uses a scene graph and culls based on bounding volumes. So if your scene is well organized then culling is efficient. BSP trees, portals, etc. would subvert the scene graph structure and have their own (pretty severe in some cases) limitations... so it's up to the game to use them if it wants to.

Personally, BSP trees kind of strike me as something that was more important back when every polygon was expensive. The performance characteristics have changed a lot in 20 years.

What are the limitations that you thought of?
I see advantages - e.g. render 100.000 polygons with few overdraw for the cell (room) my camera is inside - versus - rendering 100.000.000 polygons with a lot of overdraw (everything outside this cell/room).

For the BSP-part - I think there is something better, called kd-trees. These bsp/kd trees have advantages for level geometry similar to ā€œcanyonsā€ or ā€œcastle wallsā€. It is not only the polygons overdraw, but also a way to clip BoundingVolumes. Only the large-scale-static-geometry (walls) will be used for the tree.

I think that these parts belong to a render engine. Ogre3d incorporates a simple SceneManager system with possibility of plugins for this.
Which, according to 3D-people from the games industry is ā€œjust a little start - better would be if we could combine ā€œinsidoorsā€ and ā€œopen worldā€ SceneManager.ā€
I think this is true, you should be able to combine multiple SceneManager for combined indoor/outdoor scenes.

It is on my list for later, but I donā€™t have much experience with this.
I know what an octree is, and what the theoretical advantages of scene managers are.
I have a simple Cell-Portal-Sytem planned, which requires a lot of manual artist work to prepare the scene - this is quite the simplest possible Cell-Portal-thing I could think ofā€¦

@Ogli said: I see advantages - e.g. render 100.000 polygons with few overdraw for the cell (room) my camera is inside - versus - rendering 100.000.000 polygons with a lot of overdraw (everything outside this cell/room).
I would like to clarify. There is absolutely no overdraw occurring in the use case you mention. Overdraw means that you are drawing something over something that has been already drawn. In JME (and in any engine I can think of...) opaque objects are sorted front to back and are rendered in that order. Rendering means drawing color AND drawing depth. But when rendering a pixel, the gpu does a Depth test before to be sure the current pixel is not already occluded by a previously drawn one. So in your use case... you can have a whole world behind a wall, the only thing rendered is the wall, and that's how overdraw is avoided.

However (and I guess thatā€™s what you meant) the object are still sent on the GPU, and you still have a draw call for every object in the view (which is expensive). I say in the VIEW because JME is performing a frustum culling before sending objects to the GPU.

Having a BSP implementation to avoid this is beyond JME scope IMO. The use case you mention is not so frequent, and if the problem arise, a simple scene management (like unloading the rest of the world and loading the room) can be enough depending on the game.

The only thing that could be in JME to avoid this is what is called Occlusion Query (or Occlusion Culling). The idea is, once the frustum culling is done, to ā€œqueryā€ the view to detect entirely occluded objects and cull them.
There are a lot of drawbacks depending on the technique you use to query and the querying can turn out to be more expensive than drawing everything on some scenes.
There is a hardware implementation of this that could be interesting but here again, not always efficient (I think ogre has hardware occlusion query)

However, if I were you Iā€™d look more on this side than on the BSP tree implementation or any scene partitioning scheme.

@Ogli said: What are the limitations that you thought of? I see advantages - e.g. render 100.000 polygons with few overdraw for the cell (room) my camera is inside - versus - rendering 100.000.000 polygons with a lot of overdraw (everything outside this cell/room).

If you render your scene from near to far (as JME does) then any modern graphics card is already optimizing overdraw. Youā€™d be hard pressed to do better in CPUā€¦ though it is possible. Every approach will have some limitations for certain types of scenes.

ā€¦and anyway, managing 100,000,000 triangles is going to be painful no matter what data structure you use. Unless you can page them in and out, memory and related are going to be your biggest issues then. And if you are already paging them in and out then you are already using something simpler and ultimately faster. Plus, now your scene is organized spatialy instead of based on some arbitrary splitting schemeā€¦ which means itā€™s also easier to sort, cull, etc.

re: Limitations, BSP requires static scenes. Works best for closed/indoor scenes. Requires huge preprocessing. And so on. It is not a general solution. Every tree based solution will have some trade-offs like these. Also, for something like a BSP tree, the draw calls are often not optimal. Splitting the entire world down to convex spaces means lots of separate geometryā€¦ unless you batch them again once visibility is determinedā€¦ but then you are doing a lot of work on the CPU again.

While not always the case, in general, looser approaches that delegate more to the GPU and leave work out of the CPU (like spatially organized scene graphs) are a better general trade-off strategy. The modern GPU is extremely good at tearing through triangles. Limiting draw calls and limiting state changes are still the best general optimization strategies.

:roll:
Okay, I will answerā€¦ painful but must be doneā€¦ I think

@nehon said: I would like to clarify. There is absolutely no overdraw occurring in the use case you mention. Overdraw means that you are drawing something over something that has been already drawn. In JME (and in any engine I can think of...) opaque objects are sorted front to back and are rendered in that order. Rendering means drawing color AND drawing depth. But when rendering a pixel, the gpu does a Depth test before to be sure the current pixel is not already occluded by a previously drawn one. So in your use case... you can have a whole world behind a wall, the only thing rendered is the wall, and that's how overdraw is avoided.

However (and I guess thatā€™s what you meant) the object are still sent on the GPU, and you still have a draw call for every object in the view (which is expensive).

As if I (or any 3d coder) doesnā€™t know what a z-Buffer is and that draw calls are one of the most limiting things todayā€¦
Donā€™t get mad at meā€¦ as you already mentioned, thatā€™s what I meant. :slight_smile:

I say in the VIEW because JME is performing a frustum culling before sending objects to the GPU.

Well, yes, view frustum culling is now something that really every engine on the world should haveā€¦ You can add backface culling supportā€¦ :wink:

Having a BSP implementation to avoid this is beyond JME scope IMO. The use case you mention is not so frequent, and if the problem arise, a simple scene management (like unloading the rest of the world and loading the room) can be enough depending on the game.

I was referring to this as ā€œthe most simple Cell-Portal-thing that I can imagineā€.

The only thing that could be in JME to avoid this is what is called Occlusion Query (or Occlusion Culling). The idea is, once the frustum culling is done, to "query" the view to detect entirely occluded objects and cull them. There are a lot of drawbacks depending on the technique you use to query and the querying can turn out to be more expensive than drawing everything on some scenes. There is a hardware implementation of this that could be interesting but here again, not always efficient (I think ogre has hardware occlusion query)

However, if I were you Iā€™d look more on this side than on the BSP tree implementation or any scene partitioning scheme.

I also know what Occlusion Culling isā€¦ and there are some people using hardware oc, also in the industry. I my self can only implement software oc by now, but thatā€™s already worth a lot.
Also, using other optimizations like impostors can save more than 90% of performance loss if done right (read a PhD thesis on this subject with prove-of-concept implementation also demonstrated there).

@pspeed said: If you render your scene from near to far (as JME does) then any modern graphics card is already optimizing overdraw. You'd be hard pressed to do better in CPU... though it is possible. Every approach will have some limitations for certain types of scenes.

Should see same answer from me aboveā€¦

...and anyway, managing 100,000,000 triangles is going to be painful no matter what data structure you use. Unless you can page them in and out, memory and related are going to be your biggest issues then. And if you are already paging them in and out then you are already using something simpler and ultimately faster. Plus, now your scene is organized spatialy instead of based on some arbitrary splitting scheme... which means it's also easier to sort, cull, etc.

You are somewhat right in this point, but both my computer and my grafix card have gigas of RAM (16Gig and 3Gig or so).
That sorting and culling is what this SceneManager stuff is all about - and you would do it as a pre-compute-step of course.
So you would typically have a static scene with some dynamic items in it and then a pre-compute-step and a dynamic SceneManager in addition to that.

re: Limitations, BSP requires static scenes. Works best for closed/indoor scenes. Requires huge preprocessing. And so on. It is not a general solution. Every tree based solution will have some trade-offs like these. Also, for something like a BSP tree, the draw calls are often not optimal. Splitting the entire world down to convex spaces means lots of separate geometry... unless you batch them again once visibility is determined... but then you are doing a lot of work on the CPU again.

A lot of work is irrelevant to pre-compute-steps.
Small convex islands are a problem for bsp / kd, you are right in that regard.
Indoor: bsp, kd, cell-portal, grid-based stuff
Outdoor: paging terrain, octree, height-based occlusion, impostors for large away part of geometry, occlusion for things like mountains / skyscrapers

While not always the case, in general, looser approaches that delegate more to the GPU and leave work out of the CPU (like spatially organized scene graphs) are a better general trade-off strategy. The modern GPU is extremely good at tearing through triangles. Limiting draw calls and limiting state changes are still the best general optimization strategies.

See post above ā€¦ limit drawcalls ā€¦ is what this is all about.

But I didnā€™t yet write a SceneManager that works on the GPU - would be an interesting thing maybe ā€¦ :slight_smile:

Sorry, folks,
but the only thing that is already in engine / plugins are some Terrain-Paging and On-demand-Batching and Manual-Batching (in the SDK).
This plus simple tree-based View Frustum culling with forward rendering will not be fast enough on many machines.

But no worriesā€¦ if I find good solutions I will make them public.
Maybe beg, steal or borrow from Ogre or papers / GPU gems etc. will result in better scene management supportā€¦

tehe, right xD