Enhancements to the JME3 Rendering Engine

JhonKkk · October 6, 2023, 1:53pm

Hi, I just created a temporary repository here（https://github.com/JohnLKkk/jme3.4-temp.git）, which is jme3.4 and contains my modifications, but it is not complete. I currently don’t have time to gradually migrate the code to jme3.6. So I think this temporary repository might be useful for people who are interested.

JhonKkk · October 6, 2023, 1:55pm

I will find some time later to gradually port the code from jme3.4 to jme3.6.

JhonKkk · October 6, 2023, 2:00pm

These two folders are the main places for code modifications.

oxplay2 · October 6, 2023, 3:31pm

Thanks! Yes, as you said it will be needed port into 3.6.1 to have clean pull request into JME current version. Probably it would not work from 3.4, but im unsure.

I wish to see your work on main branch in JME

JhonKkk · October 8, 2023, 11:13am

I initially ported the Framegraph, rendering paths from jme3.4 to jme3.6.1. The first picture is Forward (SinglePass SinglePassLightBatchSize=30), the second is standard DeferredShading (there are bugs in porting TileBasedDeferredShading, fixing…), rendering 1000 point lights at 1600x900 resolution.

JhonKkk · October 8, 2023, 12:15pm

Hi @oxplay2 @sgold, I have a question. I’m currently developing on the jme3 3.6 branch. Is this the latest dev branch? Should I submit PRs on this branch?

oxplay2 · October 8, 2023, 12:44pm

Hello @JhonKkk

It seems Riccardo is absent for now and Darkchaos as i recall had some different work.

So i belive it should be fine if you want make new JME release yourself, like Sgold suggested.

I think this topic might be very usefull resource: 3.6 release and 3.6.1 release topic:

Ofc every project have different ways to make pull requests. Some ways were to create release branch first and later merge to master.

But in JME i think you create pull request directly into Master, and later create release TAG and branch from master for a release.

I see there are branches for 3.6 but i guess its related for a Fixes/subversions like 3.6.1.
@sgold or @Ali_RS if you could help clarify here would be cool

example pull request:

There is also contribution mega-thread:

But for a new release, as can notice, some fixes/features are picked to merge and then create release.
Your work itself IMO might be enough for new release itself.

JhonKkk · October 8, 2023, 1:13pm

Thank you for your guidance. I misunderstood previously. Now I know that I should adjust the code from the local 3.6 branch to the master branch. After fixing the bug in TileBasedDeferredShading, I may try to make a PR.

oxplay2 · October 8, 2023, 1:23pm

Very nice, i hope this BUG will be easy to fix for you.

Idk, if you got probe changes ready, but anyway no worry about probe changes that require SDK changes.
As i recall, it was same for new animation system / etc. First change were in jme-core and later in SDK.(but there were ofc way to manage/convert it via code first, i belive you can leave SDK changes for some of SDK folks)

sgold · October 8, 2023, 5:21pm

No. Pull request should be based on the “master” branch.

For further guidance on pull requests, please read our guidelines:

github.com

jMonkeyEngine/jmonkeyengine/blob/master/CONTRIBUTING.md

# How to contribute to jMonkeyEngine

First and foremost, you have to familiarize yourself with Git & GitHub. Dig through 
[help.github.com](https://help.github.com/), [try.github.io](http://try.github.io/) and the [gh cheat sheet](https://github.com/tiimgreen/github-cheat-sheet/blob/master/README.md) if these are new topics for you. If you'd like to contribute with something other than code, just tell us about it on the forum.

## Communication

Communication always comes first. **All** code changes and other contributions should start with the [forum](http://hub.jmonkeyengine.org/). Make a thread to explain your change and show us the important bits of your code. If the code is too long to be posted within the forum’s code tags, please paste your code in a Gist or pastebin and link to the submission in your thread. You are required to register on our website in order to create threads. (We do support login via GitHub though).

### New Contributors

Check out the [Projects](https://github.com/jMonkeyEngine/jmonkeyengine/projects/1) tab, where the team has prioritized issues that you as a new contributor can undertake that will familiarize you to the workflow of contributing. This highlights some issues the team thinks would be a good start for new contributors but you are free to contribute on any other issues or integration you wish.

When you're ready to submit your code, just make a [pull request](https://help.github.com/articles/using-pull-requests).

- In your commit log message, please refer back to the originating forum thread (example) for a ‘full circle’ reference. Also please [reference related issues](https://help.github.com/articles/closing-issues-via-commit-messages) by typing the issue hashtag.
- When committing, always be sure to run an update before you commit. If there is a conflict between the latest revision and your patch after the update, then it is your responsibility to track down the update that caused the conflict and determine the issue (and fix it). In the case where the breaking commit has no thread linked (and one cannot be found in the forum), then the contributor should contact an administrator and wait for feedback before committing.
- If your code is committed and it introduces new functionality, please edit the wiki accordingly. We can easily roll back to previous revisions, so just do your best; point us to it and we’ll see if it sticks!

p.s. We will try hold ourselves to a [certain standard](http://www.defmacro.org/2013/04/03/issue-etiquette.html) when it comes to GitHub etiquette. If at any point we fail to uphold this standard, let us know.

This file has been truncated. show original

JhonKkk · October 10, 2023, 7:48am

It took 2 days to fix bugs, this is “tile based deferred shading”(Tiled Deferred Shading – Leif Node):

However, for now, I have not fully optimized DeferredShading and TileBasedDeferredShading, they may not be very efficient, but should be compatible with existing system modules.

JhonKkk · October 10, 2023, 8:20am

github.com/jMonkeyEngine/jmonkeyengine

Enhancements to the JME3 Rendering Engine

jMonkeyEngine:master ← JohnLKkk:master

opened 08:19AM - 10 Oct 23 UTC

JohnLKkk

+6684 -185

see:[https://hub.jmonkeyengine.org/t/enhancements-to-the-jme3-rendering-engine/4…7145/27](url) ### Enhance the capabilities of the JME3.7 renderer, mainly including: 1. Introduce a basic framegraph framework for managing renderPasses, while modularizing the existing rendering modules into several core Passes for reuse; 2. Added support for render paths, including Forward (default render path), Deferred (a standard DeferredShading), and TileBasedDeferred (Tile-optimized DeferredShading). They should be compatible with most existing modules. 3. Added some code demos for RenderPath. Regarding render paths, please refer to: https://docs.unity3d.com/Manual/RenderingPaths.html ### Performance considerations The rendering overhead of real-time lights in deferred shading is proportional to the number of pixels illuminated by the light and not dependent on Scene complexity. So small Point Lights or Spot Lights are very cheap to render and if they are fully or partially occluded by Scene GameObjects then they are even cheaper. Of course, lights with shadows are much more expensive than lights without shadows. In deferred shading, shadow-casting GameObjects still need to be rendered once or more for each shadow-casting light. Furthermore, the lighting shader that applies shadows has a higher rendering overhead than the one used when shadows are disabled. ### GBuffer format: RT0, RGBA16F: Pack DiffuseColor, Ao, Alpha RT1, RGBA16F: Pack SpecularColor, FZero, Roughness RT2, RGBA16F: Pack ShadingModelId, emissiveColor RT3, RGBA32F: Pack Normal (contains Normal with TBN transform and modelNormal, using cubemap mapping) RT4, depth: Pack depth ### LightPack: There are two ways to package lightData, one of which is to use a uniform array, which should be faster, but its quantity is limited, and a large number of light sources will be drawn and accumulated through several passes to complete all light sources; Another way is to package all light source information through three texture1D, which may have some performance overhead, but supports a large amount of light source data; ### Tile-Based DeferredShading: TileLightDecode: Packs tile information, x indicates the first light source offset used by the tile (for lookup in TileLightIndex), y indicates how many lights are associated with the tile (currently no limit, i.e. unlimited number of lights), z indicates the uv coordinate offset when sampling TileLightIndex for the tile; TileLightIndex: Packs light source ids for each tile ### UseRenderPath: Enable the corresponding renderPath in the following ways： `renderManager.setCurMaxDeferredShadingLightNum(1000);// Pre-allocate a maximum value for light sources to ensure the maximum number of light sources in the scene does not exceed this value. renderManager.setRenderPath(RenderManager.RenderPath.Deferred);` ### UseFramegraph: Enable the corresponding framegraph in the following ways： `renderManager.enableFramegraph(true);`

JhonKkk · October 10, 2023, 8:40am

5000 PointLights（800x600），
Forward：

Deferred：

JhonKkk · October 10, 2023, 11:03am

I am fixing the remaining bugs, it may take some time. Afterwards, I may port the lightProbeVolume to jme3.7.

JhonKkk · October 10, 2023, 11:10am

Perhaps I should synchronize and update the JME Wiki afterwards, so that the details of related graphics techniques can be fully explained.

oxplay2 · October 10, 2023, 11:41am

Amazing work! fully optimized or not, having 5000 PointLights at 21 fps or 60+ fps (we dont even know how much if not unlock vsync) is crazy difference

Tho i still see a lot of “todo” and unfinished code there, i understand its just first pull request and LightProbeVolume gonna be added in another one. LightProbeVolume indeed its big separate topic.
Myself im always against adding any todo into core, but i understand everything will be filled with next pull request.(or changes in this one)

Were looking through commits, as i understand Forward pass will be default until we change into Deferred one.

Also seen tested with FilterPostProcessor using ToneMapFilter, but i understand other filters will work fine too.

LightProbes need Forward pass as i understand? (i could only guess based on comment in code)
I hope i understand correctly.
So i can switch like:

renderManager.setRenderPath(RenderManager.RenderPath.Forward);

Some LightProbe generation / etc code / render
and later:

renderManager.setRenderPath(RenderManager.RenderPath.Deferred);

and repeat for every probe generation?

Or is it possible to do it in some another viewport having Forward just for lightprobe generation?

But anyway if you gonna change LightProbes into LightProbeVolume then i understand its just temporary way and there it would no need switch to Forward.

There are still some chineese comments to change as noticed.

while Shader code is often black magic for me, i see LightingGBufferPack.frag or PBRLightingGBufferPack.frag are replacers for old PBRLighting.frag and Lighting.frag

what will happen with older PBRLighting.frag and Lighting.frag and verts? This older ones will still work in SinglePass Forward old way and new ones in new Passes Pipeline like Deferred one?

Shaders were always not my speciality, so someone else might also want check it out if they see anything that might break.

In my summary:

looks like TON of work, first pull request preparing Framebuffer and related passes, while leaving space/todos to finish LightProbeVolume work mostly.
From user side it is not invasive(except if anything can break in shader/etc for someone)
all need to do from user side is to change pass to deferred for example.

I hope there will be more specialist to add some opinions if they catch something important. For example @Ali_RS or anyone please also share opinions

And ofc @JhonKkk thanks for your pull request, and hope to see this todos finished fully later

oxplay2 · October 10, 2023, 11:42am

Im not sure if @mitm is still here, but if i remember correctly he did nice work with wiki before. Would just probably need explain, or you can edit wiki yourself too, since you already know what to write
As i guess based on what i remember, to edit wiki, just needed re-generate wiki pages after page changes related to JME version. (wiki support wiki per JME version)

JhonKkk · October 10, 2023, 12:01pm

Currently, some existing logic has to execute under the forward rendering path, such as the old LightProbe data baking process, which I have not yet modified to run under the deferred rendering path. Therefore, when baking LightProbe data, one can “temporarily” only switch back to the Forward rendering path, and after baking is complete, switch to any rendering path (Forward/Deferred/Tile-based Deferred).

There are still some chineese comments to change as noticed.
Yes, I will delete and unify the use of English comments in subsequent work, as well as improve the existing code comments and explanations…

Actually, “xxxxPack.vert/xxxxPack.frag” are used to pack the GBufferData data, they only do packing instead of shading. So PBRLightingGBufferPack.frag and PBRLighting.frag are not the same, although they share some reused code. PBRLighting.frag contains complex PBR lighting shading logic, while PBRLightingGBufferPack.frag only reads related textures and packs them into the 4 RTs of the GBuffer. Similarly, if you want a custom MaterialDef to run under the deferred rendering path, you may provide a GBufferPass Tech{}, then provide vs and fs to pack the GBufferData.
However, why don’t I extract the related code to reuse, such as letting PBRLightingGBufferPack.frag and PBRLighting.frag share the logic to read textures by encapsulating it in XXX.glsllib? This is likely an improvement to do next, but let’s first improve step-by-step for this 1.0 version.

Yes!

JhonKkk · October 10, 2023, 12:18pm

All known bugs in tile-based DeferredShading have been fixed. Currently, its performance is far higher than deferred shading, even when running full screen.

JhonKkk · October 10, 2023, 12:27pm

This is the core part：

and my implementation: