GLSL shader transparency problems

Troubleshooting graphics
1

I have been playing around with the Procedural planet generator demo ( http://www.jmonkeyengine.com/forum/index.php?topic=6410.0 )



I have run into a problem which I can't seem to fix though, involving the shader generated cloud effect around the planet.



This code gives the following image:


Planet p1 = new Planet(infos, generator, display.getRenderer());
p1.setLocalTranslation(0, 0, -50);
p1.updateGeometricState(-1, true);
rootNode.attachChild(p1);

Planet p2 = new Planet(infos, generator, display.getRenderer());
p2.setLocalTranslation(50, 0, -50);
p2.updateGeometricState(-1, true);
rootNode.attachChild(p2);






However, here comes the strange part


Planet p1 = new Planet(infos, generator, display.getRenderer());
p1.setLocalTranslation(50, 0, -50);  // Swapped
p1.updateGeometricState(-1, true);
rootNode.attachChild(p1);

Planet p2 = new Planet(infos, generator, display.getRenderer());
p2.setLocalTranslation(0, 0, -50);   // Swapped
p2.updateGeometricState(-1, true);
rootNode.attachChild(p2);





By swapping the order that I create the 2 planets, the shader created cloud effect becomes opaque. I'm guessing this has something to do with the draw order. If I turn off the cloud shader, then everything works as you would expect. I have tried changing the alpha blending and other options on the shader, but I can't seem to solve the problem. The zbuffer is enabled for the shaders:


        zbufferEnabledState = renderer.createZBufferState();
        zbufferEnabledState.setFunction(ZBufferState.TestFunction.LessThan);
        zbufferEnabledState.setEnabled(true);
...
        planetRenderPass.setPassState(zbufferEnabledState);
        atmoFrontRenderPass.setPassState(zbufferEnabledState);
        atmoBackRenderPass.setPassState(zbufferEnabledState);



Any ideas?  :?
2

It has to do with the order you're drawing the objects. JME has functionality in sorting objects to prevent the sort of problem you're running into, using RenderQueues, but you have to make sure your objects are set to the appropriate bucket:



planet.setRenderQueueMode(Renderer.QUEUE_TRANSPARENT)



Before objects get drawn, they get sorted so they'll be drawn back-to-front. That's why the first time around you didn't have any problems because it was already in that order.



Check out the wiki for some more info on renderqueues and transparency:



http://www.jmonkeyengine.com/wiki/doku.php/renderqueue



http://www.jmonkeyengine.com/wiki/doku.php/transparency

3

Thanks for the fast reply!



I tried planet.setRenderQueueMode(Renderer.QUEUE_TRANSPARENT), as well as setting the sphere inside the planet node to transparent, but it has no effect. The order I 'attachChild' the planets still causes differences in transparency.

4

Hmm a conundrum! The problem must lie in the shader/render passes then, unfortunately I have very rarely used those two features of jmonkey.



Also make sure that you have the blendstate applied to your renderpasses or planets (I saw a lot about zbuffer state but a fleeting reference to the blend state, although pass states override spatial states, so I'd imagine having a blendstate on your planets would be just as good).



Perhaps supply some more code in how you set up your render passes?

5

The code for the shaders/render passes comes entirly from this thread: http://www.jmonkeyengine.com/forum/index.php?topic=6410.0



Here is how the render passes are set up:



        zbufferEnabledState = renderer.createZBufferState();
        zbufferEnabledState.setFunction(ZBufferState.TestFunction.LessThan);
        zbufferEnabledState.setEnabled(true);


        backFaceCullingState = renderer.createCullState();
        backFaceCullingState.setCullFace(CullState.Face.Back);
        backFaceCullingState.setEnabled(true);

        frontFaceCullingState = renderer.createCullState();
        frontFaceCullingState.setCullFace(CullState.Face.Front);
        frontFaceCullingState.setEnabled(true);

        alphaBlendingState = renderer.createBlendState();
        alphaBlendingState.setDestinationFunction(BlendState.DestinationFunction.OneMinusSourceAlpha);
        alphaBlendingState.setSourceFunction(BlendState.SourceFunction.SourceAlpha);
        alphaBlendingState.setBlendEnabled(true);
        alphaBlendingState.setEnabled(true);
        alphaBlendingState.setTestEnabled(true);



and how they are applied:


        /* Creating each render pass: planet, atmosphere back face & atmosphere front face */
        planetRenderPass = new RenderPass();
        planetRenderPass.add(planetGeom);
        planetRenderPass.setPassState(textureState);
        planetRenderPass.setPassState(backFaceCullingState);
        planetRenderPass.setPassState(planetShader);

        atmoFrontRenderPass = new RenderPass();
        atmoFrontRenderPass.add(planetGeom);
        atmoFrontRenderPass.setPassState(frontFaceCullingState);
        atmoFrontRenderPass.setPassState(alphaBlendingState);
        atmoFrontRenderPass.setPassState(atmoShader);

        atmoBackRenderPass = new RenderPass();
        atmoBackRenderPass.add(planetGeom);
        atmoBackRenderPass.setPassState(backFaceCullingState);
        atmoBackRenderPass.setPassState(alphaBlendingState);
        atmoBackRenderPass.setPassState(atmoShader);



I think it must have something to do with the order in which the shaders are run, but I can't find any way to modify this. When I get some time, I will try dynamically un-attaching and re-attaching the planet nodes depending on their distance to the camera. I think this will fix the problem, but would make it very slow...
6

I added the following as a test to my update function:



int counter = 0;
...
{
        counter++;

        if (counter == 100) {
            sunNode.detachAllChildren();
            sunNode.attachChild(p1);
            sunNode.attachChild(p2);
        }
        if (counter > 200) {
            counter = 0;
            sunNode.detachAllChildren();
            sunNode.attachChild(p2);
            sunNode.attachChild(p1);
        }
}



This actually causes the transparency to cycle between the two images in the original post. So the order that the nodes are added to the scene graph is definitely causing which node will be transparent. This method is actually very fast and does not seem to cause any loss of FPS.

I'm guessing its a bug in the render pipeline with regards to shaders?
7

After messing around with shaders more, I think I found the reason for this. Not exactly a bug…



The planet itself has 3 render passes: planetRenderPass, atmoFrontRenderPass, atmoBackRenderPass



Normally when rendering 2 transparent spheres, it will do this:



Main Renderer
   -> Render the opaque bucket
   -> Render the transparent bucket
        -> Sort the spheres and render them correctly
   -> Render the ortho bucket
Finished



However, since each planet object has its own passes, it looks more like this (feel free to correct this if its off!)


Main Renderer

    planetRenderPass for Planet 1
        -> Render the opaque bucket
        -> Render the transparent bucket
             -> Sort the items in the bucket -- problem is here, we only have the 1 sphere in the bucket!
        -> Render the opaque bucket    
    end planetRenderPass for Planet 1
    ... atmoFrontRenderPass for Planet 1 ...
    ... atmoBackRenderPass for Planet 1 ...

    planetRenderPass for Planet 2
        -> Render the opaque bucket
        -> Render the transparent bucket
             -> Sort the items in the bucket -- problem is here, we only have the 1 sphere in the bucket!
        -> Render the opaque bucket    
    end planetRenderPass for Planet 1
    ... atmoFrontRenderPass for Planet 2 ...
    ... atmoBackRenderPass for Planet 2 ...

End Main Renderer



Wow  :-o

Doing shaders like this is expensive! We do 6 render passes now instead of 1; 3 render passes per planet, and we lose the automatic sorting of transparencies.


The best way to do this seems to be to create 3 global render passes, but I'm not sure if this is the correct way to do this. If I do the geometry (planetRenderPass) first, then the 2 shader passes after that will always draw on top of anything that has been drawn before. So anything drawn by these shaders will not be hidden by anything in front of them - No good.

Does anybody know the correct way to approach multiple shader passes across multiple objects? I'm guessing there is some method of sorting objects for render passes.
8

Correct method that the renderer should use (I think):



Main Renderer
 -> Render the opaque bucket (all objects)
 -> Render the transparent bucket (all objects)
     -> Sort all the objects
     -> Do all the render passes per object, 1 by 1, in the correct sorted order
  -> Render the ortho bucket
End Main Renderer



This is how I'm currently doing it already - except for the sorting. It would be slow with a lot of objects, even with the sorting correct...  :|