Setting up an HDR Filter

Troubleshooting
1

Hi,

I have several post process filters on my main viewport, and I would like to add HDR. As discussed in this thread, HDRRenderer and Filters do not work together.

So I would like to make my own HDR Filter, for which I need the RGBA16F format on the color texture of the main viewport. (I also use several viewports and the ComposeFilter to stitch the final image together.)

My application extends SimpleApplication. I set the frame buffer of the main viewport (which is SimpleApplication’s viewport) like this:

FrameBuffer fb=new FrameBuffer(getSettings().getWidth(), getSettings().getHeight(), 1);
Texture2D colorTexture=new Texture2D(getSettings().getWidth(), getSettings().getHeight(), Image.Format.RGBA16F);
composeFrameBuffer.addColorTexture(colorTexture);
composeFrameBuffer.setDepthBuffer(Image.Format.Depth);
viewPort.setOutputFrameBuffer(fb);

When I run the application I see a black screen. How do I tell the renderer to put this framebuffer on screen?

Since you’re the shader guru, do you have any better idea, @nehon ?

2

There’s ToneMapFilter which does HDR and works as a filter. The only downside is that it doesn’t have adaptive exposure adjustment, but that’s kind of a gimmick feature anyway…

3

Oh, I was not familiar with that filter. But it doesn’t seem to use 16 bit colors. Isn’t it true, that the point is to use a bigger range of colors to be able to apply a nice bloom? I would like to do something like Sonic Ether’s Natural Bloom shader for JME, because the built in Bloom shader just “does not bloom enough”.

4

The point of HDR is to map a much higher luminance range like what is present in the real world into a computer screen which can only support a few colors – i.e. RGB8.
jME3.1 uses the RGB111110F format for the FilterPostProcessor which should be good enough for HDR and the ToneMapFilter.

5

Thank you for your answers, I didn’t know that detail about the FPP. I now have some ideas which I will try. Thanks once more!

6

So, I’ve recently tried the ToneMap Filter but I’m not happy with it. I’d like to experiment with different tone mapping functions and therefore setup my own HDR Filter.

In order to calculate the exposure, I would like to render the scene into a single pixel. I use an approach I found in the BloomFilter class:

preHdrMaterial=new Material(manager, "MatDefs/Hdr/PreHdr.j3md");
preHdrPass=new Pass() {
@Override
public boolean requiresSceneAsTexture() {return true;}
};
preHdrPass.init(renderManager.getRenderer(), 1, 1, Image.Format.RGBA8, Image.Format.Depth, 1, preHdrMaterial);

postRenderPasses=new ArrayList<Pass>();
postRenderPasses.add(preHdrPass);

material.setTexture("Luminance", preHdrPass.getRenderedTexture());

When I try to get the result in the HDR filter fragment shader like this:

vec4 lumVal = texture2D(m_Luminance, texCoord);

then lumVal is jsut pitch black. What did I miss here?

7

I believe several people tried implementing HDR via the filter system and something didn’t work.

Also, it’s a bit hard to say what’s going on with just that part of the code.

8

This is the minimum test case. The PreHdr shader should just output a red color, however, black arrives at the ToneMap shader.

The Filter:

import com.jme3.asset.AssetManager;
import com.jme3.material.Material;
import com.jme3.post.Filter;
import com.jme3.renderer.RenderManager;
import com.jme3.renderer.ViewPort;
import com.jme3.texture.*;
import java.util.ArrayList;

public class HDRFilterTestCase extends Filter {

    private Material preHdrMaterial;
    private Pass preHdrPass;

    public HDRFilterTestCase() {
        super("HDRFilter");
    }
    
    @Override
    protected boolean isRequiresDepthTexture() {
        return false;
    }

    @Override
    protected void initFilter(AssetManager manager, RenderManager renderManager, 
     ViewPort vp, int w, int h) {
        material = new Material(manager, "MatDefs/Hdr/ToneMap.j3md");
        preHdrMaterial=new Material(manager, "MatDefs/Hdr/PreHdr.j3md");
        preHdrPass=new Pass()
        {
             @Override
             public boolean requiresSceneAsTexture() {return true;}
        };
        preHdrPass.init(renderManager.getRenderer(), 1, 1, Image.Format.RGBA8, Image.Format.Depth, 1, preHdrMaterial);
        postRenderPasses=new ArrayList<Pass>();
        postRenderPasses.add(preHdrPass);

        material.setTexture("Luminance", preHdrPass.getRenderedTexture());
    }  
   
    @Override
    protected Material getMaterial() {
        return material;
    }
}

PreHdr.j3md

MaterialDef PreHdr {
    MaterialParameters {
        Int NumSamples
        Int NumSamplesDepth
        Texture2D Texture
    }


    Technique {
        VertexShader GLSL150:   Common/MatDefs/Post/Post15.vert
        FragmentShader GLSL150: MatDefs/Hdr/PreHdr.frag

        WorldParameters {
        }

        Defines {
        TEXTURE
            RESOLVE_MS : NumSamples
            RESOLVE_DEPTH_MS : NumSamplesDepth
        }
    }


    Technique {
        VertexShader GLSL100:   Common/MatDefs/Post/Post.vert
        FragmentShader GLSL100: MatDefs/Hdr/PreHdr.frag

        WorldParameters {
        }

        Defines {
            TEXTURE
        }
    }
}

PreHdr.frag

#import "Common/ShaderLib/MultiSample.glsllib"

uniform COLORTEXTURE m_Texture;

#if __VERSION__ >= 150
in vec2 texCoord;
out vec4 outFragColor;
#else
varying vec2 texCoord;
#endif



void main()
{
    //vec4 texVal=texture2D(m_Texture, texCoord);

    vec4 texVal=vec4(1.0, 0.0, 0.0, 1.0);
    
    #if __VERSION__ >= 150
        outFragColor = texVal;
    #else
        gl_FragColor = texVal;
    #endif
}

ToneMap.j3md

MaterialDef Tone Mapper {
    MaterialParameters {
        Int NumSamples
        Int NumSamplesDepth
        Texture2D Texture
        Texture2D Luminance
    }


    Technique {
        VertexShader GLSL150:   Common/MatDefs/Post/Post15.vert
        FragmentShader GLSL150: MatDefs/Hdr/ToneMap.frag

        WorldParameters {
            WorldViewProjectionMatrix
        }

        Defines {
        TEXTURE
            RESOLVE_MS : NumSamples
            RESOLVE_DEPTH_MS : NumSamplesDepth
        }
    }


    Technique {
        VertexShader GLSL100:   Common/MatDefs/Post/Post.vert
        FragmentShader GLSL100: MatDefs/Hdr/ToneMap.frag

        WorldParameters {
            WorldViewProjectionMatrix
        }

        Defines {
            TEXTURE
        }
    }
}

ToneMap.frag

#import "Common/ShaderLib/MultiSample.glsllib"

uniform COLORTEXTURE m_Texture;
uniform sampler2D m_Luminance;

#if __VERSION__ >= 150
   in vec2 texCoord;
   out vec4 outFragColor;
#else
   varying vec2 texCoord;
#endif



void main() {
    vec4 texVal = getColor(m_Texture, texCoord);
    vec4 lumVal = texture2D(m_Luminance, texCoord);
    
    // Should be red, but is black
    vec3 toneMapped = lumVal.rgb;

    #if __VERSION__ >= 150
        outFragColor = vec4(toneMapped, texVal.a);
    #else
        gl_FragColor = vec4(toneMapped, texVal.a);
    #endif
}
9

So you are just fetching a single pixel, because your target is 1x1. That’s not going to work… If you look at the current HDRRenderer, it first does an area filter with a 64x64 target, then 4x4, and finally 1x1. Meaning it actually averages every pixel in the source to get the luminance value.

10

That was my first thought too. But if I change the size like

preHdrPass.init(renderManager.getRenderer(), 128, 128, Image.Format.RGBA8, Image.Format.Depth, 1, preHdrMaterial);

we can savely say that the output of PreHdr.frag is red.
However, m_Luminance in ToneMap.frag is still just black everywhere, and I just don’t see a reason for that.

11

This however

preHdrPass.getRenderedTexture().getImage().getData()

is returning an empty “[ ]”. So maybe it doesn’t even get drawn. I’m confused now.

12

Yippie! Got it,

There seems to be no problem to render the scene into a single pixel, BUT, as I learned today:
Never attempt to make a shader without the WorldViewProjectionMatrix in the .j3md.

WorldParameters {
    WorldViewProjectionMatrix
}

Actually this caused the output color to be black. I guess it is because the reference to the current fragment in gl_FragColor gets lost.

13

That’s odd, because the Post.vert shader doesn’t rely on it…