HDR filter use multi pass

1. Gets the average color of the current screen.
Step 1: downsample source image to quarter (for example 1280x720 → 320x180):

	postRenderPasses = new ArrayList<Pass>();
	final Material lumMat = new Material(manager, "Shaders/HDR/Lum.j3md");
	lumMat.setVector2("Resolution", new Vector2f(screenWidth / 4, screenHeight / 4));
	lumPass = new Pass() {
		@Override
		public boolean requiresSceneAsTexture()
		{
			return true;
		}

	};

	lumPass.init(renderManager.getRenderer(), screenWidth / 4, screenHeight / 4, Format.RGBA8,
			Format.Depth, 1, lumMat);
	postRenderPasses.add(lumPass);

In Lum.frag, I calculate the average color lick this:

uniform sampler2D m_Texture; //color
uniform vec2 m_Resolution;

varying vec2 texCoord;

void main() {
	vec4 color = vec4(0.);
	vec2 fragCoord = floor(texCoord * m_Resolution);
	for (float i = 0.; i < 4.; i++) {
		for (float j = 0.; j < 4.; j++) {
			color += texture2D(m_Texture, (fragCoord + vec2(i, j)) / m_Resolution);
		}
	}
	color /= 16.;
	gl_FragColor = color;
}

Preview:
Convert from:


To:

Step 2: downsample to until to 1x1 (for example 320x180 → 64x64 → 16x16 → 4x4 → 1x1):

	tex64 = lum(manager, lumPass.getRenderedTexture(), 64);
	tex16 = lum(manager, tex64, 16);
	tex4 = lum(manager, tex16, 4);
	tex1 = lum(manager, tex4, 1);

The lum method:

private Texture2D lum(AssetManager manager, final Texture texture, float width)
{
	final Material lumMat = new Material(manager, "Shaders/HDR/Lum.j3md");
	lumMat.getAdditionalRenderState().setWireframe(true);
	lumMat.setVector2("Resolution", Vector2f.UNIT_XY.mult(width));
	Pass lumPass = new Pass() {

		@Override
		public void beforeRender()
		{
			lumMat.setTexture("Texture", texture);
		}
	};
	lumPass.init(renderManager.getRenderer(), screenWidth, screenHeight, Format.RGBA8, Format.Depth,
			1, lumMat);
	postRenderPasses.add(lumPass);
	return lumPass.getRenderedTexture();
}

The "tex1 " is the average color we want.
2. Delay time in HDR, delay the average color
Seem like eye pupil accommodation, the HDR effect has a delay time. So I add deferred passes.
The whiteBuf is a first texture, it’s necessary.

	whiteBuf = manager.loadTexture("Textures/whiteBuffer.png");
	final Material deferredMat1 = new Material(manager, "Shaders/HDR/DeferredLum.j3md");
	Pass deferredPass1 = new Pass() {
		private int count;

		@Override
		public void beforeRender()
		{
			super.beforeRender();
			if (count == 0)
			{
				deferredMat1.setTexture("LastTexture", whiteBuf);
			} else
			{
				deferredMat1.setTexture("LastTexture", deferredPass2.getRenderedTexture());
			}
			count++;
		}
	};
	deferredMat1.setTexture("LumTex", tex1);
	deferredPass1.init(renderManager.getRenderer(), 1, 1, Format.RGBA8, Format.Depth, 1,
			deferredMat1);
	postRenderPasses.add(deferredPass1);

	Material deferredMat2 = new Material(manager, "Shaders/Common/DeferredBuffer.j3md");
	deferredPass2 = new Pass();
	deferredMat2.setTexture("LastTexture", deferredPass1.getRenderedTexture());
	deferredPass2.init(renderManager.getRenderer(), 1, 1, Format.RGBA8, Format.Depth, 1,
			deferredMat2);
	postRenderPasses.add(deferredPass2);

In DeferredLum.frag, I interpolate the color like this:
(0.03 is just a data I use, it affect the delay time)

uniform sampler2D m_LumTex;
uniform sampler2D m_LastTexture;

void main() {
	vec2 tc = vec2(0.);
	vec4 last = texture2D(m_LastTexture, tc);
	vec4 now = texture2D(m_LumTex, tc);

	gl_FragColor = mix(last, now, 0.03);
}

So we can get the delayed color by deferredPass2.getRenderedTexture().
3. Tone map
After get the delayed color, we use the color to change the tone map.
I use official “Common/MatDefs/Hdr/ToneMap.j3md” and change any frag code.

	final Material toneMapMat = new Material(manager, "Shaders/HDR/ToneMap.j3md");
	final Pass toneMapPass = new Pass() {

		@Override
		public boolean requiresSceneAsTexture()
		{
			return true;
		}

	};
	toneMapMat.setTexture("LumTex", deferredPass2.getRenderedTexture());
	toneMapPass.init(renderManager.getRenderer(), screenWidth, screenHeight, Format.RGBA8,
			Format.Depth, 1, toneMapMat);
	postRenderPasses.add(toneMapPass);

In ToneMap.frag, I use average color to change whitepoint.

void main() {
	vec4 color = texture2D(m_Texture, texCoord);
	if (color.a <= 0.) {
		gl_FragColor = color;
		return;
	}
	vec4 lumColor = texture2D(m_LumTex, vec2(0.));
	float gray = clamp(0.27 * lumColor.r + 0.67 * lumColor.g + 0.06 * lumColor.b, 0.1, 1.);
	vec3 toneMapped = ToneMap_Filmic(color.rgb, vec3(gray * 2.0));

	vec4 finalColor = vec4(toneMapped, color.a);
	gl_FragColor = finalColor;
}

4. Cut the dark pixeles.

	final Material cutMat = new Material(manager, "Shaders/HDR/CutLum.j3md");
	Pass cutPass = new Pass() {
		@Override
		public void beforeRender()
		{
			cutMat.setTexture("Texture", toneMapPass.getRenderedTexture());
		}
	};
	cutPass.init(renderManager.getRenderer(), screenWidth, screenHeight, Format.RGBA8, Format.Depth,
			1, cutMat);
	postRenderPasses.add(cutPass);

Preview:

5. Blur several times
I use expand, hGaussianBlur ,vGaussianBlur three passes to blur one time.

private Texture2D gaussianBlur(AssetManager manager, final Texture texture, final float downSample,
		final float blurScale, Format format)
{
	final Material expandMat = new Material(manager, "Shaders/HDR/Expand.j3md");
	final Pass expandPass = new Pass() {
		@Override
		public void beforeRender()
		{
			expandMat.setTexture("Texture", texture);
		}
	};
	expandPass.init(renderManager.getRenderer(), (int) (screenWidth / downSample),
			(int) (screenHeight / downSample), format, Format.Depth, 1, expandMat);
	postRenderPasses.add(expandPass);

	Texture2D hblur = hGaussianBlur(manager, expandPass.getRenderedTexture(), downSample, blurScale,
			format);
	Texture2D vblur = vGaussianBlur(manager, hblur, downSample, blurScale, format);
	return vblur;
}
  1. Expand is to expand bright area use max method.

     void main() {
     	vec4 expand = texture2D(m_Texture, texCoord);
     	expand = max(expand, texture2D(m_Texture, vec2(gl_FragCoord.x - 1., gl_FragCoord.y)
     			/ g_Resolution));
     	expand = max(expand, texture2D(m_Texture, vec2(gl_FragCoord.x + 1., gl_FragCoord.y)
     			/ g_Resolution));
     	expand = max(expand, texture2D(m_Texture, vec2(gl_FragCoord.x, gl_FragCoord.y - 1.)
     			/ g_Resolution));
     	expand = max(expand, texture2D(m_Texture, vec2(gl_FragCoord.x, gl_FragCoord.y + 1.)
     			/ g_Resolution));
     	gl_FragColor = expand;
     }
3 Likes

Blured texture is like this:



6.Combine
At last, we combine toneMapped texture, several blured textures.

Edited:
Add a dirt texture(Export from UE4).


Dirt texture multiply blured texture.
The result is like this:

I commit it to GitHub!

14 Likes

Thanks for the explanation , that’s pretty cool.

Not sure I understand what went on here. Could you give a brief explanation of the goal/purpose?

1 Like

I has posted git link.

1 Like

Not sure of the purpose w/HDR? This might help.

Also compare the very last screenshot (with the tapestries/flags) with the very first. Very cool!!!

Looks great, what’s the performance impact though?