Hi Monkeys,
After a reasonably long hiatus, I have returned =), and for my returning contribution, I present a Screen Space Distortion Filter (SSDF) :
This is a test scene rendering showing the effect in action in several forms : simple static box, animated Sinbad and a very basic particle fire. The scene along with the full source is available for download (~686kb). Keep in mind this is still a work in progress, so the filter has not been completely finalised. However the source is a good learning tool to accompany the explanation bellow, so i will offer it up in its current form. Oh, and the video was recorded using an older version of the source, so the download doesn’t exactly match the video (mostly just the debug view).
(fire close up)
Some other possible applications would be explosion shock waves, Matrix style bullet trails, water splashes on the screen, glass effects, water effects, Predator camouflage, heat haze… I’m sure you can think of more.
I can’t take full credit for this, @Nehon had a lot of input and helped me optimise the performance significantly.
The filter itself is pretty trivial, It uses a system similar to normal mapping, the red channel is used for x (or u) offset, from 0 - 255, with 128 (half red) being 0% offset, 0 (or no red) being -100% offset, and 255 (full red) being 100% offset. The green channel is used for y (or v) offset, and the blue channel is a multiplier. These details are still being toyed with and finalized.
For the test, I used a simple variation on the ColoredTextured material. How the colour for the offset is decided (ie what material is used), really doesn’t matter all that much. A solid colour would work and would give a uniform offset across an entire object. A simple Fresnel based shader would work great as it could be used to make the edges distort more than the middle. Animated textures, like in the particle material, could also be used, and would be effective on objects that aren’t moving, to give the displacement some movement (ripples on the surface of a pond for example).
Applying Post-Process Filter effects to only selected objects
The key to getting the filter to work, was finding a way to isolated certain objects in the scene, and applying a filter only to them. The technique I present should be handy for many shader developers, and something I have wanted the ability to do for a while now. I will try and explain how I achieved this…
Step 1: Find a way to flag or tag the items we are trying to isolate.
For this I used a custom Material Technique. The idea being that I can create a new material, or modify an existing one, to include this new technique, then when it comes to post, I can single out only objects using a material with this technique using a ForcedTechnique render pass. For the SSDF, I created a technique “ScreenSpaceDistortion”.
Step 2: Make sure the objects using the new material, don’t get rendered with the rest of the scene.
This isn’t always strictly required for all effects, but for simplicity, I do this with my new material. To achieve this, I simply changed the default Technique fragment shader, to use a new shader “Blank.frag”, which is empty :
void main() {
}
(I tried to simply remove the default Technique all together but this causes all sorts of problems, a change to the core would be required to get this to work)
At this point, our scene can be rendered without the objects we are trying to isolate in post, so we move onto the Filter itself.
Step 3 : Setup a render pass to render only objects we wish to apply the filter to (those objects using a material with the new technique).
This is done with a simple render pass, setup in the initFilter method.
distortPass = new Pass();
distortPass.init(renderManager.getRenderer(), w, h, Format.RGB8, Format.Depth, 1, true);
Step 4 :Setup our final composition shader, and pass the colour and depth information from the render pass, to the shader.
material = new Material(manager, "MatDefs/SSDFinal.j3md");
material.setTexture("DistortionTex", distortPass.getRenderedTexture());
material.setTextu
re(“DistortionDepth”, distortPass.getDepthTexture());
Step 5 : Render only our target objects.
Within postQueue :
renderManager.getRenderer().setFrameBuffer(distortPass.getRenderFrameBuffer()); // we want to render to the distortPass Frame Buffer
renderManager.getRenderer().clearBuffers(true, true, true); // clear the buffer
renderManager.setForcedTechnique("ScreenSpaceDistortion"); // set our forced technique
renderManager.renderViewPortQueues(viewPort, false); // do the render
renderManager.setForcedTechnique(null); // clear our forced technique so we dont screw up the rest of the application
Step 6: Apply our effect (or filter) to the selected target objects.
This is done within the final composition shader, SSDFinal.j3md in this case, which has access to 4 key Textures …
uniform sampler2D m_Texture; // original scene colour
uniform sampler2D m_DepthTexture; // original scene depth
uniform sampler2D m_DistortionTex; // target object colours
uniform sampler2D m_DistortionDepth; // target objects depths
It’s now in the hands of the shader developer to work their magic.
Step 7 : Compose the final render by merging the original scene with the new effect render.
In the case of the distortion filter, as with most filters, it is important to retain the depth order, so foreground objects occlude (hide) background objects. A depth test is achieved by using a step function in the final shader (basically a simple less than / greater than comparator).
this is how it looks in the SSDFinal.frag …
gl_FragColor = mix(texture2D(m_Texture, displacementCoord),origColor,step(sceneDepth,distortionDepth));
but a more generic version would be:
gl_FragColor = mix(texture2D(m_Texture, texCoord), texture2D(m_CustomObjectPassTex, texCoord),step(texture2D(m_DepthTexture,textCoord),texture2D(m_CustomObjectPassDepth,textCoord)));
… which is basically saying, at a point on the screen (texCoord), compare the depth of the original scene render and our selected target objects render depth, (step …), and which ever is closer to the camera, display that (mix…).
Step 8: ???
Step 9 : Profit!
That is about it. This technique may not be the best solution, or the cleanest, but it works well, is pretty flexible so I hope you can find some use for it
Feel free to chip in with any feedback or ideas, I hope this can help some of you better understand how post-proccessing works, and how to leverage it to your advantage =)
Cheers
James