Parallax mapping - fundamental bug?

Hi

I had a look at the example TestParallax and something immediately looked slightly wrong about the effect. After zooming about a bit and tweaking the settings to make the effect stronger I realised that in the y axis of the texture, the perspective appears “concave” whereas it should be convex. It is absolutely fine in the other (x) axis of the texture.

I have created a simpler example to more easily make the problem apparent (below). To explain as clearly as I can what the problem is, run this example and look at a brick near the bottom of the object, and you will note that you can see the brick’s southern edge’s face rather than its northern edge’s face. Note that in the texture’s x axis there is no problem - look at a brick on the left and you can see its eastern edge’s face.

N.B. the “SteepParallax” property is set true to make it easier to see the problem, but the problem exists even without SteepParallax. To verify this is not an artifact of my example, run the standard TestParallax example and up the parallax height a bit and you will see the same problem clearly.

 [java]

package jme3test.light;

import com.jme3.app.SimpleApplication;
import com.jme3.input.KeyInput;
import com.jme3.input.controls.AnalogListener;
import com.jme3.input.controls.KeyTrigger;
import com.jme3.light.DirectionalLight;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Vector3f;
import com.jme3.scene.Geometry;
import com.jme3.scene.shape.Box;
import com.jme3.texture.Texture;

public class TestSimpleParallax extends SimpleApplication implements AnalogListener {

public static void main(String[] args){
    TestSimpleParallax app = new TestSimpleParallax();
    app.start();
}
private Geometry thing;

@Override
public void simpleInitApp() {

    inputManager.addMapping("CharLeft", new KeyTrigger(KeyInput.KEY_N));
    inputManager.addMapping("CharRight", new KeyTrigger(KeyInput.KEY_M));
    inputManager.addListener(this, "CharLeft");
    inputManager.addListener(this, "CharRight");
    
    Box cube2Mesh = new Box( 1f,1f,0.01f);
    thing = new Geometry("window frame", cube2Mesh);
    thing.setLocalTranslation(0, 0.5f, 0);
        
    Material mat = assetManager.loadMaterial("Textures/Terrain/BrickWall/BrickWall2.j3m");
    
    mat.getTextureParam("DiffuseMap").getTextureValue().setWrap(Texture.WrapMode.Repeat);
    mat.getTextureParam("NormalMap").getTextureValue().setWrap(Texture.WrapMode.Repeat);
    mat.setFloat("ParallaxHeight", 0.25f);
    mat.setBoolean("SteepParallax", true);

    thing.setMaterial(mat);
    thing.setLocalScale(3f);
    rootNode.attachChild(thing);

    DirectionalLight dl = new DirectionalLight();
    dl.setDirection(new Vector3f(-1, -1, -1).normalizeLocal());
    dl.setColor(ColorRGBA.White);
    rootNode.addLight(dl);
}

public void onAnalog(String binding, float value, float tpf) 
{
    System.out.println(binding + " " + value + " " + tpf);
    if (binding.equals("CharLeft")) {
        if (value > 0) {
            System.out.println("LEFT");
            thing.rotate(0, 0, -0.01f);
        }
    } else if (binding.equals("CharRight")) {
        if (value > 0) {
            System.out.println("RIGHT");
            thing.rotate(0, 0, 0.01f);
        }
    }   
}

}

 [/java] 

Thanks
Barney

1 Like

This might actually be a bug, I always had the felling something is a bit of with my parallax mapped models, but never invested time to find out.

I’ve seen vids of great looking parallax mapping, have yet to get it looking right myself (when I say it… I mean parallax in general, be in my attempt or JME’s). Wish I could get this working properly. If someone figures it out, please tell =)

Sometime back JME switched from y-inverted normal maps to regular ones and that broke a lot of existing y-inverted cases. That being said, I can’t see how that would affect parallax mapping but I do know that when using stock JME lighting shader that I had to flip a bunch of my normal maps.

Parallax works fine in Mythruna but I use my own shaders that I forked from lighting a long time ago. Either way, tangent vectors and texture coordinates have to be setup properly. JME can take four component tangent vectors where the fourth (w) value is used to invert the vector.

I wonder if the demo is just setup poorly based on the latest “other code”.

I saw the problem not long ago…
Initially parallax mapping in JME was working ok on a moderately tesselated object. It looked completely wrong on a wide quad, because of light direction being computed in the vertex shader.
@survivor worked on some more advanced parallax mapping and posted a patch here http://hub.jmonkeyengine.org/forum/topic/patch-bug-in-steep-parallax-mapping-of-lighting-shader/ that I thought was fixing this issue.
But I may have been a bit quick on testing it, and replaced the grid for parallax with a quad and now it looks completely wrong.
If you use a grid instead of the quad for the parallax test it looks better.
I still don’t know how to fix the issue for the quad (maybe computing the light dir for each fragment but that’s gonna drain fps).

Ah, yes… I forgot about the sparse quad thing. Mythruna never has a quad bigger than 1 meter square so I wouldn’t have that particular issue.

Is this the same problem i had described at the end of my post here? http://hub.jmonkeyengine.org/forum/topic/improved-terrain-splatting-and-parallax-mapping/
If this is the problem then the fix is, invert vViewDir.y in the parallax.glsllib file (for both parallax mapping methods).

My first try at generating a patch file, I hope this works.
[java]# This patch file was generated by NetBeans IDE

Following Index: paths are relative to: D:\Development\jme3Workspaces\jmonkeyengine\trunk\engine\src\core-data\Common\ShaderLib

This patch can be applied using context Tools: Patch action on respective folder.

It uses platform neutral UTF-8 encoding and \n newlines.

Above lines and this line are ignored by the patching process.

Index: Parallax.glsllib
— Parallax.glsllib Base (BASE)
+++ Parallax.glsllib Locally Modified (Based On LOCAL)
@@ -1,5 +1,6 @@
#if (defined(PARALLAXMAP) || (defined(NORMALMAP_PARALLAX) && defined(NORMALMAP))) && !defined(VERTEX_LIGHTING)
vec2 steepParallaxOffset(sampler2D parallaxMap, vec3 vViewDir,vec2 texCoord,float parallaxScale){

  •    vViewDir.y *= -1.0;
       vec2 vParallaxDirection = normalize(  vViewDir.xy );
    
       // The length of this vector determines the furthest amount of displacement: (Ati's comment)
    

@@ -70,6 +71,7 @@
#endif
float heightScale = parallaxScale;
float heightBias = heightScale* -0.6;

  •   vViewDir.y *= -1.0;
    

\ No newline at end of file
vec3 normView = normalize(vViewDir);
h = (h * heightScale + heightBias) * normView.z;
return texCoord + (h * normView.xy);[/java]

1 Like

Happy New Year, monkeys!

I can confirm this issue. All my “RealSlimShaders” suffer from it. It must have come somewhere between RC2 and Stable because my latest, self compiled RC2 version doesn’t show the bug and my earliest Stable has it.

And since my shaders never suffered from the parallax bug mentioned above, it must be something more general like a wrong uniform or attribute passed to the shader.

RC2:

Stable:

Investigating this might be a good opportunity to stop winter sleep and jump right back into jME.

2 Likes

Hi Perjin,

Yes, it is the same problem referenced in your post, and when I apply your fix then the standard TestParallax example appears to be fixed - as indeed at first glance my simplified example appeared to be fixed.

However unfortunately that is not the whole story, since after applying your patch the effect is still broken when the textured object is rotated.

I’ve slightly modified my example below by adding a rotation of the slab object about its y-axis using comma/period keys. When you run it after patching Parallax.glsllib things initially look fixed (and indeed the fix holds up under rotation about z using n/m keys). But run it and immediately flip the slab around by holding comma, and you will see that the parallax effect is still broken. It gets more broken the more you rotate, with maximum wrongness at 180 degrees.

Afraid I have no useful input on how the shader code might be fixed, I’m fairly new to 3d :frowning:

[java]

package jme3test.light;

import com.jme3.app.SimpleApplication;
import com.jme3.input.KeyInput;
import com.jme3.input.controls.AnalogListener;
import com.jme3.input.controls.KeyTrigger;
import com.jme3.light.DirectionalLight;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Vector3f;
import com.jme3.scene.Geometry;
import com.jme3.scene.shape.Box;
import com.jme3.texture.Texture;

public class TestSimpleParallax extends SimpleApplication implements AnalogListener {

public static void main(String[] args){
    TestSimpleParallax app = new TestSimpleParallax();
    app.start();
}
private Geometry thing;

@Override
public void simpleInitApp() {

    inputManager.addMapping("CharLeft", new KeyTrigger(KeyInput.KEY_N));
    inputManager.addMapping("CharRight", new KeyTrigger(KeyInput.KEY_M));
    inputManager.addListener(this, "CharLeft");
    inputManager.addListener(this, "CharRight");
    inputManager.addMapping("CharLeftY", new KeyTrigger(KeyInput.KEY_COMMA));
    inputManager.addMapping("CharRightY", new KeyTrigger(KeyInput.KEY_PERIOD));
    inputManager.addListener(this, "CharLeftY");
    inputManager.addListener(this, "CharRightY");
    
    Box cube2Mesh = new Box( 1f,1f,0.5f);
    thing = new Geometry("window frame", cube2Mesh);
    thing.setLocalTranslation(0, 0.5f, 0);
        
    Material mat = assetManager.loadMaterial("Textures/Terrain/BrickWall/BrickWall2.j3m");
    
    mat.getTextureParam("DiffuseMap").getTextureValue().setWrap(Texture.WrapMode.Repeat);
    mat.getTextureParam("NormalMap").getTextureValue().setWrap(Texture.WrapMode.Repeat);
    mat.setFloat("ParallaxHeight", 0.25f);
    mat.setBoolean("SteepParallax", true);

    thing.setMaterial(mat);
    thing.setLocalScale(3f);
    rootNode.attachChild(thing);

    DirectionalLight dl = new DirectionalLight();
    dl.setDirection(new Vector3f(-1, -1, -1).normalizeLocal());
    dl.setColor(ColorRGBA.White);
    rootNode.addLight(dl);
}

public void onAnalog(String binding, float value, float tpf) {
    System.out.println(binding + " " + value + " " + tpf);
    if (binding.equals("CharLeft")) {
        if (value > 0) {
            System.out.println("LEFT_Z");
            thing.rotate(0, 0, -0.01f);
        }
    } else if (binding.equals("CharRight")) {
        if (value > 0) {
            System.out.println("RIGHT_Z");
            thing.rotate(0, 0, 0.01f);
        }
    } else if (binding.equals("CharLeftY")) {
        if (value > 0) {
            System.out.println("LEFT_Y");
            thing.rotate(0, -0.01f, 0);
        }
    } else if (binding.equals("CharRightY")) {
        if (value > 0) {
            System.out.println("RIGHT_Y");
            thing.rotate(0, 0.01f, 0);
        }
    }
}

}
[/java]

Thanks
Barney

1 Like

Ok, the problem is the “handedness” of the tangent which should normally come in “inTangent.w”.

Changing line 174 of “Lighting.vert” from
[java]
mat3 tbnMat = mat3(wvTangent, wvBinormal * -inTangent.w,wvNormal);
[/java]
to
[java]
mat3 tbnMat = mat3(wvTangent, wvBinormal * inTangent.w, wvNormal);
[/java]
fixes the problem.

The question @devs, @nehon is: Did the handedness change accidentally or by purpose? (TangentBinormalGenerator.java)

Hi survivor

Unless I have botched patching in your fix (and I’ve checked pretty carefully) this change does not fix the test case I posted, unfortunately.

Could you please retest with my (second) test case? Run it (it looks okay initially), but then hold comma key until the slab has rotated 180 degrees about y axis. What I see is that the bricks appear to “nod” downwards at you as each face comes into/out of view … noticeably on the initial face, and really badly on the sides and rear face. Unsurprisingly it looks worse with higher values for “ParallaxHeight” but it is still easily noticeable even for moderate heights e.g. 0.05. Doesn’t make much difference whether or not “SteepParallax” is set true.

(NB I first undid the attempted fix by Perjin, then tried it again with both changes in place, neither combination works).

If you don’t see the same problem as I do on my example, could you possibly give me the svn baseline revision number which does work for you?

Thanks
Barney

@Perjin your fix made it work again thanks.
Now I wonder why the y value of the view dir has been inverted…gonna investigate further.

Edit : didn’t saw @barney post I guess it’s still broken then

@barney: You forgot [java]TangentBinormalGenerator.generate(cube2Mesh);[/java]

I’ve created a test project with all assets included:
http://www.load.to/P4X4vFBRtd/TestSimpleParallax.zip

1 Like

Aha!

Thanks a million, it is working perfectly now. I have an example with both normals/specular and parallax mapping working and it looks brilliant - looks better than CrazyBump’s preview which is what I have been trying to recreate.

B

For the time being, it will only work with the patched Lighting.vert. Otherwise from a grazing angle it will look like this

instead of this:

The question remains: What has to change in jME: The shader or the tangent generator?

@survivor Your fix works perfectly for me.
Now that I’m comparing it to my old screenshots it looks like the normals were flipped too (the stones received light from the wrong side).

The handedness changed on purpose.
I rewrote a big part of the generator to better handle mesh with mirrored UVs thanks to @mifth constantly and tirelessly asking for it :wink:
It fixed a lot of issues with normal mapping (one being that you don’t need to invert the red channel of normal maps generated by blender anymore).

But I guess I overlooked that issue. Getting back to “inTangent.w” break standard normal mapping unfortunately. So I guess I have more digging to do in the tangent generator …sigh…

OK, @survivor’s fix and inverting the normal’s green channel (normal.y = -normal.y) again fixes the issue on both parallax and normal mapping…
I hate this guys what do we do…This can break so many games…

@nehon said: The handedness changed on purpose. I rewrote a big part of the generator to better handle mesh with mirrored UVs thanks to @mifth constantly and tirelessly asking for it ;) It fixed a lot of issues with normal mapping (one being that you don't need to invert the red channel of normal maps generated by blender anymore).

But I guess I overlooked that issue. Getting back to “inTangent.w” break standard normal mapping unfortunately. So I guess I have more digging to do in the tangent generator …sigh…

Hey if you need to test anything else i’m ready. :slight_smile:
But i see the issue is solved.

Btw, i invert normals in my shaders. #ifdef works always ok.

@nehon: There are only two solutions:

  1. Change handedness in TBG as it was in RC2.
  • no shader has to change
  • models with saved tangents stay compatible
  • you have to dig into TBG, which sounds like pain for you :slight_smile:
  1. Change handedness in shaders
    pro/con: inverse of 1.

I’d change it, so that models are compatible to ogre or other formats. Don’t know if there is a quasi standard how the tangent buffer should look like.

Edit: I guess Blender is the main asset source, so it’s best to keep that pipeline as simple as possible (like @mifth requested).