NormalMaps+Lighting Issue

mifth · June 25, 2013, 9:08am

Hi devs!

Before the release i would like to report a very ugly issue with NormalMaps + Lighting. Some artifacts appears because mirrored model(a half) and a different direction of light. Here is my screenshot with different direction of light: http://i.imgur.com/Bk4Z8DU.jpg

Possible issue:

Tangent Generator Issue
Lighting Shader issue
Light Calculation issue

I also found strange log of generated tangents.
LOG OF GENERATED TANGENTS:
jme_lightblow listening for external changes on /home/mifth/jMonkeyProjects/NormalMapTest/assets/Models/LightBlow/jme_lightblow.blend@586e062d:22b64866
Loaded asset jme_lightblow
Angle between tangents exceeds tolerance for vertex 3.
Angle between tangents exceeds tolerance for vertex 10.
Angle between tangents exceeds tolerance for vertex 24.
Angle between tangents exceeds tolerance for vertex 23.
Angle between tangents exceeds tolerance for vertex 29.
Angle between tangents exceeds tolerance for vertex 35.
Angle between tangents exceeds tolerance for vertex 47.
Angle between tangents exceeds tolerance for vertex 51.
Angle between tangents exceeds tolerance for vertex 67.
Angle between tangents exceeds tolerance for vertex 71.
Angle between tangents exceeds tolerance for vertex 770.
Angle between tangents exceeds tolerance for vertex 777.
Angle between tangents exceeds tolerance for vertex 74.
Angle between tangents exceeds tolerance for vertex 80.
Angle between tangents exceeds tolerance for vertex 78.
Angle between tangents exceeds tolerance for vertex 82.
Angle between tangents exceeds tolerance for vertex 95.
Angle between tangents exceeds tolerance for vertex 101.
Angle between tangents exceeds tolerance for vertex 98.
Angle between tangents exceeds tolerance for vertex 102.
Angle between tangents exceeds tolerance for vertex 107.
Angle between tangents exceeds tolerance for vertex 111.
Angle between tangents exceeds tolerance for vertex 124.
Angle between tangents exceeds tolerance for vertex 121.
Angle between tangents exceeds tolerance for vertex 126.
Angle between tangents exceeds tolerance for vertex 133.
Angle between tangents exceeds tolerance for vertex 137.
Angle between tangents exceeds tolerance for vertex 152.
Angle between tangents exceeds tolerance for vertex 157.
Angle between tangents exceeds tolerance for vertex 144.
Angle between tangents exceeds tolerance for vertex 150.
Angle between tangents exceeds tolerance for vertex 154.
Angle between tangents exceeds tolerance for vertex 178.
Angle between tangents exceeds tolerance for vertex 158.
Angle between tangents exceeds tolerance for vertex 181.
Angle between tangents exceeds tolerance for vertex 161.
Angle between tangents exceeds tolerance for vertex 163.
Angle between tangents exceeds tolerance for vertex 165.
Angle between tangents exceeds tolerance for vertex 167.
Angle between tangents exceeds tolerance for vertex 194.
Angle between tangents exceeds tolerance for vertex 199.
Angle between tangents exceeds tolerance for vertex 203.
Angle between tangents exceeds tolerance for vertex 209.
Angle between tangents exceeds tolerance for vertex 233.
Angle between tangents exceeds tolerance for vertex 236.
Angle between tangents exceeds tolerance for vertex 229.
Angle between tangents exceeds tolerance for vertex 237.
Angle between tangents exceeds tolerance for vertex 235.
Angle between tangents exceeds tolerance for vertex 262.
Angle between tangents exceeds tolerance for vertex 263.
Angle between tangents exceeds tolerance for vertex 281.
Angle between tangents exceeds tolerance for vertex 323.
Angle between tangents exceeds tolerance for vertex 286.
Angle between tangents exceeds tolerance for vertex 324.
Angle between tangents exceeds tolerance for vertex 293.
Angle between tangents exceeds tolerance for vertex 294.
Angle between tangents exceeds tolerance for vertex 311.
Angle between tangents exceeds tolerance for vertex 314.
Angle between tangents exceeds tolerance for vertex 321.
…
Angle between tangents exceeds tolerance for vertex 2 322.
Angle between tangents exceeds tolerance for vertex 2 321.
Angle between tangents exceeds tolerance for vertex 2 324.
Angle between tangents exceeds tolerance for vertex 2 335.
Angle between tangents exceeds tolerance for vertex 2 353.
Angle between tangents exceeds tolerance for vertex 2 363.
Angle between tangents exceeds tolerance for vertex 2 365.
unsupported key:524
unsupported key:524
File jme_lightblow.j3o saved successfully
unsupported key:154
unsupported key:154

mifth · June 25, 2013, 9:32am

Here is Another Screenshot: http://i.imgur.com/hh9dwi8.png
And here it is an example project with these examples: https://dl.dropboxusercontent.com/u/26887202/JME/NormalMapTest.zip

Lighting Shader was used for testing.

Momoko_Fan · June 25, 2013, 5:24pm

Wasn’t it working before for this model? What has changed since then?

mifth · June 25, 2013, 7:17pm

@Momoko_Fan said: Wasn't it working before for this model? What has changed since then?

Nothing changed. I just noticed it recently when i set my model with one light and without textures.

mifth · June 25, 2013, 7:22pm

@Momoko_Fan said: Wasn't it working before for this model? What has changed since then?

The model had good lighting before the full rewrite of the tangentGenerator. With that W component…

Momoko_Fan · June 26, 2013, 11:16pm

Also it depends on how the normal map was generated. If it was generated for a model that didn’t have flipped tangents then obviously its going to look incorrect in jME3 now.

mifth · June 27, 2013, 6:48am

@Momoko_Fan said: Also it depends on how the normal map was generated. If it was generated for a model that didn't have flipped tangents then obviously its going to look incorrect in jME3 now.

Is it possible to make a Boolean for flipped tangents in the TangentGenerator? As in blender and in Unity the model looks ok. This is very important feature.
Here is a screenshot from blender: http://i.imgur.com/n2LrP11.png
And here is the blend model with normalmap settings you can test: https://dl.dropboxusercontent.com/u/26887202/JME/jme_lightblow.zip

In JME if light direction is the same as camera direction, so model looks ok. But if light direction is different of the camera direction, so there becomes a problem. You can see it on my screenshots from JME:
http://i.imgur.com/Bk4Z8DU.jpg - on the left side directions(light and camera) are the same and model looks ok.
http://i.imgur.com/hh9dwi8.png - on the right side directions(light and camera) are the same and model looks ok.

I guess this is only TangentGenerator issue…

Momoko_Fan · June 27, 2013, 6:52pm

Are you sure this is caused specifically by the flipped tangents? There were many changes afterwards also. If the GLSL code multiplying by tangent.w is removed, does the model appear correct?

mifth · June 27, 2013, 7:05pm

@Momoko_Fan said: Are you sure this is caused specifically by the flipped tangents? There were many changes afterwards also. If the GLSL code multiplying by tangent.w is removed, does the model appear correct?

Yes it happens only on symmetrical models with flipped UVs… If you want i can make more tests. Anything you need i can do/test.

nehon · June 27, 2013, 10:14pm

Ok I think I fixed it.
There were some seams appearing on the model when you were really close, and at some place the normals were clearly wrong (compared to blender).

I ended up removing the way we compute parity and used the one from this site Computing Tangent Space Basis Vectors for an Arbitrary Mesh
It seems to work a lot better : no more seams (except from very very close, but those seams are also visible in blender).
And the artifacts you pointed out are gone.

Could you test it please?

mifth · June 28, 2013, 7:29am

@nehon said: Ok I think I fixed it. There were some seams appearing on the model when you were really close, and at some place the normals were clearly wrong (compared to blender).
I ended up removing the way we compute parity and used the one from this site Computing Tangent Space Basis Vectors for an Arbitrary Mesh
It seems to work a lot better : no more seams (except from very very close, but those seams are also visible in blender).
And the artifacts you pointed out are gone.

Could you test it please?

Ok, i did new tests with your new TangentGenerator. Artifacts are still here. The same issue with angle lighting is still here.
Here is my screenshot: http://i.imgur.com/3cqY7qP.png
Here is the project i tested with included code and models: https://dl.dropboxusercontent.com/u/26887202/JME/NormalMapTest_2.zip

If you need some more tests… i always ready.

nehon · June 28, 2013, 7:51am

Are you sure you regenerated the tangents with the last svn?
You need to regenerate them in code

mifth · June 28, 2013, 10:09am

@nehon said: Are you sure you regenerated the tangents with the last svn? You need to regenerate them in code

I downloaded the last commit of TangentGenerator and put it to my sources in the NormalMapTest_2.zip.
The class is called MaTangentBinormalGenerator.java.
Just run TestLight.java in from here https://dl.dropboxusercontent.com/u/26887202/JME/NormalMapTest_2.zip and you will see it.

Or i did something wrong?

Empire_Phoenix · June 28, 2013, 11:29am

Erm why you dont use the normal one? it is already there in the new version fi you use the svn version.

mifth · June 28, 2013, 2:12pm

@nehon said: Are you sure you regenerated the tangents with the last svn? You need to regenerate them in code

Ok, i played with your changes of TangentGenerator a bit and found very interesting thing. If i change lines 516, 517 in such a way i’ll get almost correct result:
[java]
// original lines
// tangent.addLocal(triangleData.tangent);
// binormal.addLocal(triangleData.binormal);

// my changes
if (givenNormal.dot(triangleData.normal) < 0) {
tangent.addLocal(triangleData.tangent);
binormal.addLocal(triangleData.binormal);
}
[/java]

Check out my screenshot with my changes: http://i.imgur.com/QKOVEga.png

The only thing i cannot find to fix is strange lighting when model is lit by side lighting. Do you have any idea?
So if we solve the issue with lighting for side lighting… it will be cool!
Jus another screenshot of the issue with my modifications: http://i.imgur.com/3K6fYth.png

mifth · June 28, 2013, 3:45pm

@nehon , @Momoko_fan i tweaked the TangentGenerator again. And it seems I almost removed seams with such tweaks:
To remove lines 516, 517 from the TangentGenerator.java:
[java]
if (givenNormal.dot(triangleData.normal) < 0) {
tangent.addLocal(triangleData.tangent);
binormal.addLocal(triangleData.binormal);
}

                if (givenNormal.dot(triangleData.normal) > 0) {
                 tangent.addLocal(triangleData.tangent.add(triangleData.tangent.mult(0.5f)));
                binormal.addLocal(triangleData.binormal);   
                }

[/java]

Here is the screenshot of the result with my changes: http://i.imgur.com/5FGoIhQ.png

nehon · June 28, 2013, 7:35pm

uh dude. On what math ground did you base this?
Maybe it fixes the issue for this model but that’s very unlikely that it works for any model.
The generator is very sensitive and based on almost magic math.

I’m gonna test your test case. I can’t see your artifacts in mine.

mifth · June 29, 2013, 11:10am

@nehon , @Momoko_fan i did many tests again and i have come to conclusion.

This way fixes mirrored UVs with normalMaps:
[java]
// lines 516, 517 of TangentBinormalGenerator.java
if (givenNormal.dot(triangleData.normal) < 0) {
tangent.addLocal(triangleData.tangent);
binormal.addLocal(triangleData.binormal);
}
[/java]

Here is the screenshot of the original TangentGenerator: http://i.imgur.com/avIbOVy.png
And here is the screenshot of my fix: http://i.imgur.com/5NixOM9.png

Can you review my fix and add it to trunk?

Thanks.

nehon · June 29, 2013, 1:05pm

I tested it yesterday night and it still makes some artifacts depending on the light angle

mifth · June 29, 2013, 1:17pm

@nehon said: I tested it yesterday night and it still makes some artifacts depending on the light angle

Can you give me your light direction coordinates?