Normal Map shading problems

EsKophan · January 30, 2016, 2:50pm

Hi everyone!

So, while working with our project Space Shift (where I am lead artist) I faced with not really cool normal map problem.
Our project is about spaceships and other hardsurf headaches…and as you might know hardsurf have to have proper normal map setup, and from here is main issue starts.

Model with same normalmaps, shade differently in blender and JME. I have a strong background in game development and I know that in most cases, problem appears because of different shading algorithms in baking software and in game engine.

So, here is demonstration of problem (I will show a comparison in 3 different engines):

All engines use exact same normal map, with same settings of lightning (except substance, there is HDRI shading). JME normal map has inverted Y (green channel).
Normal map baked in BI (Blender Internal).
Model was triangulated before baking normal map and same triangulated mesh used in all engines.

As I mentioned before, to deal with this issue we are generating normal map specially for engine by using Handplane
This program takes OS normal map and Lowpoly mesh and converts OS to Tangent for selected engine (UDK, Unity, Max, Maya…)
Also, creator of handplane explains my issue in his videos (if anyone interested)

So, this is not pleasant issue, broken geometry really show up in hardsurf.

I dont know how to bake my maps and where. JME seems to have same issue with all normalmaps generated with handplane (Max, Maya, UDK, Unity etc.) and normalmaps from Xnormal have same problem. Could you help me?

EsKophan · January 30, 2016, 2:50pm

If you need, I have prepeared some models for testing

Download them here:
Hirez/Lowrez hardsurf cube as obj and MaterialTestingSphere as .blend

To test materials (I know you are making PBR for jme, and I want to share with you this ball which I made for material testing in JME. It have inside geometry for testing alpha / glass / volume and “lightbulb monkey” have its own material, so it should be easy to test different styles of shading in same model) Hope you like it

in blend file there is 2 scenes:
First - main
Second - baking scene with hirez for cube

pspeed · January 30, 2016, 4:19pm

First question for something like this is always: did you generate tangents?

EsKophan · January 30, 2016, 4:24pm

Yes, I always batch geometry and generate tangents. Otherwise everything works even worse.

david_bernard_31 · January 30, 2016, 4:59pm

Thanks for your sample, it allow me to detect an issue in my pipeline blender → xbuf → jme

first line is MainScene
second line is BackingScene

In your zip you include obj, mtl and tga. So I guess that you import the obj in jme and not the .blend directly ? (I didn’t try those path)

EsKophan · January 30, 2016, 5:57pm

You are welcome Use it where you want.
You can delete useless models (there is 4 more in “baking” scene). I made some extra models in separate layers and scenes just for baking (If there is a problem with normal map it should be no problem to bake new ones).
Also, I put obj just for baking in external baking engine, like xnormal (not for using in JME, but I believe JME should catch obj model. It is also triangulated correctly, as models in .blend file)

I use prepared for jmonkey .bleng file with all modifiers applied and without useless scene information like AutoSmooth etc. So it must not be a problem with model.

javasabr · February 5, 2016, 3:26am

up

winhelp · February 6, 2016, 11:16am

So what is wrong? Its normal-shitty-triangle-shading
Presented model is cube_smooth from your file.

prog · February 6, 2016, 12:00pm

You should split sharp edges as there is no such a thing like having two normals per single vertex in OpenGL. In blender just mark them as sharp and as seams and then at export apply Edge Split modifier. By export I mean making a copy of a blend file where one could safely prepare a model to being loaded by JME and don’t turn future changes to said model into a mess at the same time.

BTW, I’m slowly working on plugin for blender that helps preparing model for being imported by JME, it’s based mostly around baking animations but it looks like now I have another point for checklist - to check if there are sharp edges in model and to offer to split them.

EsKophan · February 6, 2016, 4:44pm

I should have mentioned that I need correct normal map shading without split/smooth groups. Only fully-smoothed geometry + normal map.

Also:

Split groups/Smooth groups can be used in really rare situations, where normalmaps are not necessary.
Real split make vertexcount grow
It require different UV layout, not handy in some cases
Splitgroups is wrong choice for edges more than 90 degrees.

And yes, again, splitgroups in our case - is a terrible way around the real shading problem in JME.

Wow this is interesting! keep going!

EsKophan · February 6, 2016, 4:54pm

Problem in normal map shading:
Model with same normalmaps, shade differently in blender and JME and other engines (see my screenshots from first post, I pointed there on problems)

prog · February 6, 2016, 5:36pm

Ok, you’re right then. And now it’s time for you to make OpenGL render as well believe that you’re right - fix shaders and make them producing result you want.

Perjin · February 7, 2016, 1:35am

@nehon do you know which tangent space is used by the tangent generator?

My guess is, jMonkey doesn’t use Mikkelsen tangent space. In that case the solution would be to implement it. There is a c++ implementation available (i couldn’t find the originial source but the licencing of the files in the blender source doesn’t have any GPL nonsense).
If anyone wants to try this, the source for mikktspace can be found here:
https://developer.blender.org/diffusion/B/browse/master/intern/mikktspace/mikktspace.h
https://developer.blender.org/diffusion/B/browse/master/intern/mikktspace/mikktspace.c

nehon · February 7, 2016, 9:16am

I don’t think it has a name. It’s been done by an early contributor to JME and I wasn’t even working on the project at the time. I made some fixes and changes over time and I am confident that it reached a pretty stable state.
Only thing I can say is that it usually work with model from blender pretty well.

So @EsKophan, what does your UVs look like? How do you generate the tangents?
My guess is that you have mirrored uvs near the dark shaded area in your screen shot and that the tangents are not properly generated.
Just know that if you have mirrored parts in your UV set you have to pass in an extra parameter when generating tangents.
You have to call this method :

github.com

jMonkeyEngine/jmonkeyengine/blob/master/jme3-core/src/main/java/com/jme3/util/TangentBinormalGenerator.java#L134


      
           */
          private TangentBinormalGenerator() {
          }
          
          private static List<VertexData> initVertexData(int size) {
              List<VertexData> vertices = new ArrayList<>(size);
              for (int i = 0; i < size; i++) {
                  vertices.add(new VertexData());
              }
              return vertices;
          }
          
          public static void generate(Mesh mesh) {
              generate(mesh, true, false);
          }
          
          public static void generate(Spatial scene, boolean splitMirrored) {
              if (scene instanceof Node) {
                  Node node = (Node) scene;
                  for (Spatial child : node.getChildren()) {
                      generate(child, splitMirrored);

TangentBinormalGenerator.generate(spatial, true);

Note that this will likely create new vertices for the mirrored uv’s vertices.

EsKophan · February 7, 2016, 10:36am

@nehon nope, in this model I use non mirrored UV.
Also, everything is triangulated to get rid from problems with auto triangulation and artefacts with wrong triangulation.

Even more, problem occurred in non split UV (solid piece of UV) in front of the model.

nehon · February 7, 2016, 10:41am

how did you bake your normals? I mean with what software?
I’ll check your models when I’m home.

EsKophan · February 7, 2016, 11:13am

I did test baking in: Substance, Blender render, Blender Cycles, Xnormal, HandPlane(3ds max, Maya, UDK3, Unity, Source) Normalmaps from first 3 looks pretty same though there is slight difference between them (I checked the difference in raster editor)…but all bakes showed slight variety of same artefacts.

Normal map in archive is from Substance painter, I also included hirez model in “baking scene” and in obj file to make own bakes.

javasabr · February 9, 2016, 1:13pm

up

winhelp · February 12, 2016, 9:33pm

Try this:

and this: Normalmaps for the Technical Game Modeler
edit: object space map

EsKophan · February 13, 2016, 12:25pm

Thanks, but as I mentioned earlier, I don’t need split groups in my mesh, I need correct normal map shading in smoothed model, like in blender (I attached screenshots with same model and normal maps in the head of the topic) Also jmonkey seems not to support OSNormal maps.