[Committed] Correct Normapmapping with N PointLights

hey there,



according to problem: http://www.jmonkeyengine.com/forum/index.php?topic=12260.0



basically whats new is the shader with variable number of CORRECTLY reflecting lights and that the tangents are passed by shader attributes.







patch with shaders and testcase (no new graphics, … needed)  ;):



Index: src/jmetest/data/shaders/normalmap.frag
===================================================================
--- src/jmetest/data/shaders/normalmap.frag   (revision 0)
+++ src/jmetest/data/shaders/normalmap.frag   (revision 0)
@@ -0,0 +1,37 @@
+uniform sampler2D baseMap;
+uniform sampler2D normalMap;
+uniform sampler2D specularMap;
+
+varying vec3 viewDirection;
+varying vec3 lightDirections[$NL$];
+varying vec2 texcoords;
+
+void main(void)
+{
+   /* Extract colors from baseMap and specularMap */
+   vec4  baseColor = texture2D( baseMap, texcoords );
+    vec3  normal = normalize( ( texture2D( normalMap, texcoords ).xyz * 2.0 ) - 1.0 );
+   vec4  specularColor = texture2D( specularMap, texcoords );
+   
+    vec3 normalizedViewDirection = normalize( viewDirection );
+      
+   /* Sum up lighting models with OpenGL provided light/material properties */
+    vec4 totalAmbient = gl_LightModel.ambient * gl_FrontMaterial.ambient;  // init with global ambient
+    vec4 totalDiffuse;
+    vec4 totalSpecular;
+
+    //


LIGHTS
+    for(int i = 0; i < $NL$; i++) {
+        vec3 normalizedLightDirection = normalize( lightDirections[i] );
+        float NDotL = dot( normal, normalizedLightDirection );
+        vec3 reflection = normalize( ( ( 2.0 * normal ) * NDotL ) - normalizedLightDirection );
+          
+        /* Sum up lighting models with OpenGL provided light/material properties */
+        totalAmbient  += gl_FrontLightProduct[i].ambient;
+        totalDiffuse  += gl_FrontLightProduct[i].diffuse * max( 0.0, NDotL );
+        totalSpecular += gl_FrontLightProduct[i].specular * specularColor * ( pow( max( 0.0, dot( reflection, normalizedViewDirection ) ), gl_FrontMaterial.shininess ) );
+    } // for
+
+   /* Set final pixel color as sum of lighting models */
+    gl_FragColor = totalAmbient * baseColor + totalDiffuse * baseColor + totalSpecular;
+}
No newline at end of file

Property changes on: src/jmetest/data/shaders/normalmap.frag
___________________________________________________________________
Added: svn:executable
   + *

Index: src/jmetest/data/shaders/normalmap.vert
===================================================================
--- src/jmetest/data/shaders/normalmap.vert   (revision 0)
+++ src/jmetest/data/shaders/normalmap.vert   (revision 0)
@@ -0,0 +1,32 @@
+attribute vec3 modelTangent;
+
+varying vec3 viewDirection;
+varying vec3 lightDirections[$NL$];
+varying vec2 texcoords;
+
+void main(void)
+{
+    gl_Position = ftransform();
+    texcoords = (gl_TextureMatrix[0] * gl_MultiTexCoord0).xy;
+    
+    /* Get view and light directions in viewspace */
+    vec3 localViewDirection = -(gl_ModelViewMatrix * gl_Vertex).xyz;
+    
+    /* Calculate tangent info - stored in attributes */
+    vec3 normal = gl_NormalMatrix * gl_Normal;
+    vec3 tangent = gl_NormalMatrix * modelTangent;
+    vec3 binormal = cross( normal, tangent );
+    
+    /* Transform localViewDirection into texture space */
+    viewDirection.x = dot( tangent, localViewDirection );
+    viewDirection.y = dot( binormal, localViewDirection );
+    viewDirection.z = dot( normal, localViewDirection );
+    
+   for(int i = 0; i < $NL$; i++) {
+       vec3 localLightDirection = gl_LightSource[i].position.xyz + localViewDirection;
+        lightDirections[i].x = dot( tangent, localLightDirection );
+        lightDirections[i].y = dot( binormal, localLightDirection );
+        lightDirections[i].z = dot( normal, localLightDirection );
+        lightDirections[i] = normalize( lightDirections[i] );
+   } // for
+} // main
No newline at end of file

Property changes on: src/jmetest/data/shaders/normalmap.vert
___________________________________________________________________
Added: svn:executable
   + *

Index: src/jmetest/effects/TestDiffNormSpecmap.java
===================================================================
--- src/jmetest/effects/TestDiffNormSpecmap.java   (revision 0)
+++ src/jmetest/effects/TestDiffNormSpecmap.java   (revision 0)
@@ -0,0 +1,201 @@
+package jmetest.effects;
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStreamReader;
+import java.net.URL;
+import java.nio.FloatBuffer;
+import java.util.logging.Logger;
+
+import jmetest.renderer.loader.TestNormalmap;
+
+import com.jme.app.SimpleGame;
+import com.jme.image.Image;
+import com.jme.image.Texture;
+import com.jme.input.FirstPersonHandler;
+import com.jme.light.PointLight;
+import com.jme.math.FastMath;
+import com.jme.math.Vector3f;
+import com.jme.renderer.ColorRGBA;
+import com.jme.scene.TriMesh;
+import com.jme.scene.Spatial.LightCombineMode;
+import com.jme.scene.shape.Sphere;
+import com.jme.scene.state.GLSLShaderObjectsState;
+import com.jme.scene.state.MaterialState;
+import com.jme.scene.state.TextureState;
+import com.jme.system.DisplaySystem;
+import com.jme.util.TextureManager;
+import com.jme.util.geom.TangentBinormalGenerator;
+
+/**
+ * @author dhdd (Andreas Grabner)
+ */
+public class TestDiffNormSpecmap extends SimpleGame {
+    private static final Logger logger = Logger.getLogger(TestNormalmap.class.getName());
+
+    private Vector3f lightDir = new Vector3f();
+    private GLSLShaderObjectsState so;
+    private String currentShaderStr = "jmetest/data/shaders/normalmap";
+    private Sphere lightSphere1, lightSphere2, lightSphere3;
+
+    private PointLight pl1, pl2, pl3;
+
+    public static void main(String[] args) {
+        TestDiffNormSpecmap app = new TestDiffNormSpecmap();
+        app.setConfigShowMode(ConfigShowMode.AlwaysShow);
+        app.start();
+    }
+
+    protected void simpleUpdate() {
+        float spinValX1 = FastMath.sin(timer.getTimeInSeconds() * 1.0f);
+        float spinValY1 = FastMath.cos(timer.getTimeInSeconds() * 0.4f);
+        float spinValZ1 = FastMath.cos(timer.getTimeInSeconds() * 0.5f);
+        float spinValX2 = FastMath.sin(timer.getTimeInSeconds() * 0.5f);
+        float spinValY2 = FastMath.cos(timer.getTimeInSeconds() * 0.6f);
+        float spinValZ2 = FastMath.cos(timer.getTimeInSeconds() * 0.2f);
+        float spinValX3 = FastMath.sin(timer.getTimeInSeconds() * 0.4f);
+        float spinValY3 = FastMath.cos(timer.getTimeInSeconds() * 1.0f);
+        float spinValZ3 = FastMath.cos(timer.getTimeInSeconds() * 0.65f);
+
+        lightDir.set(spinValX1, spinValY1, spinValZ1).normalizeLocal();
+        lightSphere1.setLocalTranslation(lightDir.negate().multLocal(30));
+        pl1.setLocation(lightDir.negate().multLocal(30));
+
+        lightDir.set(spinValX2, spinValY2, spinValZ2).normalizeLocal();
+        lightSphere2.setLocalTranslation(lightDir.negate().multLocal(30));
+        pl2.setLocation(lightDir.negate().multLocal(30));
+
+        lightDir.set(spinValX3, spinValY3, spinValZ3).normalizeLocal();
+        lightSphere3.setLocalTranslation(lightDir.negate().multLocal(30));
+        pl3.setLocation(lightDir.negate().multLocal(30));
+    }
+
+    protected void simpleInitGame() {
+
+        cam.setAxes(new Vector3f(-1, 0, 0), new Vector3f(0, 0, 1), new Vector3f(0, 1, 0));
+        cam.setLocation(new Vector3f(0, -100, 0));
+
+        pl1 = new PointLight();
+        pl1.setAmbient(new ColorRGBA(0, 0, 0, 1));
+        pl1.setDiffuse(new ColorRGBA(0.4f, 0, 0, 1));
+        pl1.setSpecular(new ColorRGBA(1, 0, 0, 1));
+        pl1.setEnabled(true);
+
+        pl2 = new PointLight();
+        pl2.setAmbient(new ColorRGBA(0, 0, 0, 1));
+        pl2.setDiffuse(new ColorRGBA(0, 0.4f, 0, 1));
+        pl2.setSpecular(new ColorRGBA(0, 1, 0, 1));
+        pl2.setEnabled(true);
+
+        pl3 = new PointLight();
+        pl3.setAmbient(new ColorRGBA(0, 0, 0, 1));
+        pl3.setDiffuse(new ColorRGBA(0, 0, 0.4f, 1));
+        pl3.setSpecular(new ColorRGBA(0, 0, 1, 1));
+        pl3.setEnabled(true);
+
+        lightState.detachAll();
+        lightState.setGlobalAmbient(new ColorRGBA(0.2f, 0.2f, 0.2f, 1f));
+        lightState.setTwoSidedLighting(false);
+        lightState.setSeparateSpecular(true);
+        lightState.attach(pl1);
+        lightState.attach(pl2);
+        lightState.attach(pl3);
+
+        TextureState ts = display.getRenderer().createTextureState();
+
+        // Base texture
+        Texture baseMap = TextureManager.loadTexture(TestNormalmap.class.getClassLoader().getResource(
+                "jmetest/data/images/Fieldstone.jpg"), Texture.MinificationFilter.Trilinear,
+                Texture.MagnificationFilter.Bilinear);
+        ts.setTexture(baseMap, 0);
+
+        // Normal map
+        Texture normalMap = TextureManager.loadTexture(TestNormalmap.class.getClassLoader().getResource(
+                "jmetest/data/images/FieldstoneNormal.jpg"), Texture.MinificationFilter.Trilinear,
+                Texture.MagnificationFilter.Bilinear, Image.Format.GuessNoCompression, 0.0f, true);
+        ts.setTexture(normalMap, 1);
+
+        // Specular map
+        Texture specMap = TextureManager.loadTexture(TestNormalmap.class.getClassLoader().getResource(
+                "jmetest/data/images/FieldstoneSpec.jpg"), Texture.MinificationFilter.Trilinear,
+                Texture.MagnificationFilter.Bilinear);
+        ts.setTexture(specMap, 2);
+
+        Sphere model = new Sphere("sphere", new Vector3f(0, 0, 0), 50, 50, 20.0f, false);
+
+        lightSphere1 = new Sphere("light1", new Vector3f(0, 0, 0), 10, 10, 1.0f, false);
+        lightSphere1.setLightCombineMode(LightCombineMode.Off);
+        lightSphere1.setDefaultColor(ColorRGBA.red.clone());
+        lightSphere2 = new Sphere("light2", new Vector3f(0, 0, 0), 10, 10, 1.0f, false);
+        lightSphere2.setLightCombineMode(LightCombineMode.Off);
+        lightSphere2.setDefaultColor(ColorRGBA.green.clone());
+        lightSphere3 = new Sphere("light3", new Vector3f(0, 0, 0), 10, 10, 1.0f, false);
+        lightSphere3.setLightCombineMode(LightCombineMode.Off);
+        lightSphere3.setDefaultColor(ColorRGBA.blue.clone());
+        createShader(lightState.getQuantity(), model);
+
+        // Test materialstate (should be set through the import anyway)
+        MaterialState ms = display.getRenderer().createMaterialState();
+        ms.setAmbient(new ColorRGBA(1.0f, 1.0f, 1.0f, 1.0f));
+        ms.setDiffuse(new ColorRGBA(1.0f, 1.0f, 1.0f, 1.0f));
+        ms.setSpecular(new ColorRGBA(1.0f, 1.0f, 1.0f, 1.0f));
+        ms.setShininess(50.0f);
+
+        // Set all states on model
+        model.setRenderState(ts);
+        model.setRenderState(so);
+        model.setRenderState(ms);
+        model.setRenderState(lightState);
+        model.updateGeometricState(0.0f, true);
+        model.updateRenderState();
+
+        rootNode.attachChild(model);
+        rootNode.attachChild(lightSphere1);
+        rootNode.attachChild(lightSphere2);
+        rootNode.attachChild(lightSphere3);
+        
+
+        rootNode.updateGeometricState(0, true);
+        rootNode.updateRenderState();
+
+        input = new FirstPersonHandler(cam, 80, 1);
+    }
+
+    /**
+     * Loads shader from URL
+     *
+     * @param url
+     * @return String with shader
+     */
+    private String load(URL url) {
+        BufferedReader r = null;
+        try {
+            r = new BufferedReader(new InputStreamReader(url.openStream()));
+            StringBuffer buf = new StringBuffer();
+            while (r.ready()) {
+                buf.append(r.readLine()).append('n');
+            }
+            r.close();
+            return buf.toString();
+        } catch (IOException e) {
+            e.printStackTrace();
+            return null;
+        }
+    }
+
+    private void createShader(int numLights, TriMesh geometry) {
+        so = DisplaySystem.getDisplaySystem().getRenderer().createGLSLShaderObjectsState();
+        String vert = load(TestDiffNormSpecmap.class.getClassLoader().getResource(currentShaderStr + ".vert"));
+        String frag = load(TestDiffNormSpecmap.class.getClassLoader().getResource(currentShaderStr + ".frag"));
+        vert = vert.replace("$NL$", "" + numLights);
+        frag = frag.replace("$NL$", "" + numLights);
+        TangentBinormalGenerator.generate(geometry);
+        so.load(vert, frag);
+        so.setUniform("baseMap", 0);
+        so.setUniform("normalMap", 1);
+        so.setUniform("specularMap", 2);
+        FloatBuffer tangent = geometry.getTangentBuffer();
+        // the binormal is computed in the shader from tangent and normal
+        so.setAttributePointer("modelTangent", 3, false, 0, tangent);
+    }
+}

Just letting you know, this shader will requires Pixel Shader 3+ hardware. The only way to get N lights in a shader with PS2 is using multi-pass lighting.

Momoko_Fan said:

Just letting you know, this shader will requires Pixel Shader 3+ hardware. The only way to get N lights in a shader with PS2 is using multi-pass lighting.


Whats causing that? Is that the $NL$ parameter? Otherwise I see no extra changes from the original. $NL$ can be replaced with a user uniform parameter too.
timong said:

Whats causing that? Is that the $NL$ parameter? Otherwise I see no extra changes from the original. $NL$ can be replaced with a user uniform parameter too.


No, its not tne NL parameter, you can make that hardcoded too. Its also possible with uniforms but just dont do that, since uniforms are variable at runtime, which means that the GLSL compiler has to keep the for loop in tact wich is very slow. however if you do it this way (exchange a String with the number of lights before passing it to the graphicscard) the compiler unrolls the for loop and optimizes it, which will be much more performant  ;)
dhdd said:

timong said:

Whats causing that? Is that the $NL$ parameter? Otherwise I see no extra changes from the original. $NL$ can be replaced with a user uniform parameter too.


No, its not tne NL parameter, you can make that hardcoded too. Its also possible with uniforms but just dont do that, since uniforms are variable at runtime, which means that the GLSL compiler has to keep the for loop in tact wich is very slow. however if you do it this way (exchange a String with the number of lights before passing it to the graphicscard) the compiler unrolls the for loop and optimizes it, which will be much more performant  ;)

Thanks for the info!
Again, then why is it not compatible with SM2? I'm interested because just right now I'm tweaking normal / parallax map for jcrpg, but didnt check the jme update yet.

$NL$ is not causing any trouble… using a user parameter will actually cause more issues.

There are two reasons it's incompatible with PS2. First, it breaks the instruction limit (which I believe is 64) for 3 or more lights, and second, it uses too many varyings (the light direction array takes the most space).

The only way to be compatible with PS2, is by using multipass lighting. E.g rendering the model once for each light, and then blend the results.

well, okay. thanks! :slight_smile: btw, $NL$ is causing trouble on intel linux vga driver.  :frowning: but i'll try to find another solution for jcrpg. :slight_smile:

timong said:

btw, $NL$ is causing trouble on intel linux vga driver.


impossible, since the $NL$ string is never ever possibly passed to your graphics card, because you have to replace it before that:


    private void createShader(int numLights, TriMesh geometry) {
        so = DisplaySystem.getDisplaySystem().getRenderer().createGLSLShaderObjectsState();
        String vert = load(TestDiffNormSpecmap.class.getClassLoader().getResource(currentShaderStr + ".vert"));
        String frag = load(TestDiffNormSpecmap.class.getClassLoader().getResource(currentShaderStr + ".frag"));
        vert = vert.replace("$NL$", "" + numLights);
        frag = frag.replace("$NL$", "" + numLights);
        TangentBinormalGenerator.generate(geometry);
        so.load(vert, frag);
        so.setUniform("baseMap", 0);
        so.setUniform("normalMap", 1);
        so.setUniform("specularMap", 2);
        FloatBuffer tangent = geometry.getTangentBuffer();
        // the binormal is computed in the shader from tangent and normal
        so.setAttributePointer("modelTangent", 3, false, 0, tangent);
    }

Yep, i've got that since that, sorry, for not adding that. BTW, i tweaked it for my needs too, to let it support attenuation and fog, but i guess fog should be in a different shader. Attenuation might be interesting and importan though.

timong said:

BTW, i tweaked it for my needs too, to let it support attenuation


sounds good, please post a patch for the shaders and testcase. I think we can add it if the performance impact isn't a problem.