Fix for some JVM crashes(incorrect texCoords disabling in LWJGLRenderer)

I've found problem in LWJGLRenderer code (JME2), which cause JME applications to crash JVM on some hardware.



It turned out that LWJGLRenderer does not always disable all unused Textures Coords. Basically it uses following code to initialize them:


        TextureState ts = (TextureState) context.currentStates[RenderState.StateType.Texture.ordinal()];
        for (int i = 0; i < ts.getNumberOfSetTextures()
               && i < TextureState.getNumberOfFragmentTexCoordUnits(); i++) {
            TexCoords texC = g.getTextureCoords(i + offset);
// a lot of code applying texC
         }



and following code is responsible for clearing


            if (ts.getNumberOfSetTextures() < prevTextureNumber) {
                for (int i = ts.getNumberOfSetTextures(); i < prevTextureNumber; i++) {
                    if (capabilities.GL_ARB_multitexture) {
                        ARBMultitexture
                                .glClientActiveTextureARB(ARBMultitexture.GL_TEXTURE0_ARB
                                        + i);
                    }
                    GL11.glDisableClientState(GL11.GL_TEXTURE_COORD_ARRAY);
                }
            }



but prevTextureNumber is calculated incorrectly (at least I think so), it uses getNumberOfFixedUnits instead of getNumberOfFragmentTexCoordUnits (as in initialization).

So, I believe following patch should be applied to LWJGLRenderer

Index: src/com/jme/renderer/lwjgl/LWJGLRenderer.java
===================================================================
--- src/com/jme/renderer/lwjgl/LWJGLRenderer.java   (revision 4742)
+++ src/com/jme/renderer/lwjgl/LWJGLRenderer.java   (working copy)
@@ -1520,8 +1520,8 @@
             }
 
             prevTextureNumber = ts.getNumberOfSetTextures() < TextureState
-                    .getNumberOfFixedUnits() ? ts.getNumberOfSetTextures()
-                    : TextureState.getNumberOfFixedUnits();
+                    .getNumberOfFragmentTexCoordUnits() ? ts.getNumberOfSetTextures()
+                    : TextureState.getNumberOfFragmentTexCoordUnits();
         }
        
         return indicesVBO;



It has fixed JVM crashes on my machine and it seems to be quite logical. So, do you think it should be applied?

There was a bug with NumberOfFixed… topic in 08:

http://www.jmonkeyengine.com/forum/index.php?topic=9194.0



Dont know if this was fixed, too.



And i remember a bigger discussion about this, but i cant find it?



Regards