Problems with scaling in transformation hierarchy?

Alright, I think I found a quite severe issue this time… And this time I did check CVS first. :wink: Also, I couldn't find any topics on this on the forum.



Here's the problem: When I set up a transformation hierarchy using the following code (I used the indentation to make the transformation hierarchy clear here):



      Node rootNode = new Node("rootNode");
      rootNode.setLocalScale(10);
         Node transform1 = new Node("transform1");
         rootNode.attachChild(transform1);
         transform1.setLocalTranslation(0, 0, -0.11f);
//         transform1.setLocalScale(new Vector3f(1, 0.9f, 1)); // This is the critical line!
            Node transform2 = new Node("transform2");
            transform1.attachChild(transform2);
            transform2.setLocalScale(0.01f);
               Node transform3 = new Node("transform3");
               transform2.attachChild(transform3);
                  Node b1Node = new Node("b1Node");
                  transform3.attachChild(b1Node);
                  b1Node.setLocalTranslation(0, 100, -11);
                  b1Node.setLocalRotation(new Quaternion(0.7072f, 0, 0, 0.7070f));
                     Box b1 = new Box("b1", new Vector3f(-50, 0, -100), new Vector3f(50, 22, 100));
                     b1Node.attachChild(b1);
                  Node b2Node = new Node("b2Node");
                  transform3.attachChild(b2Node);
                  b2Node.setLocalTranslation(0, 97.5f, 11);
                  b2Node.setLocalRotation(new Quaternion(0.7072f, 0, 0, 0.7070f));
                     Box b2 = new Box("b2", new Vector3f(-45, 0, -97.5f), new Vector3f(45, 3, 97.5f));
                     b2Node.attachChild(b2);



...then I get the following result:



The two boxes are perfectly aligned, as you can see in the wireframe picture, if you look closely:



You'd excpect that if you uncomment the line that is commented out in the code above, the scene will simply shrink along the Y axis, right? Wrong! The result is this:



There is a gap between the two boxes! It seems there is a bug in the transformation hierachy when using subsequent translations, rotations and scalings.

I already did some reseach on this. In the method doTransforms(Spatial) of LWJGLRenderer, the separate world transformations for translation, rotation and scaling are applied in this order. The world transforms are calculated before this, but it seems that somehow the interaction between the three transformations, especially between rotation and scaling, is not taken into account correctly.


Just some wild guesses on possible solutions for this (didn't dig through every detail of jME's render cycle yet ;) ):

One solution could be the use of transformation matrices instead of separate vectors/quaternions for the three transforms (not very likely to happen, I'd say ^^ ).
Another way could be the use of OpenGL's matrix stack. At the moment, the matrix stack is pushed and popped only when rendering geometry batches (and, of course, for a few other functions such as view transforms). The stack is pushed and the three transforms are applied to the GL matrix, then the geometry is rendered, and afterwards the stack is popped again. What if the stack was also used when rendering other scene elements, that is, Spatials? When rendering a Spatial, the matrix stack would be pushed and the (local!!!) transforms applied, then the contents of the Spatial (including all children of Nodes) would be rendered, and afterwards, the matrix stack would be popped again.

I don't know if these considerations have been made before...


Can anyone please confirm if this is really a problem on the side of jME? And if not, I'd really like to know what I'm doing wrong here... ;)

Okay, I've tested that second idea (setting and resetting the transforms for each Spatial in the hierarchy separately), and it worked. Well, almost…



I added these two methods to com.jme.renderer.Renderer:



    /**
    * Sets the local transforms for the given spatial. This method should be
    * called prior to calling <code>draw(Renderer)</code> on a
    * <code>SceneElement</code> in the <code>onDraw(Renderer)</code> method of
    * <code>Spatial</code>.
    *
    * @param s
    *            The Spatial to set the transforms for
    */
    public abstract void pushTransforms(Spatial s);
   
    /**
    * Resets the transforms that have been set using
    * <code>pushTransforms(Spatial)</code>. This method should be called
    * after the calls to <code>Renderer.pushTransforms(Spatial)</code> and
    * <code>SceneElement.draw(Renderer)</code>.
    *
    * @param s
    *            The Spatial whose transforms have to be reset
    */
    public abstract void popTransforms(Spatial s);



...and implemented them in com.jme.renderer.lwjgl.LWJGLRenderer like this:


   @Override
   public void pushTransforms(Spatial s) {
        RendererRecord matRecord = (RendererRecord) DisplaySystem.getDisplaySystem().getCurrentContext().getRendererRecord();
        matRecord.switchMode(GL11.GL_MODELVIEW);
        GL11.glPushMatrix();
       
        Vector3f translation = s.getLocalTranslation();
        if (!translation.equals(Vector3f.ZERO))
            GL11.glTranslatef(translation.x, translation.y, translation.z);

        Quaternion rotation = s.getLocalRotation();
        if (!rotation.isIdentity()) {
            float rot = rotation.toAngleAxis(vRot) * FastMath.RAD_TO_DEG;
            GL11.glRotatef(rot, vRot.x, vRot.y, vRot.z);
        }
       
        Vector3f scale = s.getLocalScale();
        if (!scale.equals(Vector3f.UNIT_XYZ)) {
            GL11.glScalef(scale.x, scale.y, scale.z);
        }
   }

   @Override
   public void popTransforms(Spatial s) {
        RendererRecord matRecord = (RendererRecord) DisplaySystem.getDisplaySystem().getCurrentContext().getRendererRecord();
        matRecord.switchMode(GL11.GL_MODELVIEW);
      GL11.glPopMatrix();
   }



In LWJGLRenderer, I also commented out the contents of the methods doTransforms(Spatial) and undoTransforms(Spatial), so the transformations aren't applied doubly.

In Spatial, I changed the method onDraw(Renderer) as follows:


    public void onDraw(Renderer r) {
        int cm = getCullMode();
        if (cm == SceneElement.CULL_ALWAYS) {
            setLastFrustumIntersection(Camera.OUTSIDE_FRUSTUM);
            return;
        } else if (cm == SceneElement.CULL_NEVER) {
            setLastFrustumIntersection(Camera.INSIDE_FRUSTUM);
            r.pushTransforms(this);
            draw(r);
            r.popTransforms(this);
            return;
        }

        Camera camera = r.getCamera();
        int state = camera.getPlaneState();

        // check to see if we can cull this node
        frustrumIntersects = (parent != null ? parent.frustrumIntersects
                : Camera.INTERSECTS_FRUSTUM);


        if (cm == SceneElement.CULL_DYNAMIC && frustrumIntersects == Camera.INTERSECTS_FRUSTUM) {
            frustrumIntersects = camera.contains(worldBound);
        }

        if (frustrumIntersects != Camera.OUTSIDE_FRUSTUM) {
            r.pushTransforms(this);
            draw(r);
            r.popTransforms(this);
        }
        camera.setPlaneState(state);
    }




The results when I tried the scene again after these changes were okay, at least as far as the transformations are concerned. ;) The boxes were now still aligned and properly scaled along the Y axis. However, the lighting was wrong somehow. Here's a picture of the result:



Actually, the lighting should have looked like it did in the other example pics above. I'm also not sure if this "fix" works under all circumstances, as there are quite a few other subclasses of SceneElement which I did not check yet. I'll check on the lighting bug as soon as I find some more time...

The problem is that you are non-uniformly scaling a rotation.

Yep, but I have to scale the object directionally after a rotation.

In a similar scene for an application I'm working on, I use a model from an X3D file that is constructed like the one shown above. In this model, geometric primitves are used in order to save space. But as the primitives always have a certain orientation when they are created, I have to rotate them so they're oriented as I need them. The model that is constructed of these primitives and other objects has to be scaled along the axes separately in the application, and this is where the way in which transformations are currently being applied in jME does not work.



I've worked with other scene graphs / engines before, such as Java3D, but none of these had any problems with transformation hierarchies like the one I need here. I really didn't expect a bug of this dimension in jME. Are you sure this problem can't be fixed somehow?

I guess I'm just not convinced yet that it is indeed a bug (although, I do admit that it could be since non-uniform scaling is less prevalent.)  Can you recreate the two scenes above (commented vs. uncommented) in a model format like ASE or OBJ and verify the expected behavior in a 3d modeler tool?

I originally created the scene (a door) in 3DS max. When I exported to VRML format, converted it to X3D using the tools from the Web3D page and imported it in jME, it showed the display error. I also tried the ASE format, but the ASE loader seems to have a problem with ASE files exported from 3DS max. However, I’d consider this a bug on the side of max, as some other exports from it are also a bit… quirky. :wink:

When using the OBJ format for the models, there shouldn’t be any problem, as OBJ doesn’t store any transformations like X3D and ASE. It stores only the (already transformed) vertices, so scaling will certainly work properly.



However, I edited the door model file a bit so it already has the non-uniform scaling applied to its root node.

Here is a screenshot of the door imported in jME without the scaling (left) and one with a scaling of 0.8 along the Y axis (right):







I imported the same scene (with the Y scaling) in 3DS max, and the result was this:







As you can see, the scaling works in max, without the gap between the door and the frame.



I just sent you the model used to test the scaling in max and jME, so you can test it for yourself.

The jme and the max images don't look scaled along the same axis? To me it looks like the right scaled door is a picture of elements scaled along the horizontal axis and that each object are scaled localy around its own center. That would explain the gaps. :slight_smile:

Yeah, I had to navigate the scene before taking the screenshots, in order to display it from the same point and view angle. But of course, I wasn't able to set up exactly the same view perspective, especially in different applications. :wink:



The scenes are really scaled along the Y axis (if you want, I can mail you the model files too). But the way you describe the scaling, that each object is scaled from its own center, seems in fact to be the way it is done internally in jME. The three world transformations (translation, rotation, scale) are stored separately and also applied sepearately to the geometry batches when they are rendered. But rotation should actually change the direction in which an object is scaled, and this doesn't seem to happen. In the model file, the door and its frame are both rotated separately around the X axis by 90 degrees, and thus, the scaling is applied in the wrong direction.



If a 4x4 matrix would be used to store and accumulate the world transforms, the transformation hierarchy should work correctly. This is also the way in which transformations are computed in OpenGL. I'll see if I can find some time to test the use of matrices for the world transforms this weekend…

I think the problem is that in the lwjglrenderer's doTransforms method we apply rotation, then scale.  From some reading this morning, it appears this is fine for uniform scales, (order does not matter).  For non-uniform though, the scale needs to be before rotation. 



Swapping these locally indeed fixes your initial code demo, just not convinced yet if that change is not going to break other things… :slight_smile:

I'd also like to know if this fixes more complex transformation hierarchies too, where there are more rotations and scalings following one another than in my example…

SSSTRIKE! I just tested the use of matrices for the transformations, and at least the rendering works perfectly now, including lighting. The only thing that doesn’t work yet is the bounding volume hierarchy, but I’ll look into that.



First, a picture of the door rendered after my changes (Scaled by 0.8 along Y):







And here’s what I did. The changes made in com.jme.scene.spatial.Spatial:



First, I added some transformation matrices to Spatial:


    /** Spatial's combined local transformations */
    protected Matrix4f localTransform;
   
    /** Spatial's combined world transformations */
    protected Matrix4f worldTransform;
   
    /** Spatial's combined inverted world transformations */
    protected Matrix4f invWorldTransform;

    private final Matrix4f compMatrix = new Matrix4f();



These matrices are initialized in the Spatial's default contructor:


        localTransform = new Matrix4f();
        worldTransform = new Matrix4f();
        invWorldTransform = new Matrix4f();



Then I added the following method to Spatial, which updates the matrices according to the transformations of the Spatial and its parent:


    protected void updateWorldTransform() {
       localTransform.loadIdentity();
       localTransform.setTranslation(localTranslation);
       localRotation.toRotationMatrix(localTransform);
       compMatrix.setScaleFull(localScale);
       localTransform.multLocal(compMatrix);
       if (parent != null) {
           parent.worldTransform.mult(localTransform, worldTransform);
       } else {
           worldTransform.set(localTransform);
       }
       worldTransform.invert(invWorldTransform);
    }



This method is called in updateWorldVectors():


    public void updateWorldVectors() {
        if (((lockedMode & SceneElement.LOCKED_TRANSFORMS) == 0)) {
            updateWorldTransform();
            updateWorldScale();
            updateWorldRotation();
            updateWorldTranslation();
        }
    }



The methods updateWorldTranslation(), updateWorldRotation() and updateWorldScale() have also been changed so the translation vector, rotation quaternion and scale vector get updated from the worldTransform matrix:


    protected void updateWorldTranslation() {
       worldTransform.toTranslationVector(worldTranslation);
    }

    protected void updateWorldRotation() {
       worldTransform.toRotationQuat(worldRotation); // does not work properly
    }

    protected void updateWorldScale() {
       worldTransform.toScaleVector(worldScale); // does not work properly
    }



And finally, the methods localToWorld(...) and worldToLocal(...) also had to be changed in order to use the matrices:


    public Vector3f localToWorld( final Vector3f in, Vector3f store ) {
        if ( store == null ) store = new Vector3f();
        worldTransform.transform(in, store);
        return store;
    }

    public Vector3f worldToLocal(final Vector3f in, final Vector3f store) {
       invWorldTransform.transform(in, store);
        return store;
    }




These changes in Spatial also required some additional methods in com.jme.math.Matrix4f:


    /**
    * Sets scale/rotation part of this matrix to be a scale matrix with the
    * given scale values. The translation component is left untouched
    *
    * @param scale
    *            The scale to set this matrix to
    */
    public void setScalePart(Vector3f scale) {
       m00 = scale.x;
       m01 = 0;
       m02 = 0;
       m10 = 0;
       m11 = scale.y;
       m12 = 0;
       m20 = 0;
       m21 = 0;
       m22 = scale.z;
       m30 = 0;
       m31 = 0;
       m32 = 0;
       m33 = 1;
    }
   
    /**
    * Sets this matrix to be a scale matrix with the given scale values. The
    * translation component is reset.
    *
    * @param scale
    *            The scale to set this matrix to
    */
    public void setScaleFull(Vector3f scale) {
       loadIdentity();
       m00 = scale.x;
       m11 = scale.y;
       m22 = scale.z;
    }
   
    /**
    * Stores the scaling component of this matrix in the given Vector3f
    *
    * @param vector
    *            The Vector3f to store the scaling
    */
    public void toScaleVector(Vector3f vector) {
       vector.set(m00, m11, m22);
    }
   
    /**
    * Returns a vector containing the scaling component of this matrix
    *
    * @return A Vector3f with the scaling
    */
    public Vector3f toScaleVector() {
       return new Vector3f(m00, m11, m22);
    }
   
    /**
    * Transforms the given Vector3f with this matrix
    *
    * @param vector
    *            The Vector3f to transform
    */
    public void transform(Vector3f vector){
        float x;
        float y;
        x = m00*vector.x + m01*vector.y + m02*vector.z + m03;
        y = m10*vector.x + m11*vector.y + m12*vector.z + m13;
        vector.z =  m20*vector.x + m21*vector.y + m22*vector.z + m23;
        vector.x = x;
        vector.y = y;
    }
   
    /**
    * Transforms the given Vector3f with this matrix and stores the result in
    * the second vector
    *
    * @param vector
    *            The Vector3f to transform
    * @param store
    *            The Vector3f to store the transformed vector
    */
    public void transform(Vector3f vector, Vector3f store){
        float x;
        float y;
        x = m00*vector.x + m01*vector.y + m02*vector.z + m03;
        y = m10*vector.x + m11*vector.y + m12*vector.z + m13;
        store.z =  m20*vector.x + m21*vector.y + m22*vector.z + m23;
        store.x = x;
        store.y = y;
    }



The contents of the transform(...) methods have been taken from the Matrix4f in the Java Vecmath library


Finally, some changes in com.jme.renderer.lwjgl.LWJGLRenderer were necessary so it uses the Spatial's worldTransform matrix.

To use the matrix in org.lwjgl.opengl.GL11, a float buffer is needed:


    private FloatBuffer matrixBuffer;



This buffer is initialized in LWJGLRenderer's constructor:


        matrixBuffer = BufferUtils.createFloatBuffer(16);



In the method doTransforms(Spatial), the matrix is used:


    protected void doTransforms(Spatial t) {
        // set world matrix
        if (!generatingDisplayList || (t.getLocks() & SceneElement.LOCKED_TRANSFORMS) != 0) {
            RendererRecord matRecord = (RendererRecord) DisplaySystem.getDisplaySystem().getCurrentContext().getRendererRecord();
            matRecord.switchMode(GL11.GL_MODELVIEW);
            GL11.glPushMatrix();
            GL11.glMultMatrix(t.getWorldTransform().fillFloatBuffer(matrixBuffer, true));
        }
    }





With these changes, the scaling works fine, and also lighting and viewing. The bounding volumes currently seem to have a size of 0, so the objects get culled as soon as their center leaves the view frustum. But as I said, I'll look into that.


I'm also wondering about the impact of these changes on performance. Most of the operations used are just basic calculating operations, but in the method Matrix4f.toRotationQuat(Quaternion), a new Matrix3f is always created (which can be fixed), and in the method Quaternion.fromRotationMatrix(...) that is called by it, square roots are used. Maybe I'll also check how serious the performance impact is...


EDIT: The contents of the methods Spatial.localToWorld(...) and Spatial.worldToLocal(...) were mixed up. Contents are now correct.

Yeah, looks very close to what Eberly is doing in his most recent engine code (See his Transformation class)  I'm more inclined to replace the whole thing with a port of that class because it also includes a lot of calculation shortcut hints and could be rigged to prevent object creation as well.



I'm thinking though that this is too big of a change to do in 1.X.  hmmm

Mmm I have been thinking of this and I think that changes should do it into the repository.



Because, altough the CVS is usually quite stable, everybody knows they are using a development version. If someone is highly dependant on a specific version will probably be using a specific tag or version.



In the other hand, I this change breaks current code, it's because nobody noticed before and is using scalation wrongly, and the sooner we realize the better. Also, keeping this broken could make people keep writing incorrect code related to this issue, and as this is going to be changed eventually anyway, it think it is better to make the change now (and do not backport). In the end, the current version in the repository is a development version.



This is just an opinion, as usual.

I completely agree with jjmontes.

The purpose of updates is to make the system more robust and bug-free, leaving bugged code as-is will only make more people adjust to it.

I did some research on that bounding volume problem. It seems these two methods in Spatial are causing the trouble:


    protected void updateWorldRotation() {
       worldTransform.toRotationQuat(worldRotation); // does not work
    }

    protected void updateWorldScale() {
       worldTransform.toScaleVector(worldScale); // does not work
    }



From some reading in Eberly's book, I found out that the Rotation and scaling can't be taken from a transformation matrix that easily (chapter 4.2.2. Transformations). The matrix has to be factored in order to get the rotation and scaling separately, which is not trivial and very expensive to compute. And as long as the world rotation and scale are not calculated properly, all classes using them separately (as BoundingVolume, for example) don't work either.

I think I'll check that Transformation class of Eberly's engine. I might also be able to implement this new way of handling transforms for my application for now, but of course, it would be great if it could be implemented in jME soon. ;)

It's not really a bug, but a missing feature. The jME scenegraph supports non-uniform scale only for the leaves (geometries). Node cannot be non-uniformly scaled (with correct results). So nobody should rely on the current (mis-)behavior.



But, iirc, this cannot simply be changed by swapping scale and rotation multiplication. So my vote would be to put this new feature into the 2.0 svn.

Alright, I've been able to fix the problem with the BoundingVolumes (at least for BoundingBoxes, not sure if it works for the other types too), so I might be able to keep using non-uniform scaling for Nodes with this "hacked" version of jME.



But I'm starting to understand renanse's point. The separately stored transforms for translation, rotation and scaling are used in so many places in jME that it would be really hard to use 4x4 matrices for the world transforms consistently. Besides, for some of the functions that are using the separate transforms now, they are actually necessary, so a transformation using the whole matrix (or just its scale/rotation component) will not work.



However, for completely non-negative scaling, the separate rotation and scaling might be able to be extracted from the matrix. Afaik, the three columns of the upper 3x3 fields in the matrix represent the base vectors for the transformed coordinate system. As scaling is always applied after the rotation, the length of the vectors should be the scaling in X, Y and Z direction. The rotation could then be calculated from the three normalized vectors.

(Side note: Please correct me if I'm wrong here! I'm not a "transformation god" after all, so I might very well be wrong…)

But as I said, this will only work as long as a scaling is not negative. Besides, the calculation requires 3 square roots here, so it might seriously impact performance. For simple applications that don't require the separate rotation and scaling, the sqrts may be avoided by setting a "dirty" flag for the two, so they are only extracted from the matrix when they are needed and have not been updated since the last change of the world transform.



For my application, I don't need especially high performance and I also don't use negative scaling, so this fix might possibly work. But I don't think it would be suitable to commit a fix as "dirty" as this one to the CVS. Of course, it would be nice to have a clean implementation of non-uniform scaling in scene graph nodes asap, but if this takes time to implement properly, I'll be fine with that and agree that it should be added in version 2.0 of jME.

I'm having the same problem and I use JME 2.0.



So, I assume this is not addressed yet in JME 2.0.



Right?