BlendShapes

[nehon]

My guess is that the offset position is intended to be applied to the original vertex position and not stacked over frames. the Pose.apply() stack the offset value.
You have 2 ways to fix this

• combine the bind pose of the vertex with the offset before applying it in the apply method(you’d need to read from another buffer)
• combine the bind pose at loading (basically subtracting the bind pose to the offset) at loading time. so you don’t have to change the apply method

I like the second solution better as it avoid extra reading of the buffers at run time. (by run time i mean when the animation is playing).

[Kidow]

@nehon said: My guess is that the offset position is intended to be applied to the original vertex position and not stacked over frames. the Pose.apply() stack the offset value. You have 2 ways to fix this - combine the bind pose of the vertex with the offset before applying it in the apply method(you'd need to read from another buffer) - combine the bind pose at loading (basically subtracting the bind pose to the offset) at loading time. so you don't have to change the apply method

I like the second solution better as it avoid extra reading of the buffers at run time. (by run time i mean when the animation is playing).

Thanks nehon,

I’m not sure if I understood right and did the right thing. I collected all vertexpositions of the original mesh and subtracted it from the poseoffset. Basically like this
[java]
float new_x = original.get(index).x - poseOffset.get(index).x;
[/java]

It’s still not working. The vertices that should to be moving, are moving, but not the way they are supposed to be. Could it be the weights?

[nehon]

@Kidow said: It's still not working. The vertices that should to be moving, are moving, but not the way they are supposed to be. Could it be the weights?

Maybe i was wrong. Another possibility would be that the offset is applied several times by frame and shouldn't.

would it be possible that you post a complete patch your test case and your test data?
It would be easier to find the issue if I could debug it.
Thanks

[Kidow]

Hey there,

I think I actually found out whats not working right. But I don’t know how to fix this or where that’s coming from.

It’s the apply()-method in the Pose class
[java]
public void apply(float blend, FloatBuffer vertbuf){
for (int i = 0; i < indices.length; i++){
Vector3f offset = offsets[i];
int vertIndex = indices[i];
tempVec.set(offset).multLocal(blend);

        // acquire vertex
        BufferUtils.populateFromBuffer(tempVec2, vertbuf, vertIndex);
        System.out.println("TempVec2 : " + tempVec2.x +" / " + tempVec2.y +" / " + tempVec2.z);
        System.out.println("TempVec1 : " + tempVec.x +" / " + tempVec.y +" / " + tempVec.z);

        // add offset multiplied by factor
        tempVec2.addLocal(tempVec);
        System.out.println("After TempVec2 : " + tempVec2.x +" / " + tempVec2.y +" / " + tempVec2.z);

        // write modified vertex
        BufferUtils.setInBuffer(tempVec2, vertbuf, vertIndex);
    }
}

[/java]

The output I get is something like this:

TempVec2 : -235.37169 / 223.10895 / -1.0
TempVec1 : 0.0 / 0.09547975 / -0.0
After TempVec2 : -235.37169 / 223.20442 / -1.0

That's a bit too high. How can I fix this?

[nehon]

I think i know what’s going on, the animation is looped and the offset is applied to the vertices over and over. The bind pose should be reset when you reach the end of the animation

[Kidow]

Bindpose means the initial vertex position right? So if I’m resetting the bindpose at the end of an animation, the animation will start and end with the same vertex positions?

I took a closer look at what happens between two keyframes. Right now my animation is about 1sec long and has according to the OGRExml 32 keyframes. The setTime method in poseTrack checks between which keyframes should be interpolated at a given time. I tried to stop my animation after processing a few keyframes. And there is already the mistake.
The positions for the verticies are too high. So it’s not only happening at the animations end but also inbetween.

I tried something, I thought in the apply method of pose I had to add the difference between the previous vertexposition and the current one, instead of just adding the current position to the previous vertexposition.
That’s how it looks in code.
The applyFrame() in PoseTrack
[java]
private void applyFrame(Mesh target, int frameIndex, float weight) {
PoseFrame frame = frames[frameIndex];
VertexBuffer pb = target.getBuffer(Type.Position);
for (int i = 0; i 0){
pose.apply(poseWeight, (FloatBuffer) pb.getData(), true);
}else{
pose.apply(poseWeight, (FloatBuffer) pb.getData(), false);
}
}
[/java]

and the apply-method in Pose
[java]
public void apply(float blend, FloatBuffer vertbuf, boolean subtract){
for (int i = 0; i < indices.length; i++){
Vector3f offset = offsets[i];
int vertIndex = indices[i];
// acquire vertex
BufferUtils.populateFromBuffer(tempVec2, vertbuf, vertIndex);
tempVec.set(offset).multLocal(blend);

        if (subtract){
        	Vector3f diff = tempVec.subtract(tempVec2);
        	tempVec2.addLocal(diff);
        }else{
        	// add offset multiplied by factor
        	tempVec2.addLocal(tempVec);
        }
        System.out.println("After TempVec2 : " + tempVec2.x +" / " + tempVec2.y +" / " + tempVec2.z);
        // write modified vertex
        BufferUtils.setInBuffer(tempVec2, vertbuf, vertIndex);
    }
}

[/java]

Unfortunately it's still not working, do you have any suggestions?

[nehon]

I’ll test tonight, I still have your project code

[Kidow]

Thank you! :amused:

[nehon]

Ok so I got it to work somehow.

As you said the bind pose is the mesh data in its original form. This data is stored in another buffer. 3 Buffers actually : BindPosePosition, BondPoseNormal, BindPoseTangent.
Those bind pose buffers are generated when loading the mesh, in the mesh.generateBindPose() method. Right now the bind pose is generated only if the BoneIndex buffer exists (Easy check if the mesh is animated because until now we only had bone animations).
I removed this check so that the bind pose is always generated (note that this is wrong, the check will have to be smarter, we don’t need bind pose if the mesh is not animated).

Then I pass the BindPosePosition buffer to the Pose.apply method, along with the Position buffer. And instead of fetching the position from the position buffer i fetch it from the bind pose buffer.
Like this
[java]
public void apply(float blend, FloatBuffer vertbuf, FloatBuffer bindPose){
for (int i = 0; i < indices.length; i++){
Vector3f offset = offsets[i];
int vertIndex = indices[i];

        tempVec.set(offset).multLocal(blend);

        // acquire vertex
        BufferUtils.populateFromBuffer(tempVec2, bindPose, vertIndex);

        // add offset multiplied by factor
        tempVec2.addLocal(tempVec);

        // write modified vertex
        BufferUtils.setInBuffer(tempVec2, vertbuf, vertIndex);
    }
}

[/java]

And it works. At least, it looks like it work. I don’t know how your original animation should look so I can’t really tell. (the two top vertices of the back face of the cube moves up a bit and are closing in to the vertices of the front face).

I said “somehow” because that’s not correct.
The BindPose should be reapplied at the beginning of each frame (like it’s done for bone animation in the skeleton.resetToBind() method). and pose animation data should be applied to the position buffer (as it was done before my change).
The problem is that now the resetToBind is not a skeleton need only. I guess we should split this in several controls ( a ResetToBindControl), that would handle this operation. This way we’ll be able to mix pose and bone animation.
Also … the order will be primordial, because pose animation needs to be applied before bone animation.
Another issue to me is that normal and tangents are not taken into account at all. And it seems to me that if a vertex is moved its normal and tangent should change too. But i think we’re missing information for that.

If you want I can carry out the work from now. It’s gonna get more complicated, and you already did great.
Unless you really want to go on by yourself. Of course I’ll continue to guide you through this.

[Kidow]

Hey,

Thank you Thank you Thank you! :amused:

Somehow it doesn’t work for me. This is how it should look like

Link

And this is how it looks for me right now.

Link

Did it look the same for you? Did I miss something?

Thanks again, you are helping me a lot!

[nehon]

mhhh hard to tell, could you send me your blend file?

[Kidow]

Of course.

Link

[nehon]

Ok got it to work this time.

Still not the proper way to implement it, but here you can have a try.
this is the PoseTrack setTime code
[java]
public void setTime(float time, float weight, AnimControl control,
AnimChannel channel, TempVars vars) {
Spatial spat = control.getSpatial();
Geometry geom = findGeom(spat);
Mesh target = geom.getMesh();

    VertexBuffer bindPos = target.getBuffer(Type.BindPosePosition);
    VertexBuffer pos = target.getBuffer(Type.Position);
    FloatBuffer pb = (FloatBuffer) pos.getData();
    FloatBuffer bpb = (FloatBuffer) bindPos.getData();
    pb.clear();
    bpb.clear();
    pb.put(bpb).clear();

    
    if (time < times[0]) {
        applyFrame(target, 0, weight);
    } else if (time > times[times.length - 1]) {
        applyFrame(target, times.length - 1, weight);
    } else {
        int startFrame = 0;
        for (int i = 0; i < times.length; i++) {
            if (times[i] < time) {
                startFrame = i;
            }
        }

        int endFrame = startFrame + 1;
        float blend = (time - times[startFrame]) / (times[endFrame] - times[startFrame]);
 //       applyFrame(target, startFrame, blend * weight);
        applyFrame(target, endFrame, 1.0f);//(1f - blend) * weight);
    }
}

[/java]
applyFrame
[java]
private void applyFrame(Mesh target, int frameIndex, float weight){
PoseFrame frame = frames[frameIndex];
VertexBuffer pb = target.getBuffer(Type.Position);
for (int i = 0; i < frame.poses.length; i++){
Pose pose = frame.poses[i];
float poseWeight = frame.weights[i] * weight;

        pose.apply(poseWeight, (FloatBuffer) pb.getData());
    }

    // force to re-upload data to gpu
    pb.updateData(pb.getData());
}

[/java]
Pose.apply()
[java]
public void apply(float blend, FloatBuffer vertbuf) {
for (int i = 0; i < indices.length; i++) {
Vector3f offset = offsets[i];
int vertIndex = indices[i];
tempVec.set(offset).multLocal(blend);

        // acquire vertex
        BufferUtils.populateFromBuffer(tempVec2, vertbuf, vertIndex);

        // add offset multiplied by factor
        tempVec2.addLocal(tempVec);

        // write modified vertex
        BufferUtils.setInBuffer(tempVec2, vertbuf, vertIndex);
    }
}

[/java]
tell me if it works for you

[Kidow]

You’re my hero! Thanks soo so so much! :amused:

[nehon]

Thank YOU, you did all the hard work.

[justry]

Is it possible this could be made into a class or something that people can easily reuse without having to delve into all the intricacies under the hood?

[justry]

I’m sorry to bump, but I have a mesh which is animated with shape keys in blender, and I need to run it in jme3, and I had lost hope until I saw this thread. If someone would be so helpful as to maybe post the finished code, with some pointers on how to use, I would be very grateful.

[justry]

Ok so I followed the thread from its beginning and reconstructed the MeshLoader, PoseTrack, and Pose classes, applying all the changes mentioned. Now I just need to know how to use this to run my animation?

[dbugman]

Justry,

It seems you and I read this thread at nearly the exact same time and are both trying the same thing. I was able to do what you’re trying to do by unregistering the default MeshLoader and registering my custom MeshLoaderAdvanced class:

assetManager.unregisterLoader(MeshLoader.class);
assetManager.registerLoader(MeshLoaderAdvanced.class, "mesh.xml");

My code is able to deform the mesh, but it looks weird. I think I am near the place where Kidow was when he/she posted their long posting of code before nehon got it fixed. I have the feeling I’m close and a generic solution that doesn’t require tinkering with the internals should be possible.

One of the things I’ve found is that the indices that its trying to deform exceed the size of the mesh. It could be because the model I’m using is composed of several materials and I need to do some extra computation to determine which submesh to deform.

My model is a head model that is pretty high polygon composed of several submeshes and has 7 shapekeys. The shapekeys control facial animation. I’d like to allow for shapekey animation while also allowing skeletal animation.

Maybe if we work together we can figure out how to get this working. I’m glad there’s someone else looking at this same post, maybe we can revive interest in it.

[nehon]

Sorry guys I don’t have time to dive again into this right now.
Shoot any question you have and I’ll try to aswer though