[Mostly complete + patch] Hardware/GPU/Vertex skinning aka. matrix palette blending for jme

ghoust · August 22, 2012, 9:03pm

on my quest to optimize an mmoo I came on the next point on my list. Hardware skinning. So as usul I wanted to ask if someone has doen something in this direction for skeletal animation orr is there at least an ETA on when the official code will come in into the engine?

Sploreg · August 22, 2012, 9:37pm

It hasn’t been started to my knowledge, but I know people want it and it should get into jme eventually. If you are willing to take a stab at it I’m sure we can help provide guidance. Even if it doesn’t get into ‘core’ straight away, it can still be used as an addon until it matures. Jme3-stable release is coming up and adding new features like this will delay it too long, but we do plan to have 3.1, 3.2… releases. New features and improvements will always be added to JME.

Same response to your batched animations thread.

ghoust · August 30, 2012, 8:35pm

Ok so here’s my progress so far.

I have started on the vertex shader skinning by using the already existing code in skinning.glsllib shader code.

I have altered Lighting.j3md and Lighting.vert for optional skinning.

I have changed SkeletonControl for BoneIndex Buffer preperation from ubyte to float as attributes must be float, int, etc.

I have changed SkeletonControl to be able to switch from software to hardware skinning

I have succeeded to have a lighted, animated skinned mesh in one run using vertex shader skinning and in the other run using software skinning by commenting one line out/changing parameter from true to false. This works with oto (OgreAnim testcase) and my own models using up to 4 weights per bone and using up to 32 bones.

The todo’s on my list:

It works when sending the model through optimization, when not sent, it does look strange. I’ll have a look at the GeometryBatch optimize steps to see what needs to be adjusted right from the start.

Test if a model composed of several meshes does animate properly too (but the bug above must be fixed first)

Add Skinning to other material definitions too.

Test if I can switch from software to hardware without problems during the same one run.

Check why attribute bones[32] does work and bones[NUM_BONES] does not (perhaps just a startup naming problem)

Open for improvement(discussion)

My biggest concern is the coupling between Material and SkeletonControl for passing of the offset matrices to the shader. SkeletonControl has the bones and computes the offset matrices during update. SkeletonControl uses for targets the meshes, and does not have access to the geometries. Meshes do not contain any material references, but for passing the offset matrices to the sahder you need the material. Currently for a quick hack I just pass in the materials used by the model. But it would be nicer to have a single toggle on/toggle of method to be able to say use this, use that type of skinning without any hazzle.

Hmmm, while writing this. setSpatial does look at the Geometries anyway, so I could collect the Material references there. Well, anyway if somebody has some ideas on how to wrap this material/mesh/skeletoncontrol up nicely, please share it.

normen · August 30, 2012, 8:39pm

Exactly, as the skeletonControl always has a reference to a geometry/mesh it can most probably find some material there as well setSpatial(null) is called when the control is removed btw.

nehon · August 30, 2012, 8:49pm

Yeah you have the spatial.

Be aware thought hat the control is not always directly on a geomerty. On most imported models the control is attached to a node that contains the geometries.

So what you should do is to check if the spatial is a geometry, and grab its material, if the spatial is a node, search for all the geometry that it contains that have a material that supports skinning and/or a mesh targeted by the control’s animation.

nice progress so far, keep it on.

normen · August 30, 2012, 8:51pm

@nehon Doesn’t the AnimControl or SkeletonControl need a link to the correct geometry anyway? Or does it only have its mesh?

nehon · August 30, 2012, 8:58pm

the skeletonControl has an array of affected meshes.

Also it has a private method findTargets(Node) that returns the array of affected meshes, maybe it could be adapted/duplicated to return corresponding materials or geometries.

mifth · August 30, 2012, 9:22pm

@ghoust , have a look at here:

http://hub.jmonkeyengine.org/groups/free-announcements/forum/topic/mikumikudance-viewer/

There is gpu skinning. Possibly it will help.

ghoust · August 30, 2012, 10:46pm

Thank you for the hint, meanwhile I found it. It’s just a matter of having the right buffer types. Switching between software and hardware makes use of converting buffers from direct to heap for some. But it will give me some hints and insights how others solved it. Several parts looked similar, and some completetly different in mikumiku.

I’ll clean up the code and guess I can post it in the next week.

Also did a quick benchmark of a model composed of 9 meshes (optimized into a single, or not)

Optimized mesh | hardware skinning | vertices | tris | uniforms | objects | fps

yes | no | 2970 | 3490 | 36 | 16 | 2900-3050

yes | yes | 2970 | 3490 | 38 | 16 | 3850 -4000

no | yes | 2974 | 3492 | 170 | 27 | 2700-2800

no | no | 2974 | 3492 | 157 | 27 | 2200-2300

35 models with 164000 tris dropped the frame rate to 380-400 fps optimized on gpu, to 100-105 for unoptimized on cpu. 100 distinct models in a variety of 7 different models with 311000 tris optimized on gpu was running at 230-250 fps.

mifth · August 30, 2012, 10:59pm

wow!!! Nice result. I hope it will be i JME core.

Empire_Phoenix · August 30, 2012, 11:27pm

Yes for any game with many anitmaed stuff this will be very great

TiagoParreira · August 31, 2012, 12:16pm

This must be a really advanced topic… I tried to google out what hardware skinning is, and there’s barely anything at all! this jME thread even came on the 7th result!

ghoust · September 5, 2012, 4:18pm

not really that advanced or new (nearly 10 years old), sometimes it’s also called gpu skinning or vertex skinning perhaps this gives you more hits. this might help: http://http.developer.nvidia.com/GPUGems/gpugems_ch04.html on point 4.4 or http://http.developer.nvidia.com/CgTutorial/cg_tutorial_chapter06.html on 6.5

ghoust · September 5, 2012, 4:27pm

For anyone interrested, here’s the code as a patch to the current engine (revision 9701). The testcase it the standard OgreAnimTest, where one line was added (removing it or setting this to false will enable software skinning).Feel free to do whatever you want with the code

What is left from the points above:

Add Skinning to other material definitions too.

Test if I can switch from software to hardware without problems during the same one run.

The current configuration just uses 32 bones, would be great to test this (if vs 2.0 is supported go up with max bones to 85 etc…)

[patch]

Index: src/test/jme3test/model/anim/TestOgreAnim.java

===================================================================

— src/test/jme3test/model/anim/TestOgreAnim.java (revision 9701)

+++ src/test/jme3test/model/anim/TestOgreAnim.java (working copy)

@@ -82,6 +82,7 @@

channel.setAnim(“stand”);

geom = (Geometry)((Node)model).getChild(0);

SkeletonControl skeletonControl = model.getControl(SkeletonControl.class);

   skeletonControl.setUseHwSkinning(true);<br />

Box b = new Box(.25f,3f,.25f);
Geometry item = new Geometry("Item", b);
Index: src/core-data/Common/MatDefs/Light/Lighting.j3md
===================================================================
--- src/core-data/Common/MatDefs/Light/Lighting.j3md (revision 9701)
+++ src/core-data/Common/MatDefs/Light/Lighting.j3md (working copy)
@@ -115,6 +115,12 @@
Matrix4 LightViewProjectionMatrix3

Float PCFEdge
+
+ //Skinning
+ //vs 1.1 up to 32, vs 2+ up to 85
+ Int NumberOfBones
+ Boolean UseHwSkinning
+ Matrix4Array BoneMatrices
}

Technique {
@@ -158,6 +164,10 @@

USE_REFLECTION : EnvMap
SPHERE_MAP : EnvMapAsSphereMap
+
+ USE_HWSKINNING : UseHwSkinning
+ NUM_BONES : NumberOfBones
+ BONEMATRICES : BoneMatrices
}
}

Index: src/core-data/Common/MatDefs/Light/Lighting.vert
===================================================================
--- src/core-data/Common/MatDefs/Light/Lighting.vert (revision 9701)
+++ src/core-data/Common/MatDefs/Light/Lighting.vert (working copy)
@@ -1,3 +1,4 @@
+#import "Common/ShaderLib/Skinning.glsllib"
#define ATTENUATION
//#define HQ_ATTENUATION

@@ -132,6 +133,13 @@

void main(){
vec4 pos = vec4(inPosition, 1.0);
+ #ifdef USE_HWSKINNING
+ #if defined(VERTEX_LIGHTING)
+ Skinning_Compute(pos, inTangent, inNormal);
+ #else
+ Skinning_Compute(pos, inNormal);
+ #endif
+ #endif
gl_Position = g_WorldViewProjectionMatrix * pos;
texCoord = inTexCoord;
#ifdef SEPARATE_TEXCOORD
Index: src/core-data/Common/ShaderLib/Skinning.glsllib
===================================================================
--- src/core-data/Common/ShaderLib/Skinning.glsllib (revision 9701)
+++ src/core-data/Common/ShaderLib/Skinning.glsllib (working copy)
@@ -5,32 +5,52 @@
#endif

attribute vec4 inBoneWeight;
-attribute vec4 inBoneIndices;
+attribute vec4 inBoneIndex;
uniform mat4 m_BoneMatrices[NUM_BONES];

-void Skinning_Compute(inout vec4 position, inout vec4 normal){
- vec4 index = inBoneIndices;
- vec4 weight = inBoneWeight;
+void Skinning_Compute(inout vec4 position, inout vec3 normal){
+// vec4 index = inBoneIndex;
+// vec4 weight = inBoneWeight;

- vec4 newPos = vec4(0.0);
- vec4 newNormal = vec4(0.0);
+// vec4 newPos = vec4(0.0);
+// vec3 newNormal = vec3(0.0);

- for (float i = 0.0; i < 4.0; i += 1.0){
- mat4 skinMat = m_BoneMatrices[int(index.x)];
- newPos += weight.x * (skinMat * position);
- newNormal += weight.x * (skinMat * normal);
- index = index.yzwx;
- weight = weight.yzwx;
- }
+// for (float i = 0.0; i < 4.0; i += 1.0){
+// mat4 skinMat = m_BoneMatrices[int(index.x)];
+// newPos += weight.x * (skinMat * position);
+//// newNormal += weight.x * (skinMat * normal);
+// newNormal += weight.x * (mat3(skinMat[0].xyz,skinMat[1].xyz,skinMat[2].xyz) * normal);
+// index = index.yzwx;
+// weight = weight.yzwx;
+// }

- position = newPos;
- normal = newNormal;
-}
+// position = newPos;
+// normal = newNormal;
+

-#else
+ mat4 mat = mat4(0.0);
+
+ mat += m_BoneMatrices[int(inBoneIndex.x)] * inBoneWeight.x;
+ mat += m_BoneMatrices[int(inBoneIndex.y)] * inBoneWeight.y;
+ mat += m_BoneMatrices[int(inBoneIndex.z)] * inBoneWeight.z;
+ mat += m_BoneMatrices[int(inBoneIndex.w)] * inBoneWeight.w;
+
+ position = mat * position;
+ normal = (mat3(mat[0].xyz,mat[1].xyz,mat[2].xyz) * normal);
+}

-void Skinning_Compute(inout vec4 position, inout vec4 normal){
- // skinning disabled, leave position and normal unaltered
+void Skinning_Compute(inout vec4 position, inout vec4 tangent, inout vec3 normal){
+ mat4 mat = mat4(0.0);
+
+ mat += m_BoneMatrices[int(inBoneIndex.x)] * inBoneWeight.x;
+ mat += m_BoneMatrices[int(inBoneIndex.y)] * inBoneWeight.y;
+ mat += m_BoneMatrices[int(inBoneIndex.z)] * inBoneWeight.z;
+ mat += m_BoneMatrices[int(inBoneIndex.w)] * inBoneWeight.w;
+
+ position = mat * position;
+ tangent = mat * tangent;
+ normal = (mat3(mat[0].xyz,mat[1].xyz,mat[2].xyz) * normal);
}

+
#endif
No newline at end of file
Index: src/core/com/jme3/scene/VertexBuffer.java
===================================================================
--- src/core/com/jme3/scene/VertexBuffer.java (revision 9701)
+++ src/core/com/jme3/scene/VertexBuffer.java (working copy)
@@ -143,10 +143,11 @@
BoneWeight,

/**
- * Bone indices, used with animation (4 ubytes).
+ * Bone indices, used with animation (4 ubytes/floats).
* If used with software skinning, the usage should be
* {@link Usage#CpuOnly}, and the buffer should be allocated
- * on the heap.
+ * on the heap as a ubytes buffer. For Hardware skinning this should be
+ * either an int or float buffer due to shader attribute types restrictions.
*/
BoneIndex,

Index: src/core/com/jme3/scene/Mesh.java
===================================================================
--- src/core/com/jme3/scene/Mesh.java (revision 9701)
+++ src/core/com/jme3/scene/Mesh.java (working copy)
@@ -343,6 +343,8 @@
setBuffer(bindTangents);
tangents.setUsage(Usage.Stream);
}
+ } else {
+ //hardware setup does nothing, mesh already in bind pose
}
}

@@ -354,21 +356,66 @@
*/
public void prepareForAnim(boolean forSoftwareAnim){
if (forSoftwareAnim){
- // convert indices
+ // convert indices to ubytes on the heap or floats
VertexBuffer indices = getBuffer(Type.BoneIndex);
- ByteBuffer originalIndex = (ByteBuffer) indices.getData();
- ByteBuffer arrayIndex = ByteBuffer.allocate(originalIndex.capacity());
- originalIndex.clear();
- arrayIndex.put(originalIndex);
- indices.updateData(arrayIndex);
+ Buffer buffer = indices.getData();
+ if( buffer instanceof ByteBuffer){
+ ByteBuffer originalIndex = (ByteBuffer) buffer;
+ ByteBuffer arrayIndex = ByteBuffer.allocate(originalIndex.capacity());
+ originalIndex.clear();
+ arrayIndex.put(originalIndex);
+ indices.updateData(arrayIndex);
+ } else if(buffer instanceof FloatBuffer) {
+ FloatBuffer originalIndex = (FloatBuffer) buffer;
+ FloatBuffer arrayIndex = FloatBuffer.allocate(originalIndex.capacity());
+ originalIndex.clear();
+ arrayIndex.put(originalIndex);
+ indices.updateData(arrayIndex);
+ }

- // convert weights
+ // convert weights on the heap
VertexBuffer weights = getBuffer(Type.BoneWeight);
FloatBuffer originalWeight = (FloatBuffer) weights.getData();
FloatBuffer arrayWeight = FloatBuffer.allocate(originalWeight.capacity());
originalWeight.clear();
arrayWeight.put(originalWeight);
weights.updateData(arrayWeight);
+ } else {
+ //BoneIndex must be 32 bit for attribute type constraints in shaders
+ VertexBuffer indices = getBuffer(Type.BoneIndex);
+ Buffer buffer = indices.getData();
+ if(buffer instanceof ByteBuffer) {
+ ByteBuffer bIndex = (ByteBuffer) indices.getData();
+ final float[] rval = new float[bIndex.capacity()];
+ for (int i = 0; i < rval.length; i++) {
+ rval = bIndex.get(i);
+ }
+ clearBuffer(Type.BoneIndex);
+
+ VertexBuffer ib = new VertexBuffer(Type.BoneIndex);
+ ib.setupData(Usage.Stream,
+ 4,
+ Format.Float,
+ BufferUtils.createFloatBuffer(rval));
+ setBuffer(ib);
+ } else if( buffer instanceof FloatBuffer){
+ //BoneWeights on DirectBuffer
+ FloatBuffer originalIndices = (FloatBuffer) buffer;
+ FloatBuffer arrayIndices = BufferUtils.createFloatBuffer(originalIndices.capacity());
+ originalIndices.clear();
+ arrayIndices.put(originalIndices);
+ indices.setUsage(Usage.Stream);
+ indices.updateData(arrayIndices);
+ }
+
+ //BoneWeights on DirectBuffer
+ VertexBuffer weights = getBuffer(Type.BoneWeight);
+ FloatBuffer originalWeight = (FloatBuffer) weights.getData();
+ FloatBuffer arrayWeight = BufferUtils.createFloatBuffer(originalWeight.capacity());
+ originalWeight.clear();
+ arrayWeight.put(originalWeight);
+ weights.setUsage(Usage.Static);
+ weights.updateData(arrayWeight);
}
}

Index: src/core/com/jme3/animation/SkeletonControl.java
===================================================================
--- src/core/com/jme3/animation/SkeletonControl.java (revision 9701)
+++ src/core/com/jme3/animation/SkeletonControl.java (working copy)
@@ -4,20 +4,34 @@
*/
package com.jme3.animation;

-import com.jme3.export.*;
+import java.io.IOException;
+import java.nio.Buffer;
+import java.nio.ByteBuffer;
+import java.nio.FloatBuffer;
+import java.util.ArrayList;
+import java.util.HashSet;
+
+import com.jme3.export.InputCapsule;
+import com.jme3.export.JmeExporter;
+import com.jme3.export.JmeImporter;
+import com.jme3.export.OutputCapsule;
+import com.jme3.export.Savable;
+import com.jme3.material.Material;
import com.jme3.math.FastMath;
import com.jme3.math.Matrix4f;
import com.jme3.renderer.RenderManager;
import com.jme3.renderer.ViewPort;
-import com.jme3.scene.*;
+import com.jme3.scene.Geometry;
+import com.jme3.scene.Mesh;
+import com.jme3.scene.Node;
+import com.jme3.scene.Spatial;
+import com.jme3.scene.UserData;
+import com.jme3.scene.VertexBuffer;
import com.jme3.scene.VertexBuffer.Type;
import com.jme3.scene.control.AbstractControl;
import com.jme3.scene.control.Control;
+import com.jme3.shader.VarType;
import com.jme3.util.TempVars;
-import java.io.IOException;
-import java.nio.ByteBuffer;
-import java.nio.FloatBuffer;
-import java.util.ArrayList;

/**
* The Skeleton control deforms a model according to a skeleton,
@@ -41,14 +55,46 @@
* if they are visible in at least one camera.
*/
private boolean wasMeshUpdated = false;
-
+
+ /**
+ * Flag to enable hardware/gpu skinning, disable for software/cpu skinning
+ */
+ private boolean useHwSkinning = false;
/**
+ * Bone offset matrices, recreated each frame
+ */
+ private transient Matrix4f[] offsetMatrices;
+ /**
+ * Material references used for hardware skinning
+ */
+ private Material[] materials;
+
+ /**
* Serialization only. Do not use.
*/
public SkeletonControl() {
}

/**
+ * Sets hardware/software skinning mode. If in hardware mode all or some models display
+ * the same animation cycle make sure your materials are not identical but clones
+ * @param useHwSkinning the useHwSkinning to set
+ *
+ */
+ public void setUseHwSkinning(boolean useHwSkinning) {
+ this.useHwSkinning = useHwSkinning;
+ for(Material m : materials){
+ m.setBoolean("UseHwSkinning", useHwSkinning);
+ m.setInt("NumberOfBones", 32);
+ }
+ for (Mesh mesh : targets) {
+ if (isMeshAnimated(mesh)) {
+ mesh.prepareForAnim(!useHwSkinning); // prepare for software animation
+ }
+ }
+ }
+
+ /**
* Creates a skeleton control.
* The list of targets will be acquired automatically when
* the control is attached to a node.
@@ -75,6 +121,7 @@
return mesh.getBuffer(Type.BindPosePosition) != null;
}

+ //TODO this algorithm supposes all meshes are in a list below the node, this must not always be the case, or is it?
private Mesh[] findTargets(Node node) {
Mesh sharedMesh = null;
ArrayList<Mesh> animatedMeshes = new ArrayList<Mesh>();
@@ -114,6 +161,36 @@

return animatedMeshes.toArray(new Mesh[animatedMeshes.size()]);
}
+
+ private Material[] findMaterials(Node node) {
+
+ HashSet<Material> materials = new HashSet<Material>();
+
+ for (Spatial child : node.getChildren()) {
+ if (!(child instanceof Geometry)) {
+ continue; // could be an attachment node, ignore.
+ }
+
+ Geometry geom = (Geometry) child;
+
+ // is this geometry using a shared mesh?
+ Mesh childSharedMesh = geom.getUserData(UserData.JME_SHAREDMESH);
+
+ if (childSharedMesh != null) {
+ // Don't bother with non-animated shared meshes
+ if (isMeshAnimated(childSharedMesh)) {
+ materials.add(geom.getMaterial());
+ }
+ } else {
+ Mesh mesh = geom.getMesh();
+ if (isMeshAnimated(mesh)) {
+ materials.add(geom.getMaterial());
+ }
+ }
+ }
+
+ return materials.toArray(new Material[materials.size()]);
+ }

@Override
public void setSpatial(Spatial spatial) {
@@ -121,6 +198,7 @@
if (spatial != null) {
Node node = (Node) spatial;
targets = findTargets(node);
+ materials = findMaterials(node);
} else {
targets = null;
}
@@ -129,21 +207,27 @@
@Override
protected void controlRender(RenderManager rm, ViewPort vp) {
if (!wasMeshUpdated) {
- resetToBind(); // reset morph meshes to bind pose
+ if(useHwSkinning){

- Matrix4f[] offsetMatrices = skeleton.computeSkinningMatrices();
-
- // if hardware skinning is supported, the matrices and weight buffer
- // will be sent by the SkinningShaderLogic object assigned to the shader
- for (int i = 0; i < targets.length; i++) {
- // NOTE: This assumes that code higher up
- // Already ensured those targets are animated
- // otherwise a crash will happen in skin update
- //if (isMeshAnimated(targets)) {
- softwareSkinUpdate(targets, offsetMatrices);
- //}
- }
+ offsetMatrices = skeleton.computeSkinningMatrices();

+ hardwareSkinUpdate();
+ } else {
+ resetToBind(); // reset morph meshes to bind pose
+
+ offsetMatrices = skeleton.computeSkinningMatrices();
+
+ // if hardware skinning is supported, the matrices and weight buffer
+ // will be sent by the SkinningShaderLogic object assigned to the shader
+ for (int i = 0; i < targets.length; i++) {
+ // NOTE: This assumes that code higher up
+ // Already ensured those targets are animated
+ // otherwise a crash will happen in skin update
+ //if (isMeshAnimated(targets)) {
+ softwareSkinUpdate(targets, offsetMatrices);
+ //}
+ }
+ }
wasMeshUpdated = true;
}
}
@@ -153,13 +237,14 @@
wasMeshUpdated = false;
}

+ //only do this for software updates
void resetToBind() {
for (Mesh mesh : targets) {
if (isMeshAnimated(mesh)) {
- FloatBuffer bwBuff = (FloatBuffer) mesh.getBuffer(Type.BoneWeight).getData();
- ByteBuffer biBuff = (ByteBuffer)mesh.getBuffer(Type.BoneIndex).getData();
+ Buffer bwBuff = mesh.getBuffer(Type.BoneWeight).getData();
+ Buffer biBuff = mesh.getBuffer(Type.BoneIndex).getData();
if (!biBuff.hasArray() || !bwBuff.hasArray()) {
- mesh.prepareForAnim(true); // prepare for software animation
+ mesh.prepareForAnim(!useHwSkinning); // prepare for software animation
}
VertexBuffer bindPos = mesh.getBuffer(Type.BindPosePosition);
VertexBuffer bindNorm = mesh.getBuffer(Type.BindPoseNormal);
@@ -201,6 +286,7 @@
clone.skeleton = ctrl.getSkeleton();
// Fix animated targets for the cloned node
clone.targets = findTargets(clonedNode);
+ clone.materials = findMaterials(clonedNode);

// Fix attachments for the cloned node
for (int i = 0; i < clonedNode.getQuantity(); i++) {
@@ -288,6 +374,17 @@

}
+
+ /**
+ * Update the mesh according to the given transformation matrices
+ * @param mesh then mesh
+ * @param offsetMatrices the transformation matrices to apply
+ */
+ private void hardwareSkinUpdate() {
+ for(Material m : materials){
+ m.setParam("BoneMatrices", VarType.Matrix4Array, offsetMatrices);
+ }
+ }

/**
* Method to apply skinning transforms to a mesh's buffers
@@ -532,6 +629,8 @@
OutputCapsule oc = ex.getCapsule(this);
oc.write(targets, "targets", null);
oc.write(skeleton, "skeleton", null);
+ oc.write(useHwSkinning, "useHWSkinning", false);
+ oc.write(materials, "materials", null);
}

@Override
@@ -544,5 +643,11 @@
System.arraycopy(sav, 0, targets, 0, sav.length);
}
skeleton = (Skeleton) in.readSavable("skeleton", null);
+ useHwSkinning = in.readBoolean("useHWSkinning", false);
+ sav = in.readSavableArray("materials", null);
+ if (sav != null) {
+ materials = new Material[sav.length];
+ System.arraycopy(sav, 0, materials, 0, sav.length);
+ }
}
}

[/patch]

normen · September 5, 2012, 5:02pm

Awesome, thanks! I guess @Momoko_Fan will be happy to check this out

Momoko_Fan · September 7, 2012, 2:42pm

Thanks, that actually looks quite good @ghoust.

I have a few questions about your implementation:

The changes to Skinning_Compute are kind of interesting, have you noticed any performance improvement with your approach?
What is the reason the indices must be converted to floats to be used as indices? Using unsigned bytes should work fine, as long as normalization is disabled for that buffer.
Why is findTargets and findMaterials seperated? I think both of those should be merged. A material for skinning is a “target” too. It avoids duplicate code.
The VS1.1 / VS2.0 requirements create somewhat annoying limitations. I think if we just require a certain OpenGL version that supports high number of uniforms it will be fine. Another option is to store the bone matrices in a texture, but I don’t know how efficient that is, and it would require VTF also. EDIT: After a quick check to the GLSL specifications, it seems the minimum number of uniform components a card has to support is 1024 … Obviously this limit is too low. The ideal limit for us is 4096, but I am not sure how many cards actually support that much.
BONEMATRICES probably should not be a define, you can assume BoneMatrices are set if USE_HWSKINNING is set.

Besides this, here’s the requirements for including into core:
Unshaded.j3md must also support hardware skinning.
NumberOfBones should be set properly. To reduce the number of shaders, it can be rounded up to the largest number divisible by 10 … so a model with 32 bones and another with 38 bones would both use the shader for 40 bones
There’s no fallback for OpenGL1 … I think the flag for whether or not hardware skinning is used should be specified on the model key, since the user wouldn’t want to set it manually on each SkeletonControl they load. When SkeletonControl is deserialized from J3O, it can check that flag and apply the proper alterations to its state to support HW or SW skinning.

erlend_sh · September 13, 2012, 9:39pm

The mention to @ghoust failed so just in case he missed your reply, bump

Momoko_Fan · September 13, 2012, 11:57pm

Actually thanks for reminding me of this thread. I thought of another idea on how to handle the automatic switching … Since SkeletonControl actually updates the bone matrices in the SkeletonControl.render() method, you can check the renderer capabilities and then choose how to update the model vertices right there! That means the user doesn’t even need to know what kind of platform they are dealing with, it will choose the most appropriate method automatically. Furthermore, it can only use hardware skinning for those materials that expose the necessary material parameters. If they do not, it will just fallback to software skinning.

What do you think, @ghoust?

ghoust · September 14, 2012, 7:09pm

@erlend_sh thank you for the hint, I indeed missed the update.

@Momoko_Fan the answers:

it was purely based on the readings on optimizing skinning from the nvidia dawn demo in chapter 4.4.1 Accumulated Matrix Skinning:http://http.developer.nvidia.com/GPUGems/gpugems_ch04.html made no comparison as I assumed this to be more efficient by instruction counts
opengl does not support byte, short, or long integers, chapter 2.4.5 in language overview in the ogl shading language book. Trying to use the ubyte buffers gave straaaaange results, but this could be my nvidia card…
yes, this was my thought too. the findTarget method should also be refactored, as it only descends down to the children to find targets and does not a recoursive search. I think I left it to not change more existing code (and make the merge easier) and I have forgotten to refactor it then. Yes I agree it can be merged
yes this is 1024 uniform params in opengl which would mean for 32 4x4 matrices already 512 params used up and I wanted to keep some air for other usages. not sure where I got the vs_1_1 stuff, I read it somewhere on a site of a skinning implementation. as a suggestion. I’m not sure how to handle this, ev. allow to sizes and document this in the asset creation section.
yes, you are right, had the same idea when I ead about the rim light implementation

for the further implementation
if it’s just the one I can do this
there should be also a check about max supported # of uniforms for this, or at least some bounds checking, not sue if so may users would create models with many varying bones, but that’s why I would like to see a hard limit for vs 1.1 and 2.x so users would get a warning right at the point when the code would like to set the number of bones vs. max possible vs. suggested. If future features would require also some uniforms then this would be detected right at shader development time, if it is set dynamically this could go undetected for quite a while.
well I’m not sure 1.1 should be omitted, as Opengl ES is quite comparable to 1.1 and thus you would keep them out from a solution. currently I am also in comparison of cpu vs vertex skinning, having this always decided by the hardware would not make this possible. Yes I guess a check if the material exposes these variables for skinning at all should also be included. If the switching should be based on capabilities, is there a place this is done already, or would it be a static set during the instancing of the first skeleton control?

ghoust · September 14, 2012, 7:44pm

and what about ShowNormals and ColoredTextured materials?