[Mostly complete + patch] Hardware/GPU/Vertex skinning aka. matrix palette blending for jme

nehon · September 14, 2012, 8:33pm

deprecated.

Only Lighting and unshaded need hardware skinning.

Momoko_Fan · September 15, 2012, 1:14am

There are a few optimizations that might allow hardware skinning to be used in lower end GPUs. For example, if the animation and bind pose do not use scale (its always 1,1,1) then you can send the skinning matrices as position and quaternion (7 floats) vs a 4x4 matrix (16 floats). It allows you to have more than twice as many bones as before. So, there are ways of going around the limit, but it requires more processing and maybe work on the user part if their model does special stuff.

If you really care about supporting Android though, consider just writing a native (C++) function to handle skinning. You can even use SIMD instructions to speed it up even further. The GPU in Android devices is already shader bound in most games, so there’s not much point in giving it even more work to do.

Momoko_Fan · September 30, 2012, 5:22am

@ghoust: Just to ensure we are on track, if you implement the things I mentioned then this can go into core. The quaternion thing is optional, but its definitely critical that this works with any number of bones. We don’t want to place a hard limit of 32 bones or even 64, just because some GPUs don’t support it. We also don’t want the engine to work on some GPUs but crash on other GPUs because they cannot support that many bones. The only option remaining is to use hardware skinning when it is available and the number of bones allows it, and otherwise use software skinning.

ghoust · October 8, 2012, 8:25pm

@Momoko_Fan: thanx for reminding me of this. I have been busy with my own code the last weeks, sorry. I have changed the unshaded shader , but so far not done any tests on it. As the rest is currently not on my top prio list I will add them as time permits, but please don’t expect it within the next weeks, perhaps within the next two months or so.

Quaternions are definitely not on my list, as currently I remember it does require more gpu power. If scales are omitted you can also send them as 4x3 matrices I read somewhere.

The problem of number of bones is for my part not really solved. The problem I see is: one checks the number of vars, 1024, ok decide for hardware skinning, the developper uses a model which has say 48 bones → 768 vars used up, you feel safe, the shader works. Next day a developer adds a new shader variable for fog, glow, whatever, now the shader blows, although the capabilities say fine, you are below the bound, use hw skinning, but all effects added by #ifdef’s blow our count. Perhaps there should be a testcase which uses all available shader vars to see if that one blows. Otherwise how do you want to decide how many vars can be used for bones and how many must be left for the other vars in shaders?

TODOs:

a testcase for unshaded animation

merge findTargets an findMaterials (and make them real recursive)

use BONEMATRICES as indicator in shader if to use hw skinning

some kind of safe measurement of how many bones can be used

But as I already said, please don’t expect it in the next weeks…

Momoko_Fan · October 8, 2012, 11:06pm

OK, its totally understandable about the time.

Regarding the uniform limit, you can try to set the shader on the renderer, and if it fails (by catching the exception), not use hardware skinning. Its somewhat rudimentary, but it works …

ghoust · November 5, 2012, 5:50pm

@Momoko_Fan I’m on it now

ghoust · November 5, 2012, 9:55pm

@Momoko_Fan not sure how to handle the render test. How and when should I do this. Currently setting the bones to 400 will blow in render, way out of the control in charge. So how would I do this initialization render test?

Nov 05, 2012 10:30:11 PM com.jme3.app.Application handleError

Severe: Uncaught exception thrown in Thread[LWJGL Renderer Thread,5,main]

com.jme3.renderer.RendererException: Shader link failure, shader:Shader[numSources=2, numUniforms=20, shaderSources=[ShaderSource[name=Common/MatDefs/Light/Lighting.vert, defines, type=Vertex, language=GLSL100], ShaderSource[name=Common/MatDefs/Light/Lighting.frag, defines, type=Fragment, language=GLSL100]]] info:Vertex info

0(55) : warning C7504: OpenGL does not allow writing to input variable 'inNormal'
0(10) : error C5041: cannot locate suitable resource to bind parameter "m_BoneMatrices"
0(10) : error C5041: cannot locate suitable resource to bind parameter ""
..
...
0(10) : error C5041: cannot locate suitable resource to bind parameter ""
0(76) : error C5041: cannot locate suitable resource to bind parameter "g_WorldViewProjectionMatrix"
0(77) : error C5041: cannot locate suitable resource to bind parameter "g_WorldViewMatrix"
0(78) : error C5041: cannot locate suitable resource to bind parameter "g_NormalMatrix"
0(79) : error C5041: cannot locate suitable resource to bind parameter "g_ViewMatrix"

at com.jme3.renderer.lwjgl.LwjglRenderer.updateShaderData(LwjglRenderer.java:1086)
at com.jme3.renderer.lwjgl.LwjglRenderer.setShader(LwjglRenderer.java:1098)
at com.jme3.material.Material.renderMultipassLighting(Material.java:831)
at com.jme3.material.Material.render(Material.java:1051)
at com.jme3.renderer.RenderManager.renderGeometry(RenderManager.java:523)
at com.jme3.renderer.queue.RenderQueue.renderGeometryList(RenderQueue.java:301)
at com.jme3.renderer.queue.RenderQueue.renderQueue(RenderQueue.java:353)
at com.jme3.renderer.RenderManager.renderViewPortQueues(RenderManager.java:763)
at com.jme3.renderer.RenderManager.flushQueue(RenderManager.java:719)
at com.jme3.renderer.RenderManager.renderViewPort(RenderManager.java:983)
at com.jme3.renderer.RenderManager.render(RenderManager.java:1029)
at com.jme3.app.SimpleApplication.update(SimpleApplication.java:251)
at com.jme3.system.lwjgl.LwjglAbstractDisplay.runLoop(LwjglAbstractDisplay.java:151)
at com.jme3.system.lwjgl.LwjglDisplay.runLoop(LwjglDisplay.java:185)
at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:228)
at java.lang.Thread.run(Thread.java:722)

Nov 05, 2012 10:30:11 PM com.jme3.renderer.lwjgl.LwjglRenderer cleanup
...

ghoust · November 15, 2012, 6:55pm

@Momoko_Fan finished so far, only thing left is the testrender to get sure the hw will support the bone counts, see the FIXME notes

controlRender could be used, there the rendermanager is available, but the targets are all meshes, not the geometries perhaps change the targets in skeletonControl to geoms (then the material ref would also not be needed) at the cost of calls. then add a testrender for each geom to check if it fails if it is the first hw render try.

Anyway, here is the patch to the current code. I extended the ogreanim example to be able to switch between lighting and unshaded and hw vs sw skinning.

[patch]Index: src/test/jme3test/model/anim/TestOgreAnim.java

===================================================================

— src/test/jme3test/model/anim/TestOgreAnim.java (revision 9997)

+++ src/test/jme3test/model/anim/TestOgreAnim.java (working copy)

@@ -32,18 +32,25 @@

package jme3test.model.anim;

-import com.jme3.animation.*;

+import com.jme3.animation.AnimChannel;

+import com.jme3.animation.AnimControl;

+import com.jme3.animation.AnimEventListener;

+import com.jme3.animation.LoopMode;

+import com.jme3.animation.SkeletonControl;

import com.jme3.app.SimpleApplication;

+import com.jme3.font.BitmapText;

import com.jme3.input.KeyInput;

import com.jme3.input.controls.ActionListener;

import com.jme3.input.controls.KeyTrigger;

import com.jme3.light.DirectionalLight;

+import com.jme3.material.Material;

import com.jme3.math.ColorRGBA;

import com.jme3.math.Quaternion;

import com.jme3.math.Vector3f;

import com.jme3.scene.Geometry;

import com.jme3.scene.Node;

import com.jme3.scene.Spatial;

+import com.jme3.scene.Spatial.CullHint;

import com.jme3.scene.shape.Box;

public class TestOgreAnim extends SimpleApplication

@@ -52,6 +59,9 @@

private AnimChannel channel;

private AnimControl control;

private Geometry geom;

private SkeletonControl skeletonControl;
private Material unshaded;
private Material shaded;

public static void main(String[] args) {

TestOgreAnim app = new TestOgreAnim();

@@ -81,7 +91,11 @@

channel.setAnim("stand");

geom = (Geometry)((Node)model).getChild(0);

   SkeletonControl skeletonControl = model.getControl(SkeletonControl.class);<br />

   skeletonControl = model.getControl(SkeletonControl.class);<br />

```
 shaded = geom.getMaterial(); 
```

   unshaded = new Material(assetManager, &quot;Common/MatDefs/Misc/Unshaded.j3md&quot;);<br />

   unshaded.setColor(&quot;Color&quot;, ColorRGBA.Gray);<br />

+

Box b = new Box(.25f,3f,.25f);

Geometry item = new Geometry("Item", b);

@@ -92,8 +106,16 @@

rootNode.attachChild(model);

   inputManager.addListener(this, &quot;Attack&quot;);<br />

   inputManager.addListener(this, &quot;Attack&quot;, &quot;SkinningToggel&quot;, &quot;ShaderToggel&quot;);<br />

inputManager.addMapping("Attack", new KeyTrigger(KeyInput.KEY_SPACE));

   inputManager.addMapping(&quot;SkinningToggel&quot;, new KeyTrigger(KeyInput.KEY_R));<br />

   inputManager.addMapping(&quot;ShaderToggel&quot;, new KeyTrigger(KeyInput.KEY_T));<br />

+

   BitmapText text = new BitmapText(guiFont, false);<br />

   text.setLocalTranslation(0, settings.getHeight()-text.getLineHeight(), 0);<br />

   text.setText(&quot;Press R for SW/HW skinningnPress T for Unshaded/Lighting shadernPress spacebar to init dodge animation&quot;);<br />

   text.setCullHint(CullHint.Never);<br />

```
 guiNode.attachChild(text); 
```

}

@Override

@@ -122,6 +144,22 @@

channel.setLoopMode(LoopMode.Cycle);

channel.setSpeed(0.10f);

}

   } else if(binding.equals(&quot;SkinningToggel&quot;) &amp;&amp; value){<br />

   	skeletonControl.setUseHwSkinning(!skeletonControl.isUseHwSkinning());<br />

   } else if(binding.equals(&quot;ShaderToggel&quot;) &amp;&amp; value){<br />

   	if(geom.getMaterial().equals(shaded)) {<br />

```
 		geom.setMaterial(unshaded); 
```

   		skeletonControl.setSpatial(null);<br />

   		skeletonControl.setSpatial(geom.getParent());<br />

```
 	} else { 
```
```
 		geom.setMaterial(shaded); 
```

   		//to reassign materials + targets we toggel the target spatial<br />

   		//this is only needed for this showcase where the material of the<br />

   		//skinned geom changes, otherwise skeletonControl would know nothing<br />

```
 		//of this material change. 
```

   		skeletonControl.setSpatial(null);<br />

   		skeletonControl.setSpatial(geom.getParent());<br />

```
 	} 
```

}

}

Index: src/core-data/Common/MatDefs/Misc/Unshaded.j3md

===================================================================

— src/core-data/Common/MatDefs/Misc/Unshaded.j3md (revision 9997)

+++ src/core-data/Common/MatDefs/Misc/Unshaded.j3md (working copy)

@@ -11,6 +11,11 @@

Texture2D GlowMap

// The glow color of the object

Color GlowColor

+

```
 //Skinning 
```

  //vs 1.1 up to 32, vs 2+ up to 85<br />

```
 Int NumberOfBones 
```
```
 Matrix4Array BoneMatrices 
```

}

Technique {

@@ -27,6 +32,9 @@

HAS_LIGHTMAP : LightMap

HAS_VERTEXCOLOR : VertexColor

HAS_COLOR : Color

+

```
 	NUM_BONES : NumberOfBones 
```
```
 	BONEMATRICES : BoneMatrices 
```

}

}

Index: src/core-data/Common/MatDefs/Misc/Unshaded.vert

===================================================================

— src/core-data/Common/MatDefs/Misc/Unshaded.vert (revision 9997)

+++ src/core-data/Common/MatDefs/Misc/Unshaded.vert (working copy)

@@ -1,3 +1,4 @@

+#import "Common/ShaderLib/Skinning.glsllib"

uniform mat4 g_WorldViewProjectionMatrix;

attribute vec3 inPosition;

@@ -27,5 +28,12 @@

vertColor = inColor;

#endif

gl_Position = g_WorldViewProjectionMatrix * vec4(inPosition, 1.0);

#ifdef NUM_BONES

 vec4 pos = vec4(inPosition, 1.0);<br />

```
Skinning_Compute(pos); 
```

gl_Position = g_WorldViewProjectionMatrix * pos;<br />

#else

 gl_Position = g_WorldViewProjectionMatrix * vec4(inPosition, 1.0);<br />

#endif

+

}

No newline at end of file

Index: src/core-data/Common/MatDefs/Light/Lighting.j3md

===================================================================

— src/core-data/Common/MatDefs/Light/Lighting.j3md (revision 9997)

+++ src/core-data/Common/MatDefs/Light/Lighting.j3md (working copy)

@@ -127,6 +127,11 @@

Float PCFEdge

Float ShadowMapSize

+
```
 //Skinning 
```

  //vs 1.1 up to 32, vs 2+ up to 85<br />

```
 Int NumberOfBones 
```
```
 Matrix4Array BoneMatrices 
```

}

Technique {

@@ -170,6 +175,9 @@

USE_REFLECTION : EnvMap

SPHERE_MAP : SphereMap

+

```
 	NUM_BONES : NumberOfBones 
```
```
 	BONEMATRICES : BoneMatrices 
```

}

}

Index: src/core-data/Common/MatDefs/Light/Lighting.vert

===================================================================

— src/core-data/Common/MatDefs/Light/Lighting.vert (revision 9997)

+++ src/core-data/Common/MatDefs/Light/Lighting.vert (working copy)

@@ -1,3 +1,4 @@

+#import "Common/ShaderLib/Skinning.glsllib"

#define ATTENUATION

//#define HQ_ATTENUATION

@@ -132,6 +133,13 @@

void main(){

vec4 pos = vec4(inPosition, 1.0);

#ifdef NUM_BONES
```
 #if defined(VERTEX_LIGHTING) 
```

    Skinning_Compute(pos, inTangent, inNormal);<br />

```
 #else 
```

    Skinning_Compute(pos, inNormal);<br />

```
 #endif 
```
#endif

gl_Position = g_WorldViewProjectionMatrix * pos;

texCoord = inTexCoord;

#ifdef SEPARATE_TEXCOORD

Index: src/core-data/Common/ShaderLib/Skinning.glsllib

===================================================================

— src/core-data/Common/ShaderLib/Skinning.glsllib (revision 9997)

+++ src/core-data/Common/ShaderLib/Skinning.glsllib (working copy)

@@ -1,36 +1,67 @@

-#ifdef USE_HWSKINNING

-

-#ifndef NUM_BONES

-#error A required pre-processor define "NUM_BONES" is not set!

-#endif

+#ifdef NUM_BONES

attribute vec4 inBoneWeight;

-attribute vec4 inBoneIndices;

+attribute vec4 inBoneIndex;

uniform mat4 m_BoneMatrices[NUM_BONES];

-void Skinning_Compute(inout vec4 position, inout vec4 normal){

vec4 index = inBoneIndices;

+void Skinning_Compute(inout vec4 position){

vec4 index = inBoneIndex;

vec4 weight = inBoneWeight;

-

vec4 newPos = vec4(0.0);

vec4 newNormal = vec4(0.0);

for (float i = 0.0; i < 4.0; i += 1.0){

mat4 skinMat = m_BoneMatrices[int(index.x)];

newPos += weight.x * (skinMat * position);

   newNormal += weight.x * (skinMat * normal);<br />

index = index.yzwx;

weight = weight.yzwx;

}

position = newPos;

normal = newNormal;

}

-#else

+void Skinning_Compute(inout vec4 position, inout vec3 normal){

+// vec4 index = inBoneIndex;

+// vec4 weight = inBoneWeight;

-void Skinning_Compute(inout vec4 position, inout vec4 normal){
// skinning disabled, leave position and normal unaltered

+// vec4 newPos = vec4(0.0);

+// vec3 newNormal = vec3(0.0);

+

+// for (float i = 0.0; i < 4.0; i += 1.0){

+// mat4 skinMat = m_BoneMatrices[int(index.x)];

+// newPos += weight.x * (skinMat * position);

+//// newNormal += weight.x * (skinMat * normal);

+// newNormal += weight.x * (mat3(skinMat[0].xyz,skinMat[1].xyz,skinMat[2].xyz) * normal);

+// index = index.yzwx;

+// weight = weight.yzwx;

+// }

+

+// position = newPos;

+// normal = newNormal;

+

+

mat4 mat = mat4(0.0);

+
mat += m_BoneMatrices[int(inBoneIndex.x)] * inBoneWeight.x;
mat += m_BoneMatrices[int(inBoneIndex.y)] * inBoneWeight.y;
mat += m_BoneMatrices[int(inBoneIndex.z)] * inBoneWeight.z;
mat += m_BoneMatrices[int(inBoneIndex.w)] * inBoneWeight.w;

+
position = mat * position;
normal = (mat3(mat[0].xyz,mat[1].xyz,mat[2].xyz) * normal);

+}

+

+void Skinning_Compute(inout vec4 position, inout vec4 tangent, inout vec3 normal){
mat4 mat = mat4(0.0);

+
mat += m_BoneMatrices[int(inBoneIndex.x)] * inBoneWeight.x;
mat += m_BoneMatrices[int(inBoneIndex.y)] * inBoneWeight.y;
mat += m_BoneMatrices[int(inBoneIndex.z)] * inBoneWeight.z;
mat += m_BoneMatrices[int(inBoneIndex.w)] * inBoneWeight.w;

+
position = mat * position;
tangent = mat * tangent;
normal = (mat3(mat[0].xyz,mat[1].xyz,mat[2].xyz) * normal);

}

+

#endif

No newline at end of file

Index: src/core/com/jme3/scene/VertexBuffer.java

===================================================================

— src/core/com/jme3/scene/VertexBuffer.java (revision 9997)

+++ src/core/com/jme3/scene/VertexBuffer.java (working copy)

@@ -142,10 +142,11 @@

BoneWeight,

/**

    * Bone indices, used with animation (4 ubytes).<br />

    * Bone indices, used with animation (4 ubytes/floats).<br />

If used with software skinning, the usage should be
{@link Usage#CpuOnly}, and the buffer should be allocated

```
 * on the heap. 
```

    * on the heap as a ubytes buffer. For Hardware skinning this should be<br />

    * either an int or float buffer due to shader attribute types restrictions.<br />

*/

BoneIndex,

Index: src/core/com/jme3/scene/Mesh.java

===================================================================

— src/core/com/jme3/scene/Mesh.java (revision 9997)

+++ src/core/com/jme3/scene/Mesh.java (working copy)

@@ -342,7 +342,7 @@

setBuffer(bindTangents);

tangents.setUsage(Usage.Stream);

}

```
 } 
```

   }// else hardware setup does nothing, mesh already in bind pose<br />

}

/**

@@ -353,21 +353,69 @@

*/

public void prepareForAnim(boolean forSoftwareAnim){

if (forSoftwareAnim){

```
 // convert indices 
```

       // convert indices to ubytes on the heap or floats<br />

VertexBuffer indices = getBuffer(Type.BoneIndex);

       ByteBuffer originalIndex = (ByteBuffer) indices.getData();<br />

       ByteBuffer arrayIndex = ByteBuffer.allocate(originalIndex.capacity());<br />

```
 originalIndex.clear(); 
```

       arrayIndex.put(originalIndex);<br />

       indices.updateData(arrayIndex);<br />

       Buffer buffer = indices.getData();<br />

       if( buffer instanceof ByteBuffer){<br />

          ByteBuffer originalIndex = (ByteBuffer) buffer;<br />

          ByteBuffer arrayIndex = ByteBuffer.allocate(originalIndex.capacity());<br />

```
 originalIndex.clear(); 
```

          arrayIndex.put(originalIndex);<br />

          indices.updateData(arrayIndex);<br />

       } else if(buffer instanceof FloatBuffer) {<br />

```
 	//Floats back to bytes 
```

       	FloatBuffer originalIndex = (FloatBuffer) buffer;<br />

       	ByteBuffer arrayIndex = ByteBuffer.allocate(originalIndex.capacity());<br />

```
 	originalIndex.clear(); 
```

          for (int i = 0; i &lt; originalIndex.capacity(); i++) {<br />

          	arrayIndex.put((byte)originalIndex.get(i));<br />

```
 } 
```

           indices.updateData(arrayIndex);<br />

```
 } 
```

- // convert weights
+ // convert weights on the heap
VertexBuffer weights = getBuffer(Type.BoneWeight);
FloatBuffer originalWeight = (FloatBuffer) weights.getData();
FloatBuffer arrayWeight = FloatBuffer.allocate(originalWeight.capacity());
originalWeight.clear();
arrayWeight.put(originalWeight);
weights.updateData(arrayWeight);
+ } else {
+ //BoneIndex must be 32 bit for attribute type constraints in shaders
+ VertexBuffer indices = getBuffer(Type.BoneIndex);
+ Buffer buffer = indices.getData();
+ if(buffer instanceof ByteBuffer) {
+ ByteBuffer bIndex = (ByteBuffer) buffer;
+ final float[] rval = new float[bIndex.capacity()];
+ for (int i = 0; i < rval.length; i++) {
+ rval = bIndex.get(i);
+ }
+ clearBuffer(Type.BoneIndex);
+
+ VertexBuffer ib = new VertexBuffer(Type.BoneIndex);
+ ib.setupData(Usage.Stream,
+ 4,
+ Format.Float,
+ BufferUtils.createFloatBuffer(rval));
+ setBuffer(ib);
+ } else if( buffer instanceof FloatBuffer){
+ //BoneWeights on DirectBuffer
+ FloatBuffer originalIndices = (FloatBuffer) buffer;
+ FloatBuffer arrayIndices = BufferUtils.createFloatBuffer(originalIndices.capacity());
+ originalIndices.clear();
+ arrayIndices.put(originalIndices);
+ indices.setUsage(Usage.Stream);
+ indices.updateData(arrayIndices);
+ }
+
+ //BoneWeights on DirectBuffer
+ VertexBuffer weights = getBuffer(Type.BoneWeight);
+ FloatBuffer originalWeight = (FloatBuffer) weights.getData();
+ FloatBuffer arrayWeight = BufferUtils.createFloatBuffer(originalWeight.capacity());
+ originalWeight.clear();
+ arrayWeight.put(originalWeight);
+ weights.setUsage(Usage.Static);
+ weights.updateData(arrayWeight);
}
}

Index: src/core/com/jme3/animation/SkeletonControl.java
===================================================================
--- src/core/com/jme3/animation/SkeletonControl.java (revision 9997)
+++ src/core/com/jme3/animation/SkeletonControl.java (working copy)
@@ -31,20 +31,34 @@
*/
package com.jme3.animation;

-import com.jme3.export.*;
+import java.io.IOException;
+import java.nio.Buffer;
+import java.nio.ByteBuffer;
+import java.nio.FloatBuffer;
+import java.util.ArrayList;
+import java.util.HashSet;
+
+import com.jme3.export.InputCapsule;
+import com.jme3.export.JmeExporter;
+import com.jme3.export.JmeImporter;
+import com.jme3.export.OutputCapsule;
+import com.jme3.export.Savable;
+import com.jme3.material.Material;
import com.jme3.math.FastMath;
import com.jme3.math.Matrix4f;
import com.jme3.renderer.RenderManager;
import com.jme3.renderer.ViewPort;
-import com.jme3.scene.*;
+import com.jme3.scene.Geometry;
+import com.jme3.scene.Mesh;
+import com.jme3.scene.Node;
+import com.jme3.scene.Spatial;
+import com.jme3.scene.UserData;
+import com.jme3.scene.VertexBuffer;
import com.jme3.scene.VertexBuffer.Type;
import com.jme3.scene.control.AbstractControl;
import com.jme3.scene.control.Control;
+import com.jme3.shader.VarType;
import com.jme3.util.TempVars;
-import java.io.IOException;
-import java.nio.ByteBuffer;
-import java.nio.FloatBuffer;
-import java.util.ArrayList;

/**
* The Skeleton control deforms a model according to a skeleton,
@@ -67,15 +81,58 @@
* Used to track when a mesh was updated. Meshes are only updated
* if they are visible in at least one camera.
*/
- private boolean wasMeshUpdated = false;
+ private boolean wasMeshUpdated = false;
+ /**
+ * Flag to enable hardware/gpu skinning if available, disable for software/cpu skinning, enabled by default
+ */
+ private boolean useHwSkinning = true;

/**
+ * Bone offset matrices, recreated each frame
+ */
+ private transient Matrix4f[] offsetMatrices;
+ /**
+ * Material references used for hardware skinning
+ */
+ private Material[] materials;
+
+ /**
* Serialization only. Do not use.
*/
public SkeletonControl() {
}

/**
+ * Hint to use hardware/software skinning mode. If gpu skinning fails or is disabledIf in hardware mode all or some models display
+ * the same animation cycle make sure your materials are not identical but clones
+ * @param useHwSkinning the useHwSkinning to set
+ *
+ */
+ public void setUseHwSkinning(boolean useHwSkinning) {
+ this.useHwSkinning = useHwSkinning;
+ //next full 10 bones (e.g. 30 on 24 bones )
+ int bones = ((skeleton.getBoneCount()/10)+1)*10;
+ for(Material m : materials){
+ if(useHwSkinning) {
+ m.setInt("NumberOfBones", bones);
+//FIXME this will blow on first render, how to intercept this? m.setInt("NumberOfBones", 400);
+ } else {
+ m.clearParam("NumberOfBones");
+ }
+ }
+
+ for (Mesh mesh : targets) {
+ if (isMeshAnimated(mesh)) {
+ mesh.prepareForAnim(!useHwSkinning); // prepare for software animation
+ }
+ }
+ }
+
+ public boolean isUseHwSkinning(){
+ return useHwSkinning;
+ }
+
+ /**
* Creates a skeleton control.
* The list of targets will be acquired automatically when
* the control is attached to a node.
@@ -102,44 +159,43 @@
return mesh.getBuffer(Type.BindPosePosition) != null;
}

- private Mesh[] findTargets(Node node) {
+ private void findTargets(Node node, ArrayList<Mesh> targets, HashSet<Material> materials) {
Mesh sharedMesh = null;
- ArrayList<Mesh> animatedMeshes = new ArrayList<Mesh>();

for (Spatial child : node.getChildren()) {
- if (!(child instanceof Geometry)) {
- continue; // could be an attachment node, ignore.
- }
-
- Geometry geom = (Geometry) child;
-
- // is this geometry using a shared mesh?
- Mesh childSharedMesh = geom.getUserData(UserData.JME_SHAREDMESH);
-
- if (childSharedMesh != null) {
- // Don't bother with non-animated shared meshes
- if (isMeshAnimated(childSharedMesh)) {
- // child is using shared mesh,
- // so animate the shared mesh but ignore child
- if (sharedMesh == null) {
- sharedMesh = childSharedMesh;
- } else if (sharedMesh != childSharedMesh) {
- throw new IllegalStateException("Two conflicting shared meshes for " + node);
- }
- }
- } else {
- Mesh mesh = geom.getMesh();
- if (isMeshAnimated(mesh)) {
- animatedMeshes.add(mesh);
- }
+ if (child instanceof Geometry) {
+ Geometry geom = (Geometry) child;
+
+ // is this geometry using a shared mesh?
+ Mesh childSharedMesh = geom.getUserData(UserData.JME_SHAREDMESH);
+
+ if (childSharedMesh != null) {
+ // Don't bother with non-animated shared meshes
+ if (isMeshAnimated(childSharedMesh)) {
+ // child is using shared mesh,
+ // so animate the shared mesh but ignore child
+ if (sharedMesh == null) {
+ sharedMesh = childSharedMesh;
+ } else if (sharedMesh != childSharedMesh) {
+ throw new IllegalStateException("Two conflicting shared meshes for " + node);
+ }
+ materials.add(geom.getMaterial());
+ }
+ } else {
+ Mesh mesh = geom.getMesh();
+ if (isMeshAnimated(mesh)) {
+ targets.add(mesh);
+ materials.add(geom.getMaterial());
+ }
+ }
+ } else if( child instanceof Node){
+ findTargets((Node)child,targets, materials);
}
}

if (sharedMesh != null) {
- animatedMeshes.add(sharedMesh);
+ targets.add(sharedMesh);
}
-
- return animatedMeshes.toArray(new Mesh[animatedMeshes.size()]);
}

@Override
@@ -147,30 +203,44 @@
super.setSpatial(spatial);
if (spatial != null) {
Node node = (Node) spatial;
- targets = findTargets(node);
+
+ HashSet<Material> mats = new HashSet<Material>();
+ ArrayList<Mesh> meshes = new ArrayList<Mesh>();
+ findTargets(node, meshes, mats);
+ targets = meshes.toArray(new Mesh[meshes.size()]);
+ materials = mats.toArray(new Material[mats.size()]);
+ //FIXME: try hw skinning, will be reset to sw skinning if render call fails, this is a TODO on controlRender()
+ setUseHwSkinning(true);
} else {
targets = null;
+ materials = null;
}
}

@Override
protected void controlRender(RenderManager rm, ViewPort vp) {
if (!wasMeshUpdated) {
- resetToBind(); // reset morph meshes to bind pose
-
- Matrix4f[] offsetMatrices = skeleton.computeSkinningMatrices();
+ if(useHwSkinning){

- // if hardware skinning is supported, the matrices and weight buffer
- // will be sent by the SkinningShaderLogic object assigned to the shader
- for (int i = 0; i < targets.length; i++) {
- // NOTE: This assumes that code higher up
- // Already ensured those targets are animated
- // otherwise a crash will happen in skin update
- //if (isMeshAnimated(targets)) {
- softwareSkinUpdate(targets, offsetMatrices);
- //}
- }
+ offsetMatrices = skeleton.computeSkinningMatrices();

+ hardwareSkinUpdate();
+ } else {
+ resetToBind(); // reset morph meshes to bind pose
+
+ offsetMatrices = skeleton.computeSkinningMatrices();
+
+ // if hardware skinning is supported, the matrices and weight buffer
+ // will be sent by the SkinningShaderLogic object assigned to the shader
+ for (int i = 0; i < targets.length; i++) {
+ // NOTE: This assumes that code higher up
+ // Already ensured those targets are animated
+ // otherwise a crash will happen in skin update
+ //if (isMeshAnimated(targets)) {
+ softwareSkinUpdate(targets, offsetMatrices);
+ //}
+ }
+ }
wasMeshUpdated = true;
}
}
@@ -180,13 +250,14 @@
wasMeshUpdated = false;
}

+ //only do this for software updates
void resetToBind() {
for (Mesh mesh : targets) {
if (isMeshAnimated(mesh)) {
- FloatBuffer bwBuff = (FloatBuffer) mesh.getBuffer(Type.BoneWeight).getData();
- ByteBuffer biBuff = (ByteBuffer)mesh.getBuffer(Type.BoneIndex).getData();
+ Buffer bwBuff = mesh.getBuffer(Type.BoneWeight).getData();
+ Buffer biBuff = mesh.getBuffer(Type.BoneIndex).getData();
if (!biBuff.hasArray() || !bwBuff.hasArray()) {
- mesh.prepareForAnim(true); // prepare for software animation
+ mesh.prepareForAnim(!useHwSkinning); // prepare for software animation
}
VertexBuffer bindPos = mesh.getBuffer(Type.BindPosePosition);
VertexBuffer bindNorm = mesh.getBuffer(Type.BindPoseNormal);
@@ -223,11 +294,14 @@
Node clonedNode = (Node) spatial;
AnimControl ctrl = spatial.getControl(AnimControl.class);
SkeletonControl clone = new SkeletonControl();
- clone.setSpatial(clonedNode);
-
clone.skeleton = ctrl.getSkeleton();
- // Fix animated targets for the cloned node
- clone.targets = findTargets(clonedNode);
+
+ clone.setSpatial(clonedNode);
+
+ //removed, done already in setSpatial
+// // Fix animated targets for the cloned node
+// clone.targets = findTargets(clonedNode);
+// clone.materials = findMaterials(clonedNode);

// Fix attachments for the cloned node
for (int i = 0; i < clonedNode.getQuantity(); i++) {
@@ -315,6 +389,17 @@

}
+
+ /**
+ * Update the mesh according to the given transformation matrices
+ * @param mesh then mesh
+ * @param offsetMatrices the transformation matrices to apply
+ */
+ private void hardwareSkinUpdate() {
+ for(Material m : materials){
+ m.setParam("BoneMatrices", VarType.Matrix4Array, offsetMatrices);
+ }
+ }

/**
* Method to apply skinning transforms to a mesh's buffers
@@ -574,6 +659,7 @@
OutputCapsule oc = ex.getCapsule(this);
oc.write(targets, "targets", null);
oc.write(skeleton, "skeleton", null);
+ oc.write(materials, "materials", null);
}

@Override
@@ -586,5 +672,10 @@
System.arraycopy(sav, 0, targets, 0, sav.length);
}
skeleton = (Skeleton) in.readSavable("skeleton", null);
+ sav = in.readSavableArray("materials", null);
+ if (sav != null) {
+ materials = new Material[sav.length];
+ System.arraycopy(sav, 0, materials, 0, sav.length);
+ }
}
}[/patch]

Momoko_Fan · November 16, 2012, 12:41am

You do have the Spatial in your SkeletonControl, which in the case of animated models is a Node with all the model’s Geometries under it. What you can do is use RenderManager.preloadScene() which won’t render the model, but it would upload all meshes, textures, shaders, etc onto the GPU and thus if it cannot do that due to the shader hitting the uniform limit, it will throw an exception and then you turn on software mode and try again and possibly log a WARNING.

normen · November 16, 2012, 12:05pm

@Momoko_Fan, @ghoust: Moved this thread to the contrib repo

ghoust · December 3, 2012, 3:54pm

So here’s the patch to SkeletonControl for testing hw rendering is set on the first run and reverting to sw skinnign if hw skinning upload failed.

I left the logger out, as this would be the only place using it. If anyone should want it it should be quite obvious to put that three lines in. For anyonone who would like to test if the switchback to sw skinning works, just replace the setInt(“NumBones”, xy) call in setUseWHSkinning to something exaggerated like setInt(“NumBones”, 400);

@Momoko_Fan that would be it from my side for the hw skinning

[patch]

Index: src/core/com/jme3/animation/SkeletonControl.java

===================================================================

— src/core/com/jme3/animation/SkeletonControl.java (revision 10018)

+++ src/core/com/jme3/animation/SkeletonControl.java (working copy)

@@ -31,20 +31,35 @@

/

package com.jme3.animation;

-import com.jme3.export.;

+import java.io.IOException;

+import java.nio.Buffer;

+import java.nio.ByteBuffer;

+import java.nio.FloatBuffer;

+import java.util.ArrayList;

+import java.util.HashSet;

+

+import com.jme3.export.InputCapsule;

+import com.jme3.export.JmeExporter;

+import com.jme3.export.JmeImporter;

+import com.jme3.export.OutputCapsule;

+import com.jme3.export.Savable;

+import com.jme3.material.Material;

import com.jme3.math.FastMath;

import com.jme3.math.Matrix4f;

import com.jme3.renderer.RenderManager;

+import com.jme3.renderer.RendererException;

import com.jme3.renderer.ViewPort;

-import com.jme3.scene.*;

+import com.jme3.scene.Geometry;

+import com.jme3.scene.Mesh;

+import com.jme3.scene.Node;

+import com.jme3.scene.Spatial;

+import com.jme3.scene.UserData;

+import com.jme3.scene.VertexBuffer;

import com.jme3.scene.VertexBuffer.Type;

import com.jme3.scene.control.AbstractControl;

import com.jme3.scene.control.Control;

+import com.jme3.shader.VarType;

import com.jme3.util.TempVars;

-import java.io.IOException;

-import java.nio.ByteBuffer;

-import java.nio.FloatBuffer;

-import java.util.ArrayList;

/**

The Skeleton control deforms a model according to a skeleton,

@@ -67,15 +82,65 @@
Used to track when a mesh was updated. Meshes are only updated
if they are visible in at least one camera.

*/

private boolean wasMeshUpdated = false;

private boolean wasMeshUpdated = false;
/**

* Flag to enable hardware/gpu skinning if available, disable for software/cpu skinning, enabled by default<br />

```
*/ 
```
private boolean useHwSkinning = true;
/**

* Flag to check if we have to check the shader if it would work and on fail switch to sw skinning<br />

```
*/ 
```
private boolean triedHwSkinning = false;

/**

* Bone offset matrices, recreated each frame<br />

```
*/ 
```
private transient Matrix4f[] offsetMatrices;
/**
- Material references used for hardware skinning
*/
private Material[] materials;

+
/**

Serialization only. Do not use.

*/

public SkeletonControl() {

}

/**

- Hint to use hardware/software skinning mode. If gpu skinning fails or is disabledIf in hardware mode all or some models display
- the same animation cycle make sure your materials are not identical but clones
- @param useHwSkinning the useHwSkinning to set
*
*/
public void setUseHwSkinning(boolean useHwSkinning) {

  this.useHwSkinning = useHwSkinning;<br />

```
 this.triedHwSkinning = false; 
```

  //next full 10 bones (e.g. 30 on 24 bones )<br />

  int bones = ((skeleton.getBoneCount()/10)+1)*10;<br />

```
 for(Material m : materials){ 
```
```
 	if(useHwSkinning) { 
```

  		m.setInt("NumberOfBones", bones);<br />

```
 	} else { 
```

  		m.clearParam("NumberOfBones");<br />

```
 	} 
```
```
 } 
```

+

```
 for (Mesh mesh : targets) { 
```

       if (isMeshAnimated(mesh)) {<br />

       	mesh.prepareForAnim(!useHwSkinning); // prepare for software animation<br />

```
 } 
```
```
 } 
```
```
 if(useHwSkinning){ 
```

+

```
 } 
```
}

+
public boolean isUseHwSkinning(){
```
 return useHwSkinning; 
```
}

+
/**

Creates a skeleton control.
The list of targets will be acquired automatically when
the control is attached to a node.

@@ -102,44 +167,43 @@

return mesh.getBuffer(Type.BindPosePosition) != null;

}

private Mesh[] findTargets(Node node) {

private void findTargets(Node node, ArrayList targets, HashSet materials) {

Mesh sharedMesh = null;

   ArrayList<Mesh> animatedMeshes = new ArrayList<Mesh>();<br />

for (Spatial child : node.getChildren()) {
- if (!(child instanceof Geometry)) {
- continue; // could be an attachment node, ignore.
- }
-
- Geometry geom = (Geometry) child;
-
- // is this geometry using a shared mesh?
- Mesh childSharedMesh = geom.getUserData(UserData.JME_SHAREDMESH);
-
- if (childSharedMesh != null) {
- // Don't bother with non-animated shared meshes
- if (isMeshAnimated(childSharedMesh)) {
- // child is using shared mesh,
- // so animate the shared mesh but ignore child
- if (sharedMesh == null) {
- sharedMesh = childSharedMesh;
- } else if (sharedMesh != childSharedMesh) {
- throw new IllegalStateException("Two conflicting shared meshes for " + node);
- }
- }
- } else {
- Mesh mesh = geom.getMesh();
- if (isMeshAnimated(mesh)) {
- animatedMeshes.add(mesh);
- }
+ if (child instanceof Geometry) {
+ Geometry geom = (Geometry) child;
+
+ // is this geometry using a shared mesh?
+ Mesh childSharedMesh = geom.getUserData(UserData.JME_SHAREDMESH);
+
+ if (childSharedMesh != null) {
+ // Don't bother with non-animated shared meshes
+ if (isMeshAnimated(childSharedMesh)) {
+ // child is using shared mesh,
+ // so animate the shared mesh but ignore child
+ if (sharedMesh == null) {
+ sharedMesh = childSharedMesh;
+ } else if (sharedMesh != childSharedMesh) {
+ throw new IllegalStateException("Two conflicting shared meshes for " + node);
+ }
+ materials.add(geom.getMaterial());
+ }
+ } else {
+ Mesh mesh = geom.getMesh();
+ if (isMeshAnimated(mesh)) {
+ targets.add(mesh);
+ materials.add(geom.getMaterial());
+ }
+ }
+ } else if( child instanceof Node){
+ findTargets((Node)child,targets, materials);
}
}

if (sharedMesh != null) {
- animatedMeshes.add(sharedMesh);
+ targets.add(sharedMesh);
}
-
- return animatedMeshes.toArray(new Mesh[animatedMeshes.size()]);
}

@Override
@@ -147,30 +211,53 @@
super.setSpatial(spatial);
if (spatial != null) {
Node node = (Node) spatial;
- targets = findTargets(node);
+
+ HashSet mats = new HashSet();
+ ArrayList meshes = new ArrayList();
+ findTargets(node, meshes, mats);
+ targets = meshes.toArray(new Mesh[meshes.size()]);
+ materials = mats.toArray(new Material[mats.size()]);
+ //try hw skinning, will be reset to sw skinning if render call fails
+ setUseHwSkinning(true);
} else {
targets = null;
+ materials = null;
}
}

@Override
protected void controlRender(RenderManager rm, ViewPort vp) {
if (!wasMeshUpdated) {
- resetToBind(); // reset morph meshes to bind pose
-
- Matrix4f[] offsetMatrices = skeleton.computeSkinningMatrices();
-
- // if hardware skinning is supported, the matrices and weight buffer
- // will be sent by the SkinningShaderLogic object assigned to the shader
- for (int i = 0; i < targets.length; i++) {
- // NOTE: This assumes that code higher up
- // Already ensured those targets are animated
- // otherwise a crash will happen in skin update
- //if (isMeshAnimated(targets)) {
- softwareSkinUpdate(targets, offsetMatrices);
- //}
- }
+ if(useHwSkinning){
+ //preload scene to check if shader won't blow with too many bones
+ if(!triedHwSkinning){
+ triedHwSkinning = true;
+ try{
+ rm.preloadScene(this.spatial);
+ } catch (RendererException e){
+ //revert back to sw skinning for this model
+ setUseHwSkinning(false);
+ }
+ }
+ offsetMatrices = skeleton.computeSkinningMatrices();

+ hardwareSkinUpdate();
+ } else {
+ resetToBind(); // reset morph meshes to bind pose
+
+ offsetMatrices = skeleton.computeSkinningMatrices();
+
+ // if hardware skinning is supported, the matrices and weight buffer
+ // will be sent by the SkinningShaderLogic object assigned to the shader
+ for (int i = 0; i < targets.length; i++) {
+ // NOTE: This assumes that code higher up
+ // Already ensured those targets are animated
+ // otherwise a crash will happen in skin update
+ //if (isMeshAnimated(targets)) {
+ softwareSkinUpdate(targets[i], offsetMatrices);
+ //}
+ }
+ }
wasMeshUpdated = true;
}
}
@@ -180,13 +267,14 @@
wasMeshUpdated = false;
}

+ //only do this for software updates
void resetToBind() {
for (Mesh mesh : targets) {
if (isMeshAnimated(mesh)) {
- FloatBuffer bwBuff = (FloatBuffer) mesh.getBuffer(Type.BoneWeight).getData();
- ByteBuffer biBuff = (ByteBuffer)mesh.getBuffer(Type.BoneIndex).getData();
+ Buffer bwBuff = mesh.getBuffer(Type.BoneWeight).getData();
+ Buffer biBuff = mesh.getBuffer(Type.BoneIndex).getData();
if (!biBuff.hasArray() || !bwBuff.hasArray()) {
- mesh.prepareForAnim(true); // prepare for software animation
+ mesh.prepareForAnim(!useHwSkinning); // prepare for software animation
}
VertexBuffer bindPos = mesh.getBuffer(Type.BindPosePosition);
VertexBuffer bindNorm = mesh.getBuffer(Type.BindPoseNormal);
@@ -223,11 +311,14 @@
Node clonedNode = (Node) spatial;
AnimControl ctrl = spatial.getControl(AnimControl.class);
SkeletonControl clone = new SkeletonControl();
- clone.setSpatial(clonedNode);
-
clone.skeleton = ctrl.getSkeleton();
- // Fix animated targets for the cloned node
- clone.targets = findTargets(clonedNode);
+
+ clone.setSpatial(clonedNode);
+
+ //removed, done already in setSpatial
+// // Fix animated targets for the cloned node
+// clone.targets = findTargets(clonedNode);
+// clone.materials = findMaterials(clonedNode);

// Fix attachments for the cloned node
for (int i = 0; i < clonedNode.getQuantity(); i++) {
@@ -315,6 +406,17 @@

}
+
+ /**
+ * Update the mesh according to the given transformation matrices
+ * @param mesh then mesh
+ * @param offsetMatrices the transformation matrices to apply
+ */
+ private void hardwareSkinUpdate() {
+ for(Material m : materials){
+ m.setParam("BoneMatrices", VarType.Matrix4Array, offsetMatrices);
+ }
+ }

/**
* Method to apply skinning transforms to a mesh's buffers
@@ -574,6 +676,7 @@
OutputCapsule oc = ex.getCapsule(this);
oc.write(targets, "targets", null);
oc.write(skeleton, "skeleton", null);
+ oc.write(materials, "materials", null);
}

@Override
@@ -586,5 +689,10 @@
System.arraycopy(sav, 0, targets, 0, sav.length);
}
skeleton = (Skeleton) in.readSavable("skeleton", null);
+ sav = in.readSavableArray("materials", null);
+ if (sav != null) {
+ materials = new Material[sav.length];
+ System.arraycopy(sav, 0, materials, 0, sav.length);
+ }
}
}
[/patch]

SomethingNew · December 27, 2012, 11:28pm

Wow, so this code works great, but I’ve run across a bug. For one of my models that has multiple bones and several weights per vertex it animates as expected. For another model though that has one bone and therefore only one weight per vertex, the model is rotated and positioned incorrectly. I initially thought it was shader code, but I modified Skinning.glsllib to handle that case like this:

[java] #ifdef NUM_BONES

attribute vec4 inBoneWeight;
attribute vec4 inBoneIndex;
uniform mat4 m_BoneMatrices[NUM_BONES];

void Skinning_Compute(inout vec4 position){
#if NUM_WEIGHTS_PER_VERT == 1
position = m_BoneMatrices[int(inBoneIndex.x)] * position;
#else
mat4 mat = mat4(0.0);
mat += m_BoneMatrices[int(inBoneIndex.x)] * inBoneWeight.x;
mat += m_BoneMatrices[int(inBoneIndex.y)] * inBoneWeight.y;
mat += m_BoneMatrices[int(inBoneIndex.z)] * inBoneWeight.z;
mat += m_BoneMatrices[int(inBoneIndex.w)] * inBoneWeight.w;
position = mat * position;
#endif
}

void Skinning_Compute(inout vec4 position, inout vec3 normal){
#if NUM_WEIGHTS_PER_VERT == 1
position = m_BoneMatrices[int(inBoneIndex.x)] * position;
normal = (mat3(m_BoneMatrices[int(inBoneIndex.x)][0].xyz,
m_BoneMatrices[int(inBoneIndex.x)][1].xyz,
m_BoneMatrices[int(inBoneIndex.x)][2].xyz) * normal);
#else
mat4 mat = mat4(0.0);
mat += m_BoneMatrices[int(inBoneIndex.x)] * inBoneWeight.x;
mat += m_BoneMatrices[int(inBoneIndex.y)] * inBoneWeight.y;
mat += m_BoneMatrices[int(inBoneIndex.z)] * inBoneWeight.z;
mat += m_BoneMatrices[int(inBoneIndex.w)] * inBoneWeight.w;
position = mat * position;
normal = (mat3(mat[0].xyz,mat[1].xyz,mat[2].xyz) * normal);
#endif
}

void Skinning_Compute(inout vec4 position, inout vec4 tangent, inout vec3 normal){
#if NUM_WEIGHTS_PER_VERT == 1
position = m_BoneMatrices[int(inBoneIndex.x)] * position;
tangent = m_BoneMatrices[int(inBoneIndex.x)] * tangent;
normal = (mat3(m_BoneMatrices[int(inBoneIndex.x)][0].xyz,
m_BoneMatrices[int(inBoneIndex.x)][1].xyz,
m_BoneMatrices[int(inBoneIndex.x)][2].xyz) * normal);
#else
mat4 mat = mat4(0.0);
mat += m_BoneMatrices[int(inBoneIndex.x)] * inBoneWeight.x;
mat += m_BoneMatrices[int(inBoneIndex.y)] * inBoneWeight.y;
mat += m_BoneMatrices[int(inBoneIndex.z)] * inBoneWeight.z;
mat += m_BoneMatrices[int(inBoneIndex.w)] * inBoneWeight.w;
position = mat * position;
tangent = mat * tangent;
normal = (mat3(mat[0].xyz,mat[1].xyz,mat[2].xyz) * normal);
#endif
}
#endif
[/java]

I’ve been looking at the diff code for a while and I can’t figure it out. The one bone model animates correctly with the old software animation, but for this hardware animation code, this specific model is transformed completely off. Have any of you come across this behavior while working on this code?

The animated model with one bone is the parent node to the animated model with multiple bones and multiple weights. The multiple weighted model is animated correctly with this hardware skinning code, but the parent animated node is wrong. This works with software skinning though. Thanks for any help!

Momoko_Fan · December 28, 2012, 3:11am

There was a change done quite recently to support models that had vertices untransformed by any bones. If the first weight was set to 0, then the vertex should be passed through without any skinning. I am not sure if the hardware skinning code posted accounts for this.