[SOLVED] Can't import shape keys properly

[Horace_Mahoney]

Guys, hello. Why can’t i import my model with shape keys in gltf (from blender) properly. It’s always “RendererException: Cannot recompile shader source” or “RendererException: compile error in: ShaderSource”. But if I uncheck normals in gltf export settings, everything is fine, but the model becomes solid black and I can’t do anything.

https://ibb.co/cr378Lv

SEVERE: Uncaught exception thrown in Thread[jME3 Main,5,main]
com.jme3.renderer.RendererException: compile error in: ShaderSource[name=Common/MatDefs/Light/Lighting.vert, defines, type=Vertex, language=GLSL150]
0(459) : error C1111: non-lvalue actual parameter #2 cannot be out parameter (“norm.28”)

at com.jme3.renderer.opengl.GLRenderer.updateShaderSourceData(GLRenderer.java:1476)
at com.jme3.renderer.opengl.GLRenderer.updateShaderData(GLRenderer.java:1503)
at com.jme3.renderer.opengl.GLRenderer.setShader(GLRenderer.java:1567)
at com.jme3.material.logic.MultiPassLightingLogic.render(MultiPassLightingLogic.java:158)
at com.jme3.material.Technique.render(Technique.java:166)
at com.jme3.material.Material.render(Material.java:1026)
at com.jme3.renderer.RenderManager.renderGeometry(RenderManager.java:614)
at com.jme3.renderer.queue.RenderQueue.renderGeometryList(RenderQueue.java:266)
at com.jme3.renderer.queue.RenderQueue.renderQueue(RenderQueue.java:305)
at com.jme3.renderer.RenderManager.renderViewPortQueues(RenderManager.java:877)
at com.jme3.renderer.RenderManager.flushQueue(RenderManager.java:779)
at com.jme3.renderer.RenderManager.renderViewPort(RenderManager.java:1108)
at com.jme3.renderer.RenderManager.render(RenderManager.java:1158)
at com.jme3.app.SimpleApplication.update(SimpleApplication.java:272)
at com.jme3.system.lwjgl.LwjglAbstractDisplay.runLoop(LwjglAbstractDisplay.java:151)
at com.jme3.system.lwjgl.LwjglDisplay.runLoop(LwjglDisplay.java:197)
at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:232)
at java.base/java.lang.Thread.run(Thread.java:834)

My simpleInitApp():
public void simpleInitApp() {
setTimer(new EraseTimer());

    viewPort.setBackgroundColor(ColorRGBA.LightGray);
    rootNode.addLight(new DirectionalLight(new Vector3f(-1, -1, -1).normalizeLocal()));

    Spatial model = assetManager.loadModel("Models/untitled.glb");

    File storageFolder = JmeSystem.getStorageFolder();
    file = new File(storageFolder.getPath() + File.separator + "g8m.j3o");
    BinaryExporter be = new BinaryExporter();
    try {
        be.save(model, file);
    } catch (IOException e) {
        e.printStackTrace();
    }

    assetManager.registerLocator(storageFolder.getPath(), FileLocator.class);
    model2 = assetManager.loadModel("g8m.j3o");
    model2.setLocalTranslation(0, 0, 0);
    model2.depthFirstTraversal(new SceneGraphVisitor() {
    @Override
    public void visit(Spatial spatial) {
        if (spatial instanceof Geometry) {
            if (((Geometry) spatial).getMesh().hasMorphTargets()) {
                System.out.println("Found morphs: " + ((Geometry) spatial).getMesh().getMorphTargets().length);
                myGeom = (Geometry)spatial;
                weights = new float[((Geometry) spatial).getMesh().getMorphTargets().length]; 
                myGeom.setMorphState(weights);
            }
        }
    }
});

    Material sphereMat = new Material(assetManager,
    "Common/MatDefs/Light/Lighting.j3md");
    sphereMat.setBoolean("UseMaterialColors",true);
    model2.setMaterial(sphereMat);
    model2.setLocalTranslation(0, 0, 8);
    rootNode.attachChild(model2);

}

After unchecking normals:

or i can get:
SEVERE: Uncaught exception thrown in Thread[jME3 Main,5,main]
com.jme3.renderer.RendererException: Cannot recompile shader source
at com.jme3.renderer.opengl.GLRenderer.updateShaderSourceData(GLRenderer.java:1363)
at com.jme3.renderer.opengl.GLRenderer.updateShaderData(GLRenderer.java:1503)
at com.jme3.renderer.opengl.GLRenderer.setShader(GLRenderer.java:1567)
at com.jme3.material.logic.MultiPassLightingLogic.render(MultiPassLightingLogic.java:158)
at com.jme3.material.Technique.render(Technique.java:166)
at com.jme3.material.Material.render(Material.java:1026)
at com.jme3.renderer.RenderManager.renderGeometry(RenderManager.java:614)
at com.jme3.renderer.queue.RenderQueue.renderGeometryList(RenderQueue.java:266)
at com.jme3.renderer.queue.RenderQueue.renderQueue(RenderQueue.java:305)
at com.jme3.renderer.RenderManager.renderViewPortQueues(RenderManager.java:877)
at com.jme3.renderer.RenderManager.flushQueue(RenderManager.java:779)
at com.jme3.renderer.RenderManager.renderViewPort(RenderManager.java:1108)
at com.jme3.renderer.RenderManager.render(RenderManager.java:1158)
at com.jme3.app.SimpleApplication.update(SimpleApplication.java:272)
at com.jme3.system.lwjgl.LwjglAbstractDisplay.runLoop(LwjglAbstractDisplay.java:151)
at com.jme3.system.lwjgl.LwjglDisplay.runLoop(LwjglDisplay.java:197)
at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:232)
at java.base/java.lang.Thread.run(Thread.java:834)

[Ali_RS]

Hi

@Horace_Mahoney

Material sphereMat = new Material(assetManager, “Common/MatDefs/Light/Lighting.j3md”);

Shape keys are not supported in Lighting.j3md material, you should use PBRLighting.j3md.

@Horace_Mahoney

error C1111: non-lvalue actual parameter #2 cannot be out parameter

Yeah, that’s because morph animation (shape key) is currently broken in JME 3.3.0-beta1. Please see this issue for details :

Common/MatDefs/Light/PBRLighting.vert exception in 3.3.0-beta1(after switch from alpha5)

It’s already solved in master branch so you need to compile engine from the master branch or wait for 3.3.0-beta2 release.

[Horace_Mahoney]

@Ali_RS

PBRLighting

Thank you for your answer.
I changed the material, but still have an exception:

com.jme3.renderer.RendererException: compile error in: ShaderSource[name=Common/MatDefs/Light/PBRLighting.vert, defines, type=Vertex, language=GLSL150]
0(417) : error C1111: non-lvalue actual parameter #2 cannot be out parameter (“norm.11”)

at com.jme3.renderer.opengl.GLRenderer.updateShaderSourceData(GLRenderer.java:1476)

[Ali_RS]

@Horace_Mahoney

I changed the material, but still have an exception:

Please read my above post.

[Horace_Mahoney]

Oh, im sorry. Thanks for help

[oxplay2]

was mentioned that need use GLTF exporter too?

Also please note shape keys can be used manually via code,
but i was unable to find a way how to use Blender shape keys animations.
(maybe it was fixed by GLTF/etc - i didnt check it last time)

Anyway you can do things like lipsync, my example:

character lipsync - jMonkeyEngine.

Like Ali already said, you just need use 3.3-alpha5 or 3.3-beta2 version or above. In blender i dont remember, but i belive you also needed to have/remove “current shape key action selected” to make GLTF export them, but they might also already fix it (since it was long time ago). Same as i noticed they fixed GLTF exporter animation sampling, so no need bake IK animations manually no more.

[Horace_Mahoney]

Actually was inspired by your example a couple of days ago. It is a pity that it is not described in the documentation.

[mitm]

@Horace_Mahoney

It is a pity that it is not described in the documentation.

It does now.
https://wiki.jmonkeyengine.org/jme3/external/blender/blender_gltf.html#shape-keys

[oxplay2]

here is full link that explain even more(about lipsync itself):

LipSync / Sound amplitude/data listener

For anyone who will look how to do it. Here simple free solution! Found a nice app, that is generally under MIT. It works on both Windows and Linux and seems like work very well. You just do(Windows): .\rhubarb.exe -o output.txt .\tmp\guess-what-2.wav or(Linux) ./rhubarb -o output.txt ./tmp/guess-what-2.wav and you get letters where each letter mean some mouth shape(described on github page) example: 0.00 X 0.05 B 0.19 D 0.40 B 0.54 F 1.03 B 1.31 X 1.46 X so later j…

about shape keys, @mitm i see added nice documentation.
There are missing some subpoints like:

• currently only manual usage of shape keys allowed (due to gltf or jme animation not export?)
• shape keys can be selected by index(but there was commit to select them by name, idk if its in beta versions already)

[mitm]

@oxplay2

currently only manual usage of shape keys allowed

Manual as in how?

@oxplay2

shape keys can be selected by index

Again, how to do so?

I haven’t used gltf animations with shape keys before. I like the concept.

[oxplay2]

Manual as in how?

i mean you cant do animComposer.setCurrentAnimation(“shapeKeyAnimation”) but you need do something similar like morphs.get(i).setWeight(weight).

Again, how to do so?

I haven’t used gltf animations with shape keys before. I like the concept.

im not in work now, but it was like .get(i) just to get morph. There was commit for getting by Name, so it is also probably now available.

[Ali_RS]

@oxplay2

i mean you cant do animComposer.setCurrentAnimation(“shapeKeyAnimation”)

Of course, you can do it.

To get sure I made a simple test in Blender 2.81 and added two shape key animations to it. And I can play it in JME using animComposer.setCurrentAnimation() just fine.

I can upload the Gltf file with the example code in case you want to take a look.

Note I am using Gltf addon v1.2.33.

[oxplay2]

oh, when i were playing arround i remember not only i was unable to play shape key animations.

Maybe it was fixed in later time.

Good to know! You could just show Blender configuration, maybe i also had something wrong there.

[Ali_RS]

@oxplay2

Maybe it was fixed in later time.

Yeah, probably.

@oxplay2

Good to know! You could just show Blender configuration, maybe i also had something wrong there.

I created a simple test based on this tutorial on YT:

How To Create Shape Keys In Blender 2.8

and then:
DopeSheet → ShapeKeyEditor → PushDown

That’s it.

[oxplay2]

thanks! good to know it works.

Then i dont feel like GLTF / animation system have issues at all. It works fine, without issues now.
(earlier IK also needed to be baked, now GLTF “sample animations” works fine for IK animations thats nice)

[mitm]

Added the video. Nice.

[Ali_RS]

Thought to mention this just in case, I am not the author of video. It is done by someone else.

[oxplay2]

yeah, i noticed this, initially also thought its yours, but title definetly refer only to Blender.

But still, it would be nice have someone in JME who have good voice / courage to make YT video tutorial series too. (more like a advertisment of engine, not only just to help new people)

[Ali_RS]

@oxplay2

initially also thought its yours

Sorry that my post was misleading, I edited my post.