Zombiegirl: GltfLoader vs MonkeyWrench

capdevon · November 14, 2024, 7:35pm

Hello everyone,
could you help me understand why the hair in the Zombie gltf model is not rendered?
Here is the original model downloadable from the Mixamo site.

Here is the gltf model rendered via the latest versions of GltfLoader and MonkeyWrench.

Here are the details of the geometries and materials of the model:

g[454] "ZombieGirl_Body-1" bucket=Transparent cull=Inherit/Dynamic
       mat"ZombieGirl_Material" def"PBR Lighting" dTest,dWrite,NOwireframe,blend=Alpha with 15 parms:
         BackfaceShadows: false
         BaseColor: rgb=0.8 a=0
         BaseColorMap: Models/Zombiegirl/gltf/zombie_diffuse.png (Flipped) (Mipmapped) mag:Bilinear min:BilinearNoMipMaps aniso:0 s:Repeat t:Repeat
         Emissive: rgb=0
         EmissiveIntensity: 1
         EmissivePower: 3
         Glossiness: 1
         Metallic: 0
         NormalMap: Models/Zombiegirl/gltf/zombie_normal.png (Flipped) (Mipmapped) mag:Bilinear min:BilinearNoMipMaps aniso:0 s:Repeat t:Repeat
         NormalType: 1
         ParallaxHeight: 0.05
         Roughness: 0.301511
         Specular: rgb=1
         UseSpecularAA: true
         UseVertexColor: true

Here is the java code I used to load the model:

        jme3utilities.debug.Dumper dumper = jme3utilities.debug.new Dumper();
        dumper.setDumpCull(true);
        dumper.setDumpBucket(true);
        dumper.setDumpMatParam(true);
        dumper.setDumpUser(true);

        if (useMonkeyWrench) {
            String[] extensions = { "3ds", "3mf", "blend", "bvh", "dae", "fbx", "glb", "gltf",
                    "lwo", "meshxml", "mesh.xml", "obj", "ply", "stl" };
            assetManager.registerLoader(LwjglAssetLoader.class, extensions);
            
            int ppFlags = Assimp.aiProcess_CalcTangentSpace
                    | Assimp.aiProcess_JoinIdenticalVertices
                    | Assimp.aiProcess_Triangulate
                    | Assimp.aiProcess_GenNormals
                    | Assimp.aiProcess_ValidateDataStructure
//                    | Assimp.aiProcess_RemoveRedundantMaterials
                    | Assimp.aiProcess_SortByPType;
            
            LwjglAssetKey key = new LwjglAssetKey(MODEL_PATH, ppFlags);
            key.setVerboseLogging(true);
            
            Spatial model = assetManager.loadModel(key);
            rootNode.attachChild(model);
            dumper.dump(model);
            
        } else {
            Spatial model = assetManager.loadModel(MODEL_PATH);
            rootNode.attachChild(model);
            dumper.dump(model);
        }

Could you tell me if this is a defect?
Or how to correct the material values to render the hair in a more believable and artistically acceptable way?

The 3D model files are downloadable from my github here: Zombiegirl

Note: It is not excluded that the Mixamo model may contain errors.
I would like to have an expert opinion on the result obtained. @sgold

Thanks for your help

Pixelapp · November 14, 2024, 10:04pm

Usually authoring programs add alpha blending to the hair to avoid harsh transitions.

I see this flag in your material: blend=Alpha.

As I develop in JMonkey Three types of alpha situations arise.

Opaque: No alpha.

Alpha clip: Used for leaves or grass. When you just want to NOT draw some parts of the image.

Alpha blend: This is used heavily for terrains. When adding roads, grass and dirt to a single mesh.

I suggest you load the model to blender and cycle between these three options so you learn how they work.

I used this troubleshooting technique just a few weeks ago to load a model that I exported from MakeHuman. The hair was showing translucent which was incorrect.

This is what my grass material looks like in JMonkey:

Material My Material : Common/MatDefs/Light/Lighting.j3md {
MaterialParameters {
Shininess : 1
DiffuseMap : “Models/sample_path/trees-02.png”
ParallaxHeight : 0.05
BackfaceShadows : false
AlphaDiscardThreshold : 0.2
}
AdditionalRenderState {
FaceCull Off
}
}

In this case a tackle the problem by using the alpha clip technique.

The material shows ‘AlphaDiscardThreshold : 0.2’. So it’s clipping any alpha below 0.2.

These settings should provide good enough results for your character’s hair.

You could achieve more complex results by using alpha blend. But I don’t suggest you use it just yet since you are learning.

sgold · November 16, 2024, 7:10pm

could you help me understand why the hair in the Zombie gltf model is not rendered?
Here is the original model downloadable from the Mixamo site.
Here is the gltf model rendered via the latest versions of GltfLoader and MonkeyWrench.

I can try. However, at this point I’m already confused.

The Mixamo site doesn’t offer a glTF download for this model, so I don’t know where your glTF came from.
GltfLoader and MonkeyWrench are 2 completely different things. I’m unsure which asset pipeline was used to generate your screenshot.
Your GitHub repo includes both glTF and FBX files, so I’m unsure which one you want me to look at. The FBX file in the repo is version 7700 (aka 7.7), which isn’t supported by either GltfLoader or MonkeyWrench, so hopefully you’re importing the glTF not the FBX.

As you’re probably aware, JME’s rendering pipeline has significant limitations when it comes to rendering transparent geometries: Alpha/Transparency Sorting, Your Z-buffer, and You . If you’re hitting up against those limitations, there isn’t much the importers can do.

I’m still investigating…

EDIT: I loaded the glTF from your repo into the glTF viewer at https://gltf-viewer.donmccurdy.com/ . It looks like the hair isn’t visible in the glTF, so I think there’s a problem with the conversion to glTF:

capdevon · November 17, 2024, 12:00pm

Hi @sgold ,

Thank you for your time and patience. I apologize for the previous confusion and will be more specific going forward.

I’m currently importing a 3D model in GLTF format. To achieve the desired result, it was sufficient to change the alpha value of the BaseColor parameter to 1, leaving the default values for BlendMode = Alpha and QueueBucket = Transparent. I even wrote a Materials editor built into the SDK to test all possible combinations of parameters.

I was not aware of this limitations, thanks for bringing it to my attention, I will keep it in mind.

However, due to recent changes in the GltfLoader with the latest engine version, I’m considering switching to MonkeyWrench in my pipeline.

As these tools have distinct approaches, I’ve noticed differences in the Material parameters for the Zombie’s hair (eg: Metallic, EmissiveIntensity, UseVertexColor…)

I’m curious which tool would provide more reliable and accurate Material parameter values to help identify potential bugs.
To facilitate comparison, I’ve attached both GltfLoader and MonkeyWrench output files for the ZombieGirl_Body_1 geometry Material.

Thank you for your expertise and assistance.

GltfLoader:

Material ZombieGirl_Body_1 : Common/MatDefs/Light/PBRLighting.j3md {

    MaterialParameters {
      EmissivePower : 3.0
      BackfaceShadows : false
      Roughness : 0.30151135
      EmissiveIntensity : 2.0
      Glossiness : 1.0
      BaseColor : 0.8 0.8 0.8 0.0
      ParallaxHeight : 0.05
      Metallic : 0.4
      NormalType : 1.0
      Emissive : 0.0 0.0 0.0 1.0
      NormalMap : WrapRepeat_S WrapRepeat_T MinBilinearNoMipMaps "Models/Zombiegirl/gltf/zombie_normal.png"
      BaseColorMap : WrapRepeat_S WrapRepeat_T MinBilinearNoMipMaps "Models/Zombiegirl/gltf/zombie_diffuse.png"
      Specular : 1.0 1.0 1.0 1.0
    }

    AdditionalRenderState {
      PointSprite On
      Blend Alpha
    }
}

MonkeyWrench:

Material MAT_ZombieGirl_Body-1 : Common/MatDefs/Light/PBRLighting.j3md {

    MaterialParameters {
      EmissivePower : 3.0
      BackfaceShadows : false
      Roughness : 0.30151135
      EmissiveIntensity : 1.0
      Glossiness : 1.0
      BaseColor : 0.8 0.8 0.8 0.0
      ParallaxHeight : 0.05
      Metallic : 0.0
      NormalType : 1.0
      Emissive : 0.0 0.0 0.0 1.0
      NormalMap : Flip WrapRepeat_S WrapRepeat_T MinBilinearNoMipMaps "Models/Zombiegirl/gltf/zombie_normal.png"
      BaseColorMap : Flip WrapRepeat_S WrapRepeat_T MinBilinearNoMipMaps "Models/Zombiegirl/gltf/zombie_diffuse.png"
      UseVertexColor : true
      Specular : 1.0 1.0 1.0 1.0
    }

    AdditionalRenderState {
      PointSprite On
      Blend Alpha
    }
}

Edit:
I don’t understand much about the format of the gltf file, but it seems that the correct Metallic value is 0.4f, instead MonkeyWrench reads it as 0f.

"materials": [
        {
            "name": "ZombieGirl_body_Material",
            "alphaMode": "OPAQUE",
            "extras": {
                "fromFBX": {
                    "shadingModel": "Phong",
                    "isTruePBR": false
                }
            },
            "normalTexture": {
                "index": 0,
                "texCoord": 0
            },
            "pbrMetallicRoughness": {
                "baseColorTexture": {
                    "index": 1,
                    "texCoord": 0
                },
                "baseColorFactor": [
                    0.800000011920929,
                    0.800000011920929,
                    0.800000011920929,
                    1.0
                ],
                "metallicFactor": 0.400000005960464,
                "roughnessFactor": 0.301511347293854
            }
        },
        {
            "name": "ZombieGirl_Material",
            "alphaMode": "BLEND",
            "extras": {
                "fromFBX": {
                    "shadingModel": "Phong",
                    "isTruePBR": false
                }
            },
            "normalTexture": {
                "index": 0,
                "texCoord": 0
            },
            "pbrMetallicRoughness": {
                "baseColorTexture": {
                    "index": 2,
                    "texCoord": 0
                },
                "baseColorFactor": [
                    0.800000011920929,
                    0.800000011920929,
                    0.800000011920929,
                    0.0
                ],
                "metallicFactor": 0.400000005960464,
                "roughnessFactor": 0.301511347293854
            }
        }
    ],

sgold · November 17, 2024, 6:59pm

I see what you mean about the “metallicFactor” being different in the materials imported by MonkeyWrench. I re-ran the import with verbose logging and confirmed that Assimp read a metallicFactor of 0.4 .

Looking at the logic of MaterialBuilder.createJmeMaterial() I see there’s an override that zeroes out PBR’s “Metallic” parameter when vertex colors are present in the mesh:

github.com

stephengold/MonkeyWrench/blob/039eba5081c9142e925a2e740b1452d89493a09b/lib/src/main/java/com/github/stephengold/wrench/MaterialBuilder.java#L298-L304


      
          // Use vertex colors if they're available:
          if (jmeMesh.getBuffer(VertexBuffer.Type.Color) != null) {
              result.setBoolean("UseVertexColor", true);
              if (isPbr) {
                  result.setFloat("Metallic", 0f);
              }
          }

By adding a println(), I confirmed that the setFloat() above is executed for each geometry in the Zombiegirl model.

I’m not a PBR expert, so I can’t say for certain whether this logic is correct. To me, the side-by-side renders of Zombiegirl with Metallic=0.4 and Metallic=0 look very similar.

“Git blame” indicates the logic was added on 8 October 2023 to address a bug in the BoxVertexColors test (from the glTF-Sample-Models repo). I re-ran that test with the setFloat() commented out, and it failed. (The GltfLoader in JME 3.7.0-stable passes that test.)

Bottom line: I don’t understand why GltfLoader and MonkeyWrench give different material parameters, and I’m not yet convinced it’s a problem. What do you think?

zzuegg · November 17, 2024, 8:08pm

The gltf specs say that color_0 is used as linear multiplier for baseColor. I don’t see why metallic should be set to 0 in this case. Not sure how close jmes pbr shader follows the specs

I checked the shader and it does implement vertex color 0 as specified.

I would expect the difference on a more metallic object to be more visible.

sgold · November 18, 2024, 3:21am

I’ll look into what GltfLoader is doing to pass BoxVertexColor without zeroing out the Metallic parameter.

zzuegg · November 18, 2024, 6:39am

At a first glance it just sets useVertexColor to true, in both, the mesh and the material.

In the wrench lib it seems you are not setting the boolean on the mesh. (I have checked materialbuilder and meshbuilder). Thats a difference. Not sure if it fixes the issue.

sgold · November 18, 2024, 10:19am

it seems you are not setting the boolean on the mesh.

I’ll look into what GltfLoader is doing to pass BoxVertexColor without zeroing out the Metallic parameter.

On the BoxVertexColors test, MonkeyWrench does set UseVertexColor=true, and GltfLoader does set Metallic=0.

MonkeyWrench sets the following parameters:

BackfaceShadows: false
BaseColor: rgb=1
Emissive: rgb=0
EmissiveIntensity: 1
EmissivePower: 3
Glossiness: 1
Metallic: 0
NormalType: -1
ParallaxHeight: 0.05
Roughness: 1
Specular: rgb=1
UseSpecularAA: true
UseVertexColor: true

while GltfLoader sets:

BackfaceShadows: false
BaseColor: rgb=1
EmissiveIntensity: 2
EmissivePower: 3
Glossiness: 1
Metallic: 0
NormalType: -1
ParallaxHeight: 0.05
Roughness: 1
Specular: rgb=1
UseSpecularAA: true
UseVertexColor: true

The only matparams that differ are Emissive and EmissiveIntensity. If Metallic=1, the test fails.

EDIT: Unlike Zombiegirl, BoxVertexColors doesn’t specify “metallicFactor”. When Assimp reads BoxVertexColors.gltf, it sets “$mat.metallicFactor” to 1. So apparently Assimp thinks metallicFactor defaults to 1.

zzuegg · November 18, 2024, 11:03am

Strictly spoken the specs state that the default value for metallic is 1.

However as some issues at the official tracker states:

github.com/KhronosGroup/glTF

The default values for metalness

opened 06:22PM - 03 Sep 17 UTC

closed 08:00PM - 23 Jan 19 UTC

javagl

specification

The default values for textures that are not present are defined to be 1.0, acco…rding to [the spec](https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#metallic-roughness-material) > If a texture is not given, all respective texture components within this material model are assumed to have a value of 1.0 The default value for `pbrMetallicRoughness.metallicFactor` is defined to be 1.0 as well (see https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#pbrmetallicroughnessmetallicfactor ) Yet, in the implementations that I have seen so far (e.g. also in the reference implementation at https://github.com/KhronosGroup/glTF-WebGL-PBR/blob/master/shaders/pbr-frag.glsl ), the computations for the diffuse colors usually involve lines like these: float metallic = u_MetallicRoughnessValues.x; metallic = valuesSampledFromMRTexture.b * metallic; diffuseColor *= 1.0 - metallic which basically means that a material that has **all default** values will barely be visible: Its diffuse color will be multiplied with 0.0. For example, an "empty", "default" material that only refers to a texture... ``` { "pbrMetallicRoughness" : { "baseColorTexture" : { "index" : 0 } } } ``` will basically be invisible. In order to see *anything*, one has to set the `metallicFactor` to 0.0: ``` { "pbrMetallicRoughness" : { "baseColorTexture" : { "index" : 0 }, "metallicFactor" : 0.0 } } ``` This doesn't seem like a sensible default to me. However, I know that it will hardly make sense to try and change this now, considering the implications on ratification etc. I just wanted to point this out, and ask whether there is a convincing reason of why these defaults have been chosen (except for some sort of "consistency" with other defaults, where the 1.0 makes perfectly sense)

the default value does not make sense since it will set the base color to 0.0.0

diffuseColor *= 1.0 - metallic

I have checked the BoxVertexColors.gltf and it does not seem to have any material parameters. At this point it seems eighter there is somewhere another hidden spec that changes the default values if vertex colors are used, or the khronos renderer does not follow the spec. Imho it should render as black cube if i read the specs correctly.

sgold · November 18, 2024, 11:10am

https://gltf-viewer.donmccurdy.com/ renders BoxVertexColors as a color cube:

zzuegg · November 18, 2024, 11:26am

So does the kronos renderer. but it has set metallic to 1. So i guess they use a different calculation.

The question is, if we want to follow the spec, or if we are happy with the results if the tests render correctly. At least, both importers should produce the same result to be consistent.

sgold · November 18, 2024, 11:54am

At least, both importers should produce the same result to be consistent.

The goal of MonkeyWrench is utility, not consistency with GltfLoader.

GltfLoader has a slew of known bugs and defects:

I have difficulty believing BoxVertexColors could be inconsistent with the glTF spec. To me, it seems more likely there’s a defect in the PBRLighting material and both GltfLoader and MonkeyWrench are working around that defect by setting Metallic=0.

zzuegg · November 18, 2024, 12:01pm

Point taken. With consitency i was referring to this usecase.
I agree that prblighting does something outside of the specs. If its due a missing feature, or due performance reasons it is to be explored.
At this point i cannot guess how a change to such a basic property will impact every model imported prior to this future fix. We will probably have to add a similar workaround in the j3o loader if we care about compability.

sgold · November 18, 2024, 12:14pm

Returning to the OP’s request:

It’s difficult to say which tool is more accurate for importing glTFs. On some models, MonkeyWrench does better. On others, GltfLoader might be preferred.

capdevon · November 21, 2024, 3:33pm

Thanks @sgold @zzuegg , I appreciate you taking the time to investigate my concern.