Light Control Fixes and GLTF Punctual Light and Unlit Material Extensions

Markil3 · January 9, 2021, 9:17pm

Continuing the discussion from [SOLVED] Issues converting unlit textures in GLTF 2.0 from Blender 2.8:

After trying to create a skybox for my game, I found out that jMonkey’s GLTF importer didn’t support unlit materials. Since I needed it at the time, I created a plugin that would export any unlit materials to use JME’s Unshaded.j3md. After that, I also worked a bit to allow jMonkey to import lights via the KHR_lights_punctual plugin. I made a fork with the two relevant commits, if anyone wants to check them out.

At the moment, the main issue is that shadows are rather wonky for point lights (either shadows don’t show up at all, or shadows will show up through different layers). Also, many of the lights I export from Blender end up being pretty powerful in JME, so I had the importer scale the spot and point lights down as needed. However, they may need more tweaking.

github.com

Markil3/jmonkeyengine/blob/8306bf8d730cfe663aaa17a960b3b4b34eb5e87b/jme3-plugins/src/gltf/java/com/jme3/scene/plugins/gltf/PunctualExtensionLoader.java#L95


lightOb = el.getAsJsonObject();
color = GltfUtils.getAsColor(lightOb, "color", ColorRGBA.White).multLocal(Optional.ofNullable(lightOb.get("intensity")).map(JsonElement::getAsFloat).orElse(1F));
switch (lightOb.get("type").getAsString()) {
    case "directional":
        light = new DirectionalLight(new Vector3f(), color);
        break;
    case "spot":
        /*
         * The high values that Blender exports can be pretty powerful in JME. Scale them down a bit.
         */
        light = new SpotLight(new Vector3f(), new Vector3f(), color.mult(1F / 1000F));
        if (lightOb.has("range")) {
            ((SpotLight) light).setSpotRange(lightOb.get("range").getAsFloat());
        }
        ((SpotLight) light).setSpotInnerAngle(lightOb.get("spot").getAsJsonObject().get("innerConeAngle").getAsFloat());
        ((SpotLight) light).setSpotOuterAngle(lightOb.get("spot").getAsJsonObject().get("outerConeAngle").getAsFloat());
        break;
    case "point":
        /*
         * The high values that Blender exports can be pretty powerful in JME. Scale them down a bit.
         */

tlf30 · January 9, 2021, 9:24pm

I submitted a PR for this a while ago: Added KHR_lights_punctual gltf extension by tlf30 · Pull Request #1443 · jMonkeyEngine/jmonkeyengine · GitHub

I have not fully tested it yet. Perhaps you may be interested in it. If you see any improvements please let me know as I want to get this merged in next week.

Markil3 · January 9, 2021, 9:51pm

The lights aren’t displaying with either my test scene or yours.

One issue I am seeing is that Blender will export its lights as a node containing the positioning data, along with a subnode containing the actual light data (or rather, the link to the actual light type). Since jmonkey’s lights only work on objects that are within the name node that they are in, I had my importer just delete those nodes and add the light to the parent of the positioning node.

Scene
	Camera
	Floor - Material[name=Floor, def=PBR Lighting, tech=null]
	Lamp - Material[name=Metal, def=PBR Lighting, tech=null]
	Point
		Point_Orientation
			Point - com.jme3.light.PointLight@300e0c18
	Lamp - Material[name=Metal, def=PBR Lighting, tech=null]
	Point.001
		Point.001_Orientation
			Point.001 - com.jme3.light.PointLight@4201a936
	Floor - Material[name=Floor, def=PBR Lighting, tech=null]
	Cylinder.001 - Material[name=Metal, def=PBR Lighting, tech=null]
	Spot
		Spot_Orientation
			Spot - com.jme3.light.SpotLight@3958275a
	Sphere - Material[name=Disko, def=PBR Lighting, tech=null]
	Sun
		Sun_Orientation
			Sun - DirectionalLight[name=Sun, direction=(-0.0, -0.0, -1.0), color=Color[-0.10587825, -0.10587825, -0.10587825, 0.42457518], enabled=true]
	Lamp - Material[name=Metal, def=PBR Lighting, tech=null]
	Point.002
		Point.002_Orientation
			Point.003 - com.jme3.light.PointLight@6c5780d7
	Lamp - Material[name=Metal, def=PBR Lighting, tech=null]
	Point.003
		Point.003_Orientation
			Point.002 - com.jme3.light.PointLight@2cf92f89
	Cylinder.001 - Material[name=Metal, def=PBR Lighting, tech=null]
	Spot.001
		Spot.001_Orientation
			Spot.002 - com.jme3.light.SpotLight@637e6b39
	Sphere - Material[name=Disko, def=PBR Lighting, tech=null]
	Cylinder.002 - Material[name=Shadeless, def=PBR Lighting, tech=null]

On the other hand, your test scene just has that single node containing the extension data. However, your importer will put the light within that single node, so that doesn’t appear to work, either.

scene
	point_light
		Point - com.jme3.light.PointLight@34adff56
	directional_light
		Directional - DirectionalLight[name=Directional, direction=(-0.0, -0.0, -1.0), color=Color[-0.094108455, -0.084697604, -0.06587592, 0.5945001], enabled=true]
	MODEL_ROUNDED_CUBE_PART_1 - Material[name=Rounded Cube Material, def=PBR Lighting, tech=null]

Considering that the GLTF standard appears to need a node in order to implement a light, I would recommend at minimum removing the node that stores the light data and just add the light to its parent. As for Blender scenes, though, I’m not sure. I have noticed before that Blender does wrap its stuff within extra nodes when I try to the use jme3-blender module, so this may just be a quirk with Blender exports in general and outside the control of jMonkey.

Wait, does this mean we are starting 3.4 releases next week, or is it just something you want to get out of the way?

tlf30 · January 9, 2021, 9:59pm

Interesting, do you know why?

That was something I had noticed, but considering that people are working on support for linked nodes in gltf from blender, I do not think it is wise to remove nodes. The nodes have basically no extra overhead, and users should expect to see them if they are familiar with gltf. If there are no impacts with removing the node as far as linked objects in blender are concerned, then perhaps they can be removed, but otherwise I can see more issues with the node missing, when it does not gain anything to remove it.

I will be back to a computer with a gpu in it next week and can do more testing, but after next week i will be gone for three weeks before I can look at it again.

Markil3 · January 9, 2021, 10:07pm

It is because the lights are attached within the nodes that contain the light data. Lights only shine on objects that are within the same node as the light (hence why lights are usually attached to the root node, so they can shine on everything). Since the lights are attached to a subnode, the light can’t really shine on anything.

Well, we should at least attach the light to the node’s parent (often the scene node) then.

github.com

tlf30/jmonkeyengine/blob/e2ea80eb69de0c608eb578d3ada014f7ffe8d222/jme3-plugins/src/gltf/java/com/jme3/scene/plugins/gltf/LightsPunctualExtensionLoader.java#L189


      
              PointLight pointLight = new PointLight();
              pointLight.setName(name);
              pointLight.setColor(color);
              pointLight.setRadius(range);
          
              return pointLight;
          }
          
          private void addLight(Node node, int light) {
              if (lightDefinitions.containsKey(light)) {
                  node.addLight(lightDefinitions.get(light));
              } else {
                  throw new AssetLoadException("KHR_lights_punctual extension accessed undefined light at index " + light);
              }
          }
          
          private ColorRGBA lumensToColor(ColorRGBA color, float lumens) {
              ColorRGBA brightnessModifier = lumensToColor(lumens);
              return color.mult(brightnessModifier);
          }

If I understand your code correctly, changing that line to node.getParent().addLight(lightDefinitions.get(light)); should do the trick, at least for your scenes.
As for anything exported from Blender, some extra work will have to be done from the game developer, but that’s just Blender.

tlf30 · January 9, 2021, 10:18pm

Ah, i had forgotten that. That makes sense. Hmm. I will to think about the node issue but I think you are correct. Perhaps only attach to the model parent? But idk yet.

Markil3 · January 9, 2021, 10:20pm

For my implementation, I just attached it to the light node’s parent. That way, just in case the user wants to limit what nodes the light will be effective on for whatever reason, that behavior will be consistent with JME.
Of course, JME’s lighting behavior is a bit unusual, so attaching it to the scene itself for consistency with every 3D editor makes sense as well.

tlf30 · January 9, 2021, 10:39pm

Yeah, even for my use I would want it on the root node. But I would not expect a imported model to modify anything outside of its node, much less the root node. Too bad there is no way to pass settings to the importer and make it configurable.

Perhaps @pspeed or @sgold may have an idea.

Edit: I will make your proposed change to my PR tonight so we can test it, would you be willing to test it before I get back to a modern computer next week?

Markil3 · January 9, 2021, 10:53pm

Yeah, just give me a few minutes…

tlf30 · January 9, 2021, 10:56pm

Ok, I’m probably over an hour before I get to a computer, currently on a phone. But feel free to make the change and test. How does the FBX importer do lights? or does it even support lights?

Markil3 · January 9, 2021, 11:08pm

The good news is that lights are properly present.
The bad news is that, a lot of the time, they seem too weak to even be visible.
I think it may need a bit of tweaking (the lumenToColor method, in particular).
Of course, the brightness calculations were rather wonky for my implementation as well, so it isn’t just your code.

Markil3 · January 9, 2021, 11:09pm

I’m not familiar with this.

tlf30 · January 9, 2021, 11:18pm

Yeah, I winged it based off how the shader does it, but I had a feeling that it would be a bit off.

sgold · January 10, 2021, 12:05am

I’m no expert on asset importers, but I believe there’s a way to pass settings. It involves defining a subclass of ModelKey that includes the settings, then passing that object to AssetManager.loadAsset() instead of a bare String. The AssetLoader then uses AssetInfo.getKey() to access the settings.

tlf30 · January 10, 2021, 1:38am

Do you have any input on which node the light should be attached?

tlf30 · January 10, 2021, 1:45am

I think that is a good idea, I will get investigate adding a GltfModelKey that will support settings for extensions as well as the gltf plugin itself.

pspeed · January 10, 2021, 3:49am

People see spatial.addLight() and think that the are somehow “attaching” the light to the node. That’s not the case.

In JME, the only way to attach a light to a node is with a control. That node will then position, rotate, etc. the light.

What spatial.addLight() is doing is saying “I want this spatial afffected by this light.” It doesn’t matter where that light is “attached” in the scene graph… and you can make any random spatial in the scene be affected by that light with addLight(). Scene graph structure doesn’t matter, really.

I think if you want to unwind the best way to deal with lights, it’s important to consider this distinction. Having the imported lights be a child of a node may be appropriate… but then they should also all be added to the root of the model at minimum… which is not the same as “attaching”.

If the user wants to promote those lights to their whole scene then they can grab them from the root node of the model.

tlf30 · January 14, 2021, 2:17am

@Markil3 sorry for the delay. I’m finally back home for a few days!
I have made some changes. I now attach the light to the model root. The light node now has a LightControl attached to it linked to the light.

The developer will have to move the light to the desired node for lighting within the scene. The main disadvantage to this approach is that the lights will still be in the scene if the model is removed and the lights were moved to a node outside of the model.

Can you please test it and make sure it works as desired on your end?

Markil3 · January 15, 2021, 10:21pm

I’m not sure if I’m doing something wrong, or if there is something else going on. I copy your test scene from jme3-plugins/src/test/resources/gltf into jme3-examples/src/main/resources and run the code below, but all I get is this black cube.

However, by playing with your lumensToColor methods and having them just return the base color without fooling around with intensity at all, I am able to achieve at least the red point light:

Also, in your scene, the directional light doesn’t show up until I add a “matrix” property to the node (I copied the one from the point light).
In short, I think the main thing is to fool around with the lumensToColor methods, otherwise the lights are simply just too weak.

package jme3test.light;

import com.jme3.app.SimpleApplication;
import com.jme3.input.controls.ActionListener;
import com.jme3.light.DirectionalLight;
import com.jme3.light.Light;
import com.jme3.light.PointLight;
import com.jme3.light.SpotLight;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Vector3f;
import com.jme3.renderer.queue.RenderQueue;
import com.jme3.scene.Geometry;
import com.jme3.scene.Node;
import com.jme3.scene.Spatial;
import com.jme3.scene.control.Control;
import com.jme3.shadow.DirectionalLightShadowRenderer;
import com.jme3.shadow.EdgeFilteringMode;
import com.jme3.shadow.PointLightShadowRenderer;
import com.jme3.shadow.SpotLightShadowRenderer;

public class TestGltfPunctual extends SimpleApplication implements ActionListener {
    private Node scene;
    public static void main(String[] args) {
        TestGltfPunctual testUnlit = new TestGltfPunctual();
        testUnlit.start();
    }

    @Override
    public void simpleInitApp() {
        final int SHADOWMAP_SIZE = 2048;
        Node scene;
        DirectionalLightShadowRenderer dlsr;
        SpotLightShadowRenderer slsr;
        PointLightShadowRenderer plsr;

        this.cam.setLocation(new Vector3f(0, 10, 20));

        scene = (Node) this.getAssetManager().loadModel("gltf/lights/lights.gltf");
//        scene.setShadowMode(RenderQueue.ShadowMode.CastAndReceive);
        this.rootNode.attachChild(scene);
//        for (Light light : scene.getLocalLightList()) {
//            System.out.println("Adding processor for " + light);
//            if (light instanceof DirectionalLight) {
//                dlsr = new DirectionalLightShadowRenderer(assetManager, SHADOWMAP_SIZE, 3);
//                dlsr.setLight((DirectionalLight) light);
//                dlsr.setLambda(0.55f);
//                dlsr.setShadowIntensity(0.8f);
//                dlsr.setEdgeFilteringMode(EdgeFilteringMode.Nearest);
//                viewPort.addProcessor(dlsr);
//            }
//            else if (light instanceof SpotLight) {
//                slsr = new SpotLightShadowRenderer(assetManager, 512);
//                slsr.setLight((SpotLight) light);
//                slsr.setShadowIntensity(0.8f);
//                slsr.setShadowZExtend(100);
//                slsr.setShadowZFadeLength(5);
//                slsr.setEdgeFilteringMode(EdgeFilteringMode.PCFPOISSON);
//                viewPort.addProcessor(slsr);
//            }
//            else if (light instanceof PointLight) {
//                plsr = new PointLightShadowRenderer(assetManager, SHADOWMAP_SIZE);
//                plsr.setLight((PointLight) light);
//                plsr.setEdgeFilteringMode(EdgeFilteringMode.PCFPOISSON);
//                plsr.setShadowZExtend(100);
//                plsr.setShadowZFadeLength(5);
//                plsr.setShadowIntensity(0.8f);
//                // plsr.setFlushQueues(false);
//                //plsr.displayFrustum();
//                viewPort.addProcessor(plsr);
//            }
//        }
//        this.viewPort.detachScene(this.rootNode);
//        this.viewPort.attachScene(scene);

        this.debugShadowMode(scene);

        ColorRGBA skyColor = new ColorRGBA(0.5f, 0.6f, 0.7f, 0.0f);

        flyCam.setMoveSpeed(20);
        viewPort.setBackgroundColor(skyColor.mult(0.9f));
    }

    private void debugShadowMode(Spatial scene) {
        debugShadowMode(scene, 0);
    }

    private void debugShadowMode(Spatial scene, int level) {
        Control control;
        for (int i = 0; i < level; i++) {
            System.out.print('\t');
        }
        System.out.print(scene.getName());
        if (scene instanceof Geometry)
        System.out.print(" - " + ((Geometry) scene).getMaterial());
        System.out.println();
        for (Light child : scene.getLocalLightList()) {
            for (int i = 0; i < level + 1; i++) {
                System.out.print('\t');
            }
            System.out.println(child.getName() + " - " + child);
            /*
             * Adding this line doesn't seem to make a difference.
             */
            this.rootNode.addLight(child);
        }
        for (int i = 0, l = scene.getNumControls(); i < l; i++) {
            for (int j = 0; j < level + 1; j++) {
                System.out.print('\t');
            }
            control = scene.getControl(i);
            System.out.println(control);
        }
        if (scene instanceof Node) {
            for (Spatial child : ((Node) scene).getChildren()) {
                debugShadowMode(child, level + 1);
            }
        }
    }

    @Override
    public void onAction(String name, boolean isPressed, float tpf) {
    }
}

Output:

scene
	Point - com.jme3.light.PointLight@63138cc2
	Directional - DirectionalLight[name=Directional, direction=(-0.0, -0.0, -1.0), color=Color[-0.094108455, -0.084697604, -0.06587592, 0.5945001], enabled=true]
	point_light
		com.jme3.scene.control.LightControl@4d43450a
	directional_light
		com.jme3.scene.control.LightControl@60765650
	MODEL_ROUNDED_CUBE_PART_1 - Material[name=Rounded Cube Material, def=PBR Lighting, tech=null]

tlf30 · January 15, 2021, 10:57pm

In the particular instance of the test example, the light will be inside the cube, IDK if that would cause it to not show, perhaps it would be better to have a different test model.

Can you elaborate on the matrix to make the ambient light work, I’m not sure what you mean.

I do believe that the calculation for the intensity may be off. That is what the shader is using so I am not sure why it is off. Do you have another model to test with?