NPE when Ray casting with InstancedNode

Hello,

I am trying to cast pick rays to collide with parts of my scene graph, which includes an InstancedNode, but get a NullPointerException due to missing world bounds.

Is this expected?

Simple example - click to cause the NPE:

import com.jme3.app.SimpleApplication;
import com.jme3.collision.CollisionResult;
import com.jme3.collision.CollisionResults;
import com.jme3.font.BitmapText;
import com.jme3.input.MouseInput;
import com.jme3.input.controls.ActionListener;
import com.jme3.input.controls.MouseButtonTrigger;
import com.jme3.light.AmbientLight;
import com.jme3.light.PointLight;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Ray;
import com.jme3.math.Vector3f;
import com.jme3.scene.Geometry;
import com.jme3.scene.instancing.InstancedNode;
import com.jme3.scene.shape.Box;
import com.jme3.scene.shape.Sphere;


// Based on distance from camera, swap in/out more/less detailed geometry to/from an InstancedNode.
public class TestInstancedNodeAttachDetachWithPicking extends SimpleApplication {
    public static void main(String[] args) {
        TestInstancedNodeAttachDetachWithPicking app = new TestInstancedNodeAttachDetachWithPicking();
        app.setShowSettings(false);  // disable the initial settings dialog window
        app.start();
    }

    private InstancedNode instancedNode;

    private Vector3f[] locations = new Vector3f[10];
    private Geometry[] spheres = new Geometry[10];
    private Geometry[] boxes = new Geometry[10];

    @Override
    public void simpleInitApp() {
        addPointLight();
        addAmbientLight();

        Material material = createInstancedLightingMaterial();

        instancedNode = new InstancedNode("theParentInstancedNode");
        rootNode.attachChild(instancedNode);

        // create 10 spheres & boxes, positioned along Z-axis successively further from the camera
        for (int i = 0; i < 10; i++) {
            Vector3f location = new Vector3f(0, -3, -(i*5));
            locations[i] = location;

            Geometry sphere = new Geometry("sphere", new Sphere(16, 16, 1f));
            sphere.setMaterial(material);
            sphere.setLocalTranslation(location);
            sphere.getMesh().createCollisionData();
            instancedNode.attachChild(sphere);       // initially just add the spheres to the InstancedNode
            spheres[i] = sphere;

            Geometry box = new Geometry("box", new Box(0.7f, 0.7f, 0.7f));
            box.setMaterial(material);
            box.setLocalTranslation(location);
            box.getMesh().createCollisionData();
            boxes[i] = box;
        }
        instancedNode.instance();

        flyCam.setMoveSpeed(30);


        addCrossHairs();

        // when you left-click, print the distance to the object to system.out
        inputManager.addMapping("leftClick", new MouseButtonTrigger(MouseInput.BUTTON_LEFT));
        inputManager.addListener(new ActionListener() {
            @Override
            public void onAction(String name, boolean isPressed, float tpf) {
                if( isPressed ) {
                    CollisionResult result = pickFromCamera();
                    if( result != null ) {
                        System.out.println("Distance = "+result.getDistance());
                    }
                }
            }
        }, "leftClick");
    }

    @Override
    public void simpleUpdate(float tpf) {
        // Each frame, determine the distance to each sphere/box from the camera.
        // If the object is > 25 units away, switch in the Box.  If it's nearer, switch in the Sphere.
        // Normally we wouldn't do this every frame, only when player has moved a sufficient distance, etc.


        for (int i = 0; i < 10; i++) {
            Vector3f location = locations[i];
            float distance = location.distance(cam.getLocation());

            instancedNode.detachChild(boxes[i]);
            instancedNode.detachChild(spheres[i]);

            if( distance > 25.0f ) {
                instancedNode.attachChild(boxes[i]);
            } else {
                instancedNode.attachChild(spheres[i]);
            }
        }

        instancedNode.instance();
    }

    private Material createInstancedLightingMaterial() {
        Material material = new Material(assetManager, "Common/MatDefs/Light/Lighting.j3md");
        material.setBoolean("UseMaterialColors", true);
        material.setBoolean("UseInstancing", true);
        material.setColor("Ambient", ColorRGBA.Red);
        material.setColor("Diffuse", ColorRGBA.Red);
        material.setColor("Specular", ColorRGBA.Red);
        material.setFloat("Shininess", 1.0f);
        return material;
    }

    private void addAmbientLight() {
        AmbientLight ambientLight = new AmbientLight(new ColorRGBA(0.2f, 0.2f, 0.2f, 1.0f));
        rootNode.addLight(ambientLight);
    }

    private void addPointLight() {
        PointLight pointLight = new PointLight();
        pointLight.setColor(ColorRGBA.White);
        pointLight.setRadius(100f);
        pointLight.setPosition(new Vector3f(10f, 10f, 0));
        rootNode.addLight(pointLight);
    }

    private void addCrossHairs() {
        BitmapText ch = new BitmapText(guiFont, false);
        ch.setSize(guiFont.getCharSet().getRenderedSize()+4);
        ch.setText("+"); // crosshairs
        ch.setColor(ColorRGBA.White);
        ch.setLocalTranslation( // center
                settings.getWidth() / 2 - ch.getLineWidth() / 2,
                settings.getHeight() / 2 + ch.getLineHeight() / 2, 0);
        guiNode.attachChild(ch);
    }

    private CollisionResult pickFromCamera() {
        CollisionResults results = new CollisionResults();
        Ray ray = new Ray(cam.getLocation(), cam.getDirection());
        instancedNode.collideWith(ray, results);
        return results.getClosestCollision();
    }
}

The stack trace that I get is:

java.lang.NullPointerException
	at com.jme3.collision.bih.BIHTree.collideWithRay(BIHTree.java:414)
	at com.jme3.collision.bih.BIHTree.collideWith(BIHTree.java:471)
	at com.jme3.scene.Mesh.collideWith(Mesh.java:1035)
	at com.jme3.scene.Geometry.collideWith(Geometry.java:472)
	at com.jme3.scene.Node.collideWith(Node.java:615)
	at com.codealchemists.jme.test.instancing.TestInstancedNodeAttachDetachWithPicking.pickFromCamera(TestInstancedNodeAttachDetachWithPicking.java:148)
	at com.codealchemists.jme.test.instancing.TestInstancedNodeAttachDetachWithPicking.access$000(TestInstancedNodeAttachDetachWithPicking.java:23)
	at com.codealchemists.jme.test.instancing.TestInstancedNodeAttachDetachWithPicking$1.onAction(TestInstancedNodeAttachDetachWithPicking.java:77)
	at com.jme3.input.InputManager.invokeActions(InputManager.java:171)
	at com.jme3.input.InputManager.onMouseButtonEventQueued(InputManager.java:448)
	at com.jme3.input.InputManager.processQueue(InputManager.java:867)
	at com.jme3.input.InputManager.update(InputManager.java:917)
	at com.jme3.app.LegacyApplication.update(LegacyApplication.java:724)
	at com.jme3.app.SimpleApplication.update(SimpleApplication.java:246)
	at com.jme3.system.lwjgl.LwjglAbstractDisplay.runLoop(LwjglAbstractDisplay.java:153)
	at com.jme3.system.lwjgl.LwjglDisplay.runLoop(LwjglDisplay.java:193)
	at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:234)
	at java.lang.Thread.run(Thread.java:745)

When I set a breakpoint, the worldbounds of the InstancedGeometry under the InstancedNode is null, which seems to be the cause - but updating model bounds or geometry states of the InstancedNode doesn’t seem to help.

This is on windows, with jMonkeyEngine 3.3.2-stable.

Thanks,

Duncan

99% sure that one of the down sides of instancing is that you can’t do picking. Sort of like not being able to do accurate picking for GPU hardware skinning.

It’s a non-trivial problem to solve and I see no evidence of any code trying to solve it.

Probably when things are close enough to pick, you probably want that level of detail to no longer be instancing.

(Note that batching shouldn’t have this issue but does increase memory overhead.)

Hmmm… given that a collision result is going to have to give you a hit geometry and there is nothing intercepting the collideWith() to the mesh (which is useless)… then I’m 100% sure that picking doesn’t work with instancing.

Edit: Batching suffers from that same ‘which geometry did I pick?’ problem.

1 Like

Before getting to the main problem, I noticed there is an issue with your code.

Geometry sphere = new Geometry("sphere", new Sphere(16, 16, 1f));

in the above code, you are creating a new mesh (Sphere(16, 16, 1f)) for each geometry, this is wrong, you should reuse the same mesh instance on all similar geometries. The same goes for the box geometries.

See TestInstanceNode example:

so you should change your code like this:

        Sphere s = new Sphere(16, 16, 1f);
        Box b = new Box(0.7f, 0.7f, 0.7f);
        // create 10 spheres & boxes, positioned along Z-axis successively further from the camera
        for (int i = 0; i < 10; i++) {
            Vector3f location = new Vector3f(0, -3, -(i*5));
            locations[i] = location;

            Geometry sphere = new Geometry("sphere", s);
            sphere.setMaterial(material);
            sphere.setLocalTranslation(location);
           
            instancedNode.attachChild(sphere);       // initially just add the spheres to the InstancedNode
            spheres[i] = sphere;

            Geometry box = new Geometry("box", b);
            box.setMaterial(material);
            box.setLocalTranslation(location);
            
            boxes[i] = box;
        }

this should also fix the raycasting issue but there is an edge case that problem will still happen and I guess it is when an InstancedGeometry becomes empty (in your example all become boxes or all become spheres) in this case, empty InstancedGeometry’s world bound will be set to null which probably is the reason for that NPE.

I think we should update InstancedNode to remove the InstancedGeometry if it has no instance in it or maybe just set it’s bound to zero instead of null?

Apparently, it works (at least I can verify that in OP’s test) :slightly_smiling_face:

In the way JME is doing it, each individual instances are still in the scene graph but they just get culled so ray can still pick them.

1 Like

Here is how I find collidables in instanced node

public void findTargets(Ray ray, CollisionResults results) {
		// scene is InstancedNode
		List<Spatial> boulders = scene.getChildren().stream().filter(spatial -> {
			boolean hasBound = spatial.getWorldBound() != null;

			// instanced spatials have name like "mesh-1528824395"
			// where prefix "mesh-" is constant
			boolean isGameObject = !spatial.getName().startsWith("mesh-");

			// this is better check
			// boolean hasGameObjectControl = spatial.getControl(HealthControl.class) != null;

			return hasBound && isGameObject;

			// return hasGameObjectControl;
		}).collect(Collectors.toList());

		// not really efficient, still work for me...
		boulders.forEach(boulder -> boulder.collideWith(ray, results));
	}

hope this helps

2 Likes

Yet another way would be to override collideWith() on InstancedGeometry and ignore collision checking there as InstancedGeometry is not meant to get participated in raycasting anyway.

    @Override
    public int collideWith(Collidable other, CollisionResults results) {
        return 0;
    }

I tried this and it fixes the NPE mentioned by OP.

But won’t that prevent the children (that are apparently still there taking up time, etc. but just culled) from getting collisions?

I wasn’t sure if OP wanted collisions on these or not. I tend to arrange my scene graph into ‘objects that are pickable’ and ‘everything else’. So I assume others do also… but that’s probably a bad assumption on my part.

No, the children are parented to InstancedNode, not InstancedGeometry so overriding InstancedGeometry.collideWith() will not affect those.

Sorry, I misread.

1 Like

I can submit a PR if everyone is Ok with this change. :slightly_smiling_face:

Edit:

3 Likes

That’s a good call, I should have spotted that.

Thanks to @walfram for the workaround, which works for me.

If my original expectation was right - that you should be able to collideWith any Node (regardless of whether it’s an InstancedNode, and the fact that it may have InstancedGeometry children shouldn’t cause it to throw an NPE) - then I support @Ali_RS’s fix PR.

I’m impressed with the quality of responses on this forum, really impressed. Thank you everyone, it’s much appreciated.

1 Like

And leaving the question aside of whether you should get useful results or not (it appears you will), the NPE was wrong either way.