[SOLVED] NPE with InstancedNode and Shadow Filter

Hello,

I’m trying to use DirectionalLightShadowFilter with InstancedNode, which seems to be causing an NPE due to InstancedGeometry sometimes having no worldBound, similar to this (solved) issue: NPE when Ray casting with InstancedNode

Exception:

java.lang.NullPointerException
at com.jme3.shadow.ShadowUtil.updateShadowCamera(ShadowUtil.java:487)
at com.jme3.shadow.DirectionalLightShadowRenderer.getOccludersToRender(DirectionalLightShadowRenderer.java:194)
at com.jme3.shadow.AbstractShadowRenderer.renderShadowMap(AbstractShadowRenderer.java:427)
at com.jme3.shadow.AbstractShadowRenderer.postQueue(AbstractShadowRenderer.java:412)
at com.jme3.shadow.AbstractShadowFilter.postQueue(AbstractShadowFilter.java:116)
at com.jme3.post.FilterPostProcessor.postQueue(FilterPostProcessor.java:241)
at com.jme3.renderer.RenderManager.renderViewPort(RenderManager.java:1103)
at com.jme3.renderer.RenderManager.render(RenderManager.java:1158)
at com.jme3.app.SimpleApplication.update(SimpleApplication.java:272)
at com.jme3.system.lwjgl.LwjglAbstractDisplay.runLoop(LwjglAbstractDisplay.java:153)
at com.jme3.system.lwjgl.LwjglDisplay.runLoop(LwjglDisplay.java:193)
at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:234)
at java.lang.Thread.run(Thread.java:745)

The example code below is the same as that for NPE when Ray casting with InstancedNode but with a DirectionalLightShadowFilter added.

When you move around the scene, everything is initially fine, until you travel forward or backward enough, then the NPE occurs.

import com.jme3.app.SimpleApplication;
import com.jme3.light.AmbientLight;
import com.jme3.light.DirectionalLight;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Vector3f;
import com.jme3.post.FilterPostProcessor;
import com.jme3.renderer.queue.RenderQueue;
import com.jme3.scene.Geometry;
import com.jme3.scene.Mesh;
import com.jme3.scene.instancing.InstancedNode;
import com.jme3.scene.shape.Box;
import com.jme3.scene.shape.Sphere;
import com.jme3.shadow.DirectionalLightShadowFilter;
import com.jme3.shadow.EdgeFilteringMode;

// Based on distance from camera, swap in/out more/less detailed geometry to/from an InstancedNode.
public class TestInstancedNodeAttachDetachWithShadowFilter extends SimpleApplication {
	public static void main(String[] args) {
		TestInstancedNodeAttachDetachWithShadowFilter app = new TestInstancedNodeAttachDetachWithShadowFilter();
		app.setShowSettings(false);  // disable the initial settings dialog window
		app.start();
	}

	private FilterPostProcessor filterPostProcessor;
	private InstancedNode instancedNode;

	private Vector3f[] locations = new Vector3f[10];
	private Geometry[] spheres = new Geometry[10];
	private Geometry[] boxes = new Geometry[10];

	@Override
	public void simpleInitApp() {
		filterPostProcessor = new FilterPostProcessor(assetManager);
		getViewPort().addProcessor(filterPostProcessor);

		addDirectionalLight();
		addAmbientLight();

		Material instancingMaterial = createLightingMaterial(true, ColorRGBA.LightGray);

		instancedNode = new InstancedNode("theParentInstancedNode");
		instancedNode.setShadowMode(RenderQueue.ShadowMode.CastAndReceive);
		rootNode.attachChild(instancedNode);

		// create 10 spheres & boxes, along the z-axis, successively further from the camera
		Mesh sphereMesh = new Sphere(32, 32, 1f);
		Mesh boxMesh = new Box(0.7f, 0.7f, 0.7f);
		for (int z = 0; z < 10; z++) {
			Vector3f location = new Vector3f(0, -3, -(z * 4));
			locations[z] = location;

			Geometry sphere = new Geometry("sphere", sphereMesh);
			sphere.setMaterial(instancingMaterial);
			sphere.setLocalTranslation(location);
			instancedNode.attachChild(sphere);       // initially just add the spheres to the InstancedNode
			spheres[z] = sphere;

			Geometry box = new Geometry("box", boxMesh);
			box.setMaterial(instancingMaterial);
			box.setLocalTranslation(location);
			boxes[z] = box;
		}

		instancedNode.instance();


		Geometry floor = new Geometry("floor", new Box(20, 0.1f, 40));
		floor.setMaterial(createLightingMaterial(false, ColorRGBA.Yellow));
		floor.setLocalTranslation(5, -5, 0);
		floor.setShadowMode(RenderQueue.ShadowMode.Receive);
		rootNode.attachChild(floor);

		flyCam.setMoveSpeed(30);
	}

	@Override
	public void simpleUpdate(float tpf) {
		// Each frame, determine the distance to each sphere/box from the camera.
		// If the object is > 25 units away, switch in the Box.  If it's nearer, switch in the Sphere.
		// Normally we wouldn't do this every frame, only when player has moved a sufficient distance, etc.

		for (int i = 0; i < 10; i++) {
			Vector3f location = locations[i];
			float distance = location.distance(cam.getLocation());

			instancedNode.detachChild(boxes[i]);
			instancedNode.detachChild(spheres[i]);

			if (distance > 25.0f) {
				instancedNode.attachChild(boxes[i]);
			} else {
				instancedNode.attachChild(spheres[i]);
			}
		}

		instancedNode.instance();
	}

	private Material createLightingMaterial(boolean useInstancing, ColorRGBA color) {
		Material material = new Material(assetManager, "Common/MatDefs/Light/Lighting.j3md");
		material.setBoolean("UseMaterialColors", true);
		material.setBoolean("UseInstancing", useInstancing);
		material.setColor("Ambient", color);
		material.setColor("Diffuse", color);
		material.setColor("Specular", color);
		material.setFloat("Shininess", 1.0f);
		return material;
	}

	private void addAmbientLight() {
		AmbientLight ambientLight = new AmbientLight(new ColorRGBA(0.1f, 0.1f, 0.1f, 1.0f));
		rootNode.addLight(ambientLight);
	}

	private void addDirectionalLight() {
		DirectionalLight light = new DirectionalLight();

		light.setColor(ColorRGBA.White);
		light.setDirection(new Vector3f(-1, -1, -1));

		DirectionalLightShadowFilter dlsf = new DirectionalLightShadowFilter(assetManager, 1024 * 8, 3);
		dlsf.setLight(light);
		dlsf.setEdgeFilteringMode(EdgeFilteringMode.PCFPOISSON);
		filterPostProcessor.addFilter(dlsf);

		rootNode.addLight(light);
	}
}

The ShadowUtil code iterates over potential shadow receivers, and gets their worldBound / bounding volume, which seems to be where the problem lies. Should it ignore any cases where the worldBound is null, or am I doing something silly in the code above that causes this problem?

for (int i = 0; i < receivers.size(); i++) {
	// convert bounding box to light's viewproj space
	Geometry receiver = receivers.get(i);
	BoundingVolume bv = receiver.getWorldBound();
	BoundingVolume recvBox = bv.transform(viewProjMatrix, vars.bbox);	// <--- NPE happens here

	if (splitBB.intersects(recvBox)) {
		//Nehon : prevent NaN and infinity values to screw the final bounding box
		if (!Float.isNaN(recvBox.getCenter().x) && !Float.isInfinite(recvBox.getCenter().x)) {
			receiverBB.mergeLocal(recvBox);
			receiverCount++;
		}
	}
}

This is on windows, with jMonkeyEngine 3.3.2-stable.

Thanks,

Duncan

2 Likes

What if you set shadow mode to RenderQueue.ShadowMode.Off on all InstancedGeometries!!

Like this?

instancedNode.instance();

for (Spatial s : instancedNode.getChildren()) {
	if (s instanceof InstancedGeometry) {
		s.setShadowMode(RenderQueue.ShadowMode.Off);
	} else {
		s.setShadowMode(RenderQueue.ShadowMode.CastAndReceive);
	}
}

Same thing happens, no change.

Seems so, or it should not be included in the receivers list in the first place.

maybe should be filtered here?

I thought this check should already drop it

camera.contains(child.getWorldBound()) != Camera.FrustumIntersect.Outside)

but it seems the camera will consider it inside by default if world bound is null! Strange!

Same thing happens, no change.

Apologies, I made a mistake here. When I tried your suggestion, it stopped the NPE, but now we get no shadows from any of the spheres or boxes.

Since then, I have figured out how to build jme3-core from the v3.3 branch and get it into my (maven) project. I’ve changed ShadowUtil:487 to check whether the bv is null and ignore those receivers, and it appears to work. I guess the InstancedGeometries sometimes have a boundingVolume? Shall I submit a patch, or should I give a bit more time for others to look at this?

2 Likes

Yes, please go ahead and submit your patch. We can keep it open for a while in case someone wants to give more thought. Thanks.

Sometimes it will have “Null” boundingVolume and that is when all its instances are removed. :slightly_smiling_face:

1 Like

I spoke too soon - the change does work, in that shadows appear for these InstancedGeometries and they look right (to me), however after a time, I eventually get this error:

com.jme3.renderer.RendererException: Attempting to upload empty buffer (limit = 0), that's an error
	at com.jme3.renderer.lwjgl.LwjglGL.checkLimit(LwjglGL.java:22)
	at com.jme3.renderer.lwjgl.LwjglGL.glBufferData(LwjglGL.java:74)
	at com.jme3.renderer.opengl.GLRenderer.updateBufferData(GLRenderer.java:2688)
	at com.jme3.renderer.opengl.GLRenderer.setVertexAttrib(GLRenderer.java:2836)
	at com.jme3.renderer.opengl.GLRenderer.renderMeshDefault(GLRenderer.java:3117)
	at com.jme3.renderer.opengl.GLRenderer.renderMesh(GLRenderer.java:3167)
	at com.jme3.material.logic.DefaultTechniqueDefLogic.renderMeshFromGeometry(DefaultTechniqueDefLogic.java:67)
	at com.jme3.material.logic.DefaultTechniqueDefLogic.render(DefaultTechniqueDefLogic.java:95)
	at com.jme3.material.Technique.render(Technique.java:166)
	at com.jme3.material.Material.render(Material.java:1026)
	at com.jme3.renderer.RenderManager.renderGeometry(RenderManager.java:598)
	at com.jme3.renderer.queue.RenderQueue.renderGeometryList(RenderQueue.java:266)
	at com.jme3.renderer.queue.RenderQueue.renderShadowQueue(RenderQueue.java:275)
	at com.jme3.shadow.AbstractShadowRenderer.renderShadowMap(AbstractShadowRenderer.java:439)
	at com.jme3.shadow.AbstractShadowRenderer.postQueue(AbstractShadowRenderer.java:412)
	at com.jme3.shadow.AbstractShadowFilter.postQueue(AbstractShadowFilter.java:116)
	at com.jme3.post.FilterPostProcessor.postQueue(FilterPostProcessor.java:241)
	at com.jme3.renderer.RenderManager.renderViewPort(RenderManager.java:1103)
	at com.jme3.renderer.RenderManager.render(RenderManager.java:1158)
	at com.jme3.app.SimpleApplication.update(SimpleApplication.java:272)
	at com.jme3.system.lwjgl.LwjglAbstractDisplay.runLoop(LwjglAbstractDisplay.java:153)
	at com.jme3.system.lwjgl.LwjglDisplay.runLoop(LwjglDisplay.java:193)
	at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:234)
	at java.lang.Thread.run(Thread.java:745)

Maybe skipping the receiver when it has no boundingVolume is not the right way forward. Perhaps InstancedGeometries shouldn’t appear in the receivers list in the first place? I’ll give it some more thought.

I can prevent the limit=0 exception by changing DefaultTechniqueDefLogic.java, but I lack the understanding of the internals of the engine to know whether that’s a good idea or not. It seems like this situation shouldn’t arise in the first place.

public static void renderMeshFromGeometry(Renderer renderer, Geometry geom) {
    Mesh mesh = geom.getMesh();
    int lodLevel = geom.getLodLevel();
    if (geom instanceof InstancedGeometry) {
        InstancedGeometry instGeom = (InstancedGeometry) geom;

        // --- duncanj: experimental code --------------------
        if( instGeom.getTransformUserInstanceData().getData().limit() != 0 ) {    // duncanj: prevent limit=0
            renderer.renderMesh(mesh, lodLevel, instGeom.getActualNumInstances(),
                    instGeom.getAllInstanceData());
        } else {
            System.out.println("Prevented 0 limit render bug");
        }
        // ----------------------------------------------------
    } else {
        renderer.renderMesh(mesh, lodLevel, 1, null);
    }
}

All ideas appreciated!

Submitted a patch that should solve this issue and various other issues with InstanceGeometry. Also added two test you provided for picking and shadow using instancing.

2 Likes

That works really well, thank you. It makes everything look amazing! There is one remaining bug that I can see - when you are close to an object sometimes the shadow disappears. The TestInstancedNodeAttachDetachWithShadowFilter example shows this, when you fly alongside the spheres and move forward - the shadows individually disappear, even though they’re still within the visible bounds of the viewport/scene.

disappearing-shadows

Fly forwards from here, and you’ll see what I mean. Is the shadow being culled along with the sphere, because the sphere is moving outside the visible bounds, even though the shadow is still within it?

1 Like

Yes, geometries that are outside the camera frustum will be removed from transformInstanceData buffer, and thus they won’t be rendered anymore so they won’t have a shadow.

Not sure how we can solve this issue. I do not know if there is an easy way to check if an instance shadow is still visible and don’t remove it from the instance data buffer.

I think the simple (but non-efficient) way would be to turn off instance culling in InstancedGeometry when used with shadow so all instances will be rendered but this will largely impact performance.

@pspeed any thought?

Many thanks for the PR and merge into master, this fixes the bugs perfectly. Marking as [SOLVED].

2 Likes