How to get depthTexture correctly?

carpenter · January 29, 2023, 12:55pm

I used three method to obtain depth buffer with no use

firstly I used manual method with this code:

    int width = getContext().getSettings().getWidth();
	int height = getContext().getSettings().getHeight();
	Camera reflectionCam = new Camera(width, height);
	reflectionCam.setFrustumPerspective(getCamera().getFov(), getCamera().getAspect(), 0.01f, 1000);
	reflectionCam.setLocation(new Vector3f(0, 0, -6.f));
	ViewPort reflectionView = renderManager.createPreView("reflectionView", getCamera());
	reflectionView.setEnabled(true);
	reflectionView.setClearFlags(true, true, true);
	reflectionView.attachScene(getRootNode());
	renderFrameBuffer = new FrameBuffer(width, height, 1);
	 //setup framebuffer to use texture
	renderFrameBuffer.setMultiTarget(true);
	reflectionView.setOutputFrameBuffer(renderFrameBuffer);
	depthTexture = new Texture2D(width, height, Format.Depth);
 	renderedTexture = new Texture2D(width, height, Format.RGBA8);
 	renderFrameBuffer.setDepthTarget(FrameBufferTarget.newTarget(depthTexture));
 	renderFrameBuffer.addColorTarget(FrameBufferTarget.newTarget(renderedTexture));
 	reflectionView.setBackgroundColor(ColorRGBA.Cyan);
 	reflectionView.setOutputFrameBuffer(renderFrameBuffer);

As you can see the colored target worked successfully whereas the depth one wasn’t

the second method I was trying to get depthtexture by using filter like waterfilter and fogfilter:

FogFilter fogFilter = new FogFilter();
FilterPostProcessor filterPostProcessor  = new FilterPostProcessor(assetManager);
filterPostProcessor.addFilter(fogFilter);
viewPort.addProcessor(filterPostProcessor);
depthTexture = filterPostProcessor.getDepthTexture();
		
Image img = renderedTexture.getImage();

but it return null

The third method by making custom filter and custom filterpostprocessor with no use
so please how to get depth texture in easy way possible?

Ali_RS · January 29, 2023, 2:39pm

It will be set when FilterPostProcessor is initialized.

github.com

jMonkeyEngine/jmonkeyengine/blob/5ae543cc6f35cd93b53eb511cb912f91e210749f/jme3-core/src/main/java/com/jme3/post/FilterPostProcessor.java#L190


      
           */

          private void initFilter(Filter filter, ViewPort vp) {

              filter.setProcessor(this);

              if (filter.isRequiresDepthTexture()) {

                  if (!computeDepth && renderFrameBuffer != null) {

                      depthTexture = new Texture2D(width, height, depthFormat);

                      renderFrameBuffer.setDepthTarget(FrameBufferTarget.newTarget(depthTexture));

                  }

                  computeDepth = true;

                  filter.init(assetManager, renderManager, vp, width, height);

                  filter.setDepthTexture(depthTexture);

              } else {

                  filter.init(assetManager, renderManager, vp, width, height);

              }

          }

          

          /**

           * renders a filter on a fullscreen quad

           * @param r

           * @param buff

           * @param mat

pspeed · January 29, 2023, 3:16pm

But only if one of the filters requires it:

carpenter · January 29, 2023, 3:57pm

I used both waterfilter and fogfilter and enabled them but when I used filterPostProcessor.getDepthTexture(); it return null for depthTexture.getImage()
So if I burden you somehow can you give a working code when you have time, it’ll also benefit the future user

pspeed · January 29, 2023, 5:30pm

From where? Where did you call it? When did you call it?

…depth texture clearly works in a filter post processor else water wouldn’t work, fog wouldn’t work, etc…

If you expect the depth texture to be available “at any random time of your choosing” then you might be disappointed.

You might also try looking in the JME examples as I recall that there are a few of them that render various depth buffers in little debug displays.

Ali_RS · January 29, 2023, 6:35pm

       fogFilter = new FogFilter() {
            @Override
            protected void setDepthTexture(Texture depthTexture) {
                super.setDepthTexture(depthTexture);

                // Grab depth texture and use it for good!
                System.out.println("depthTexture=" + depthTexture);
            }
        };

Not the best solution of course!

carpenter · January 30, 2023, 8:13am

I tried your code it produced null images for both renderedTextured and depthTextured

FogFilter fogFilter = new FogFilter() {
		protected void setDepthTexture(Texture depthTexture) {
			super.setDepthTexture(Main.this.depthTexture);
			Main.this.depthTexture = (Texture2D) depthTexture;
		}
		protected boolean isRequiresDepthTexture() {
			return true;
		}
		@Override
		protected void setRenderedTexture(Texture2D renderedTexture) {
			super.setRenderedTexture(renderedTexture);
			Main.this.renderedTexture = renderedTexture;
		}		
	};
	fogFilter.setEnabled(true);
	
 	FilterPostProcessor filterPostProcessor = new FilterPostProcessor(assetManager);
 	filterPostProcessor.addFilter(fogFilter);
 	viewPort.addProcessor(filterPostProcessor);

And I implemented the filter class directly with this code with the same result for both textures

Filter filter = new Filter() {
 		private Pass reflectionPass;
 	    protected Spatial reflectionScene;
 	    protected Spatial rootScene = rootNode;
 	    protected ViewPort reflectionView;
 	    private Camera reflectionCam;
 	    private int reflectionMapSize = 512;
 	    private Material material;
		@Override
		protected void initFilter(AssetManager manager, RenderManager renderManager, ViewPort vp, int w, int h) {
			rootScene = vp.getScenes().get(0);

	        if (reflectionScene == null) {
	            reflectionScene = rootScene;
	        }
	        reflectionPass = new Pass();
	        reflectionPass.init(renderManager.getRenderer(), reflectionMapSize, reflectionMapSize, Format.RGBA8, Format.Depth,1,true);
	        reflectionCam = new Camera(reflectionMapSize, reflectionMapSize);
	        reflectionView = new ViewPort("reflectionView", reflectionCam);
	        reflectionView.setClearFlags(true, true, true);
	        reflectionView.attachScene(reflectionScene);
	        reflectionView.setOutputFrameBuffer(reflectionPass.getRenderFrameBuffer());
	        renderedTexture = (Texture2D) reflectionPass.getRenderFrameBuffer().getColorTarget().getTexture();
	        material = new Material(manager, "Common/MatDefs/Water/Water.j3md");
		}
		
		@Override
		protected Material getMaterial() {
			return material;
		}
		@Override
		protected boolean isRequiresDepthTexture() {
			return true;
		}
		@Override
		public void setDepthTexture(Texture depthTexture) {
			super.setDepthTexture(depthTexture);
			Main.this.depthTexture = (Texture2D) depthTexture;
		}
		@Override
		protected void setRenderedTexture(Texture2D renderedTexture) {
			super.setRenderedTexture(renderedTexture);
			Main.this.renderedTexture = renderedTexture;
		}
	};
	filter.setEnabled(true);
FilterPostProcessor filterPostProcessor = new FilterPostProcessor(assetManager);
 	filterPostProcessor.addFilter(filter);
 	viewPort.addProcessor(filterPostProcessor);

However with direct code I can produce renderedTexture but not depthTexture where am I doing wrong?

int width = getContext().getSettings().getWidth();
	int height = getContext().getSettings().getHeight();
	Camera reflectionCam = new Camera(width, height);
	reflectionCam.setFrustumPerspective(getCamera().getFov(), getCamera().getAspect(), 0.01f, 1000);
	reflectionCam.setLocation(new Vector3f(0, 0, -6.f));
	ViewPort reflectionView = renderManager.createPreView("reflectionView", getCamera());
	reflectionView.setEnabled(true);
	reflectionView.setClearFlags(true, true, true);
	reflectionView.attachScene(getRootNode());
	renderFrameBuffer = new FrameBuffer(width, height, 1);
	reflectionView.setOutputFrameBuffer(renderFrameBuffer);
	depthTexture = new Texture2D(width, height, Format.Depth);
 	renderedTexture = new Texture2D(width, height, Format.RGBA8);
 	renderFrameBuffer.setDepthTarget(FrameBufferTarget.newTarget(depthTexture));
 	renderFrameBuffer.addColorTarget(FrameBufferTarget.newTarget(renderedTexture));
 	reflectionView.setOutputFrameBuffer(renderFrameBuffer);

Ali_RS · January 30, 2023, 8:28am

Add a System.out.println(depthTexture) inside setDepthTexture method just like I did and we will see if it is null or not:

fogFilter = new FogFilter() {
            @Override
            protected void setDepthTexture(Texture depthTexture) {
                super.setDepthTexture(depthTexture);

                System.out.println("depthTexture=" + depthTexture);
            }
        };

carpenter · January 30, 2023, 8:38am

I did this inside setDepthTexture()

Main.this.depthTexture = (Texture2D) depthTexture;

Ali_RS · January 30, 2023, 8:41am

Please do this and let me know what it prints in the console.

carpenter · January 30, 2023, 8:45am

It produces nothing at all
Update
I set one in setRenderedTexture
it also produces nothing as well

Ali_RS · January 30, 2023, 8:51am

Please show the complete code.

I tested and it prints this in the console in my case

depthTexture=Texture2D[name=null, image=Image[size=832x624, format=Depth]]

carpenter · January 30, 2023, 8:55am

Ok my bad
for depthTextured
depthTextureTexture2D[name=null, image=Image[size=640x480, format=Depth]]
but none for setRenderedTexture

Ali_RS · January 30, 2023, 9:07am

So it is not null you just needed to wait for Filter to be initialized properly before you can use Main.this.renderedTexture.

in another word you can not access it from simpleInitApp because the filter is not yet initialized.

carpenter · January 30, 2023, 9:11am

So where and when and how I can access it
do I need to execute the code in update or render methods after period of time?

Ali_RS · January 30, 2023, 9:17am

You can check it in the update loop until it becomes available then use it. I have never used it before so I do not know if you also need to consider anything else.

carpenter · January 30, 2023, 10:02am

I made a CustomFogFilter

public class CustomFogFilter extends FogFilter {
		private Texture2D renderedTexture;
		private Texture2D depthTexture;
	@Override
	protected void initFilter(AssetManager manager, RenderManager renderManager, ViewPort vp, int w, int h) {
		super.initFilter(manager, renderManager, vp, w, h);
        renderedTexture = (Texture2D) this.defaultPass.getRenderFrameBuffer().getColorTarget().getTexture();
	}
	
	@Override
	protected Material getMaterial() {
		return material;
	}
	@Override
	protected boolean isRequiresDepthTexture() {
		return true;
	}
	@Override
	protected boolean isRequiresSceneTexture() {
		return true;
	}
	@Override
	public void setDepthTexture(Texture depthTexture) {
		super.setDepthTexture(depthTexture);
		System.out.println("depthTexture "+depthTexture);
		this.depthTexture = (Texture2D) depthTexture;
	}
	@Override
	protected void setRenderedTexture(Texture2D renderedTexture) {
		super.setRenderedTexture(renderedTexture);
		this.renderedTexture = renderedTexture;
		System.out.println("renderedTexture "+renderedTexture);
	}
	public Texture2D getDepthTexture() {
		return depthTexture;
	}
	public Texture2D getRenderedTexture() {
		return renderedTexture;
	}

And I made this code in Update method

loops++;
		if(loops>100) {
			if(filter.getDepthTexture().getImage()!=null) {
				picture2.setTexture(assetManager, filter.getDepthTexture(), false);
			}
			if(filter.getRenderedTexture().getImage()!=null) {
				picture1.setTexture(assetManager, filter.getRenderedTexture(), false);
			}
		}

There is no Null error any more but empty images

zzuegg · January 30, 2023, 12:39pm

Some pitfalls when using depth textures:

Make sure it is really not working. Only because it is all white does not imply that. It is expected to be white not very far away from the camera.

You cannot simply copy the image to another object and use that at some random time. It will still refer to the same image you are rendering/clearing

Make sure your do not render a full screen quad with depth write enabled as long as you have bound the depth texture.

The gl specs do not allow reading and writing to the same texture at any given time. Result is not defined and may be correct or wrong or partially wrong depending on the driver/gpu.

carpenter · January 30, 2023, 12:51pm

Thank you for you response
Doesn’t the depth texture updated frequently
There is no full screen quad (I can post the full code if you want)
How could I obtain depth texture then? there is no working example anywhere!

pspeed · January 30, 2023, 12:55pm

What is it that you are actually tying to do? ie: what do you need the depth texture for?