FrameBuffers and stencil operations?

Is there support for stencil buffers when rendering to a FBO in jME?

Code:
mat.getAdditionalRenderState().setStencil(true, StencilOperation.Keep, StencilOperation.Keep, StencilOperation.Keep, StencilOperation.Keep, StencilOperation.Keep, StencilOperation.Keep, TestFunction.Never, TestFunction.Never);
If I've got this stencilbuffer thingy right then geometries using the material above should not be rendered.
TestFunctions says the pixel should always fail the stencil test, but all the geometry still ends up on my screen.

I'm rendering to a FrameBuffer with a depth texture (Format.Depth) and there is no way for me to attach a stencil buffer and afaik the depthbuffer and stencil buffer is sharing space in hardware =S

Is my understanding of this wrong?

You can now use Format.Depth24Stencil8

1 Like

Great news! Does one have to enable stencil in the AppSettings? Noticed a setting for it there…

@kwando said:
Great news! Does one have to enable stencil in the AppSettings? Noticed a setting for it there...

You only need to enable stencil there if you want to use it on the main framebuffer. If you want to use it on an FBO (as you mentioned in your post), you have to specify Format.Depth24Stencil8 in the FrameBuffer depth buffer setup.

@Momoko_Fan



Sure if this works correctly? I’cant create a Format.Depth24Stencil8 as a depth texture. I’m having a hard time thinking my Macbook Pro HD6750M GPU should not support this =S



2012-apr-21 14:37:07 com.jme3.renderer.lwjgl.LwjglRenderer setFrameBuffer

=== OpenGL FBO State ===

Context doublebuffered? false

FBO ID: 1

ALLVARLIG: === jMonkeyEngine FBO State ===

Is proper? true

Is bound to draw? true

Is bound to read? true

FrameBuffer[format=1280x720x1, drawBuf=mrt]

Draw buffer: GL_COLOR_ATTACHMENT0

Depth => TextureTarget[format=Depth24Stencil8]

Read buffer: GL_COLOR_ATTACHMENT0

== Renderbuffer Depth ==

Color(0) => TextureTarget[format=RGBA16F]

RB ID: -1

Is proper? false

Type: Texture

Color(1) => TextureTarget[format=RGBA16F]

== Renderbuffer Color0 ==

RB ID: -1

Color(2) => TextureTarget[format=RGBA16F]

Is proper? false



Type: Texture

== Renderbuffer Color1 ==

RB ID: -1

Is proper? false

Type: Texture

== Renderbuffer Color2 ==

2012-apr-21 14:37:07 com.jme3.app.Application handleError

RB ID: -1

Is proper? false

Type: Texture

ALLVARLIG: Uncaught exception thrown in Thread[LWJGL Renderer Thread,5,main]

java.lang.IllegalStateException: Framebuffer has erronous attachment.

at com.jme3.renderer.lwjgl.LwjglRenderer.checkFrameBufferError(LwjglRenderer.java:1325)

at com.jme3.renderer.lwjgl.LwjglRenderer.setFrameBuffer(LwjglRenderer.java:1599)

at mygame.v2.DMonkeySceneProcessor.postQueue(DMonkeySceneProcessor.java:138)

at com.jme3.renderer.RenderManager.renderViewPort(RenderManager.java:1122)

at com.jme3.renderer.RenderManager.render(RenderManager.java:1168)

at com.jme3.app.SimpleApplication.update(SimpleApplication.java:254)

at com.jme3.system.lwjgl.LwjglAbstractDisplay.runLoop(LwjglAbstractDisplay.java:149)

at com.jme3.system.lwjgl.LwjglDisplay.runLoop(LwjglDisplay.java:182)

at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:223)

at java.lang.Thread.run(Thread.java:680)



And there seems to many System.out.printlns in there that shouldn’t be there…

After some research I’m not that sure anymore, damn…

Sorry I made a small mistake before. It should work fine now (I tested it)

Hi,

I’m trying to use stencil with framebuffer (to mask light volume in a deferred rendering pass). And I also try to use TestFunction.Never to test “the stencil test”. But witout success.

As a small sample I edit the sample jme3test.post.TestRenderToTexture with :
[java]
public Texture setupOffscreenView(){

//setup framebuffer to use texture
offBuffer.setDepthBuffer(Format.Depth24Stencil8);

Material material = assetManager.loadMaterial(“Interface/Logo/Logo.j3m”);
material.getAdditionalRenderState().setStencil(true,
StencilOperation.Keep, StencilOperation.Keep, StencilOperation.Keep,
StencilOperation.Keep, StencilOperation.Keep, StencilOperation.Keep,
TestFunction.Never, TestFunction.Never
);

return offTex;
}
[/java]
full ‘modified’ source available at stencil and jme test · GitHub

And expect to have the logo hidden (black or white), but stencil seems to have no impact.
What I did wrong ?

EDIT: I use jme-3.0.10

XxxRenderer doesn’t manage Depth24Stencil8.
To workaround the issue:

  1. I copy LwjglRenderer and modify :
    [java]
    private int convertAttachmentSlot(int attachmentSlot) {
    // can also add support for stencil here
    if (attachmentSlot == -100) {
    return GL_DEPTH_ATTACHMENT_EXT;
    } else if (attachmentSlot == -101) {
    return GL_DEPTH_STENCIL_ATTACHMENT;
    } else if (attachmentSlot < 0 || attachmentSlot >= 16) {
    throw new UnsupportedOperationException("Invalid FBO attachment slot: " + attachmentSlot);
    }

     return GL_COLOR_ATTACHMENT0_EXT + attachmentSlot;
    

    }
    [/java]

  2. use refection to change the attachmentSlot
    [java]
    public static void setDepthStencilAttachment(FrameBuffer fb) {
    Field field = FrameBuffer.class.getDeclaredField(“depthBuf”);
    field.setAccessible(true);
    RenderBuffer depthBuf = (RenderBuffer) field.get(fb);
    depthBuf.slot = -101;
    }
    [/java]

  3. setup depth
    [java]
    fb.setDepthBuffer(Format.Depth24Stencil8);
    FrameBufferHack.setDepthStencilAttachment(fb);
    [/java]

full workaround available at : hack: patch LwjglRenderer to allow GL_DEPTH_STENCIL_ATTACHMENT (via d… · davidB/jme3_ext_deferred@474510a · GitHub (if someone else is interested)

Team member, do you want a PR (or is it useless due to Renderer refactor) ?

@kwando, do you find a workaround for mac “unsupported image format ‘Depth24Stencil8’” ? (2 users reported me the error)

  • 2010 Macbook Pro with latest OSX
  • Macbook Pro (with Iris Pro GFX)
@david.bernard.31 said: @kwando, do you find a workaround for mac "unsupported image format 'Depth24Stencil8'" ? (2 users reported me the error)
  • 2010 Macbook Pro with latest OSX
  • Macbook Pro (with Iris Pro GFX)

It will not work on Mac, because jME3 requires OpenGL 3.0+ for Depth24Stencil8 which is not exposed unless a core OpenGL 3.2 context is requested. The other option is to use EXT_packed_depth_stencil, but that one has somewhat different semantics and might be a bit too much work to support both ways. The alternative is to request jME3 to create a core 3.2 context via AppSettings.setRenderer(AppSettings.LWJGL_OPENGL3), but then all pre GLSL 1.5 shaders won’t work.

From https://developer.apple.com/opengl/capabilities/ , mac 10.9 should support opengl 3.3 on every hardware.

I currently use a LwjglRendererCustom (a copy of LwjgRenderer + support for Stencil buffer like describe above in this thread). How to force OpenGL 3 I don’t see where the check opengl 3 requested / opengl 2 available is done. (I only used GLSL 1.5+)

@david.bernard.31 said: From https://developer.apple.com/opengl/capabilities/ , mac 10.9 should support opengl 3.3 on every hardware.

I currently use a LwjglRendererCustom (a copy of LwjgRenderer + support for Stencil buffer like describe above in this thread). How to force OpenGL 3 I don’t see where the check opengl 3 requested / opengl 2 available is done. (I only used GLSL 1.5+)


The alternative is to request jME3 to create a core 3.2 context via AppSettings.setRenderer(AppSettings.LWJGL_OPENGL3), but then all pre GLSL 1.5 shaders won’t work.

I can’t use AppSettings.setRenderer(AppSettings.LWJGL_OPENGL3) because i use a custom renderer. My question is how with a custom renderer, where to look?

You mean beyond the results of:
find . -name "*.java" | xargs grep LWJGL_OPENGL3
?

Thanks, I missed LwjglContext.createContextAttribs() during my first search (I should not debug with fog in brain) :stuck_out_tongue:

package com.jme3.renderer.lwjgl;

import org.lwjgl.opengl.ContextAttribs;
import com.jme3.system.lwjgl.LwjglDisplay;

public class LwjglDisplayCustom extends LwjglDisplay {
	protected ContextAttribs createContextAttribs() {
		ContextAttribs attr = new ContextAttribs(3, 2);
		attr = attr.withProfileCore(true).withForwardCompatible(true).withProfileCompatibility(false);
		if (settings.getBoolean("GraphicsDebug")) {
			attr = attr.withDebug(true);
		}
		return attr;
	}

	@Override
	protected void initContextFirstTime() {
		//super.initContextFirstTime();
		renderer = new LwjglRendererCustom();
		((LwjglRendererCustom)renderer).initialize();

		// Init input
		if (keyInput != null) {
			keyInput.initialize();
		}

		if (mouseInput != null) {
			mouseInput.initialize();
		}

		if (joyInput != null) {
			joyInput.initialize();
		}
	}

}

I’ll package a new version, and hope for a mac user to test and share feedback.