StencilState

Troubleshooting graphics
1

Hi guys!

I'm new to the forum and i'm italian, so excuse me for possible language mistakes  :stuck_out_tongue:

Now, coming to the problem, i can't figure out how does the StencilState work.



I started with a simple example taken from an OpenGl book, here it is:


glClearStencil(0.0f);
glEnable(GL_STENCIL_TEST);
glClear(GL_COLOR_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);

// All drawing commands fail the stencil test, and are not
// drawn, but increment the value in the stencil buffer.
glStencilFunc(GL_NEVER, 0x0, 0x0);
glStencilOp(GL_INCR, GL_INCR, GL_INCR);

glBegin(GL_LINE_STRIP);
//....Draw a spiral pattern...doesn't matter
glEnd();

// Now, allow drawing, except where the stencil pattern is 0x1
// and do not make any further changes to the stencil buffer
glStencilFunc(GL_NOTEQUAL, 0x1, 0x1);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);

// Now draw red bouncing square
// (x and y) are modified by a timer function
glColor3f(1.0f, 0.0f, 0.0f);
glRectf(x, y, x + rsize, y - rsize);

// All done, do the buffer swap
glutSwapBuffers();



Now what i got from this few lines is that at the beginning the test always fails and so the spiral pattern is not rendered on the screen but only rendered in the Stencil Buffer, which writes 1 where the spiral is drawn;
with  glStencilFunc(GL_NOTEQUAL, 0x1, 0x1);  the test fails where 1 is written and so we get a red square with a spiral pattern "cut" into it.  hope this is right  :roll:

Coming to Jme i tried to make an example like this one; I created 2 squares in OrthoMode, here they are:



Now i would like to achieve this effect using the StencilState:



Here is the code:


protected void initGame()
{
        //root Node
        Node quad;
        quad = new Node("Nodo Ortho");
       
        //nodes for the 2 squares
        Node NodeRed;
        Node NodeYellow;
        NodeRed = new Node("Nodo Rosso");
        NodeYellow = new Node("Nodo Giallo");

        //stencil stettings
        StencilState stencil;
        stencil = display.getRenderer().createStencilState();
        quad.setRenderState(stencil);
 
        //the two squares
        Quad Redquad;
        Quad Yellowquad;

        Redquad = new Quad("Quadrato Rosso",200,200);
        Redquad.setLocalTranslation(width/2,height/2,0);
        Redquad.setDefaultColor(ColorRGBA.red);
      
        Yellowquad = new Quad("Quadrato Giallo",100,100);
        Yellowquad.setLocalTranslation(width/2,height/2,0);
        Yellowquad.setDefaultColor(ColorRGBA.yellow);
      
        //attach the two quads
        NodeRed.attachChild(Redquad);
        NodeYellow.attachChild(Yellowquad);

       //attach the nodes to the root
       quad.setRenderQueueMode(Renderer.QUEUE_ORTHO);
       quad.attachChild(NodeRed);
       quad.attachChild(NodeRed);

       quad.updateGeometricState(0.0f, true);
       quad.updateRenderState();
}
       



Now if i draw the quad Node i get what you can see above; let's add the Stencil in the Render method:


   protected void render(float interpolation)
   {
      //clear buffers
      display.getRenderer().clearBuffers();
      display.getRenderer().clearStencilBuffer();
      
      stencil.setEnabled(true);
      //i think that for this example i need a single sided buffer, right?
      stencil.setUseTwoSided(false);
      
      //SAME AS THE EXAMPLE ON THE BOOK
      stencil.setStencilFunction(StencilFunction.Never);
      stencil.setStencilReference(0x0);
      stencil.setStencilMask(0x0);
      stencil.setStencilOpFail(StencilOperation.Increment);
      stencil.setStencilOpZFail(StencilOperation.Increment);
      stencil.setStencilOpZPass(StencilOperation.Increment);
      
      //i draw the YellowQuad...should not be rendered on the screen, but only in the stencil, which writes 1
       //only where the Yellow square is drawn (hope so)
      display.getRenderer().draw(NodeYellow);
      
      //don't know if this is necessary
      quad.updateRenderState();
      
      //SAME AS THE EXAMPLE ON THE BOOK, let the test fail where 1 is written
      stencil.setStencilFunction(StencilFunction.NotEqualTo);
      stencil.setStencilReference(0x1);
      stencil.setStencilMask(0x1);
      stencil.setStencilOpFail(StencilOperation.Keep);
      stencil.setStencilOpZFail(StencilOperation.Keep);
      stencil.setStencilOpZPass(StencilOperation.Keep);
      quad.updateRenderState();
      
      //draw the redQuad
      display.getRenderer().draw(NodeRed);
      
   }



The problem is that ONLY the Red square is rendered on the screen, no hole inside  :x

Now maybe i missed something there, but there's a lack of documentation about the StencilState and so i had to figured it out by myself.

Can you guys please help me?
Thanks a lot  :D







2

I have the same problem!! Does anyone have any examples using stencil state?? I've tried to look at the surces of jme but I'm a total noob!!

3

The problem is that in jME, the draw order of objects is undefined. In other words, adding two objects to a node, you don't know which one of the objects will be drawn first.

However, you're lucky, because you're using the ORTHO render queue mode, that one sorts objects using their Z order value. Set the z order using Spatial.setZOrder().