Stencil Buffer Outlines

Troubleshooting graphics
1

I’d like to outline an object that’s selected. I’ve found a tutorial with exactly the effect I want:







Here’s the code they used to achieve the above result:



glClearStencil(0);

glClear(GL_STENCIL_BUFFER_BIT);



// Render the mesh into the stencil buffer.



glEnable(GL_STENCIL_TEST);



glStencilFunc(GL_ALWAYS, 1, -1);

glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);



RenderMesh();



// Render the thick wireframe version.



glStencilFunc(GL_NOTEQUAL, 1, -1);

glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);



glLineWidth(3);

glPolygonMode(GL_FRONT, GL_LINE);



RenderMesh();



Unfortunately I’m not sure how to implement it in jMonkey, seeing as it hides away lots of the low level OpenGL functions. I’m guessing I should probably be putting some code in simpleRender(RenderManager rm) so that I can do some custom rendering, but I’m not sure.



The best I can do at the moment is render a wirefame over the top of the selected geometry:



public void simpleRender(RenderManager rm) {

ViewPort vp = new ViewPort(“Selection Overlay”, cam);

vp.attachScene(selected);



Material selectionMat = new Material(assetManager, “Common/MatDefs/Misc/Unshaded.j3md”);

selectionMat.setColor(“Color”, ColorRGBA.White);



rm.setForcedMaterial(selectionMat);

selected.getMesh().setLineWidth(1);

selected.getMesh().setMode(Mode.Lines);

rm.renderViewPortRaw(vp);

rm.setForcedMaterial(null);

selected.getMesh().setMode(Mode.Triangles);

}



Would anybody be able to give me any pointers on where I should go next on this quest?

2

All these GL functions are deprecated in OpenGL3 anyway, so you best go with shaders. The Comic outline shader basically does something similar, maybe you want to look into that. Its a post effect afaik though, maybe going via the material shader would be better but I don’t know. I think you might find some solutions on the forum for this too. I’d simply go with a second half-transparent solid-colored geometry with the same mesh thats slightly larger and put this as a selection marker.

3

Thanks mate – I’ll have a look at shaders.

4

A bit hackish maybe but you could use two viewports, one with a outline effect and one without… I guess…