Problems with Post filtering & depth buffers (for cartoon edges)

Hey all,

Trying to experiment with cartoon edges in 3089. I’m having lots of trouble getting the CartoonEdgeFilter to work properly & I think it has to do with the depth texture. Here is the original scene:

… and here is the depth buffer (using a modified CartoonEdge.frag that just takes texture2D(m_DepthTexture, texCoord).r as the fragColor):

… you can just barely make out the opening, but the definition is just really poor, which is why I believe I am having so much trouble. You can’t even make out the stuff passed the door. I only significantly notice differences in the depth buffer/texture when really close to things that have a huge depth difference, and that is where CartoonEdges show up:

(notice the black line on the left of that wall, which has a big depth difference pass the wall)

I’ve tried messing with the CartoonEdgeFilter’s sensitivty and thresholds, but it is very difficult to find something that works because I believe the data in the depth buffer is being messed with, somehow…

Any help will be appreciated!

  • Phr00t

How are you interpreting the depth buffer? It requires some particular magic math to turn it into human-understandable depth values.

See this:
http://code.google.com/p/jmonkeyengine/source/browse/trunk/engine/src/core-effects/Common/MatDefs/Post/DepthOfField.frag#16

1 Like

That did it! Now the depth looks as I’d expect it to… now to work on a custom Cartoon edge shader… thank you!