Graphic Novel Filter - Code Posted

Here is what it does:











Let me know if others are interested and I’ll finish it up. Oh… worth mentioning… this can be blended over normal textures/lighting.

7 Likes

Nice, it looks cool. Probably nice for people wanting to mark games as “draft only” as well like the Napkin LAF for java :wink:

1 Like

My plan was to create a version of the lighting shader that renders using dots… like you see in comic books. I think it would be sick if you could play a 3d comic :wink:

I’m interested in this filter even more than in your SSAO. Paste it here, please.

Is the performance of the filter better than your ssao?

@mifth said:
I'm interested in this filter even more than in your SSAO. Paste it here, please.
Is the performance of the filter better than your ssao?


Yes... actually close to twice as fast than the SSAO filter.

Really nice.



I could see also using this along with some photoshop-style filters to create some in-game screen shots that look like hand sketches. I’ve been doing some of these by hand now to use as transition screens later… would be nice to be able to dump them right from the game. :slight_smile:

A little bit of creative material manipulation… and you really are in a comic. I like this one!





1 Like

It would be interesting to add some kind of crosshatch filter for the darker grays. :slight_smile:



So is this just a standard post-process filter or does it require other special setup?

@pspeed said:
It would be interesting to add some kind of crosshatch filter for the darker grays. :)

So is this just a standard post-process filter or does it require other special setup?


I was wanting to do just that (cross hatch... actually... was going for just single direction diagonal lines.. but still)... but have no clue where to start.

And... no weird setup.... just a standard filter. I'll post the code tonight. 2 material definitions, 2 frag shaders, and the post-process filter. I'll dump the Main.java, the model and textures as links as well... though, I know you know where they are... but for others.

Here be the code:



First the Material Definition files (x2)



graphicNovel.j3md

[java]MaterialDef GraphicNovel {



MaterialParameters {

Int NumSamples

Int NumSamplesDepth

Texture2D DepthTexture

Texture2D Texture

Texture2D Normals

Vector2 FrustumNearFar

Vector3 FrustumCorner

Float SampleRadius

Float Intensity

Float Scale

Float Bias

Float FalloffStart

Float FalloffAmount

Vector3Array Samples

}



Technique {

VertexShader GLSL120: Common/MatDefs/Post/Post.vert

FragmentShader GLSL120: Shaders/graphicNovel.frag



WorldParameters {

WorldViewProjectionMatrix

WorldViewMatrix

Resolution

}

}







Technique FixedFunc {

}

}[/java]



graphicNovelComp.j3md

[java]MaterialDef GraphicNovelComp {



MaterialParameters {

Int NumSamples

Int NumSamplesDepth

Texture2D Texture

Texture2D EdgeMap

Texture2D PatternMap

Texture2D DepthTexture

Vector2 FrustumNearFar

Boolean UseEdges

Boolean UseEdgesOnly

Float XScale

Float YScale

}



Technique {

VertexShader GLSL120: Common/MatDefs/Post/Post.vert

FragmentShader GLSL120: Shaders/graphicNovelComp.frag



WorldParameters {

WorldViewProjectionMatrix

WorldViewMatrix

Resolution

}



Defines {

RESOLVE_MS : NumSamples

RESOLVE_DEPTH_MS : NumSamplesDepth

}

}







Technique FixedFunc {

}

}[/java]



Here be the shaders (x2):



graphicNovel.frag

[java]varying vec2 texCoord;

uniform vec2 g_Resolution;

uniform vec2 m_FrustumNearFar;

uniform vec3 m_FrustumCorner;

uniform sampler2D m_Texture;

uniform sampler2D m_Normals;

uniform sampler2D m_DepthTexture;

uniform float m_SampleRadius;

uniform float m_Intensity;

uniform float m_Scale;

uniform float m_Bias;

uniform float m_FalloffStart;

uniform float m_FalloffAmount;

uniform vec3[12] m_Samples;

float depthv;

const float edgeFactor = 0.075;



vec3 getPosition(in vec2 uv){

depthv = texture2D(m_DepthTexture,uv).r;

float depth = (2.0 * m_FrustumNearFar.x) / (m_FrustumNearFar.y + m_FrustumNearFar.x - depthv* (m_FrustumNearFar.y-m_FrustumNearFar.x));

float x = mix(-m_FrustumCorner.x, m_FrustumCorner.x, uv.x);

float y = mix(-m_FrustumCorner.y, m_FrustumCorner.y, uv.y);

return depth* vec3(x, y, m_FrustumCorner.z);

}



vec3 getNormal(in vec2 uv){

return normalize(texture2D(m_Normals, uv).xyz * 2.0 - 1.0);

}



vec3 getRandom(in vec2 uv){

float rand = (fract(uv.x*(g_Resolution.x/2.0))0.25)+(fract(uv.y(g_Resolution.y/2.0))0.5);

return normalize(vec3(rand,rand,rand));

}



vec3 reflection(in vec3 v1,in vec3 v2){

vec3 result = 2.0 * dot(v2, v1) * v2;

result = v1 - result;

return result;

}



float findEdges(in vec2 tc, in vec3 pos, in vec3 norm){

vec3 diff = getPosition(tc) - pos;

vec3 v = normalize(diff);

float d = length(diff) * m_Scale;

return step(0.00002,d) * max(0.0, dot(norm, v) - m_Bias) * (1.0 / (1.0 + d)) * (m_Intensity+edgeFactor) * smoothstep(0.00002,0.0027,d);

}



void main(){

float result;

vec3 position = getPosition(texCoord);



if(depthv==1.0){ gl_FragColor=vec4(1.0); return; }



vec3 normal = getNormal(texCoord);

vec3 rand = getRandom(texCoord);



float edges = 0.0;

float rad = m_SampleRadius/position.z+edgeFactor;



int iterations = 12;

for (int j = 0; j < iterations; ++j){

vec3 coord2 = reflection(vec3(m_Samples[j]), rand) * vec3(rad
0.5);

edges += findEdges(texCoord + coord2.xy * 0.05, position, normal) * (0.25-edgeFactor);

}

edges /= float(iterations) * (2.35-edgeFactor) / 2;

result = 1.0-edges;



gl_FragColor = vec4(vec3(result),1.0);

}[/java]



graphicNovelComp.frag

[java]uniform sampler2D m_Texture;

uniform sampler2D m_DepthTexture;

uniform sampler2D m_EdgeMap;

uniform sampler2D m_PatternMap;

uniform vec2 g_Resolution;

uniform bool m_UseEdges;

uniform bool m_UseEdgesOnly;

uniform vec2 m_FrustumNearFar;

varying vec2 texCoord;



void main(){

vec4 edges = texture2D( m_EdgeMap,texCoord);

vec4 color = texture2D(m_Texture,texCoord);



if (!m_UseEdges && !m_UseEdgesOnly)

gl_FragColor = color;

else if (m_UseEdges && m_UseEdgesOnly)

gl_FragColor = edges;

else

gl_FragColor = color*edges;

}[/java]



Here is the Filter:



GraphicNovelFilter.java

[java]package mygame;



import com.jme3.asset.AssetManager;

import com.jme3.export.InputCapsule;

import com.jme3.export.JmeExporter;

import com.jme3.export.JmeImporter;

import com.jme3.export.OutputCapsule;

import com.jme3.material.Material;

import com.jme3.math.Vector2f;

import com.jme3.math.Vector3f;

import com.jme3.post.Filter;

import com.jme3.renderer.RenderManager;

import com.jme3.renderer.Renderer;

import com.jme3.renderer.ViewPort;

import com.jme3.shader.VarType;

import com.jme3.texture.Image.Format;

import com.jme3.texture.Texture;

import java.io.IOException;

import java.util.ArrayList;



public class GraphicNovelFilter extends Filter{

private Pass normalPass;

private Vector3f frustumCorner;

private Vector2f frustumNearFar;

private Vector3f[] samples = {

new Vector3f(1.0f, 0.0f, 1.0f),

new Vector3f(-1.0f, 0.0f, 1.0f),

new Vector3f(0.0f, 1.0f, 1.0f),

new Vector3f(0.0f, -1.0f, 1.0f),

new Vector3f(1.0f, 0.0f, 0.0f),

new Vector3f(-1.0f, 0.0f, 0.0f),

new Vector3f(0.0f, 1.0f, 0.0f),

new Vector3f(0.0f, -1.0f, 0.0f),

new Vector3f(1.0f, 0.0f, -1.0f),

new Vector3f(-1.0f, 0.0f, -1.0f),

new Vector3f(0.0f, 1.0f, -1.0f),

new Vector3f(0.0f, -1.0f, -1.0f)

};

private float sampleRadius = 0.55f;

private float intensity = 42.5f;

private float scale = 0.005f;

private float bias = 0.025f;

private boolean useEdgesOnly = false;

private boolean useEdges = true;

private Material edgeMat;

private Pass edgePass;

private float falloffAmount = 5f, falloffStart = 100f;

private float downSampleFactor = 1f;



RenderManager renderManager;

ViewPort viewPort;



public GraphicNovelFilter() {

super("GraphicNovelFilter");

}

public GraphicNovelFilter(float sampleRadius, float intensity, float scale, float bias) {

this();

this.sampleRadius = sampleRadius;

this.intensity = intensity;

this.scale = scale;

this.bias = bias;

}



@Override

protected boolean isRequiresDepthTexture() {

return true;

}



@Override

protected void postQueue(RenderQueue renderQueue) {

Renderer r = renderManager.getRenderer();

r.setFrameBuffer(normalPass.getRenderFrameBuffer());

renderManager.getRenderer().clearBuffers(true, true, true);

renderManager.setForcedTechnique("PreNormalPass");

renderManager.renderViewPortQueues(viewPort, false);

renderManager.setForcedTechnique(null);

renderManager.getRenderer().setFrameBuffer(viewPort.getOutputFrameBuffer());

}



@Override

protected Material getMaterial() {

return material;

}



@Override

protected void initFilter(AssetManager manager, RenderManager renderManager, ViewPort vp, int w, int h) {

this.renderManager = renderManager;

this.viewPort = vp;



int screenWidth = w;

int screenHeight = h;

postRenderPasses = new ArrayList<Pass>();



normalPass = new Pass();

normalPass.init(renderManager.getRenderer(), (int) (screenWidth / downSampleFactor), (int) (screenHeight / downSampleFactor), Format.RGBA8, Format.Depth);



frustumNearFar = new Vector2f();



float farY = (vp.getCamera().getFrustumTop() / vp.getCamera().getFrustumNear()) * vp.getCamera().getFrustumFar();

float farX = farY * ((float) screenWidth / (float) screenHeight);

frustumCorner = new Vector3f(farX, farY, vp.getCamera().getFrustumFar());

frustumNearFar.x = vp.getCamera().getFrustumNear();

frustumNearFar.y = vp.getCamera().getFrustumFar();



edgeMat = new Material(manager, "MatDefs/graphicNovel.j3md");

edgeMat.setTexture("Normals", normalPass.getRenderedTexture());

edgeMat.setFloat("SampleRadius", sampleRadius);

edgeMat.setFloat("Intensity", intensity);

edgeMat.setFloat("Scale", scale);

edgeMat.setFloat("Bias", bias);

edgeMat.setFloat("FalloffStart", falloffStart);

edgeMat.setFloat("FalloffAmount", falloffAmount);

edgeMat.setVector3("FrustumCorner", frustumCorner);

edgeMat.setVector2("FrustumNearFar", frustumNearFar);

edgeMat.setParam("Samples", VarType.Vector3Array, samples);



edgePass = new Pass() {

@Override

public boolean requiresDepthAsTexture() {

return true;

}

};



edgePass.init(renderManager.getRenderer(), (int) (screenWidth / downSampleFactor), (int) (screenHeight / downSampleFactor), Format.RGBA8, Format.Depth, 1, edgeMat);

edgePass.getRenderedTexture().setMinFilter(Texture.MinFilter.Trilinear);

edgePass.getRenderedTexture().setMagFilter(Texture.MagFilter.Bilinear);



postRenderPasses.add(edgePass);



material = new Material(manager, "MatDefs/graphicNovelComp.j3md");

material.setTexture("EdgeMap", edgePass.getRenderedTexture());

material.setVector2("FrustumNearFar", frustumNearFar);

material.setBoolean("UseEdges", useEdges);

material.setBoolean("UseEdgesOnly", useEdgesOnly);

}



public float getBias() {

return bias;

}

public void setBias(float bias) {

this.bias = bias;

if (edgeMat != null) {

edgeMat.setFloat("Bias", bias);

}

}



public float getIntensity() {

return intensity;

}

public void setIntensity(float intensity) {

this.intensity = intensity;

if (edgeMat != null) {

edgeMat.setFloat("Intensity", intensity);

}



}



public float getSampleRadius() {

return sampleRadius;

}

public void setSampleRadius(float sampleRadius) {

this.sampleRadius = sampleRadius;

if (edgeMat != null) {

edgeMat.setFloat("SampleRadius", sampleRadius);

}



}



public float getScale() {

return scale;

}

public void setScale(float scale) {

this.scale = scale;

if (edgeMat != null) {

edgeMat.setFloat("Scale", scale);

}

}



public boolean isuseEdges() {

return useEdges;

}

public void setuseEdges(boolean useEdges) {

this.useEdges = useEdges;

if (material != null) {

material.setBoolean("useEdges", useEdges);

}



}



public boolean isuseEdgesOnly() {

return useEdgesOnly;

}

public void setuseEdgesOnly(boolean useEdgesOnly) {

this.useEdgesOnly = useEdgesOnly;

if (material != null) {

material.setBoolean("useEdgesOnly", useEdgesOnly);

}

}



public void updateFalloff(float density, float distance) {

falloffStart = distance;

falloffAmount = density;

if (edgeMat != null) {

edgeMat.setFloat("FalloffStart", distance);

edgeMat.setFloat("FalloffAmount", density);

}

}



public void toggleEdgeFilter() {

if (!useEdgesOnly && useEdges) { // GraphicNovelFilter Disabled

useEdgesOnly = false;

useEdges = false;



} else if (useEdgesOnly && useEdges) { // GraphicNovelFilter Edge Map Only

useEdgesOnly = false;

useEdges = true;

} else if (!useEdgesOnly && !useEdges) { // GraphicNovelFilter Blended

useEdgesOnly = true;

useEdges = true;

}

if (material != null) {

material.setBoolean("useEdges", useEdges);

material.setBoolean("useEdgesOnly", useEdgesOnly);

}

}



@Override

public void write(JmeExporter ex) throws IOException {

super.write(ex);

OutputCapsule oc = ex.getCapsule(this);

oc.write(sampleRadius, "sampleRadius", 5.1f);

oc.write(intensity, "intensity", 1.5f);

oc.write(scale, "scale", 0.2f);

oc.write(bias, "bias", 0.1f);

}



@Override

public void read(JmeImporter im) throws IOException {

super.read(im);

InputCapsule ic = im.getCapsule(this);

sampleRadius = ic.readFloat("sampleRadius", 5.1f);

intensity = ic.readFloat("intensity", 1.5f);

scale = ic.readFloat("scale", 0.2f);

bias = ic.readFloat("bias", 0.1f);

}

}[/java]



If anyone needs it… here is the simpleapp…

Main.java

[java]package mygame;



import com.jme3.app.SimpleApplication;

import com.jme3.light.AmbientLight;

import com.jme3.light.DirectionalLight;

import com.jme3.material.Material;

import com.jme3.math.ColorRGBA;

import com.jme3.math.Quaternion;

import com.jme3.math.Vector2f;

import com.jme3.math.Vector3f;

import com.jme3.post.FilterPostProcessor;

import com.jme3.renderer.RenderManager;

import com.jme3.scene.Geometry;

import com.jme3.scene.Node;

import com.jme3.texture.Texture;



/**

  • test
  • @author normenhansen

    */

    public class Main extends SimpleApplication {

    Geometry model;

    Node node;

    public static void main(String[] args) {

    Main app = new Main();

    app.start();

    }



    @Override

    public void simpleInitApp() {

    cam.setLocation(new Vector3f(68.45442f, 8.235511f, 7.9676695f));

    cam.setRotation(new Quaternion(0.046916496f, -0.69500375f, 0.045538206f, 0.7160271f));





    flyCam.setMoveSpeed(50);



    Material mat = new Material(assetManager, "Common/MatDefs/Light/Lighting.j3md");

    Texture diff = assetManager.loadTexture("Textures/BrickWall.jpg");

    diff.setWrap(Texture.WrapMode.Repeat);

    Texture norm = assetManager.loadTexture("Textures/BrickWall_normal.jpg");

    norm.setWrap(Texture.WrapMode.Repeat);

    mat.setBoolean("UseMaterialColors", true);

    // mat.setBoolean("VertexLighting", true);

    mat.setBoolean("HighQuality", true);

    mat.setColor("Ambient", new ColorRGBA(0.8f,0.8f,0.8f,1.0f));

    mat.setColor("Diffuse", new ColorRGBA(3.8f,3.8f,3.8f,1.0f));



    mat.setTexture("DiffuseMap", diff);

    mat.setTexture("NormalMap", norm);

    mat.setFloat("Shininess", 2.0f);





    AmbientLight al = new AmbientLight();

    al.setColor(new ColorRGBA(1.8f, 1.8f, 1.8f, 1.0f));

    rootNode.addLight(al);



    DirectionalLight sun = new DirectionalLight();

    sun.setDirection(new Vector3f(.2f, -1f, .2f));

    sun.setColor(new ColorRGBA(0.6f, 0.6f, 0.6f, 1.0f));

    rootNode.addLight(sun);



    model = (Geometry) assetManager.loadModel("Models/Sponza.j3o");

    model.getMesh().scaleTextureCoordinates(new Vector2f(2, 2));



    model.setMaterial(mat);



    rootNode.attachChild(model);



    FilterPostProcessor fpp = new FilterPostProcessor(assetManager);

    GraphicNovelFilter gnf = new GraphicNovelFilter(0.55f,42.5f,0.005f,0.025f); //(6.940201f, 19.928635f, 1f, 0.25f);

    fpp.addFilter(gnf);



    viewPort.addProcessor(fpp);

    }



    @Override

    public void simpleUpdate(float tpf) {

    //TODO: add update code

    }



    @Override

    public void simpleRender(RenderManager rm) {

    //TODO: add render code

    }

    }[/java]



    The assets I am using are all in test. You can either change the file paths in Main… or copy them to the Models & Textures directories under Assets.



    Models:

    Sponza.j3o



    Textures:

    BrickWall.jpg

    BrickWall_normal.jpg



    There are a couple bits in the mat defs that are leftover / unused declarations… but, mostly… this is all cleaned up. Let me know if I missed something.
3 Likes

Very nice.

What are exactly the difference with the CartoonEdgeFilter?

It adds some self occlusion on plane areas?

@nehon said:
Very nice.
What are exactly the difference with the CartoonEdgeFilter?
It adds some self occlusion on plane areas?


I haven't looked at how the CartoonEdgeFilter is done, so I hope I answer this correctly.

This uses gnats-ass 3-axis reflection... setting the sample radius relatively high, the intensity VERY high and then scaling the back the results to find both edges and object borders. It's based off the detail pass of the SSAO filter I am working on. I just removed the second pass that does the broader occlusion, and whacked out the setting to trace edges from all sides.

I honestly thought it was going to take a little more than it did to come up with these results. I realized it was going to be easy when I discovered I had test code in the SSAO map shader that was knocking out CONSIDERABLE amounts of detail. The code was a placeholder for distance falloff & I had forgotten about it (embarrassing...).

Oh… btw.



I tried setting up the sample directions (the vector array) in the shader as a constant and it renders a single frame and then locks up. Any idea why this would happen? Is that why you pass this into the original SSAO filter?

@t0neg0d said:
I was wanting to do just that (cross hatch... actually... was going for just single direction diagonal lines.. but still)... but have no clue where to start.


A texture based cross-hatch filter might look ok. Find a repeating crosshatch texture and then just mix it in based on some value. Finding the appropriate texture could be tricky. The math to simulate one might not be hard but I haven't really thought about it.

I look forward to trying what you have, though.

very nice!