CartoonEdgeFilter ignores my terrain. Why?

Hi all, I am trying out the CartoonEdgeFilter in my game!

It looks good! However there is one problem: It ignores my terrain.

Hence, the terrain does not have a cartoon edge. And, even worse, the cartoon edge appears through the terrain!!

The terrain is created like this:

  public void setUpTerrain(){
  //assetManager.clearCache();
  treeShape = null;
  if (terrain != null){
      terrain.removeFromParent();
      
  }
  if (landscape != null){
      
      bulletAppState.getPhysicsSpace().remove(landscape);
      landscape.destroy();
  }
  while (CURRENT_MAP == 0 || CURRENT_MAP == 1000){
    //Do nothing
    try{
    Thread.sleep(100);
    }catch (Exception e2){
        
    }
    System.out.println("Waiting for server to choose the map!");
}
  if (CURRENT_MAP <= 4){
      if (guiNode.hasChild(pic1)){
         pic1.removeFromParent();
     }
  mat_terrain = new Material(assetManager, 
        "Common/MatDefs/Terrain/Terrain.j3md");
//mat_terrain.setFloat("Shininess", 5f);

//We need to wait until the server has chosen a map. Waiting here...

 //mat_terrain.setBoolean("isTerrainGrid", false);
//mat_terrain.setBoolean("useTriPlanarMapping", false);
// mat_terrain.setBoolean("WardIso", true);
//mat_terrain.setFloat("Shininess", 0);
 /** 1.1) Add ALPHA map (for red-blue-green coded splat textures) */
mat_terrain.setTexture("Alpha", assetManager.loadTexture(
        "Textures/Terrain/splat/" + CURRENT_MAP + "/Alpha.jpg"));
int anisotropic = 16;
float tile = 1.1f;
/** 1.2) Add GRASS texture into the red layer (Tex1). */
String grassTex = "Textures/Terrain/splat/" + CURRENT_MAP + "/Grass.png";
String rockTex = "Textures/Terrain/splat/" + CURRENT_MAP + "/Dirt2.png";
String dirtTex = "Textures/Terrain/splat/" + CURRENT_MAP + "/Sand.png";
if (false){ //JME textures, don't use these at the 
     grassTex = "Textures/Terrain/grass.jpg";
     rockTex = "Textures/Terrain/road.jpg";
     dirtTex = "Textures/Terrain/dirt.jpg";
     tile*=2;
}
Texture normM = assetManager.loadTexture("Textures/Terrain/road_normal.png");
normM.setWrap(WrapMode.Repeat);
Texture grass = assetManager.loadTexture(
        grassTex);
if (!minimalQuality){
grass.setAnisotropicFilter(anisotropic);
normM.setAnisotropicFilter(anisotropic);
}
grass.setWrap(WrapMode.Repeat);
mat_terrain.setTexture("Tex1", grass);
mat_terrain.setFloat("Tex1Scale", 32f * tile);
//mat_terrain.setTexture("NormalMap", normM);
/** 1.3) Add DIRT texture into the green layer (Tex2) */
Texture dirt = assetManager.loadTexture(
        dirtTex);
if (!minimalQuality){
dirt.setAnisotropicFilter(anisotropic);
}
dirt.setWrap(WrapMode.Repeat);
mat_terrain.setTexture("Tex2", dirt);
mat_terrain.setFloat("Tex2Scale", 32f * tile);
//mat_terrain.setTexture("NormalMap_1", normM);
/** 1.4) Add ROAD texture into the blue layer (Tex3) */
Texture rock = assetManager.loadTexture(
        rockTex);
if (!minimalQuality){
rock.setAnisotropicFilter(anisotropic);
}
rock.setWrap(WrapMode.Repeat);
mat_terrain.setTexture("Tex3", rock);
mat_terrain.setFloat("Tex3Scale", 24f * tile);
//mat_terrain.setTexture("NormalMap_2", normM);
  }else{
      setUpTerrainTextures(assetManager);
  }
/** 3. We have prepared material and heightmap. 
 * Now we create the actual terrain:
 * 3.1) Create a TerrainQuad and name it "my terrain".
 * 3.2) A good value for terrain tiles is 64x64 -- so we supply 64+1=65.
 * 3.3) We prepared a heightmap of size 512x512 -- so we supply 512+1=513.
 * 3.4) As LOD step scale we supply Vector3f(1,1,1).
 * 3.5) We supply the prepared heightmap itself.
 */
int patchSize = 65 / 5;
terrain = assetManager.loadModel("Textures/Terrain/splat/" + CURRENT_MAP + "/MeshVLow.obj");//new TerrainQuad("Map", patchSize, 513, heightmap.getHeightMap());
/** 4. We give the terrain its material, position & scale it, and attach it. */
terrain.setMaterial(mat_terrain);
 //mat_terrain.setBoolean("useTriPlanarMapping", false);
 //mat_terrain.setBoolean("WardIso", true);
terrain.setLocalTranslation(-2500, 100, -2500);
int mu = 1;
terrain.setLocalScale(0.5f * mu, 0.25f * mu, 0.5f * mu);
collideable.attachChild(terrain);
collideable.attachChild(trees);
rootNode.attachChild(collideable);
/** 5. The LOD (level of detail) depends on were the camera is: */
//TerrainLodControl control = new TerrainLodControl(terrain, getCamera());
//control.setTerrain(terrain);
//terrain.addControl(control);
//stateManager.attach(bulletAppState);
    bulletAppState.setDebugEnabled(false);
CollisionShape sceneShape =
        CollisionShapeFactory.createMeshShape(terrain);
landscape = new RigidBodyControl(sceneShape, 0f);
landscape.setFriction(1f);
terrain.addControl(landscape);
bulletAppState.getPhysicsSpace().add(landscape);

/* //Set up JME's town.zip as a test world.
assetManager.registerLocator("town.zip", ZipLocator.class);
Spatial sceneModel = assetManager.loadModel("main.scene");
sceneModel.setLocalScale(20f);
sceneModel.setLocalTranslation(-500, 150, 0);
sceneModel.setShadowMode(ShadowMode.CastAndReceive);
// We set up collision detection for the scene by creating a
// compound collision shape and a static RigidBodyControl with mass zero.
CollisionShape sceneShape2 =
        CollisionShapeFactory.createMeshShape((Node) sceneModel);
RigidBodyControl landscape2 = new RigidBodyControl(sceneShape2, 0);
sceneModel.addControl(landscape2);
collideable.attachChild(sceneModel);
bulletAppState.getPhysicsSpace().add(landscape2);
 */

//Set ground for the two flags
Vector3f lor = Ray(new Vector3f(RED_FLAG.x, 50000, RED_FLAG.z), new Vector3f(0, -1, 0));

Vector3f lob = Ray(new Vector3f(BLUE_FLAG.x, 50000, BLUE_FLAG.z), new Vector3f(0, -1, 0));

RED_FLAG = lor;
BLUE_FLAG = lob;
needsLoad = false; //Terrain does not need loading
health = 0; //Reset health (if a new terrain is loaded, this means you need to respawn!).
}

Yes, the terrain code is messy and needs tidying up and some is copied from tutorials. Don’t kill me please. :stuck_out_tongue:

I just don’t understand why this this issue is happening.

The terrain uses a custom material and shader (slightly modified from Lighting.j3md).

Edit: issue only occurs if CURRENT_MAP <= 4.

How are you setting up the filter ? I use cartoon edge filter and it works on my terrain.

fpp.addFilter(new CartoonEdgeFilter());

I mean the actual settings and order you added the processor. The order you add the filters can drastically change the output. This is the order I add mine.

Shadow filter → Water Filter → Edge Filter → Fog filter

private void initEdgeFilter(){
        if (gameManager.app.getRenderer().getCaps().contains(Caps.GLSL100)){
            filter.setNumSamples(4);
            stencilFilter = new CartoonEdgeFilter();
            stencilFilter.setEdgeColor(ColorRGBA.Black);
            stencilFilter.setEdgeWidth(control.getStencilEdgeWidth());
            stencilFilter.setEdgeIntensity(control.getStencilEdgeIntensity());
            setEnableStencil(control.isEnableStencil());  
            filter.addFilter(stencilFilter);
        }
    }

I see your point, but the order of the filter is not the problem. But if it helps, the screenshot was before the fog & FXAA & Translucent but after the Water & Shadow.

Shadow filter → Water Filter → Edge Filter → Fog filter

The order you add them is EXTREMELY important. I learned through trial and error. The above order gave me the least amount of problems. It looks like your water filter is mixing the transparency with your edge filter.

On a side note if you add a water filter with all white colors after you edge filter it will make your scene look like a black and white comic book but the frame rate will be awful.

Don’t add the edge filter after the water filter this will cause them to mix and will give you an extreme drop in frame rate.

1 Like

I understand your point, and thanks. Though it does not help to solve the current situation.

(And also putting the shadow filter before water looks horrible. And putting water before shadow looks horrible. So meh. ;()

Here is a sample of my render set up.

I had similar problems where my scene was rendering just like the one in the photo. I solved it by playing the filter shuffle. What water filter are you using basic or advanced and have you tried running your scene minus the water filter ?

Here are several things to check if you have a custom material and want the edge filter to work:
It must render depth, it must have the NormalPass technique in the j3md.

1 Like

Look what I found … maybe it’s in the wrong bucket? :slight_smile:

(from the Wiki)

Put the Geometry (not the Material!) in the appropriate render queue bucket.
Objects in the translucent bucket (e.g. particles) are not affected by
SceneProcessors (e.g. shadows). Objects in the transparent bucket (e.g.
foliage) are affected by SceneProcessors (e.g. shadows).
geo.setQueueBucket(Bucket.Translucent);
geo.setQueueBucket(Bucket.Transparent);

I have worked out that the issue occurs with Terrain.j3md only. Is this what the issue is?