Using fog to mask the sharp edge at the edge of the rendered world

I’m trying to use fog to mask the edge of my rendered world. I’ve been following the test fog example to create a FogFilter but I’m not sure I understand what the FogDistance and FogDensity exactly means. In particular the FogDistance, since the fog seems neither to begin nor end at the FogDistance.

The following is a screenshot of fog on a scene with boxes at 10m intervals with fogDistance set to 40 and fogDensity set to 2

Imgur

The fog in this image was set up as follows

[java]
FilterPostProcessor fpp=new FilterPostProcessor(assetManager);
//fpp.setNumSamples(4);
int numSamples = getContext().getSettings().getSamples();
if( numSamples > 0 ) {
fpp.setNumSamples(numSamples);
}
FogFilter fog=new FogFilter();
fog.setFogColor(new ColorRGBA(0.9f, 0.9f, 0.9f, 1.0f));
fog.setFogDistance(40);
fog.setFogDensity(2f);
fpp.addFilter(fog);
viewPort.addProcessor(fpp);
[/java]

Ideally what I want is a relatively sharp fog that (for example) starts at 35m at no fog and ends at 45m with completely opaque fog, is this the correct technique to use for this?

1 Like

Lets try the screenshot again

As a side note; why can’t I edit my first post but I can edit every other one?

As a side response… :wink:

Type in the url of the thread without the post number but with `/edit’ added
http://hub.jmonkeyengine.org/forum/topic/using-fog-to-mask-the-sharp-edge-at-the-edge-of-the-rendered-world/edit

Edit: When I get frustrated, I tend to just hit things with bricks until they work (or my arm gets tired) :stuck_out_tongue:

With the fog, I’d play around with both setting tied to a key mapping to see the effects real-time :slight_smile:

That’s roughly how (without the real-time and with a really annoying delay between iterations) I am roughing in the behavior of various aspects of the Aurora engine, particularly the effects and txi (texture directives).

1 Like

@foxhavendesigns thanks for the tip regarding /edit!

Playing around with vaules I am able to get a relatively sharp fog by having extreme values for both distance and density. But that doesn’t seem right somehow. Additionally I want to set the render distance as being user defined so I would ideally like not to have to play around with the numbers for every supported render distance

[java]fog.setFogDistance(1000000000);
fog.setFogDensity(15000000f);[/java]

Also; its still not as sharp as I would like and I’m hitting the maximum value for an integer if I go much higher on the FogDistance. What I’m looking for would be cubes 1-3 basically unfogged, cube 4 foggy, cube 5+ invisible.

You may need to customize a shader to get this effect and use vertex based fog instead of post-proc fog.

OpenGL’s implementation of Fog has multiple types of density:

Camera To Distance (as linear or exponential)
Distance To Infinity (as linear or exponential)
Linear (between two distances)
Exponential (JME’s Fog Filter uses this one–starting from the Camera’s position)

http://www.opengl.org/sdk/docs/man2/xhtml/glFog.xml

You may want to look into implementing these options as an alternative.

Also, depending on the view the user has into your rendered world, the Fog Filter (or shader based alternative) may not be what you want to use to accomplish this.

2 Likes
@richtea said: I'm hitting the maximum value for an integer if I go much higher on the FogDistance.

I realize this doesn’t address all your concerns, but please note that FogFilter.setFogDistance() takes a float parameter, so you’re not limited to Integer.MAX_VALUE .

Interesting, I’ll have a play and post what I come up with

Ok, I’ve come up with a Mark-1 edge fog using a fragment shader (shaders are awesome)

In this image fog begins at 40m and is 100% at 50m. The fog start (fogDistanceNear) can be placed anywhere and will increase linearly towards 100% for at the fogDistanceFar (which must be the camera Far Frustrum)

Filter
[java]
import com.jme3.asset.AssetManager;
import com.jme3.export.InputCapsule;
import com.jme3.export.JmeExporter;
import com.jme3.export.JmeImporter;
import com.jme3.export.OutputCapsule;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.post.Filter;
import com.jme3.renderer.RenderManager;
import com.jme3.renderer.ViewPort;
import java.io.IOException;

/**

  • A filter to render a fog effect

  • @author Rémy Bouquet aka Nehon
    */
    public class EdgeFogFilter extends Filter {

    private ColorRGBA fogColor = ColorRGBA.White.clone();
    //private float fogDensity = 0.7f;
    private float fogDistanceFar = 1000;
    private float fogDistanceNear = 800;
    //private float fractionToFog = 0.1f;
    /**

    • Creates a FogFilter
      */
      public EdgeFogFilter() {
      super(“FogFilter”);
      }

    /**

    • Creates edge fog to mask the edge of the world
    • @param fogColor
    • @param fogDensity
    • @param fogDistanceNear
    • @param fogDistanceFar
      */
      public EdgeFogFilter(ColorRGBA fogColor, float fogDensity,float fogDistanceNear, float fogDistanceFar) {
      this();
      this.fogColor = fogColor;
      this.fogDistanceFar = fogDistanceFar;
      this.fogDistanceNear=fogDistanceNear;
      }

    @Override
    protected boolean isRequiresDepthTexture() {
    return true;
    }

    @Override
    protected void initFilter(AssetManager manager, RenderManager renderManager, ViewPort vp, int w, int h) {
    material = new Material(manager, “Materials/EdgeFog.j3md”);
    //material = new Material(manager, “Common/MatDefs/Post/Fog.j3md”);
    material.setColor(“FogColor”, fogColor);
    material.setFloat(“FogDistanceFar”, fogDistanceFar);
    material.setFloat(“FogDistanceNear”, fogDistanceNear);
    }

    @Override
    protected Material getMaterial() {

     return material;
    

    }

    /**

    • returns the fog color
    • @return
      */
      public ColorRGBA getFogColor() {
      return fogColor;
      }

    /**

    • Sets the color of the fog
    • @param fogColor
      */
      public void setFogColor(ColorRGBA fogColor) {
      if (material != null) {
      material.setColor(“FogColor”, fogColor);
      }
      this.fogColor = fogColor;
      }

    /**

    • returns the fog Far distance, the point of 100% fog and the far frustrum
    • @return
      */
      public float getFogFarDistance() {
      return fogDistanceFar;
      }

    /**

    • must be the camera far frustrum, point at which fog is 100%
    • @param fogDistanceFar
      */
      public void setFogFarDistance(float fogDistanceFar) {
      if (material != null) {
      material.setFloat(“FogDistanceFar”, fogDistanceFar);
      }
      this.fogDistanceFar = fogDistanceFar;
      }

    /**

    • Point at which it begins to get foggy. Everything closer than this
    • is 0% foggy
    • @param fogDistanceNear
      */
      public void setFogNearDistance(float fogDistanceNear) {
      if (material != null) {
      material.setFloat(“FogDistanceNear”, fogDistanceNear);
      }
      this.fogDistanceNear = fogDistanceNear;
      }

    @Override
    public void write(JmeExporter ex) throws IOException {
    super.write(ex);
    OutputCapsule oc = ex.getCapsule(this);
    oc.write(fogColor, “fogColor”, ColorRGBA.White.clone());
    oc.write(fogDistanceFar, “fogDistanceFar”, 1000);
    oc.write(fogDistanceNear, “fogDistanceNear”, 800);
    }

    @Override
    public void read(JmeImporter im) throws IOException {
    super.read(im);
    InputCapsule ic = im.getCapsule(this);
    fogColor = (ColorRGBA) ic.readSavable(“fogColor”, ColorRGBA.White.clone());
    fogDistanceFar = ic.readFloat(“fogDistanceFar”, 1000);
    fogDistanceNear = ic.readFloat(“fogDistanceNear”, 1000);
    }
    }[/java]

Material - EdgeFog.j3md

[java]
MaterialDef Fade {

MaterialParameters {
    Int NumSamples
    Int NumSamplesDepth
    Texture2D Texture
    Texture2D DepthTexture
    Vector4 FogColor;
    Float FogDistanceFar;
    Float FogDistanceNear;
}

Technique {
    VertexShader GLSL150:   MatDefs/EdgeFog15.vert
    FragmentShader GLSL150: MatDefs/EdgeFog15.frag

    WorldParameters {
    }

    Defines {
        RESOLVE_MS : NumSamples
        RESOLVE_DEPTH_MS : NumSamplesDepth
    }
}

}
[/java]

Fragment Shader - EdgeFog15.frag

[java]
#import “Common/ShaderLib/MultiSample.glsllib”

uniform COLORTEXTURE m_Texture;
uniform DEPTHTEXTURE m_DepthTexture;

uniform vec4 m_FogColor;
uniform float m_FogDistanceFar;
uniform float m_FogDistanceNear;
in vec2 texCoord;

vec2 m_FrustumNearFar=vec2(1,m_FogDistanceFar);
const float LOG2 = 1.442695;

void main() {

   vec4 texVal = getColor(m_Texture, texCoord);
   float fogVal = getDepth(m_DepthTexture,texCoord).r; //goes as 1/z
   
   float Near=1;
   float Far=m_FogDistanceFar;
   
   //between 0 and 1, linear
   float depth= (2.0 * Near) / (Far + Near - fogVal * (Far - Near));

   //between 0 and 1, linear
   float fogStart=m_FogDistanceNear/m_FogDistanceFar;
   float fogExistsOverDistance=1-fogStart;

   float depthThroughFog=(depth-fogStart)/fogExistsOverDistance;
   float fogIntensity=depthThroughFog;
   fogIntensity=clamp(fogIntensity,0.0,1.0);
   
   gl_FragColor =mix(texVal,m_FogColor,fogIntensity);

}
[/java]

Vertex Shader - EdgeFog15.vert (same as Post15.vert)

[java]
in vec4 inPosition;
in vec2 inTexCoord;

out vec2 texCoord;

void main() {
vec2 pos = inPosition.xy * 2.0 - 1.0;
gl_Position = vec4(pos, 0.0, 1.0);
texCoord = inTexCoord;
}
[/java]

Setup within project

The fragment shaders, vertex shaders and material should be placed as follows:

.

Usage example

[java]public class Main extends SimpleApplication {

private FilterPostProcessor fpp;

public static void main(String[] args) {

    Main app = new Main();
    app.start();
}

public void simpleInitApp() {
    this.flyCam.setMoveSpeed(10);
    Node mainScene=new Node();
    cam.setLocation(new Vector3f(0f, 0f, 0f));
    cam.lookAt(new Vector3f(0f, 0f, 1f), Vector3f.UNIT_Y);
    
    float farFrustrum=50;

    cam.setFrustumFar(farFrustrum);
    
    for(int i=1;i<100;i++){
        createBoxAtPosition(new Vector3f(5,0,10*i));
    }

    rootNode.attachChild(mainScene);

    fpp=new FilterPostProcessor(assetManager);
    //fpp.setNumSamples(4);
    int numSamples = getContext().getSettings().getSamples();
    if( numSamples > 0 ) {
        fpp.setNumSamples(numSamples);
    }
    
    EdgeFogFilter fog=new EdgeFogFilter();
    fog.setFogColor(new ColorRGBA(0.9f, 0.9f, 0.9f, 1.0f));
    //fog.setFogNearDistance(40);
    fog.setFogFarDistance(farFrustrum);
    fog.setFogNearDistance(30);
    fpp.addFilter(fog);
    viewPort.addProcessor(fpp);
}

private void createBoxAtPosition(Vector3f position){
    Box b = new Box(1, 1, 1); // create cube shape
    Geometry geom = new Geometry("Box", b);  // create cube geometry from the shape
    Material mat = new Material(assetManager,
      "Common/MatDefs/Misc/Unshaded.j3md");  // create a simple material
    mat.setColor("Color", ColorRGBA.Blue);   // set color of material to blue
    geom.setMaterial(mat);                   // set the cube's material
    geom.setLocalTranslation(position);
    rootNode.attachChild(geom);              // make the cube appear in the scene
    
    
}

}[/java]

Issues

setFogFarDistance must be set as being the camera FarFrustrum. or else the maths goes wrong. If I could get the FarFrustrum as a uniform within my fragment shader I could remove this dependancy.

Its currently using depth from the camera plane rather than distance from the camera so as you turn your view left and right things come in and out of fog at the corner of your view.

1 Like