Fog Filter Questions - Code added for possible depth fix

I’ve been think… which is usually painful… and dangerous for those around me.



I switched from using the fog filter to shader based fog for 2 reasons.



The first (and most obvious) was the lack of being able to set a start distance. This is a one uniform fix to the shader that renders the fog.



The other was that it rendered over the sky bucket, which kinda defeated the purpose of a skybox…



My first question is… the sky bucket is rendered at depth 1.0 always, correct? So, I can exclude this from the filter easily enough?



My next question is for anyone who has had problems with transparency and this filter… I’m interested in compiling a list of issues to revisit using this instead of shader based fog. The shader based fog has it’s own set of issues with transparency.



And this leads me to my last question… why is there a transparency bucket and translucent bucket? Both have issues with render order in some cases (which I totally understand is a case by case basis… and their is no way to account for every circumstance… as one will destroy the fix for another).



The reason I ask why is… what was the original purpose of these? They were added to fix certain issues… what were they? Is there a list of these some place? I seem to have better success rendering transparent objects directly to the opaque bucket. /shrug



Thanks in advance!

Actually… it seems the fog filter does allow for a start distance… why do I remember this not working correctly. Probably because I was not working correctly :wink:

I take it back. The fog distance is not working correctly. :frowning:

My understanding is that transparent bucket is need to render transparent objects at the right time and order. It renders after the opaque bucket and sky bucket… and orders the objects back to front. And then post processing is run on that.



The translucent bucket is for when you need transparent objects that are run after post processing… like then you have things that need to render on top of post-processed water.

1 Like

Ah… I see… the fog distance isn’t the start distance… the distance is working correctly… just not what I would have wanted is all. I would think a start distance would be more useful as some games use fog to set visual boundaries and don’t necessarily want fog rendered over everything from starting from their camera position. I’m guessing you would only need to do is pass in the camera frustum. to calc the start distance.



Not that anyone really cares, but is there a way to request changes to these filters? I can implement the changes for my purposes… buuuuut…



There are a million reasons why a start distance would be better than not… the other 1.0 distance flag is a no-brainer. No reason for a sky box, if you can never see it. And blending techniques for fog to sky are easier than not to implement yourself.

Transparent vs translucent bucket.

It’s all about the rendering order, geometry sorting and processors. Render goes like this for each viewport:

  • for each processor call preFrame
  • Dispatching each geom in corresponding renderQueue (one for each Bucket) and building shadow queues
  • for each processor call postQueues
  • Rendering OpaqueBucket with object sorted front to back.
  • Rendering SkyBucket with depth forced to 1.0. this means every object of this bucket will be far away and behind everything
  • Rendering TransparentBucket with object sorted back to front.
  • for each processor call postFrame
  • Rendering TranslucentBucket with objects sorted back to front.



    The translucent bucket is rendered at the end. That’s where you put transparent object that you don’t want to be affected by post processing ( shadows or what ever). self light emitting particle emitters are a good examples (A fire).

    Post processors are not applied to this bucket with one exception : the FilterPostProcessor.

    The filter post processor hijack the rendering process and renders a full screen quad with the texture of the scene applied on it.

    Once it’s done the depth buffer is 0, so it’s impossible to render a queue over it with proper depth test.

    So if you use a FilterPostProcessor you have to add at the end of your filter stack the TranslucentBucketFilter. It will handle the translucent bucket rendering in the stead of the RenderManager. (of course the correct depth information is passed to the filter).

    The nice side effect is that if you want to apply a post filter to your translucent bucket (like bloom for example) you can just push up the translucent bucket filter in the filter stack.
4 Likes

So now about the fog filter, i don’t really remember but the fog distance is supposed to be what you want.

So maybe there is an issue.

@nehon said:
- Rendering OpaqueBucket with object sorted back to front.


The opaque bucket is rendered front to back to minimize overdraw. Transparent bucket is rendered back to front so that blending happens properly.
@nehon said:
So now about the fog filter, i don't really remember but the fog distance is supposed to be what you want.
So maybe there is an issue.


Oh... I think there is an issue then. The way the code reads now, it seems the fog is rendered from m_FogDistance to 1.0... which is correct. But, I am seeing results of the fog being rendered from 0.0 to either m_FogDistance or 1.0... hard to tell which, but I think it is m_FogDistance (going from memory here). This is more and more noticable as you increase m_FogDensity.

I'll grab some screenshots in bit.

I remember how I noticed this…



I was trying to use this fogFactor equation as a basis for doing distance falloff and couldn’t seem to get proper results. The falloff would happen starting from 0.0 no matter what I did.



Does it have something to do with the NearFar frustum being set to (1.0, m_FogDistance)… being that m_FogDistance is… say… 800f. Wouldn’t setting the far distance to the camera’s actual culling distance work?



Just a thought.

This should fix it:



EDIT: m_FrustumNearFar.x needs to be fractioned somehow to fix the scale… but I’m not sure how. The basic idea is there though. / 4.0 seems to place the fog in the correct depth… but… I think someone smarter than me needs to review this.



[java]float depth= (m_FrustumNearFar.x / 4.0) /

(m_FrustumNearFar.y - fogVal *

(m_FrustumNearFar.y));[/java]

Here is a screenshot comparison of the original and the fix using the following settings:



[java]FogFilter fog=new FogFilter();

fog.setFogColor(new ColorRGBA(0.3f, 0.5f, 0.3f, 1.0f));

fog.setFogDistance(800f);

fog.setFogDensity(12.0f);

fpp.addFilter(fog);[/java]



The original:





With the fix:

1 Like

If anyone is interested… here is an updated version with both the fix and a new uniform for excluding skybox rendering:



NOTE: This ONLY fixes the GLSL100 version.

ALSO: These are pointing to local copies… soooo… the Material Def file would go in MatDef, the .frag file would go into Shaders… you can put the filter where ever you like.



FogFilter.java:

[java]

/*

  • Copyright © 2009-2012 jMonkeyEngine
  • All rights reserved.

    *
  • Redistribution and use in source and binary forms, with or without
  • modification, are permitted provided that the following conditions are
  • met:

    *
    • Redistributions of source code must retain the above copyright
  • notice, this list of conditions and the following disclaimer.

    *
    • Redistributions in binary form must reproduce the above copyright
  • notice, this list of conditions and the following disclaimer in the
  • documentation and/or other materials provided with the distribution.

    *
    • Neither the name of ‘jMonkeyEngine’ nor the names of its contributors
  • may be used to endorse or promote products derived from this software
  • without specific prior written permission.

    *
  • THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
  • "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
  • TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
  • PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
  • CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
  • EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
  • PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
  • PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
  • LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
  • NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
  • SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

    */



    import com.jme3.asset.AssetManager;

    import com.jme3.export.InputCapsule;

    import com.jme3.export.JmeExporter;

    import com.jme3.export.JmeImporter;

    import com.jme3.export.OutputCapsule;

    import com.jme3.material.Material;

    import com.jme3.math.ColorRGBA;

    import com.jme3.math.Vector2f;

    import com.jme3.post.Filter;

    import com.jme3.renderer.RenderManager;

    import com.jme3.renderer.ViewPort;

    import java.io.IOException;



    /**
  • A filter to render a fog effect
  • @author Rémy Bouquet aka Nehon

    */

    public class FogFilter extends Filter {



    private ColorRGBA fogColor = ColorRGBA.White.clone();

    private float fogDensity = 0.7f;

    private float fogDistance = 1000;

    private boolean excludeSky = false;



    /**
  • Creates a FogFilter

    */

    public FogFilter() {

    super("FogFilter");

    }



    /**
  • Create a fog filter
  • @param fogColor the color of the fog (default is white)
  • @param fogDensity the density of the fog (default is 0.7)
  • @param fogDistance the distance of the fog (default is 1000)

    */

    public FogFilter(ColorRGBA fogColor, float fogDensity, float fogDistance) {

    this();

    this.fogColor = fogColor;

    this.fogDensity = fogDensity;

    this.fogDistance = fogDistance;

    }



    @Override

    protected boolean isRequiresDepthTexture() {

    return true;

    }



    @Override

    protected void initFilter(AssetManager manager, RenderManager renderManager, ViewPort vp, int w, int h) {

    material = new Material(manager, "MatDefs/Fog.j3md");

    material.setColor("FogColor", fogColor);

    material.setFloat("FogDensity", fogDensity);

    material.setFloat("FogDistance", fogDistance);

    material.setBoolean("ExcludeSky", excludeSky);

    }



    @Override

    protected Material getMaterial() {



    return material;

    }





    /**
  • returns the fog color
  • @return

    */

    public ColorRGBA getFogColor() {

    return fogColor;

    }



    /**
  • Sets the color of the fog
  • @param fogColor

    */

    public void setFogColor(ColorRGBA fogColor) {

    if (material != null) {

    material.setColor("FogColor", fogColor);

    }

    this.fogColor = fogColor;

    }



    /**
  • returns the fog density
  • @return

    */

    public float getFogDensity() {

    return fogDensity;

    }



    /**
  • Sets the density of the fog, a high value gives a thick fog
  • @param fogDensity

    */

    public void setFogDensity(float fogDensity) {

    if (material != null) {

    material.setFloat("FogDensity", fogDensity);

    }

    this.fogDensity = fogDensity;

    }



    /**
  • returns the fog distance
  • @return

    */

    public float getFogDistance() {

    return fogDistance;

    }



    /**
  • the distance of the fog. the higer the value the distant the fog looks
  • @param fogDistance

    */

    public void setFogDistance(float fogDistance) {

    if (material != null) {

    material.setFloat("FogDistance", fogDistance);

    }

    this.fogDistance = fogDistance;

    }



    /**
  • Sets the exclude sky flag
  • @param excludeSky

    */

    public void setExcludeSky(boolean excludeSky) {

    if (material != null) {

    material.setBoolean("ExcludeSky", excludeSky);

    }

    this.excludeSky = excludeSky;

    }



    /**
  • returns the exclude sky flag
  • @return

    */

    public boolean getExcludeSky() {

    return excludeSky;

    }



    @Override

    public void write(JmeExporter ex) throws IOException {

    super.write(ex);

    OutputCapsule oc = ex.getCapsule(this);

    oc.write(fogColor, "fogColor", ColorRGBA.White.clone());

    oc.write(fogDensity, "fogDensity", 0.7f);

    oc.write(fogDistance, "fogDistance", 1000);

    oc.write(excludeSky, "excludeSky", false);

    }



    @Override

    public void read(JmeImporter im) throws IOException {

    super.read(im);

    InputCapsule ic = im.getCapsule(this);

    fogColor = (ColorRGBA) ic.readSavable("fogColor", ColorRGBA.White.clone());

    fogDensity = ic.readFloat("fogDensity", 0.7f);

    fogDistance = ic.readFloat("fogDistance", 1000);

    excludeSky = ic.readBoolean("excludeSky", false);

    }





    }[/java]



    Material definition file (Fog.j3md):

    [java]MaterialDef Fog {



    MaterialParameters {

    Int NumSamples

    Int NumSamplesDepth

    Texture2D Texture

    Texture2D DepthTexture

    Vector4 FogColor;

    Float FogDensity;

    Float FogDistance;

    Boolean ExcludeSky : false

    }



    Technique {

    VertexShader GLSL100: Common/MatDefs/Post/Post.vert

    FragmentShader GLSL100: Shaders/Fog.frag



    WorldParameters {

    WorldViewProjectionMatrix

    }

    }



    Technique FixedFunc {

    }



    }[/java]



    Shader (Fog.frag):

    [java]uniform sampler2D m_Texture;

    uniform sampler2D m_DepthTexture;

    varying vec2 texCoord;



    uniform vec4 m_FogColor;

    uniform float m_FogDensity;

    uniform float m_FogDistance;

    uniform bool m_ExcludeSky;



    const float LOG2 = 1.442695;



    void main() {

    vec2 m_FrustumNearFar=vec2(1.0,m_FogDistance);

    vec4 texVal = texture2D(m_Texture, texCoord);

    float fogVal = texture2D(m_DepthTexture,texCoord).r;

    float depth= (m_FrustumNearFar.x / 4.0) /

    (m_FrustumNearFar.y - fogVal *

    (m_FrustumNearFar.y));



    float fogFactor = exp2( -m_FogDensity * m_FogDensity * depth * depth * LOG2 );

    fogFactor = clamp(fogFactor, 0.0, 1.0);

    if (m_ExcludeSky) {

    if (depth < 1.0)

    gl_FragColor = mix(m_FogColor,texVal,fogFactor);

    else

    gl_FragColor = texVal;

    } else

    gl_FragColor = mix(m_FogColor,texVal,fogFactor);

    }[/java]
2 Likes

Also… a quick FYI. There are a few filters that use this same code… the Fade filter comes to mind, off the top of my head.

I haven’t looked at all of the code but:


@t0neg0d said:
fog.setFogDistance(800f);


Should mean that fog is 100% at 800 meters. This is very common since many times you want to use fog to hide that your scene is only rendering out to 800 meters. If you need a separate near start distance then that's a different setting.
@pspeed said:
I haven't looked at all of the code but:

Should mean that fog is 100% at 800 meters. This is very common since many times you want to use fog to hide that your scene is only rendering out to 800 meters. If you need a separate near start distance then that's a different setting.


But if that was the case... you would always have fog starting at 0.0 and never be able to create fog effects aside from that.. only cut out more and more distance. In many games, fog starts out at a particular distance and then quickly ramps. This fix does both. And if I read what @nehon said above properly... this is what it was supposed to do originally.

Think about it in reverse... a falloff starts at a particular distance and then ramps into nothing depending on the intensity. Fog is just a colored version of falloff.

Well yes near and far would be both very usefull. (as i try to hide for example that my terrain ends after 10km, so having fog start at 7 and max at 9.5 would be perfect.

Oh… as well… if you consider intensity in the equation… this would be possible setting for cutting off at 800f with totally different effects in between the camera and the max fog.



setDistance(780f);

setDensity(24f)



By 800f nothing is visible… but there is NO effect between the camera and 780f;



setDistance(400f);

setDensity(12f);



By 800f nothing is visible… but now your fog gradually increases from 400f to 800f.



I think you get the idea… etc, etc.



EDIT: These are pseudo figures… not sure what the exact setting would be… but… /shrug

You need both if you are not going to have a max distance… traditionally “fog distance” has meant “the distance where fog is 100%”. Maybe nehon was confused.



For those who want more fake looking fog then you can also provide a near distance… but the far distance is the critical one. Users shouldn’t have to do math and guesswork to try to figure out what the max fog distance will be from “min” and “density” since many games that need fog will have a hard stop at a certain distance. And since technically density should not be applied linearly, it becomes even harder to guess where fog is 100% for some near + density values.

@pspeed said:
You need both if you are not going to have a max distance... traditionally "fog distance" has meant "the distance where fog is 100%". Maybe nehon was confused.

For those who want more fake looking fog then you can also provide a near distance... but the far distance is the critical one. Users shouldn't have to do math and guesswork to try to figure out what the max fog distance will be from "min" and "density" since many games that need fog will have a hard stop at a certain distance. And since technically density should not be applied linearly, it becomes even harder to guess where fog is 100% for some near + density values.


I'll update this to use both and peeps can use this as an option if needed.