FXAA filter removes transparency

Ali_RS · December 5, 2019, 2:07pm

Hi

I noticed using FXAA filter in my icon generator causes the generated PNG to have no transparency.

Diving into shader code I noticed here in the fragment shader

jMonkeyEngine/jmonkeyengine/blob/98f6d326e1d062aa4e1103e4729c02f0c0f16cc1/jme3-effects/src/main/resources/Common/MatDefs/Post/FXAA.frag#L88


    }
    else
    {
        return rgbB; 
    }
}

void main()
{

    gl_FragColor = vec4(FxaaPixelShader(posPos, m_Texture, g_ResolutionInverse), 1.0);
}

it sets gl_FragColor alpha to 1.0, is this done intentionally?

or it should change to this:

vec4 texVal = texture2D(m_Texture, texCoord);
gl_FragColor = vec4(FxaaPixelShader(posPos, m_Texture, g_ResolutionInverse), texVal.a);

Note, not a problem for me as I am using regular AA sampling, I just wanted to know if this needs to be fixed in the shader.

joliver82 · April 7, 2021, 4:37pm

Your change looks good but I don’t know the inner workings and passes of the FXAA filter and if this change could harm the effect result. I’ll have a look at it later today and give you more feedback

joliver82 · April 8, 2021, 11:23am

Just checked the filter code, and it’s just a simple pass one. I’ve not tested it but setting proper alpha shouldn’t change the rendering result in this case.

I had a similar issue using SSAO wiht the screen capture thing you help me with here

When adding the filter the background went black and I had to add the following code

if(texture2D(m_Normals, texCoord).a < 0.1)
  discard

at the beginning of ssao.frag main: jmonkeyengine/ssao.frag at 98f6d326e1d062aa4e1103e4729c02f0c0f16cc1 · jMonkeyEngine/jmonkeyengine · GitHub

I’m using the exact same shader for regular SSAO filtering on the rendered viewport without issues

Ali_RS · April 8, 2021, 11:44am

Thanks for the feedback. Will submit the patch soon.

Edit: