Alpha To Coverage does't work properly with some post-process filters

Alpha To Coverage looks great, but with any filter, that does something with depth, smooth edges become dark.



I noticed it with PSSM Filter, but same issue appear with Fog or Depth of Field filters.



Screenshots of PSSM filer:

https://dl.dropbox.com/u/8845134/dev/ATC.jpg



and Fog:

https://dl.dropbox.com/u/8845134/dev/ATC_Fog.jpg

Yes, based on my reading, alpha to coverage is really not a solution for semi-transparent objects. It’s really for creating proper edges for textures that have some pixels fully transparent.



Even in your first pictures, I think your fuzz ball thing has issues with depth sorting causing bits of the background to show through where they shouldn’t.



I don’t think there is a solution for this problem but I’m not an expert on “alpha to coverage”.

I just found this thread:

http://hub.jmonkeyengine.org/groups/graphics/forum/topic/multi-sampling-issues/



Now I trying disable MSAA, but getting NullPointerException :



[java]

public static void main(String[] args) {

AppSettings settings = new AppSettings(false);

settings.setSamples(1);

Main app = new Main();

app.setSettings(settings);

app.start();

}

[/java]

My understanding is that alpha to coverage only works when multisampling is on.



You get a null pointer exception because AppSettings/AppSettingsDialog is fairly stupid about unset defaults. So when you pass false to the AppSettings constructor it’s basically a useless object until some of the settings have been configured that cause NPEs in AppSettingsDialog. I think it’s a bug but I haven’t looked into it.



Pass true instead and you will not get the NPE. But when you turn off AA then you will also effectively be disabling AlphaToCoverage I think.

@pspeed said:
Pass true instead and you will not get the NPE. But when you turn off AA then you will also effectively be disabling AlphaToCoverage I think.


Yeah, that's exactly what's happening.

Okay then, I'll go to Plan-B, and make my own alpha dithering in fragment shader.

But, how to make AppSettings, with user config and without NPE? Set all the defaults? I have almost exact same code (setting title and splash image) in another project, and it works.
@bananaft said:
Yeah, that's exactly what's happening.

Okay then, I'll go to Plan-B, and make my own alpha dithering in fragment shader.

But, how to make AppSettings, with user config and without NPE? Set all the defaults? I have almost exact same code (setting title and splash image) in another project, and it works.


It's stupid, but you somehow have to detect that it's the first time and let it load defaults. Or set all of the options. If it's not the first time then you can pass false because then it will load the last settings.

In Mythruna, I'm already storing other preferences with the preferences API so it was easy to detect the first run and pass true.

Just to sum up, here is how plan-B (alpha dithering in fragment shader) looks like. Clearly not as good as Alpha To Coverage.

https://dl.dropbox.com/u/8845134/dev/z10.png