When I apply an alpha m_ColorMap to a model’s submesh, the rest of the submesh is ignored and disappears. This diagram explains it all:
Is this a glitch or is there a workaround? I guess it has something to do with render passes, since the only culling is for the camera frustum. Can I somehow force the rest of the geometry to display?
Can you provide a test case that shows the issue?
I think this is a problem with not having sorted triangles within single geometry.
Basically, jme3 should sort geometries for you - but only on geometry versus geometry level. Within single geometry, order of drawing the triangles is more or less random. This means that even with transparent textures, you might write to depth buffer and hide things behind.
Basically, you have two choices. Easy one is to disable depth buffer write and use some kind of additive blending.
[java]
mat.getAdditionalRenderState().setDepthWrite(false);
mat.getAdditionalRenderState().setBlendMode(BlendMode.AlphaAdditive);
[/java]
this makes order not important. Even plain Alpha blend mode works sometimes reasonably.
Harder option is to sort triangles on each frame yourself, in reverse distance from camera - you may take a look at ParticleComparator/ParticleTriMesh for how it is done. In any case, it will work only for reasonably shaped objects, all bets are off when you start intersecting triangles (or even intersecting them on plane projected from your screen).
Please provide the test case and you will see a solution very soon 
http://www.megaupload.com/?d=JZDCNE1A
(The .java file is inside the folder.)
The blue rectangles should display through the transparent horizontal plane, but they disappear, even though the monkey shows through. This doesn’t happen if the planes don’t intersect, so my original illustration was completely incorrect.
[java]mat.getAdditionalRenderState().setDepthWrite(false);
mat.getAdditionalRenderState().setBlendMode(BlendMode.AlphaAdditive);[/java]
This does fix the disappearing problem, although there’s still something odd going on with the shading. In the example, the rectangles appear black instead of blue.
Also, I’ve been vertically mirroring all the images I use as textures to get them to line up with the UV map. Apparently the loader loads from the wrong end of the image.
Thanks for looking into this!
The answer is that there’s no easy solution to this problem.
There are several things you can do though, first is alpha-test (getAdditionalRenderState().setAlphaTest(true)). This would make the blending less-smooth, but at least now you will be able to see through fully-transparent objects.
Another thing you can do, which I think is the best solution if you can support it, is to use alpha-to-coverage:
[java]GL11.glEnable(ARBMultisample.GL_SAMPLE_ALPHA_TO_COVERAGE_ARB);[/java]
You can’t use it with post-processing filters, and you have to enable multi-sampling for it to work
(in the jME3 display settings dialog, click the “Disabled” combobox and select number of samples)
It is not a built-in jME3 feature yet, because it needs multisampling and I want it to work with filters as well, so we are trying to add full multisampling support for filters now and until that happens, you’ll have to use direct OpenGL calls to enable this feature.
You probably knew this, but with getAdditionalRenderState().setAlphaTest(true) if there are multiple stacked layers the transparent area becomes progressively less transparent as the layers stack. I guess when it’s mixing the opacity it adds a certain value to the alpha which should stay at 0. I can make a test case if necessary (stacked planes that should be transparent are translucent.)
GL11.glEnable(ARBMultisample.GL_SAMPLE_ALPHA_TO_COVERAGE_ARB); works perfectly though. Thanks!
You can now use RenderManager.setAlphaToCoverage(true); in last SVN