Issue with transparency while splatting texture on terrain

[methusalah]

Hi !

I have an issue on OpenRTS with texture splatting. I would like to have transparency on the terrain (I admit, it’s strange ! I can explain )

here is a test case with a hand made alpha map and the resulting splatted texture :

Note that only two diffuse maps are added to the terrain’s material, so only the Red and Green channels ares used.

The second texture, splatted by the Green value of the alpha map, is a bit transparent unless it is maxed ! And even at the max value, the zone is surrounded by a transparent halo. We can see the bluish color of the ground throught it.

The terrain’s material has an additionnal render state :

material.getAdditionalRenderState().setBlendMode(BlendMode.Alpha);

and the terrain’s geometry has the transparent queue bucket :

 geometry.setQueueBucket(Bucket.Transparent);

Can’t understand why… any idea?

[methusalah]

Another picture of a non antialiazed shape in the alpha map. red is 255/0/0 and green is 0/255/0. No gradient.

[guillaumevds]

What happens when you change the AlphaMap’s texture resolution ? (from very low to very high like 4096^2)

EDIT: please don’t put the terrain in the Transparent bucket. It has nothing to do with what you want to achieve (if I understand correctly what you want to achieve)
EDIT2: When I zoom into your second image your AlphaMap looks antialiased.

[pspeed]

@methusalah

(I admit, it’s strange ! I can explain )

…then you probably should.

[methusalah]

@pspeed

…then you probably should.

I have two different needs :

• the engine will allow underground assets like craters, pits, wells, breaks… The floor has to be transparent to make these assets visible. See below an exemple from Stracraft 2.
• I have a fully transparent cover above the terrain, were we can draw special transparent textures. This is used to have dead leafs, visible roots, pavements, stracraft’s “creep”… continuously above many terrain textures (for that specific need I alos have a color’s blending issue but it’s another subject).

@guillaumevds

What happens when you change the AlphaMap’s texture resolution ?

I’ll try it this soon.

@guillaumevds

please don’t put the terrain in the Transparent bucket

Isn’t it necessary to achieve transparency? I’m not sure to have tested without. Could you give some details about it?

@guillaumevds

your AlphaMap looks antialiased.

After check, it’s not. It’s pixel perfect. But i’ve zommed the sample alphamap on the image I provided and it appear antialiased.

Exemple of glass floor in Starcraft 2:

[pspeed]

Well, I suspect you will have to hack the shader then as I guess it is transferring alpha directly which is something it sounds like you normally want… except in the cases where you don’t want it. It will be interesting to see how you figure out where you want alpha and where you don’t.

Also, you should avoid the transparent bucket because it will cause ordering issues with every other transparent geometry you try to add. You will end up having to control your render order very closely with a custom GeometryComparator… and in general it’s always going to be best to draw the terrain first. Buckets just control render order.

Edit: the alpha showing through could also be a problem with the math in the shader, I guess… maybe it’s doing something that causes the values not to add to 1 when there is overlap.

[jmaasing]

As pspeed says, the terrain shader can’t do what you ask of it. It uses the alpha channel for other purposes. Actually you probably don’t want to use terrain at all for SC2-maps. I guess that ‘regular’ models will be less pain for SC2-maps.

[methusalah]

@pspeed

it is transferring alpha directly which is something it sounds like you normally want

Could you provide more details? For me, into the transition zone between two textures, you only have to draw a part of the first and a part of the other according to a gradient. The two textures have their alpha to 1 so the blended color should have alpha to 1 too, shouldn’t it?

@pspeed

how you figure out where you want alpha and where you don’t

By having transparency on the texture to draw. I use it for the cover layer and it works. For the terrain texture, I have a 1x1 pixel texture with color (0, 0, 0, 0). I acheive my glass floor by drawing this special texture like on the image bellow.

@pspeed

and in general it’s always going to be best to draw the terrain first.

I understand. But is there a simple way to draw all my underground assets before the terrain?

@pspeed

Edit: the alpha showing through could also be a problem with the math in the shader, I guess… maybe it’s doing something that causes the values not to add to 1 when there is overlap.

Does this sound like a fixable jMonkey’s shader issue?

@jmaasing

Actually you probably don’t want to use terrain at all for SC2-maps. I guess that ‘regular’ models will be less pain for SC2-maps.

I’me a bit confuse here. OpenRTS uses a home made mesh, extending the regular JME’s Mesh class. The texture splatting is achieved with the TerrainMaterial. It doesn’t use the TerrainQuad if it’s what you’re talking about.

[guillaumevds]

I think I would achieve the glass floor effect by modifying the TerrainLighting shader.
As for the border problem, please have a look at how the variable “diffuseColor” is computed, maybe you’ll find a small hack giving the expected result

This is probably where you want to look :

#ifdef DIFFUSEMAP_1
  vec4 diffuseColor1 = texture2D(m_DiffuseMap_1, texCoord * m_DiffuseMap_1_scale);
  diffuseColor = mix( diffuseColor, diffuseColor1, alphaBlend.g );
  #ifdef DIFFUSEMAP_2
    vec4 diffuseColor2 = texture2D(m_DiffuseMap_2, texCoord * m_DiffuseMap_2_scale);
    diffuseColor = mix( diffuseColor, diffuseColor2, alphaBlend.b );
    #ifdef DIFFUSEMAP_3
      vec4 diffuseColor3 = texture2D(m_DiffuseMap_3, texCoord * m_DiffuseMap_3_scale);
      diffuseColor = mix( diffuseColor, diffuseColor3, alphaBlend.a );

You might also define the specularColor there for example if you want to use the third diffusemap for a wet materials (look blood in my game :p)

[jmaasing]

Oh right, it doesn’t use TerrainQuad, then why not just leave a hole in the mesh where the glass floors are and render a separate glass mesh in the hole? The other way is to hack the shader to use alpha the way you want.

[methusalah]

@guillaumevds

maybe you’ll find a small hack giving the expected result

Thanks ! I’ve never hacked the standard jMonkey shaders. I will have a look at it, but it seems that the gardient effect is done more deeply into the code. I can’t understand why the blending of two 0% transparent textures produces a transparent one.

@jmaasing

then why not just leave a hole in the mesh

There are two reasons :

• The mesh is a tile based map and the holes would be 1x1 square. I need more precision.
• The transparency should not be boolean. In some cases the texture of the ground have to garduatly mix with the texture of the underground asset (see picture).

[methusalah]

Does somebody know where I can find the code involved in the gradient effect that appear at the border between two textures for terrain shader?

The one visible on this pic :

Thanks !

[pspeed]

@methusalah

here is a test case with a hand made alpha map

From your first post, what do you mean by “hand made alpha map”?

In your other posts with the simpler red/green texture can you post that texture?

[methusalah]

@pspeed

what do you mean by “hand made alpha map”?

In OpenRTS, the three alpha maps of the terrain material are computed by an atlas generator. But to present a minimalist test case, I’ve create a test alpha map mannually.

      mat.setTexture("AlphaMap", am.loadTexture("textures/testalphamap.png"));
//    mat.setTexture("AlphaMap", new Texture2D(new Image(Image.Format.RGBA8, atlas.getWidth(), atlas.getHeight(), atlas.getBuffer(0))));

The material is created with only two diffuses, using the red and blue channel. Here is it :

And the result inside the engine :

[methusalah]

I believe that for each pixel, the textures to draw receive the corresponding alpha map’s channel’s value as their own alpha, so that they can be blended like any other overlaying textures.

I would like to know where is the involved code to understand.

For exemple, seen in TerrainLighting.frag :

vec4 diffuseColor = tex0 * alphaBlend.r;

I multiply all channels of the texture by the alpha map value.

If I have 50% of a (0, 255, 0, 255) colored pixel, I will obtain a (0, 127, 0, 127) pixel. I create transparency where there is none.

I guess this mix function has the same behavior :

diffuseColor = mix( diffuseColor, tex1, alphaBlend.g );

[guillaumevds]

Please try with that same AlphaMap texture but with a res of 4096^2 (instead of this tiny texture) and see if it changes the transparency on the borders.

[methusalah]

@guillaumevds

Please try with that same AlphaMap texture but with a res of 4096^2 (instead of this tiny texture) and see if it changes the transparency on the borders.

I tried, sorry that I haven’t give feedback. The probleme is the same, but with a very thin gradient between to extremes. For partially applied textures, the probleme is exactly the same.

[guillaumevds]

I have the same “problem” in my game but that “problem” is in fact making my game looking better on lower terrain’s AlphaMap texture size
EDIT: I’d like to know why and how it’s happening.
EDIT2: Maybe it has something to do with some kind of alpha discard treshold. No time to look at that for the next few days tho.

[pspeed]

This is essentially the issue… you have to decide how you want to indicate real alpha and then set the resulting alpha to that, overriding and multiplied alpha you get from the other layers.

[Momoko_Fan]

Probably the terrain shader wasn’t designed to work with alpha blending. It was never tested, so not really guaranteed to work.
Since all it takes to cause the problem is alpha blending, that implies the the output color alpha is set to a value below 1, causing the sky to show through. So probably if you were to set gl_FragColor.a = 1.0; at the end of the terrain shader you would solve this.