[SOLVED] Strange shining problem

Hi,

After 2 days of hard work I finished the deferred rendering for my game. Almost everything is ok. There are shadows, there is a small boost in performance.
The only thing that need to be fixed is a strange shining on the surfaces. Depending on the view direction the screen looks or or there is an shining visible. I messed up something, now I need to advice where to look at the code, wrong calculation of which value could cause such effect? The models and all textures was not changed.

Please pardon the stupidity of my question but … isn’t what you describe as a shine just the “specular” part of the shader ?

I also think that you should have an issue with your specular, because specular are compute from eye position.
Maybe some issue with view direction and eye direction (eyePos - pixelPos)

The problem is that I checked all specular related operations twice. I wrote here in act of despair, but for now I found the bug.

In my solution I’m writing data into G buffer. I mean all object related data: diffuse, specular, normal… But because my lack of 3D math skill I’m writing object’s tangent and binormal data as well. Because I have no idea how to use them on the first pass.

Having such data in G buffer, the second pass is obvious:

vec3 wvTangent  = texture2D(m_WoViTanRender, newTexCoord).xyz;
vec3 wvBinormal = texture2D(m_WoViBinRender, newTexCoord).xyz;
mat3 tbnMat = mat3(wvTangent, wvBinormal, wvNormal);
vec3 vViewDir  = -wvPos.xyz * tbnMat;
vLightDir.xyz = (vLightDir.xyz * tbnMat).xyz;

But I passed the tangent and binormal using Format.RGB8 texture. Switching to Format.RGB16F solved the problem. Now I wonder if there is a way to code/encode them somehow. Like I can do it with normal, which allows me to use only 2 color channels of RGB8 texture:

vec2 encodeNormal(in vec3 n)
{
    vec2 enc = normalize(n.xy) * (sqrt(-n.z*0.5+0.5));
    enc = enc*vec2(0.5)+vec2(0.5);
    return enc;
}
vec3 decodeNormal(in vec4 enc)
{
    vec4 nn = enc * vec4(2.0,2.0,0.0,0.0) + vec4(-1.0,-1.0,1.0,-1.0);
    float l = dot(nn.xyz, -nn.xyw);
    nn.z = l;
    nn.xy *= sqrt(l);
    return nn.xyz * vec3(2.0) + vec3(0.0,0.0,-1.0);
} 

Any idea?

Nice you find, the bug.

Sorry I can’t help you about tangent, binormal. In my WIP deferred pipeline, I encode normal in RGB8 via oct16 algo to keep the maximum precision and avoid some issue when using normal in other pass (shadow, light, ao final shading …)

vec3 encodeNormal(in vec3 normal){
 return snorm12x2_to_unorm8x3(float32x3_to_oct(normal));
}

vec3 decodeNormal(in vec3 unorm8x3Normal){
 return oct_to_float32x3(unorm8x3_to_snorm12x2(unorm8x3Normal));
 //return hemioct_to_float32x3(unorm8x3_to_snorm12x2(intNormal.rgb));
}

vec3 readNormal(in sampler2D normalBuffer, in vec2 uv) {
	vec3 intNormal = texture(normalBuffer, uv).rgb;
	return normalize(decodeNormal(intNormal));
}

you can see the rest of the code of at:

https://github.com/davidB/jme3_ext_deferred/blob/master/src/main/resources/ShaderLib/UnitVector.glsllib.ftl

Unfortunately those functions can’t be used for tangents/binormals. But right now I got an acceptance of the new renderer from my team member. He have better eye than me and told me that it looks ok, besides some small differences.

Anyway, I advice you to use this encode/decode for normal (see the links at the top of the source for why).
GG for your Deferred render pipeline. Do you write it from scrach ?

I’ll try to replace the current functions. Thank you.
The origin of my current rendering lies deep in the JME’s shaders. Some time ago @Perjin gave me his reworked version of JME’s shaders, which was an combination of lighting and shadowcasting. After that I managed to apply there a lots of optimizations and customizations. The actual version is just a next rework of an existing code. I’m too weak in 3d math to write something like that from scratch.