[SOLVED] Simarboreal PBR Shaders & shadow issue

I am using the simarboreal PBR leaf and tree shaders that were created with JmonkeyBuilder’s simarboreal plugin, and it seems as though both the leaf and tree MatDefs do not have proper preshadow techniques, which is causing the trees and leaves to create self-casting shadow artifacts.

I figured that I should start this thread, rather than continuing in the other thread about shadows, since @pspeed has informed me that this specific shadow problem is related to the preshadow technique. (However this is a new shader concept to me and I may require some guidance fixing the PBR tree shaders to work with shadows)

I believe I must have messed up something with my modified version of the shader.
I updated my MatDef files to include the proper .vert files in the preshadow technique, but trees are still creating shadow artifacts as they sway.

Here is my .j3md file for the PBR tree shader, which was originally based off of Javasabre’s simarboreal shaders for the JMB Simarboreal plugin.

MaterialDef AfflictedTrees {

MaterialParameters {


    Boolean UseVertexColorsAsSunIntensity
    Float StaticSunIntensity

    Vector4 ProbeColor

    Boolean BrightenIndoorShadows


    // Alpha threshold for fragment discarding
    Float AlphaDiscardThreshold (AlphaTestFallOff)

    //metalness of the material
    Float Metallic : 1.0
    //Roughness of the material
    Float Roughness : 1.0
    // Base material color
    Color BaseColor : 1.0 1.0 1.0 1.0
    // The emissive color of the object
    Color Emissive
    // the emissive power
    Float EmissivePower : 3.0
    // the emissive intensity
    Float EmissiveIntensity : 2.0

    // BaseColor map
    Texture2D BaseColorMap

    // Metallic map
    Texture2D MetallicMap -LINEAR

    // Roughness Map
    Texture2D RoughnessMap -LINEAR

    //Metallic and Roughness are packed respectively in the b and g channel of a single map
    Texture2D MetallicRoughnessMap -LINEAR

    // Texture of the emissive parts of the material
    Texture2D EmissiveMap

    // Normal map
    Texture2D NormalMap -LINEAR

    // the light map is a gray scale ao map, on ly the r channel will be read.
    Boolean LightMapAsAOMap

    //The type of normal map: -1.0 (DirectX), 1.0 (OpenGl)
    Float NormalType : -1.0

    // For Spec gloss pipeline
    Boolean UseSpecGloss
    Texture2D SpecularMap
    Texture2D GlossinessMap
    Texture2D SpecularGlossinessMap
    Color Specular : 1.0 1.0 1.0 1.0
    Float Glossiness : 1.0

    Vector4 ProbeData

    // Prefiltered Env Map for indirect specular lighting
    TextureCubeMap PrefEnvMap -LINEAR

    // Irradiance map for indirect diffuse lighting
    TextureCubeMap IrradianceMap -LINEAR

    //integrate BRDF map for indirect Lighting
    Texture2D IntegrateBRDF -LINEAR

    // Parallax/height map
    Texture2D ParallaxMap -LINEAR

    //Set to true is parallax map is stored in the alpha channel of the normal map
    Boolean PackedNormalParallax

    //Sets the relief height for parallax mapping
    Float ParallaxHeight : 0.05

    //Set to true to activate Steep Parallax mapping
    Boolean SteepParallax

    // Set to Use Lightmap
    Texture2D LightMap

    // Set to use TexCoord2 for the lightmap sampling
    Boolean SeparateTexCoord

    //shadows
    Int FilterMode
    Boolean HardwareShadows

    Texture2D ShadowMap0
    Texture2D ShadowMap1
    Texture2D ShadowMap2
    Texture2D ShadowMap3
    //pointLights
    Texture2D ShadowMap4
    Texture2D ShadowMap5

    Float ShadowIntensity
    Vector4 Splits
    Vector2 FadeInfo

    Matrix4 LightViewProjectionMatrix0
    Matrix4 LightViewProjectionMatrix1
    Matrix4 LightViewProjectionMatrix2
    Matrix4 LightViewProjectionMatrix3
    //pointLight
    Matrix4 LightViewProjectionMatrix4
    Matrix4 LightViewProjectionMatrix5
    Vector3 LightPos
    Vector3 LightDir

    Float PCFEdge
    Float ShadowMapSize

    // For hardware skinning
    Int NumberOfBones
    Matrix4Array BoneMatrices

    //For instancing
    Boolean UseInstancing

    //For Vertex Color
    Boolean UseVertexColor

    Boolean BackfaceShadows : false

    // Wind related parameters
    Vector3 WorldNoiseOffset
    Texture2D WindNoise
    Boolean UseWind
    Float FlexHeight : 2.0
    Float TrunkFlexibility : 1.0
    Float BranchFlexibility : 1.0


    Boolean UseTriplanarPlagueMapping

        Int PlaguedMapScale : 8
        Texture2D AfflictionTexture
        Texture2D PlaguedAlbedoMap
        Texture2D PlaguedNormalMap -LINEAR
        Texture2D PlaguedRoughnessMetallicMap -LINEAR
    Vector3 TileLocation
    Float TileWidth : 512.0


    Boolean Bark

    Boolean UseFog
    Color FogColor
    Vector2 LinearFog
    Float ExpFog
    Float ExpSqFog

}

Technique {
    LightMode SinglePassAndImageBased

    VertexShader GLSL110 GLSL150:   MatDefs/vert/AfflictedPBRTrees.vert
    FragmentShader GLSL110 GLSL150: MatDefs/frag/pbrGrass.frag

    WorldParameters {
        WorldViewProjectionMatrix
        ViewProjectionMatrix
        NormalMatrix
        WorldNormalMatrix
        WorldViewMatrix
        ViewMatrix
        CameraPosition
        WorldMatrix
        Time
    }

    Defines {

        USE_FOG : UseFog
        FOG_LINEAR : LinearFog
        FOG_EXP : ExpFog
        FOG_EXPSQ : ExpSqFog

        AFFLICTIONTEXTURE : AfflictionTexture
        PLAGUEDALBEDOMAP: PlaguedAlbedoMap
        PLAGUEDNORMALMAP : PlaguedNormalMap
        PLAGUEDROUGHNESSMETALLICMAP : PlaguedRoughnessMetallicMap

        PROBE_COLOR : ProbeColor

        USE_VERTEX_COLORS_AS_SUN_INTENSITY : UseVertexColorsAsSunIntensity
        STATIC_SUN_INTENSITY : StaticSunIntensity
        BRIGHTEN_INDOOR_SHADOWS : BrightenIndoorShadows

        TILEWIDTH : TileWidth
        TILELOCATION : TileLocation

        AO_MAP : LightMapAsAOMap

        BARK: Bark
        USE_TRIPLANAR_PLAGUE_MAPPING : UseTriplanarPlagueMapping


        BASECOLORMAP : BaseColorMap
        NORMALMAP : NormalMap
        METALLICMAP : MetallicMap
        ROUGHNESSMAP : RoughnessMap
        EMISSIVEMAP : EmissiveMap
        EMISSIVE : Emissive
        SPECGLOSSPIPELINE : UseSpecGloss
        PARALLAXMAP : ParallaxMap
        NORMALMAP_PARALLAX : PackedNormalParallax
        STEEP_PARALLAX : SteepParallax
        LIGHTMAP : LightMap
        SEPARATE_TEXCOORD : SeparateTexCoord
        DISCARD_ALPHA : AlphaDiscardThreshold
        NUM_BONES : NumberOfBones
        INSTANCING : UseInstancing
        USE_PACKED_MR: MetallicRoughnessMap
        USE_PACKED_SG: SpecularGlossinessMap
        SPECULARMAP : SpecularMap
        GLOSSINESSMAP : GlossinessMap
        NORMAL_TYPE: NormalType
        VERTEX_COLOR : UseVertexColor

        // Wind related
        USE_WIND : UseWind
    }
}

Technique PreShadow {

    VertexShader GLSL110 :   MatDefs/TreePreShadow.vert
    FragmentShader GLSL100 : Common/MatDefs/Shadow/PreShadow.frag


    WorldParameters {
        WorldViewProjectionMatrix
        WorldViewMatrix
        WorldMatrix
        ViewMatrix
        ViewProjectionMatrix
        Time
    }

    Defines {
        DISCARD_ALPHA : AlphaDiscardThreshold
        NUM_BONES : NumberOfBones
        INSTANCING : UseInstancing

          // Wind related
        USE_WIND : UseWind
    }

    ForcedRenderState {
        FaceCull Off
        DepthTest On
        DepthWrite On
        PolyOffset 5 3
        ColorWrite Off
    }
}

Technique PostShadow {
    VertexShader GLSL100 GLSL150:   Common/MatDefs/Shadow/PostShadow.vert
    FragmentShader GLSL100 GLSL150: Common/MatDefs/Shadow/PostShadow.frag

    WorldParameters {
        WorldViewProjectionMatrix
        WorldMatrix
        ViewProjectionMatrix
        ViewMatrix
    }

    Defines {
        HARDWARE_SHADOWS : HardwareShadows
        FILTER_MODE : FilterMode
        PCFEDGE : PCFEdge
        DISCARD_ALPHA : AlphaDiscardThreshold
        SHADOWMAP_SIZE : ShadowMapSize
        SHADOWMAP_SIZE : ShadowMapSize
        FADE : FadeInfo
        PSSM : Splits
        POINTLIGHT : LightViewProjectionMatrix5
        NUM_BONES : NumberOfBones
        INSTANCING : UseInstancing
        BACKFACE_SHADOWS: BackfaceShadows
    }

    ForcedRenderState {
        Blend Modulate
        DepthWrite Off
        PolyOffset -0.1 0
    }
}

Technique PreNormalPass {

    VertexShader GLSL100 :   Common/MatDefs/SSAO/normal.vert
    FragmentShader GLSL100 : Common/MatDefs/SSAO/normal.frag

    WorldParameters {
        WorldViewProjectionMatrix
        WorldViewMatrix
        NormalMatrix
        ViewProjectionMatrix
        ViewMatrix
    }

    Defines {
        NUM_BONES : NumberOfBones
        INSTANCING : UseInstancing
    }
}

Technique Glow {

    VertexShader GLSL100 GLSL150:   Common/MatDefs/Misc/Unshaded.vert
    FragmentShader GLSL100 GLSL150: Common/MatDefs/Light/Glow.frag

    WorldParameters {
        WorldViewProjectionMatrix
        ViewProjectionMatrix
        ViewMatrix
    }

    Defines {
        NEED_TEXCOORD1
        HAS_GLOWMAP : GlowMap
        HAS_GLOWCOLOR : GlowColor
        NUM_BONES : NumberOfBones
        INSTANCING : UseInstancing
    }
}

}

and here is the SinglePass .vert file for the shader where the wind code happens - i believe the code for windiness is the same as simarboreal’s original Phong .vert shader

#import "Common/ShaderLib/GLSLCompat.glsllib"
#import "Common/ShaderLib/Instancing.glsllib"
#import "Common/ShaderLib/Skinning.glsllib"

#ifdef USE_WIND
    uniform float g_Time;
#endif

#ifdef INSTANCING
#else
    #define worldMatrix g_WorldMatrix
#endif

#import "MatDefs/TreeWind.glsllib"

uniform vec4 m_BaseColor;
uniform vec4 g_AmbientLightColor;

varying vec2 texCoord;

#ifdef SEPARATE_TEXCOORD
    varying vec2 texCoord2;
    attribute vec2 inTexCoord2;
#endif

varying vec4 Color;

attribute vec3 inPosition;
attribute vec2 inTexCoord;
attribute vec3 inNormal;

#ifdef VERTEX_COLOR
    attribute vec4 inColor;
#endif

varying vec3 wNormal;
varying vec3 wPosition;
#if defined(NORMALMAP) || defined(PARALLAXMAP)
    attribute vec4 inTangent;
    varying vec4 wTangent;
#endif


#ifdef USE_FOG
varying float fogDistance;
uniform vec3 g_CameraPosition;
#endif

void main() {
    
    vec4 modelSpacePos = vec4(inPosition, 1.0);
    vec3 modelSpaceNorm = inNormal;
  
    
    
    wPosition = TransformWorld(modelSpacePos).xyz;
    
    
	

    #if (defined(NORMALMAP) || defined(PARALLAXMAP)) && !defined(VERTEX_LIGHTING)
        vec3 modelSpaceTan = inTangent.xyz;
    #endif

    #ifdef NUM_BONES
        #if defined(NORMALMAP) && !defined(VERTEX_LIGHTING)
            Skinning_Compute(modelSpacePos, modelSpaceNorm, modelSpaceTan);
        #else
            Skinning_Compute(modelSpacePos, modelSpaceNorm);
        #endif
    #endif

    #ifdef USE_WIND
        // some simple wind
        float windStrength = 0.75;

        // Need to know the model's ground position for noise basis
        // otherwise the tree will warp all over the place and it
        // will look strange as the trunk stretches and shrinks.
        vec4 groundPos = worldMatrix * vec4(0.0, 0.0, 0.0, 1.0);

        // Wind is applied to world space
        vec4 wPos = worldMatrix * modelSpacePos;

        wPos.xyz += calculateWind(groundPos.xyz, wPos.xyz - groundPos.xyz, windStrength);
        gl_Position = g_ViewProjectionMatrix * wPos;
    #else
        gl_Position = TransformWorldViewProjection(modelSpacePos);
    #endif

    texCoord = inTexCoord;

    #ifdef SEPARATE_TEXCOORD
        texCoord2 = inTexCoord2;
    #endif


    wNormal = TransformWorldNormal(modelSpaceNorm);

    #if defined(NORMALMAP) || defined(PARALLAXMAP)
        wTangent = vec4(TransformWorldNormal(modelSpaceTan),inTangent.w);
    #endif

    Color = m_BaseColor;

    #ifdef VERTEX_COLOR
        Color *= inColor;
    #endif
    
      #ifdef USE_FOG
        fogDistance = distance(g_CameraPosition, (g_WorldMatrix * modelSpacePos).xyz);
    #endif
}

Thanks in advance for any help, and for any advice/knowledge about the PreShadow technique and how it works :slightly_smiling_face:

I just noticed that there is not a PreShadow fragment shader that supports the PBR materials, and this was causing half of my problem, specifically the leaf shader which was casting blocky shadows, and was ignoring the transparency because it was trying to look for an m_DiffuseMap rather than the m_BaseColorMap.

I submitted a pull request to add a new PreShadowPBR.frag file to the same place where the other pre/post shadow shaders are. Would it also be important to create new post shadow fragment shaders for PBR?

https://github.com/jMonkeyEngine/jmonkeyengine/pull/1264

If this addition is okay, then I can also submit another request to update the existent PBRLighting.j3md to use this new PreShadowPBR frag shader.

So you’re saying the JME PBR shadows are broken right now? Just with transparency?

If so, good catch.

I’m not sure what the post shadow technique is used for but maybe just for the shadow renderer and not the filter? (I always tend to use the filter so maybe it has never come up for me.)

1 Like

could you please prepare some TestCase for the issue?

because im unsure what is broken, shadows work fine for me - tree leafs(quad with transparent texture)

Which makes one wonder… if the frag shader is referencing the wrong texture then how is it working? Or is it even used?

i think im wrong, he said exactly about PBR issue and i remember it was working for me when i were still using Lightning, now i have very blurry shadow so i dont even see difference.

yes looks like i have same issue as @yaRnMcDonuts

so i understand you also changed:

Technique PreShadow {

to use new frag
and it works fine? then IMO it gonna be quick and good fix :slight_smile:

1 Like

yes this would just be a quick edit to one line in the PBRLighting.frag shader, although I didn’t want to submit that as a second pull request until I was sure that it my file addition is okay with the core devs.

I could try to make a test case, but it sounds like you managed to reproduce and confirm the issue for your PBR models

Here’s a screenshot of the leaf and fern shadows before I implemented my pull request into my own local branch, compared to afterwards

before the change:
https://i.imgur.com/j3QXgg3.png

after:
https://i.imgur.com/UNPCw5Y.png

4 Likes

Nice looking scene, too.

I think a PR for JME would be good if you have the time.

1 Like

Thank you :slightly_smiling_face:

I should be able to submit a full PR sometime tonight. I’ll ammend the initial PR to update the PBRLighting.j3md file, and also include a a PostShadowPBR.frag alongside the PreShadowPBR.frag shader.

Edit:
I couldn’t figure out how to change the first PR, so I closed it and submitted a new one with all 3 changes.
https://github.com/jMonkeyEngine/jmonkeyengine/pull/1265

4 Likes

Thanks for the PR :slightly_smiling_face:

Sorry if I am wrong, I am not familiar with how the shadow renderer works, but shouldn’t there be a new PostShadow.j3md and PreShadow.j3md created for PBR also to use the new PostShadowPBR.frag and PreShadowPBR.frag as well?

I was wondering this as well. My initial thought would be to assume so.

However, I do not know too much about shadows myself. And I am not sure where (if anywhere) the two .j3md files you mention (PostShadow.j3md and PreShadow.j3md) are used in the engine.

But if they are used somewhere, then I think it could be important to create PBR versions of these, and also update the java code somewhere to choose the proper one depending on whether your’e using PBR or not. But I don’t know much so I am just guessing here.

1 Like

i also dont know where this are used, but issue was just with transparency that was solved by preshadow.frag so i dont think other files would require fixes.

also i think the fix could be done in preshadow.frag without creating preshadowPBR.frag to check texture?

do you think this could just work? can you check it?

because imo its much better than dupplicating code in new file.

#import "Common/ShaderLib/GLSLCompat.glsllib"
varying vec2 texCoord;

#ifdef DISCARD_ALPHA
   #ifdef COLOR_MAP
      uniform sampler2D m_ColorMap;
   #elseif BASECOLORMAP
      uniform sampler2D m_BaseColorMap;
   #else    
      uniform sampler2D m_DiffuseMap;
   #endif
    uniform float m_AlphaDiscardThreshold;
#endif


void main(){
   #ifdef DISCARD_ALPHA
       #ifdef COLOR_MAP
            if (texture2D(m_ColorMap, texCoord).a <= m_AlphaDiscardThreshold){
                discard;
            }
       #elseif BASECOLORMAP
            if (texture2D(m_BaseColorMap, texCoord).a <= m_AlphaDiscardThreshold){
                discard;
            }
       #else    
            if (texture2D(m_DiffuseMap, texCoord).a <= m_AlphaDiscardThreshold){
                discard;
            }
       #endif
   #endif

   gl_FragColor = vec4(1.0);
}

I did a quick copy/paste of your suggestion into my local code, and I couldn’t get it to work for both the phong and pbr leaf shader at the same time. I shuffled around the #ifdefs and also made sure to update the BASECOLORMAP define in the PreShadow technique, but I still couldn’t get it to work with both the phong and pbr shader consistently.

My initial thought was to avoid changing the old PreShadow.frag shader, since I don’t typically use the Phong shader at all anymore to be certain if it is working correctly, so I did not want to make changes anywhere that could break shadows for anyone else that uses Lighting.j3md.

1 Like

yes i understand, thats safe and proper way too.

its like choosing safe way or not dupplicated code way.

But i think phong will be supported very long time or like forever, so could be better to have same for both if possible :slight_smile: (we will test it anyway, right?)

@yaRnMcDonuts your PR is merged. Thanks for the fix.

I also submitted another PR to remove an extra PostShadow Technique from PBRLighting :
https://github.com/jMonkeyEngine/jmonkeyengine/pull/1273

Would you mind to test it in your example scene you posted above to see if it does not break anything, please?

4 Likes

Sorry for the late reply its been a hectic week.

I tested it with your changes though and nothing appears to be broken :slightly_smiling_face:

1 Like

Thank you so much, Ryan :slightly_smiling_face:

1 Like