My next pathetic attempt with shadows - help

Troubleshooting graphics
41

Yes, I’m going to have the same light level on the corners where wall meets the ceiling/floor. I’m still playing with the formula trying to find the compromise between my ‘ambient’ and the shadow effect given by the vector’s math. It’s just a matter of time to find the good solution, because now I understand the mechanism and I know where to look at.
The promising solution is this one:

		float val = 1.0 - max(0.0, dot(norm, lightdir));
		return (1.0 - (val * val)) * 0.6 + 0.4;

combined with a bit darker bitmaps on both floor and ceiling,

The more important thing now is the unwanted shadow artifact that appear on the gate and other surfaces. Hoping that @nehon made any progress.

42
@FrozenShade said: The promising solution is this one: 
		float val = 1.0 - max(0.0, dot(norm, lightdir));
		return (1.0 - (val * val)) * 0.6 + 0.4;

That’s a fun one to unravel if you know what those values actually mean. :slight_smile: The val * val part is the only piece that can’t be deconstructed directly to an angle easily but it can kind of be visualized.

43

It’s after midnight here and playing is the only thing that left to me now. dot(norm, lightdir) is the cosine between these two vectors so now val is the sinus. To be honest some of my attempts are just blind shots.

44

I hope that this one is final: http://postimg.org/image/vlkbky7ed/

I put a ‘vLightDir.w’ factor into the formula - this was, from the very beginning, the missing thing. I know it is used later in the ATTENUATION part, but it seems that it fits here too :wink:

		float val = 1.0 - max(0.0, dot(norm, lightdir)); 
		return ((1.0 - (vLightDir.w * val)) + vLightDir.w * 0.3) * 0.85;

1/3 of thinking, 1/3 of blind shot, 1/3 of luck.

45
@FrozenShade said: It's after midnight here and playing is the only thing that left to me now. dot(norm, lightdir) is the cosine between these two vectors so now val is the sinus. To be honest some of my attempts are just blind shots.

Being pedantic as it doesn’t really matter to your issue, but should matter to your long term understanding… but 1 - cosine is not sine.

cos^2 + sin^2 = 1
sin^2 = 1 - cos^2
sin = sqrt(1 - cos^2)

1 - cos is not simply described. For example, if the angle was 45 degrees then the resulting angle is 72 degrees. 0 becomes 90 and 90 becomes 0 but all other angles are kind of skewed. 15 degrees becomes 88 degrees. 75 degrees becomes 42 degrees. I’m tempted to plot the curve to see what it looks like. There is probably a stretched reflection point in there somewhere. (Ah, reflects at 60 degrees… cos = 0.5) 60 degrees becomes 60 degrees.

In your latest code:

@FrozenShade said: 
		float val = 1.0 - max(0.0, dot(norm, lightdir)); 
		return ((1.0 - (vLightDir.w * val)) + vLightDir.w * 0.3) * 0.85;

Just reworking the math a bit to try to understand it…
val = 1.0 - max(0.0, dot(norm, lightdir);
…let’s call that…
val = 1.0 - cosTheta

1.0 - (vLightDir.w * val)
…let’s call that…
1.0 - (w * val)

so:
1.0 - (w * (1.0 - cosTheta))
or:
1.0 + (w * (1.0 - cosTheta) * -1.0)
or:
1.0 + (w * (cosTheta - 1))
or:
1.0 + (w * costTheta) - w

So now all together:
(1.0 + (w * cosTheta) - w + (w * 0.3)) * 0.85

Let’s see what it looks like as cosines:
(cos(90) + (w * cosTheta) - w + (w * cos(72.5))) * cos(31.7)

Anyway, as fun as that was… it isn’t really illuminating. :slight_smile: The result is used as a percentage, sort of… so the cosine can get lost early. Though I guess the reduction still works nicely that way:
(1.0 + atten * diffuseFactor - atten + (atten * 0.3)) * 0.85

…which is:
0.85 + (0.85 * atten * diffuseFactor) - (atten * 0.85) + (atten * 0.3 * 0.85)
…
0.85 + (0.85 * atten * diffuseFactor) - (atten * 0.85) + (atten * 0.255)
…
0.85 + atten * ((0.85 * diffuseFactor) - 0.85 + 0.255)
…
0.85 + atten * ((0.85 * diffuseFactor) - 0.595)

Which is a much simpler version of the same thing (if my math is right).

Hmmm… I maybe can do even better…
0.85 + (atten * ((0.85 * diffuseFactor) - 0.595))
…is also:
0.85 + (atten * 0.85 * ((0.85 * diffuseFactor)/0.85 - 0.595/0.85))
…
0.85 + (atten * 0.85 * (diffuseFactor - 0.595/0.85))
…
0.85 + (atten * 0.85 * (diffuseFactor - 0.7))

…and I think we can go one more because that duplicate 0.85 now bugs me…
0.85 * (1 + (atten * (diffuseFactor - 0.7)))

It’s much easier to see what the characteristics are now… and if you ever decide you want to clamp the overflow at something other than the final result you have sensible places to do it now. Though it’s all a bit ‘trial and error’ random so it’s hard to say what would make sense.

So if diffuseFactor is between 0 and 1, with normally 0 meaning no light at all and 1 meaning fully facing the light… it seems like now that is put into the range of -0.7 to 0.3, respectively. Then multiplied by distance which will also be between 0 and 1. So, whatever the angle was, it will be pushed towards 45 degrees as distance increases. ie: at max distance, the result there is always 0 (which was originally 0.7, acos(0.7) is a bit more than 45 degrees). So anyway, the angle is now relative to ~45 degrees and tapered based on distance. And the (1 + foo) * 0.85 part is moving that deflection to 31 degrees.

I guess it looks like you will never go below 0 but for direct light close to the surface you will spike above 1 by as much as 0.15 (if atten is ever 1.0). If this actually does occur then post-clamping may create noticeable circle where the light goes flat. In which case, you could premultiply the (atten * (diffuseFactor - 0.7)) part by 0.5 to prevent it, I guess. It would make the dark places brighter, though. Maybe given the atten values it’s not an issue anyway and perhaps the spike is visually desirable.

Interesting thing you arrived at by blind luck. :slight_smile:

46

Ok, seems like fifteen years after the high school I just forgot everything. Sad… It’s time for me to go back to the books.

47

@nehon
http://s23.postimg.org/obke8ukej/3ddu32.png
Call me ‘dumb’, but I really don’t know why I played only with the first PolyOffset value. The screen shows shadow artifacts with PolyOffset 0.9 39 in PreShadow.
Hope this is important…

48

Well I never really got what the values stand for exactly… :stuck_out_tongue:
There are 2 polyoffset to consider though, the one in the prePass (when rendering shadow maps) and the one in the post pass (when rendering shadows in the scene)

49

Did you manage to reproduce the problem? Do you have any idea what to do? Is there any chance to have it working correctly?
Are there any special instructions that we can follow creating the geometry to avoid such issues?

As far As I see you are the best one here, so if You don’t know then who might?

Now I don’t know what to do, to continue my work on the project or to rewrite everything to, for example c# and use another engine. I’m not skilled in shader programming, even with required math I have some problems. I don’t want to be stuck for months with tons of books learning the shaders from it’s basics.
That’s why I picked up the graphics engine rather than writing everything on my own from scratch.

50

I didn’t have time to test it. There are a lot of parameters to consider and making a test case will take time. If you could provide me a test case it will make things faster.

That’s nice of you to say I’m the best around but that’s far from deserved. However I made the shadows system of JME3 and I know my way around it. Yet every single issue is a new one and I’m not a genius.

I think you’re delusional if you think that if you pick another engine you won’t have the same kind of issues. There is no such thing as a push button game engine, even the best ones requires knowledge of 3D graphics and programming language. What you’re trying to do is pretty complex and won’t be easy in any engine.
Besides, you did great so far. What you did to have the lights attenuate shadows cast by other lights is already very advanced. On a side note, learning shaders is hardly a waste of time if you want to get into computer graphics. They are the base of everything today, and I wouldn’t consider making a game without using custom made shaders, because the “All inclusives” shaders that we provide cannot fit every single need one would have for a game.

Now your issue is definitely a z fighting issue, and I think you won’t be able to find a good compromise with the polygon offset to fit every geometries in your scene.
Did you consider using the shadowFilter instead of the shadowRenderer? It renders shadows as a post process so you wouldn’t have zfighting issues. However it also implies that you modify the filter shader to take the other lights into account as you did for the renderer.

51

Right now I’m performing more and more tests, I’m a bit surprised of the results so I need to carefully confirm these results with more tests. I just want to give you as much info as I can. If is necessary I’m ready to give you (but only you) my current sources (via email/skype) and assets. I don’t want to show the code to the public because it’s too dirty right now.

I see people on some forums/portals presenting their work in other engines. I don’t have knowledge about the problems they meet, I only see the results of their work, sometimes described with ‘did it in 3 days of work…’ comment. And come on, first few things on the 3d scene, nothing complex and such problems… I’m not blaming anyone nor the JME engine itself, but it’s a bit discouraging. And I’m aware that this is free and open source engine, which, for me is a big advantage.
Making customs, like this one with the shadows is ok for me, but I just don’t like situations where I’m helpless.

No, I think that using shadowFilter might not be necessary right now… but let’s wait for my results.

EDIT: it was just hope that I have found something interesting. Only confirmed that this was the shadows that causes the problem. Currently I’m trying to modify the ShadowFilter. We will see the results soon.

EDIT2: It’s even worse with the filter: http://postimg.org/image/ha2laetrv/
And so far I’m not even sure if I’ll be able to use my shadow’s code with the filter. So, @nehon you can contact me via email
XXXXXXXXXXXX I’ll send you my sources.

EDIT3: mail address removed

52

Next interesting screen showing the shadows problem: http://postimg.org/image/exz391dan/
Compare the shadows on the wall and on the dark side of the sphere.
Looks like the artifacts shows on the shadow’s source, not the surface on which shadows are cast.

53

I don’t see artifacts on the last shot. What’s the problem

54

On the upper screen - there are shadowy lines on two triangles for example
On the bottom screen the shadows are not visible - just to compare which the upper one - which dark pixels comes from shadows, and which ones from the lighting.

55

This is not the same issue. This is due to shadow map rasterisation on a surface parallel to the light direction. This is inevitable. You have to use edge filtering (like PCFPOISSON or PCF4 or 8) or increase the shadowmap resolution to alleviate the issue. But of course at the expense of some fps.

56

Ok, thank you. I’ll make some experiments.

57

Today again we took the problem of shadows.

Check the image: http://s3.postimg.org/x0ojj7vur/3ddu35.png
The lower part contains two images with the old value (0.9 3) of PolyOffset in PreShadow. You can see the shadowy noise.
The upper part contains the scene with adjusted PolyOffset value (0.9 5) and removed textures. The only wrong thing are the lines. I’ve fond that the lines on the doorpost are caused by the grate. If you open the door (the grate is moved up) most of these lines disappear. Different lines are visible from various distances.
There are another lines too, you can see them where objects intersect each other. Wall with column, column with doorpost etc.
Further PolyOffset changes have no effect.
We also found that the density of these lines depends on the angle of view.

I can tell that our project is going very well. So far there artifacts are the only thing that keeps me awake at night. @nehon did you find something that can help?

58

Nope, to be honest, I downloaded your project zip, but I’ve been discouraged to try because it’s an eclipse project. Setting it up back in the SDK would have take too long. I have plenty other things on my plate and can’t spend too much time on this.

In the mean time though, I’ve talked about the issue with Kirill, and somehow he said “looks like the shadows are perpendicular to the wall”. Actually, the issue might be that they are parallel to the wall. Axis aligned shadows always did strange things back when we only had directional light shadows. And from the look of your map, there is a good chance that it’s grid aligned and axis aligned, am I wrong?.
For a point light, the problem is that shadow cams directions are always axis aligned, so I guess we hit the same issue. The weird thing is that, I don’t have the issue in the stock test case…
If you made your own PointLightShadowRenderer, you can try to slightly rotate the shadow cams on each axis and see what it does. Rotate them from the exact same amount though or you’ll have seams in the projected shadows.

59

To be honest I don’t like JME’s SDK but I love Eclipse. Anyway, it is just an IDE, never thought that it would be a problem, the code runs the same way…

You are right, we have a pure grid here, everything is parallel or perpendicular.
I’ll try that trick with shadow cams, I’ll write the results as soon as possible. Thank you.

60

Ok, something went wrong…

As far as I understand I need to rotate every camera by QUARTER_PI on the same axis I rotate the player’s cam, so the every wall will be at 45 degrees to the cam.

            q1 = shadowCams[i].getRotation();
            q2 = new Quaternion();
            q2.fromAngleAxis(FastMath.QUARTER_PI, Vector3f.UNIT_Y);
            shadowCams[i].setRotation(q2.mult(q1));

This code produces such result: http://postimg.org/image/9997zl4d9/

Anyway, the dark lines still exists. Help :frowning:

EDIT: @nehon I’ve already run the project in JME’s SDK without any problem, it took me 2 minutes to do that. Use ‘Import Project’, change JDK to 7 and just add the ‘assets’ folder. Run the file com.dungeongame.gameclient.DungeonClient.java
That’s all. Of course if you want I’ll send you the converted one.