Improved billboarding? Rotation matrix making billboards look at camera?

Troubleshooting graphics
1

Hi guys,

// TL;DR’able explaination :stuck_out_tongue:
I’ve been searching on Google some ways I could improve the billboard shader in my project. I found that instead of simply projecting the billboard in the opposite camera orientation, I could improve it by (ofc) applying a rotation matrix so that it looks at the camera. This will most certainly be slower to process for the GPU but it will give a more realistic feel to the player, effectively not rotating the billboards when the players move the mouse to aim around them. That’s one thing I did not like about the simple billboard implementation I had before: if you were standing next to a tree, for example on the right side of a tree and aim at the right side, the tree would rotate and completely fill the camera frustum until you aim on the right enough for the tree to go on the back of your head if you will making it disappear out of your sight.

// The problem! <— :smiley:
With my new billboard shader, it ALMOST works, but as you can see on the screenshot, it only works if the camera is at Z+ or /before/ the billboard. On the second screenshot you see that I only turned 180 degrees and aimed in the opposite direction to see billboards behind me. There are none, until I go forward past the billboard’s Z value, then it appears. All billboards having a Z position value /lower/ than the camera’s are oriented perfectly at the camera, but the ones having a Z position value /higher/ than the camera are not showing, probably because they’re facing backwards.

Screenshot #1 facing at Z-

Screenshot #2 facing at Z+

This is the vertex shader so far:

[java]
// pos = inPosition | texCoord = inTexCoord

    vec3 toCamera = normalize(pos.xyz - vec3(g_CameraPosition.x, pos.y, g_CameraPosition.z));
    vec3 t = normalize(cross(normalize(pos.xyz), toCamera));
    vec3 c = cross(toCamera, t);
    mat4 r = mat4(1.0, c.x, toCamera.x, 0.0,    // m11, m21, m31, m41
                  t.y, 1.0, toCamera.y, 0.0,    // m12, m22, m32, m42
                  t.z, c.z, 1.0       , 0.0,    // m13, m23, m33, m43
                  0.0, 0.0, 0.0       , 1.0); // m14, m24, m34, m44

    vec4 model_offset_pos = normalize(vec4(0.0, pos.y, 0.0, 1.0));
    float offset = texCoord.x - 0.5;  // (make it -0.5 to 0.5)
    model_offset_pos.x += offset * inTexCoord2.x;
    normalize(model_offset_pos);
    pos = (r * model_offset_pos) + (pos);

    gl_Position = g_WorldViewProjectionMatrix * pos;

[/java]

Thx for all your inputs.

2

(SORRY CAN’T EDIT THE OP)

Screenshot #2 facing at Z+

My god, I don’t know what’s happening with JME3 web hosting service today, but it’s constantly timing out! it’s soooo slowwww… :frowning:

3

My grass billboards (and tree billboards) in the IsoSurface stuff all face the camera position and not the camera orientation. Maybe you could look at those. The grass is y-axis aligned but the leaves of the trees are screen aligned… both use camera position and not orientation.

I don’t make a matrix, though. It’s just a cumbersome way of doing two vector projections (with the cost of a third and fourth thrown in for no good reason).

Basic idea: figure out what direction (in world space) to project X and Y… then project… then apply view transform.

Pseudo code:
wPos = g_WorldMatrix * inPos;
…project wPos…
gl_Position = g_ViewProjectionMatrix * wPos;

4

Hi Paul! Thx for helping.

No, that’s exactly what I had before and I’m stepping away from it because I realized it’s EFFECTIVE but it doesn’t make sense in all situations actually. e.g.: If you’re standing near a tree and move your mouse to aim elsewhere, the tree rotates, as a matter of fact ALL trees rotate even those far away, which doesn’t make sense. It causes problems with the shadowing, some look thin while they should not, etc… That’s what I tried to explain in the OP. What I want - and I’m sure we can accomplish that once and for all - is that the billboards always face the camera and NOT align with the camera’s orientation (which is what you use and what I had before). Do you understand my point of view? (no pun intended lol)

As you saw in my screenshots, the matrix I posted does like half the job. I realized while waiting for the JMonkey hosting provider to reboot the server (lol) that I could mirror the matrix diagonal by passing -1.0 instead of 1.0 and it would render billboards having Z position value higher than the camera. Here’s the code I have so far and a new screenshot of what it’s doing… I’m near having a wonderful solution!.. but not quite there yet:

[java]
vec3 toCamera = normalize(worldPos.xyz - vec3(g_CameraPosition.x, worldPos.y, g_CameraPosition.z));
vec3 t = normalize(cross(normalize(pos.xyz), toCamera));
vec3 c = cross(toCamera, t);
float d = (worldPos.z > g_CameraPosition.z ? -1.0 : 1.0); // <-- This makes the other half work
mat4 r = mat4(d , c.x, toCamera.x, 0.0, // m11, m21, m31, m41
t.y, d , toCamera.y, 0.0, // m12, m22, m32, m42
t.z, c.z, d , 0.0, // m13, m23, m33, m43
0.0, 0.0, 0.0 , 1.0); // m14, m24, m34, m44
vec4 model_offset_pos = normalize(vec4(0.0, worldPos.y, 0.0, 1.0));
float offset = texCoord.x - 0.5; // (make it -0.5 to 0.5)
model_offset_pos.x += offset * inTexCoord2.x;
normalize(model_offset_pos);
pos = (r * model_offset_pos) + (pos);
[/java]

Screenshot with the new rotation matrix:

BTW, I just found a tutorial that explains exactly what I want to implement… it’s called CYLINDRICAL BILLBOARDS and there doesn’t seem to be a lot of GLSL implementations around… been searching for hours. I only get theory, but I think I can manage to implement my own based on this: OpenGL @ Lighthouse 3D - Billboarding Tutorial

5

Ok, I’m out I think since you didn’t even really read what I posted. Good luck with your efforts.

@.Ben. said: If you're standing near a tree and move your mouse to aim elsewhere, the tree rotates, as a matter of fact ALL trees rotate even those far away, which doesn't make sense.

Because I already said my approach doesn’t do this and I believe I repeated it twice. It’s based on camera POSITION and not camera rotation. ie: it does not change when you rotate the camera because it is not based on the camera. Since it is based on camera position it is not affected by rotation because rotation is not used. The rotation is not used so only position affects the facing. So rotating the camera does not affect the rotation of the billboards because position is used in stead of rotation. The camera rotation is never taken into account so it cannot possibly affect the rotation of the billboards. Since rotation of the camera isn’t used (because position is used) then rotating the camera with the mouse (or by any other means) does not change the rotation of the billboards.

Only moving does.

Does that make sense?

6

https://code.google.com/p/simsilica-tools/source/browse/trunk/IsoSurface/assets/MatDefs/Grass.vert#167

1 Like
7

Or:
https://code.google.com/p/simsilica-tools/source/browse/trunk/SimArboreal/assets/MatDefs/AxisBillboardLighting.vert#184

…with a bit less magic.

8
@pspeed said: Ok, I'm out I think since you didn't even really read what I posted. Good luck with your efforts.

I did read what you posted:

Pseudo code:
wPos = g_WorldMatrix * inPos;
…project wPos…
gl_Position = g_ViewProjectionMatrix * wPos;

But I had something similar to this before and billboards would react to camera orientation, I misread the g_ViewProjectionMatrix tough, I swear I read it as g_ProjectionMatrix. Anyhow, I’m glad that you posted another 3 messages to let me know I misread something in your code. I’ll definitely try it right now and post my results back in a short while. Thx for the headsup.

9

Hi again, did not have luck with the AxisBillboardingLighting.vert solution. I pretty much copied and renamed variables to what I had in my shader:

[java]
vec4 wPosition = g_WorldMatrix * pos;
vec3 wNormal = (g_WorldMatrix * vec4(inNormal, 0.0)).xyz;
vec3 dir = normalize(wPosition.xyz - g_CameraPosition);
vec3 offset = normalize(cross(dir, wNormal));
wPosition.xyz += offset * inTexCoord2.x;
vec3 wvPosition = (g_ViewMatrix * wPosition).xyz;
gl_Position = g_ViewMatrix * g_ProjectionMatrix * wPosition;
[/java]

This gave me nothing visible, I tried to circle around just to make sure… nothing.

I’m now trying the other suggestion in the grass.vert file. BRB.

10

Alright, so… I implemented the other suggestion in the grass.vert file you mentionned. This is what the vertex shader looks like now:

[java]
vec4 modelSpacePos = vec4(inPosition,1.0);
vec3 modelSpaceNorm = inNormal;

    vec3 wPos = (g_WorldMatrix * modelSpacePos).xyz;
    vec3 cameraOffset = wPos - g_CameraPosition;
    float vDistance = length(cameraOffset);
    vec3 cameraDir = cameraOffset / vDistance;
    vec3 posOffset = normalize(vec3(-cameraDir.z, 0.0, cameraDir.x));

    float texFract = fract(texCoord.x);
    float offsetLength = (texFract * 2.0) - 0.5;
    float texY = abs(offsetLength) * 2.0;
    float normalProjectionLength = texY - 0.25;
    float size = inTexCoord2.x;

    modelSpacePos.xyz += modelSpaceNorm * normalProjectionLength * size;
    wPos = (g_WorldMatrix * modelSpacePos).xyz;
    wPos += posOffset * offsetLength * size;
    gl_Position = g_ViewMatrix * g_ProjectionMatrix * vec4(wPos, 1.0);

[/java]

Unfortunately, it’s the same as the other solution, it’s just not drawing any billboards it seems. Any idea what’s wrong?

11

Hey hey, look at what we have here. I had inverted the matrix multiplication order. You can clearly see that I’m still a GLSL noob… still in my first year, but after swapping those 2 matrix in order, it looks very good! This is my vertex shader now:

[java]
vec4 modelSpacePos = vec4(inPosition,1.0);
vec3 modelSpaceNorm = inNormal;

    vec3 wPos = (g_WorldMatrix * modelSpacePos).xyz;
    vec3 cameraOffset = wPos - g_CameraPosition;
    float vDistance = length(cameraOffset);
    vec3 cameraDir = cameraOffset / vDistance;
    vec3 posOffset = normalize(vec3(-cameraDir.z, 0.0, cameraDir.x));

    float offsetLength = texCoord.x - 0.5;  // (make it -0.5 to 0.5)
    float texY = abs(offsetLength) * 1.0;
    float normalProjectionLength = texY - 0.5;
    float size = inTexCoord2.x;

    modelSpacePos.xyz += modelSpaceNorm * normalProjectionLength * size;
    wPos = (g_WorldMatrix * modelSpacePos).xyz;
    wPos += posOffset * offsetLength * size;
    gl_Position = g_ProjectionMatrix * g_ViewMatrix * vec4(wPos, 1.0);

[/java]

… and here’s what it looks like! 360 degrees and the circle of billboards around the camera looks so much more smoother than the one in the OP.

I +1’ed you, this was very helpful and it works flawlessly on all angles. The problem is that I still don’t quite understand all of it. I spent nearly all my saturday searching on Google and trying various rotation matrix solutions, but this is exactly what does the trick really. Nothing else is required. It’s perfect. Thx again. BTW, for next people that come here, if you’d like to explain your code to people, I think it would be useful too as a side conversation. Else, thanks again.

12

All it is doing is projecting (moving) the point out in world space perpendicular to the camera. When there is a constant up axis (like 0,1,0) this is trivial as the cross product will give you the perpendicular vector. No need for extra matrices or anything else as you just need to move down one vector.

…and in this case, because we know the up vector is the y axis, we can do a little inversion as:
vec3 posOffset = normalize(vec3(-cameraDir.z, 0.0, cameraDir.x));

…and avoid the cross product altogether. For the trees, I used a cross product because I rotate them around their own axis (defined by the normal) instead of always up. The reason you may not have seen anything is because your mesh may not have been setup right. (It needs normals for example unless you replace that vector with (0,1,0))

13
@pspeed said: The reason you may not have seen anything is because your mesh may not have been setup right. (It needs normals for example unless you replace that vector with (0,1,0))

No, it does have normal and it does display correctly, the reason as I wrote in my last post was because I had the matrices multiplication reversed. Instead of g_ProjectionMatrix * g_ViewMatrix * pos, I had this: g_ViewMatrix * g_ProjectionMatrix * pos

… BTW am I right to think it would be faster to add ProjectionViewMatrix to the J3MD file and use the g_ProjectionViewMatrix like you’re doing?

14
@.Ben. said: No, it does have normal and it does display correctly, the reason as I wrote in my last post was because I had the matrices multiplication reversed. Instead of g_ProjectionMatrix * g_ViewMatrix * pos, I had this: g_ViewMatrix * g_ProjectionMatrix * pos

… BTW am I right to think it would be faster to add ProjectionViewMatrix to the J3MD file and use the g_ProjectionViewMatrix like you’re doing?

Yes, two matrix multiplies is of course more expensive than one.