# Calculating vertex normals of custom mesh

Hi,

I am building a custom mesh and I was wondering if there is a way to auto-calculate vertex normals. I have all the buffers set up and inserting some vectors for normals works as expected.
Could the TangentBinormalGenerator be of any use for this? I know what tangents and binormals are but I’m not sure if this generator could be of any use for vertex normals…
I guess that there is some method to do this somewhere in the API for the terrain, etc.
If there is no method already, maybe someone can give me a hint with the maths

No, in fact the tangent binormal generator will need normals already to generate tangents.

The thing is that it largely depends on what kind of normals you want. There is no one math answer. We’d even have to know more about the mesh you are building to even make suggestions, really.

Edit: and 9 times out of 10, the code generating the mesh has a better idea of what the normals should be than any kind of post processing.

1 Like

Ok, I understand that it’s highly dependant on the use case.
I was hoping that there would be some kind of method that takes into acount a given vertex and it’s neighbouring vertices and calculates a normal vector from these. The final result would be a smoothed shading effect on the mesh. I guess that a similar approach has to be taken for the terrain to be smooth shaded. Maybe I’ll have to dig into the source code there…

@Pauli said: Ok, I understand that it's highly dependant on the use case. I was hoping that there would be some kind of method that takes into acount a given vertex and it's neighbouring vertices and calculates a normal vector from these. The final result would be a smoothed shading effect on the mesh. I guess that a similar approach has to be taken for the terrain to be smooth shaded. Maybe I'll have to dig into the source code there...

Here is the general idea. It uses a diamond square for simplification, though the general idea applies to most circumstances:

[java]
Vector3f v0 = new Vector3f(); // Will store the vert you are calculating for
Vector3f v1 = new Vector3f(); // 1st surrounding vert (in clockwise fashion)
Vector3f v2 = new Vector3f(); // 2nd surrounding vert
Vector3f v3 = new Vector3f(); // 3rd surrounding vert
Vector3f v4 = new Vector3f(); // 4th surrounding vert
Vector3f r1, r2, r3, r4, normal;

// Next populate (.set) the temp verts
// I’ll leave this up to you

// Next subtract the center vert (v0)
v1.subtractLocal(v0);
v2.subtractLocal(v0);
v3.subtractLocal(v0);
v4.subtractLocal(v0);

// Next get the normalized cross product for each surrounding vert
r1 = v1.cross(v2).normalize();
r2 = v2.cross(v3).normalize();
r3 = v3.cross(v4).normalize();
r4 = v4.cross(v1).normalize();

// Lastly, get the sum of the above cross products
normal.set(r1);
[/java]

1 Like
@Pauli said: Ok, I understand that it's highly dependant on the use case. I was hoping that there would be some kind of method that takes into acount a given vertex and it's neighbouring vertices and calculates a normal vector from these. The final result would be a smoothed shading effect on the mesh. I guess that a similar approach has to be taken for the terrain to be smooth shaded. Maybe I'll have to dig into the source code there...

How are you generating the mesh? What does it look like?

You certainly could average the normals as above but that doesn’t always work well… in fact sometimes it will look really bad depending on what the mesh looks like. I could make other random suggestions also but it would be much quicker if we knew what the mesh looks like. Otherwise it becomes a game of “guess the letter between 1 and 100”.

@pspeed said: How are you generating the mesh? What does it look like?

You certainly could average the normals as above but that doesn’t always work well… in fact sometimes it will look really bad depending on what the mesh looks like. I could make other random suggestions also but it would be much quicker if we knew what the mesh looks like. Otherwise it becomes a game of “guess the letter between 1 and 100”.

100 to 1 it is either a cube or terrain… When is it anything other than a cube or terrain? Oh wait… it could be rocks or trees or impostors! Which is back to cubes and terrain. =) Soooo… if the answer falls under the “terrain” category, Use the above.

Source data still matters.

Sorry for the late reply guys, I was on vacation.

@t0neg0d: Thanks, that looks exactly like the method I was looking for. I will try this as soon as possible!

@pspeed: The mesh actually could be assumed to be a terrain mesh. I’m working on some kind of a dynamic cloth thing.
And I’m well aware of the fact that if it would be something like a cube I had to use some other method, because it would have hard edges. But to think of my mesh as a terrain simplifies things a lot.

@Pauli said: Sorry for the late reply guys, I was on vacation.

@t0neg0d: Thanks, that looks exactly like the method I was looking for. I will try this as soon as possible!

@pspeed: The mesh actually could be assumed to be a terrain mesh. I’m working on some kind of a dynamic cloth thing.
And I’m well aware of the fact that if it would be something like a cube I had to use some other method, because it would have hard edges. But to think of my mesh as a terrain simplifies things a lot.

Then it should be pretty easy. If it turns out that the frequency of your data is higher than the normals would show (as in my sample pic) then you can always bias the normals to one side or another. (If you bias left then the odd normals in my pic would point a bit left and the even ones would point a bit right since the left sample is higher… don’t know if I’m explaining right.) It’s all an approximation anyway.

For terrain, I’ve often just gone ahead and calculated the normals just against the west and north points. In my case, it was better to have shadows that were a bit off (hard to notice anyway) than to have areas that looked flat because the frequency was too high for proper shadows. I think in Mythruna’s current “far terrain” low-res terrain I do that, too.

@pspeed said: For terrain, I've often just gone ahead and calculated the normals just against the west and north points. In my case, it was better to have shadows that were a bit off (hard to notice anyway) than to have areas that looked flat because the frequency was too high for proper shadows. I think in Mythruna's current "far terrain" low-res terrain I do that, too.

That’s a neat idea. Is the distance terrain different than the terrain at a low LoD?

@t0neg0d said: That's a neat idea. Is the distance terrain different than the terrain at a low LoD?

Close up it’s a normal block world. Far away (from local clip out to a bit over 1 km) it’s just a low res height map of the block world. Because it is lower res, many times the normals would point nearly straight up if I calculate “proper” ones. Using just north and west makes for more dramatic shadows anyway which is often nicer for the far terrain especially in contrast to how the block world portion already has high contrast because of the block sides.

1 Like