The API docs for intersectWherePlanarQuad() claim it returns the plane hit coords as t, u, v.
This example shows that the ray will register a hit beyond the right side of the defined quad (as looking from the ray's PoV). So what does it return in the loc storage vector? Well, :D u will exceed 1.0f, which it should never do(?).
Note that it's only positive u that gives a problem – neg. u and both pos. and neg. v work perfectly, with no hit registered on the plane.
Please could this be fixed? Thanks!
//These points outline a set of 4 walls, each of which will be a plane. Room is 5x5.
outlinePoints = new Vector3f
new Vector3f(2.5f,0,-2.5f), //ne corner
new Vector3f(2.5f,0,2.5f), //se corner
new Vector3f(-2.5f,0,2.5f), //sw corner
new Vector3f(-2.5f,0,-2.5f) //nw corner
//This ray shouldn't hit the north wall, because it's shoting a full 1.5 units beyond the right edge, but it does.
Ray ray = new Ray(new Vector3f(4.0f, -.5f, 0), Vector3f.UNIT_Z.negate());
Vector3f point1, point2, point3, hit = Vector3f.ZERO;
Plane planes = new Plane[outlinePoints.length];
//Create each plane and check ray against it
for (int index = 0; index < outlinePoints.length; index++)
lastIndex = index == 0? room.foundationOutlinePoints.length - 1 : index - 1;
//plane ranges between p1 and p2 in x and z and 0 and -1 in y
point1 = room.foundationOutlinePoints[lastIndex];
point2 = room.foundationOutlinePoints[index];
point3 = point1.add(Vector3f.UNIT_Y.negate()); //the bottom left point of the proposed quad
boolean result = ray.intersectWherePlanarQuad(point1, point2, point3, hit);
if (!result ) hit = new Vector3f();
Main.logger.info("Ray intersects plane? "+result);
Main.logger.info("Ray intersects plane at "+hit.x +", "+ hit.y +", "+ hit.z);
EDIT: Also worth pointing out that the storage vector remains unchanged on a fail, meaning it retains the same value it had before, if the variable is reused. I don't know if this is the correct way, but it might be worth nulling it when there is no hit? Maybe I'm wrong here.