Making a top down isometric shooter, need some help with hit detection

So, I’m trying to do ray-casting for hit detection in my shooter, but the ray always seems to be off from the coordinates I’m sending it. I cant for the life of me figure out where I’m making an error, other than that it could have something to do with the camera angle.

Below I have pasted in the code for my shoot() method, I’m also drawing a line for testing that should show where the ray is, but the mark that shows up on the targets is always some small angle off from where the line shows.


private void shoot(){

//BUG: Ray does not line up perfectly with line

CollisionResults results = new CollisionResults();

Vector2f mousePosNoOff = new Vector2f();

//Converts from screen position to a coord grid of -1.0 to +1.0

mousePosNoOff.x = (cursorPos.x - settings.getWidth()/2)/(settings.getWidth()/2);

mousePosNoOff.y = (cursorPos.y - settings.getHeight()/2)/(settings.getHeight()/2);

//Convert to polar coords so I can add 45 degrees

float polarAngle = mousePosNoOff.getAngle() - FastMath.PI/4;

float polarMag = FastMath.sqrt(FastMath.pow(mousePosNoOff.x, 2) + FastMath.pow(mousePosNoOff.y, 2));

Vector2f mousePos = new Vector2f(polarMag * FastMath.cos(polarAngle),

polarMag * FastMath.sin(polarAngle));

Vector3f playerPos = player.getWorldPos();

Vector3f rayCoords = new Vector3f(((mousePos.x * 300) + playerPos.x),

((mousePos.y * 300) + playerPos.y),


Ray ray = new Ray(playerPos, rayCoords);


//Line that I’m using for testing - it should have the same coords as ray

Mesh lineMesh = new Mesh();



lineMesh.setBuffer(VertexBuffer.Type.Position, 3, new float[]{








lineMesh.setBuffer(VertexBuffer.Type.Index, 2, new short[]{ 0, 1 });




shootables.collideWith(ray, results);

if (results.size() > 0) {


Collisions? " + results.size() + "
float dist = results.getClosestCollision().getDistance();
Vector3f pt = results.getClosestCollision().getContactPoint();
String hit = results.getClosestCollision().getGeometry().getName();
System.out.println(" You shot " + hit + " at " + pt + ", " + dist + " wu away.");
if(pt != null){
} else {

What am I doing wrong? Why is the ray not lined up with the line?
If anyone needs more info, the full source is here:


Why you do all that fancy stuff with normalizing to 0-1 and using polar coords?

when you make a ray:

Ray ray = new Ray(playerPos, rayCoords);

you dont give it 2 points, you give it a starting point and a direction, you can use “cam.getDirection()” for the second parameter, in this case, since your doing “top down”, you should try giving it a normal vector of the angle your mouse makes in relation to your character…

Vector3f rayCoords = new Vector3f(((mousePos.x * 300) + playerPos.x), ((mousePos.y * 300) + playerPos.y), 1f);

that “1f” might very well be your problem, since that’s a direction, NOT a position, that is slightly moving the angle 1f to z, so if playerPos.z is 1f, the ray will slowly move up to 2f…

and will reach 2f after

x = playerPos.x + (((mousePos.x * 300) + playerPos.x)

and y = playerPos.y + ((mousePos.y * 300) + playerPos.y)

and after that it will keep moving z+=1 at that same rate…

then again, you should make your life easier and make a normal vector instead of using rayCoords as it is…

also, i see your using parts of “hello_picking” tutorial… i recommend you read it again and try to understand it fully…

hope this helps you…