# Mouse picking (yet another thread)

So I’ve followed the Mouse Picking tutorial (as well as dugg through all the various “mouse picking” topics on this forum) and now have the following code:

```ray = new Ray(); Vector3f pos = cam.getWorldCoordinates(new Vector2f(mouseX, mouseY), 0f).clone(); Vector3f dir = cam.getWorldCoordinates(new Vector2f(mouseX, mouseY), 1f).clone(); dir.subtractLocal(pos).normalizeLocal(); ray.setOrigin(pos); ray.setDirection(dir); scene.collideWith(ray, results); for (int i = 0; i < results.size(); i++) { // For each hit, we know distance, impact point, name of geometry. float dist = results.getCollision(i).getDistance(); Vector3f pt = results.getCollision(i).getWorldContactPoint(); String hit = results.getCollision(i).getGeometry().getName(); infoPanel.setText(i + ": " + hit + " at " + pt + ", " + dist + “units away”); }```

The above displays the coordinates of the resulting collision. The problem that I am having (and I suspect it has to do with the z coordinates of vectors pos and dir) is that the ray seems to shoot off at an angle. That is, when I click the water in front of me, it may note that in fact the ray collided with the sky (and vice versa). Also, unless I move the camera, the points that I click appear to remain the same.

Summarizing: I have a problem converting the 2D mouse coordinates into 3D. Any tips would be greatly appreciated. See screenshots below:

Mouse pointer is highlighted by a black dot.

Thanks

Just theorizing (i didn’t test picking) :

you have the screen space coordinates of the mouse (x,y), and you want the coordinates of the point picked in world space.

What i’d do : multilply the invProjMatrix by the mouse coordinates. set the z value of the resultant vector with the length of the ray. this should give you the coordinates of the point in view space.

To get them in world space multiply the invWorldMatrix with the view space coordinates.

When you say “et the z value of the resultant vector with the length of the ray.” do you mean “as”? Or do you mean “multiply with”?

Edit: Sorry, I don’t quite follow the logic behind this (my maths skills are rather rusty).

don’t worry it’s a tricky part, i had nightmares about it before

i mean “as”

look at this wiki page

Look at the “Spaces and Matrices” chapter.

Your mouse coordinates are in the Projection Space. you need to translate them into worldspace.

But, with a simple transformation you won’t be able to restore the z coordinate.

The rays is shot in view space. The view space is the camera space with its origin at the position of the camera.

the z axis in view space goes straight forward. So when you shoot the ray, the length of the ray is the z coordinate of the intersecting point in view space. (i assume here that the direction of the ray is the direction of the cam).

mhhh, reading back the tutorial, this line Vector3f pt = results.getCollision(i).getWorldContactPoint(); should return what you need.

I don’t get what is the problem

lol, a nightmare indeed. Thanks. I will read through it though.

I already use `Vector3f pt = results.getCollision(i).getWorldContactPoint(`). The problem is that no matter where I click on the screen, it always returns the same point:

1. I click on the screen. Returns a point. I click somewhere else; still returns the same point.
2. I move. I click. I get a different point. Now, I continue clicking anyway on the screen, and I will still receive the same point, until I start moving again (so, if, for example, I click at mouse(x=0, y=0) I get the same result than if I click at mouse(x=150, y=100). Hence I suspect that there is a problem with the z axis somewhere. Below the entire code:

``` private void onMouseClicked() { ray = new Ray(); Vector2f mouseCords = new Vector2f(mouseX, mouseY); Matrix4f matrix = cam.getProjectionMatrix(); matrix.mult(mouseCords.toArray(null)); Vector3f pos = cam.getWorldCoordinates(mouseCords, 0).clone(); Vector3f dir = cam.getWorldCoordinates(mouseCords, 1f).clone(); dir.subtractLocal(pos).normalizeLocal(); ray.setOrigin(pos); ray.setDirection(dir); scene.collideWith(ray, results); for (int i = 0; i < results.size(); i++) { // For each hit, we know distance, impact point, name of geometry. float dist = results.getCollision(i).getDistance(); Vector3f pt = results.getCollision(i).getWorldContactPoint(); String hit = results.getCollision(i).getGeometry().getName(); System.out.println("* Collision #" + i); System.out.println(" You shot " + hit + " at " + pt + “, " + dist + " wu away.”); infoPanel.setText(i + ": " + hit + " at " + pt + ", " + dist + “units away”); } }```

EDIT: I said MOVE but meant changing the viewport using the mouse wheel.

If it helps you, here is my method for picking and it works rather fine:

``` Vector2f mouseCoords = new Vector2f(mouseX, mouseY); CollisionResults results = new CollisionResults(); Vector3f pos = cam.getWorldCoordinates(mouseCoords, 0).clone(); Vector3f dir = cam.getWorldCoordinates(mouseCoords, 0.3f).clone(); dir.subtractLocal(pos).normalizeLocal(); Ray ray = new Ray(pos, dir); ```

Are you sure you update your mouseX and mouseY properly in the mouse motion listener callback?

@InShadow: Thanks, but that’s what I have…well, had (I increased 0.3f to 1f). It gives me a collision detection alright. Just not the right one

@nehon: Yup, 100% sure. Here my code:

``` public class MouseInputListener implements RawInputListener { private InfoPanel infoPanel; private Camera cam; private Ray ray; private CollisionResults results; private Node scene; private boolean mouseClicked; private boolean mousePressed; private int mouseX, mouseY; public MouseInputListener(InfoPanel infoPanel, Camera camera, Node scene) { this.infoPanel = infoPanel; this.cam = camera; this.scene = scene; results = new CollisionResults(); mouseClicked = false; mousePressed = false; mouseX = mouseY = 0; } public void onJoyAxisEvent(JoyAxisEvent evt) { } public void onJoyButtonEvent(JoyButtonEvent evt) { } public void onKeyEvent(KeyInputEvent evt) { } public void onMouseButtonEvent(MouseButtonEvent evt) { if (evt.isPressed()) { mousePressed = true; if (mouseClicked) { mouseClicked = false; } } else if (evt.isReleased()) { if (mousePressed) { mouseClicked = true; onMouseClicked(); } } } public void onMouseMotionEvent(MouseMotionEvent evt) { mouseX = evt.getX(); mouseY = evt.getY(); } private void onMouseClicked() { ray = new Ray(); Vector2f mouseCords = new Vector2f(mouseX, mouseY); Matrix4f matrix = cam.getProjectionMatrix(); matrix.mult(mouseCords.toArray(null)); Vector3f pos = cam.getWorldCoordinates(mouseCords, 0).clone(); Vector3f dir = cam.getWorldCoordinates(mouseCords, 1f).clone(); dir.subtractLocal(pos).normalizeLocal(); ray.setOrigin(pos); ray.setDirection(dir); scene.collideWith(ray, results); for (int i = 0; i < results.size(); i++) { // For each hit, we know distance, impact point, name of geometry. float dist = results.getCollision(i).getDistance(); Vector3f pt = results.getCollision(i).getWorldContactPoint(); String hit = results.getCollision(i).getGeometry().getName(); System.out.println("* Collision #" + i); System.out.println(" You shot " + hit + " at " + pt + “, " + dist + " wu away.”); infoPanel.setText(i + ": " + hit + " at " + pt + ", " + dist + “units away”); } // 5. Use the results (we mark the hit object) // if (results.size() > 0) { // CollisionResult closest = results.getClosestCollision(); //// mark.setLocalTranslation(closest.getWorldContactPoint()); //// scene.attachChild(mark); // } } } ```