Mouse picking accuracy problem

Hi!
I’m writing a map editor using jMonkeyEngine that supports editing terrain (just like the jME SDK). And I have a problem. The problem’s name’s float accuracy I think.

When I’m staying close to the terrain object, everything’s ok: http://i.imgur.com/bjGyxcc.png
Problem starts when I’m getting further… The brush’s position starts to getting inaccurate and - what’s even worse - it starts shaking… http://i.imgur.com/3ch9yYf.png

I know that these 2 screens don’t describe my problem perfectly, but it’s the most I can offer at this moment I think.

My code used to do mouse picking is just a standard one:

	public static CollisionResults pick(Collidable collidable,
			Vector2f mousePos, Camera cam) {
		Vector3f pos = cam.getWorldCoordinates(
				new Vector2f(mousePos.x, mousePos.y), 0).clone();
		Vector3f dir = cam
				.getWorldCoordinates(new Vector2f(mousePos.x, mousePos.y), 1)
				.subtractLocal(pos).normalizeLocal();

		return pick(collidable, pos, dir);
	}

	public static CollisionResults pick(Collidable collidable, Vector3f origin,
			Vector3f direction) {
		CollisionResults results = new CollisionResults();
		Ray ray = new Ray(origin, direction);
		collidable.collideWith(ray, results);

		return results;
	}

Interesting (and maybe important) fact is that when I replace new Vector2f(mousePos.x, mousePos.y), 1) with new Vector2f(mousePos.x, mousePos.y), .1f) it’s even more inaccurate and shaking much more. That’s why I think it MAY BE a float precission problem.

Another fact is that when I scaled the terrain to .1f, .1f, .1f, the problem doesn’t occur (or, to be more exact: it isn’t as much noticeable as when using normally-scaled terrain).

Please help me.
Regards

How much further is further?

I think we need a video of the symptoms and some hard numbers on what you are doing, scales etc.

<cite>@zarch said:</cite> How much further is further?
First symptoms of "shaking" appears at about 1000 units. At about 2500 units brush shaking is really annoying. Then it's only worse.
I think we need a video of the symptoms and some hard numbers on what you are doing, scales etc.
Ok, I'll try to record the video that'll image the problem. Terrain's local and world scale is Vector3f(1, 1, 1). Terrain size is 1024x1024. If you want any other numbers, just name them, I'll post everything. :)

Ok, I recorded the video. Here it is. I know it’s not the perfect quality, but I think it’s good enough to describe my problem.

I don’t really see the problem - you mean the way the brush jitters a bit as you zoom out?

Yeah, it’s exactly what I mean. And I think it’s not “a bit”. At the length of ~6500 units the brush shakes at almost entire area of 256x256 terrain quad.

You may, of course, think it’s not big deal - but it kinda is. This jittering affects the transform gizmo as well. When you go further of it, it starts blinking (it is highlighted with mouse cursor). I know it’s not really important - but it’s really annoying.

<cite>@m4tx said:</cite> Interesting (and maybe important) fact is that when I replace new Vector2f(mousePos.x, mousePos.y), 1) with new Vector2f(mousePos.x, mousePos.y), .1f) it's even more inaccurate and shaking much more. That's why I think it MAY BE a float precission problem.

Instead of 1 or .1f, try 10000f.

I tried it, but it doesn’t work at all - the brush doesn’t show up.

I also tried something like this:

Vector3f dir = cam
				.getWorldCoordinates(new Vector2f(mousePos.x, mousePos.y), 10000f)
				.subtractLocal(pos).divideLocal(10_000f).normalizeLocal();

But it doesn’t work either.

Does it only jitter while you are zooming or does it still happen when the zooming stops?

The brush doesn’t jitter when the zooming stops, BUT it is inaccurate anyway.
Short video
As you can see, the brush “jumps”.

It certainly looks like a rounding or precision error. I wonder if its because you are casting an extremely long ray from your mouse - so tiny differences at the start of the ray end up being massive differences at the end.

Is zooming out this far something you absolutely need to be able to do?

<cite>@zarch said:</cite> Is zooming out this far something you absolutely need to be able to do?
Hmm, yeah, I think so... I'm wondering about scaling rootNode to .1f, .1f, .1f; I think this will solve the problem - but maybe there's some other way to fix that?

EDIT:
Hm, but scaling rootNode will cause, for example, TerrainMonkey to look inappropriate - the waves will be too big. :confused:

Try doing the test with really small screen resolution and with largest you can.

You may find the problem is much worse at low resolutions.

Hmm, I don’t think so. When I scaled the window to very low size, the problem appears anyway - it’s a bit less noticeable, but still noticeable. The editor is prepared to work with 1024x768 resolution or higher - and the accuracy problem seems to appear in 1024x768, so my problem isn’t solved anyway… :E

Maybe your ray is colliding with your brush sphere first sometimes.

I wish it were true :frowning: But I’m colliding ray with terrain only.

Honestly, the only solution I can think of at the moment is to average the pointer position over time and move towards it smoothly rather than “jumping”. That will cosmetically improve things.

Someone else might have better ideas though.