CursorEvents with localscale

So I have some UI’s in the ingame world

Now I wanted to start inplementing input for them via the cursorevents,

The good news is it seems to fit, if the element is not scaled.
The bad news is, similar to real computer screens, mine support different resolutions via an autoselect.

So the actual ui node is scaled to fit the supposed display.

Now I get the problem, that cursor events are completly off to somewhere else, and I have the feeling that they might not take scaling into consideration.

Should they already work with scaling? was this ever tested? What points are good candidates to start looking for where it starts to go wrong. (aka what part is supposed to do the actual ray? test in the ui.

PickEventSession is what’s doing the actual collideWith() calls… but it’s really just using JME’s normal collision stuff. Perhaps this is another case where JME is not dealing with non-uniform scaling properly? Or is your screen scaled uniformly?

I scale my UIs all the time in the guiNode and there is no problem. So it might be something specific to the 3D world.

Yes, they are non uniform, using a uniform scale greatly increases the pick results, but they are still of by a little:)

Anyway quite likely that I will remove/replace the picking in a fork for myself anyway, cause I already do a eyetrace every frame, no reason to do two (especially since they are kinda costly)

Lemur’s picking is modular. You could turn it off or just manage your own PickEventSession. It’s actually designed that way.

I use PickEventSession to manage the mouse delivery for things I render off screen through viewports. (Like the curved book pages in my Mythruna UI.) So I know that part works at least. :slight_smile:

If you figure out the scaling issue then let me know… but really the 3D pick locations should be properly in world coordinates. Maybe there is a worldToLocal() needed on your end somewhere or a deeper bug in JME itself.

My guess is that the same trickery that is used for transforming normals (e.g. inverse-transpose matrix) should be used for the ray collision here?

Hm ok, I notices a few strange things.

I have the cursor grabbed by the input manager, so I kinda assumed that it would be at the center of the screen (apparently it is not). This probably kinda explains my noticed disconnect between ingame movement and cursor position.

Now, how would I correctly create such a behaviour in JME?
Do I have to change the input manager to always return a neutral cursorpos, or is there a cleaner way build in to do this?

From a Lemur perspective, you could avoid using the MouseAppState and just manage your own PickEventSession that doesn’t use a real cursor position and only ever sets the center of the screen.

MouseAppState… all it does is forward the cursor position and RawInputListener button events to a PickEventSession.

Perfect, thats it :slight_smile:

I however had to modify a few parts of Lemur, is there a cleaner way to get the mouseState than editing the source?

	protected void onUIChange() {
		inputRequired = false;
		for (Spatial child : guiNode.getChildren()) {
			if (child instanceof IRequiresInput) {
				inputRequired = true;
				break;
			}
		}
		GuiGlobals.getInstance().mouseState.setEnabled(inputRequired);
		inputManager.setCursorVisible(inputRequired);
	}

and in simpleUpdate if the input is not used:

GuiGlobals.getInstance().mouseState.getSession().cursorMoved(getActualDisplayWidth() / 2, getActualDisplayHeight() / 2);

For this i had to change getSession() and cursorMoved to public.

Uh… getState(MouseState.class) ?

Actually, my suggestion was to not even use the state if you aren’t using it. But I guess then you’d have to forward your own button events. Still, there’s not much code in MouseState and you are not needing half of it.

Haha XD
I really should not probgramm at 5am anymore :slight_smile:

Anyway main reasons I want to keep it, is that it will manage all the 2d parts, and if it ever changes internally, I do not have to care at all about that, I only have to deal with the 3d part then.

I will look at making it more extensible.

Alternately, you could go with my original plan and instead of removing the app state you could just remove the root node registration from it so that it would just handle the 2D aspects.

Then what I could maybe do is factor out the internal listener that forwards the button events so it’s reusable. Though honestly I guess that code isn’t very onerous to duplicate.

Could you use a CursorEventControl on the root Element and then from there determine which child Element the cursor is over? CursorEventControl forwards the CollisionResult to your listener so you can grab the world space hit coordinate and convert it to local space then loop through the child Elements to see which one the cursor is over.