How to get projected DropDown to work

My interface is a Lemur interface that was rendered offline and than projected on a cube that rotates to show different in game menus:

(styling and layout have not been my main focus the last few weeks.)

To capture buttons being clicked, this code works:
        MouseEventControl.addListenersToSpatial(guiBox, new DefaultMouseListener() {
            protected void click( MouseButtonEvent event, Spatial target, Spatial capture ) {
            System.out.println("I've been clicked:" + target);

Here guiBox is the Geometry of the cube the menu is projected on.

But if you look carefully, you also see a dropdown box (Lemur calls it a Selector). And it is not clear to me how I can get that DropDown to respond to my mouse. Where to go from here?

General outline of what you need to do:

  • Create a PickEventSession that you will use to manage the UI events for your guiBox
  • Add your off screen viewport as the collision root on the pick event session
  • Register a CursorListener (not MouseListener) to your guiBox.
  • translate the CursorEvent location to locations on your off-screen UI and forward those events to the PickEventSession

That is basically all you need to get the mouse working with the GUI. The last step is only tricky for projected geometry where the click location is not an obvious mapping to the texture UI… but in your case that should be straight forward.

Your next problem will be that the selector popup will show up in the regular guiNode (and in the wrong place). To fix that you will set PopupState’s gui node to your root. (Edit: note that things might get weird here as I believe selector will try to make sure that the popup fits on screen and will be confused by the dimensions, etc…)

Ok… trying to follow these steps… this is what I got now:

        PickEventSession pickEvent = new PickEventSession();
        pickEvent.addCollisionRoot(guiNode, viewport);
        CursorEventControl.addListenersToSpatial(guiBox, new DefaultCursorListener() {
            protected void click(CursorButtonEvent event, Spatial target, Spatial capture) {
                System.out.println("I've been clicked:" + target);
                Vector2f v = event.getLocation();
                utils.LOG("Mouse:" + v.x + " , " + v.y);
                pickEvent.buttonEvent(0, (int) v.x, (int) v.y, event.isPressed());

Where guiNode is the rootnode Lemur is attached to. And guiBox is the box onto which the offline viewport is projected to.

  • I get unexpected values for the coords, I think what I am seeing are screen coords. I think I should search for UV coords of the collission but the are not in the collision result. Or am I missing something?
  • And where do I get the right button ID? Event has an isPressed method, but I see no method to obtain which button was pressed.

Those coords are screen UI coords just like for the mouse events… but the cursor events ALSO have the actual collision information that you can pull apart to see where things were actually clicked on your object and sort out position from that.


No idea why I missed event.getButtonIndex()…

Anyway, I think I found a way to get the UVS, but for that I need the collision. And somehow event.getCollision() returns null.

The button click doesn’t know the collision but the last cursor motion will have it.

Somehow I can not get Ray.intersectWherePlanar to work. It keeps returning false. Even in a case where I know the ray is getting collision results, intersectsWherePlanar does not give me the coords I need.

Vector2f click2d = app.getInputManager().getCursorPosition();
Vector3f click3d = cam.getWorldCoordinates(
                new Vector2f(click2d.x, click2d.y), 0f)
Vector3f dir = cam.getWorldCoordinates(new Vector2f(click2d.x, click2d.y), 1f).subtractLocal(click3d).normalizeLocal();
Ray ray = new Ray(click3d, dir);
app.getRootNode().collideWith(ray, results);


Vector3f w=new Vector3f();
Triangle tri=results.getClosestCollision().getTriangle(null);
ray.intersectWherePlanar(tri, w);

This is code that helps to determine which guy I select. It works. I can select guys.
I injected the last three lines, knowing this ray works. But still I get false returned from intersectWhenPlanar and the vector w stays zero.

Why are you trying to do this and what does it have to do with this topic?

You have a cursor listener. You are notified about cursor move events. Save the collision data aside from those events to use in the button click events.

Probably for your cube simply running worldToLocal on the contact point will give you the information you need to know where on the cube was clicked.

You have a cursor listener. You are notified about cursor move events. Save the collision data aside from those events to use in the button click events.

That one I got. I save getContactPoint to the Vector3f v in the move-event. In the click event I put these coords in pickEvent.buttonEvent:

pickEvent.buttonEvent(0, (int) v.x, (int) v.y, event.isPressed());

This does not work. I also tried putting v through guiBox.worldToLocal. But still no results.

You need to translate the point in 3D space to where it is on whatever face they are clicking on.

I know less than zero about your specific mesh/geometry/etc. setup so I can only guess. But usually the math to convert a “3D point on a cube” to a “2D point on a face” is trivial. When it is not then you have to get calculations involved with triangles and barycentric texture coordinates, etc…

If you know what that cube face represents in your 2D UI then you should be able to directly convert a 3D point to that 2D UI.

The UV route would be preferable since it the cube has one single texture - so just multiply them with the texture size and I got the coords I need.

Calculating the relative position on the 3D square could work too. I’ll have try.

Probably one line of math in the end.

Is a whole page of math.

You were right, in case of a cube it is not that hard. I am now sure I have the right coords ranging from {0,0} to {1024,1024}. However the interface still does not respond to clicks.

I set up PickEvent like this:

        pickEvent = new PickEventSession();

Where guiNode is the root node to which Lemur is attached, and viewport is the offscreen viewport where the texture is rendered.

I feed the calculated coords to buttonEvent like this:
pickEvent.buttonEvent(_button, x, y, true);

And you pass through all of the other events, too? ie: not just button events.

I think so, on pickEventSession I only see ButtonEvent and CursorMoved.

This is the related code:

        pickEvent = new PickEventSession();
        CursorEventControl.addListenersToSpatial(guiBox, new DefaultCursorListener() {
            protected void click(CursorButtonEvent event, Spatial target, Spatial capture) {                
                pickEvent.buttonEvent(event.getButtonIndex(), guiX, guiY, event.isPressed());
            public void cursorMoved(CursorMotionEvent event, Spatial target, Spatial capture ) {   
                Vector3f m=event.getCollision().getContactPoint();
                guiBox.worldToLocal(m, v);
                pickEvent.cursorMoved(guiX, guiY);

And here I translate the coords to the gui coords:

    int guiX=0, guiY=0;
    void calcGuiCoords(Vector3f _hitCoords){
        // place outside later on
        float SQSize=((float)SCREENSIZE/3);
        float SQHalf=SQSize/2;

        switch (currCubeFace){
            case 0:



click() is a helper method and not part of the listener interface. You miss a bunch of events this way but especially the button down event.

Javadoc here for the interface:

You need to implement and forward the move AND the down and up. Click is only giving you the up.


@pspeed, thanks for your patience and advice! I am very happy with how this works out. The options list with some styling will look great!