3D Scene Spatial Event Support Through the GUI Library

This started with a need posted from @sgold dealing with leveraging the ToolTips feature and sort of grew from there… however, I wanted people using the library to be aware of the addition of event support. Here is the post from the other thread that gives a basic rundown of what it is and how to use it:

To start, the following events will be handled (I want a bit more time to consider keyboard and touch events… specifically multi-touch)

MouseButtonListener
MouseWheelListener
MouseFocusListener
MouseMovementListener

Here is a quick vid of mouse focus + contextual right-click menus for scene objects:

[video]http://youtu.be/Gqhq5LXksCo[/video]

The code to make the above work was grueling:

ExtendedNode.java
[java]
import com.jme3.scene.Node;
import tonegod.gui.listeners.MouseButtonListener;
import tonegod.gui.listeners.MouseFocusListener;

/**
*

  • @author t0neg0d
    */
    public abstract class ExtendedNode extends Node implements MouseFocusListener, MouseButtonListener { }
    [/java]

Main.java
[java]
import com.jme3.app.SimpleApplication;
import com.jme3.input.event.MouseButtonEvent;
import com.jme3.input.event.MouseMotionEvent;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Vector2f;
import com.jme3.renderer.RenderManager;
import com.jme3.scene.Geometry;
import com.jme3.scene.Mesh;
import com.jme3.scene.Node;
import com.jme3.scene.shape.Box;
import com.jme3.scene.shape.Sphere;
import tonegod.gui.controls.menuing.Menu;
import tonegod.gui.core.*;

/**

  • test

  • @author t0neg0d
    */
    public class Main extends SimpleApplication {
    private Screen screen;
    private Menu menu, menu2;

    public static void main(String[] args) {
    Main app = new Main();
    app.start();
    }

    @Override
    public void simpleInitApp() {
    setupCamera();
    createGUIScreen();
    layoutGUI();

     flyCam.setDragToRotate(true);
     inputManager.setCursorVisible(true);
    

    }

    private void setupCamera() {
    inputManager.setCursorVisible(true);
    flyCam.setMoveSpeed(50);
    }

    private void createGUIScreen() {
    screen = new Screen(this);
    screen.setUse3DSceneSupport(true);
    guiNode.addControl(screen);
    }

    private void layoutGUI() {
    createMenus();

     Box box = new Box(1,1,1);
     Sphere sphere = new Sphere(12,12,1);
     
     Material mat = new Material(assetManager,"Common/MatDefs/Misc/Unshaded.j3md");
     mat.setColor("Color", ColorRGBA.Blue);
     
     Node n = this.getNewGUINode(box, mat.clone(), menu);
     Node n2 = this.getNewGUINode(sphere, mat.clone(), menu2);
     n2.setLocalTranslation(3,0,0);
     
     rootNode.attachChild(n);
     rootNode.attachChild(n2);
    

    }

    private void createMenus() {
    menu = new Menu(screen, Vector2f.ZERO, false) {
    @Override
    public void onMenuItemClicked(int index, Object value, boolean isToggled) {

     	}
     };
     menu.addMenuItem("Contextual Box Menu Item 1", 0, null);
     menu.addMenuItem("Contextual Box Menu Item 2", 1, null);
     menu.addMenuItem("Contextual Box Menu Item 3", 2, null);
     menu.addMenuItem("Contextual Box Menu Item 4", 3, null);
     screen.addElement(menu);
     
     menu2 = new Menu(screen, Vector2f.ZERO, false) {
     	@Override
     	public void onMenuItemClicked(int index, Object value, boolean isToggled) {
     		
     	}
     };
     menu2.addMenuItem("Contextual Sphere Menu Item 1", 0, null);
     menu2.addMenuItem("Contextual Sphere Menu Item 2", 1, null);
     menu2.addMenuItem("Contextual Sphere Menu Item 3", 2, null);
     menu2.addMenuItem("Contextual Sphere Menu Item 4", 3, null);
     screen.addElement(menu2);
    

    }

    private Node getNewGUINode(Mesh mesh, Material mat, final Menu menu) {
    Geometry geom = new Geometry();
    geom.setMesh(mesh);
    ExtendedNode node = new ExtendedNode() {
    public void onGetFocus(MouseMotionEvent evt) {
    ((Geometry)getChild(0)).getMaterial().setColor(“Color”, ColorRGBA.Yellow);
    evt.setConsumed();
    }
    public void onLoseFocus(MouseMotionEvent evt) {
    ((Geometry)getChild(0)).getMaterial().setColor(“Color”, ColorRGBA.Blue);
    evt.setConsumed();
    }
    public void onMouseLeftPressed(MouseButtonEvent evt) { }
    public void onMouseLeftReleased(MouseButtonEvent evt) { }
    public void onMouseRightPressed(MouseButtonEvent evt) { }
    public void onMouseRightReleased(MouseButtonEvent evt) {
    menu.showMenu(null, screen.getMouseXY().x, screen.getMouseXY().y-menu.getHeight());
    evt.setConsumed();
    }
    };
    node.attachChild(geom);
    node.setMaterial(mat);
    return node;
    }

    @Override
    public void simpleUpdate(float tpf) { }
    @Override
    public void simpleRender(RenderManager rm) { }
    }
    [/java]

Lastly, I wanted to note how to leverage custom cursors for your 3D scene spatials. When using the MouseFocusListener, in onGetFocus, add:

[java]
screen.setForcedCursor(StyleManager.CursorType.HAND); // Or whatever cursor/custom cursor you want
[/java]

And in onLoseFocus, add:

[java]
screen.releaseForcedCursor();
[/java]

Everything else is fairly self-explanatory. But, do feel free to ask questions if you decide to use this for event handling in your game.

Screen.java has been updated for this and the changes committed.

3 Likes

jmonkeyplatform-contributions rev 1446 hasn’t showed up in the Plugin Portal yet. In fact, neither has rev 1445 from May 4th, so maybe the distribution process broken.

I pulled tonegodgui rev 513 from Google Code, rebuilt my local copy of the GUI library, and verified that screen.setUse3DSceneSupport(true) and screen.setForcedToolTip() work together as expected in my maze game.

I notice that getEventNode() looks only at the first scene in the default viewport. Because my game has two overlapping viewports, I don’t think I can use the 3D scene support as it stands—which is fine with me because I already have my own picking routines in place.

1 Like
@sgold said: jmonkeyplatform-contributions rev 1446 hasn't showed up in the Plugin Portal yet. In fact, neither has rev 1445 from May 4th, so maybe the distribution process broken.

I pulled tonegodgui rev 513 from Google Code, rebuilt my local copy of the GUI library, and verified that screen.setUse3DSceneSupport(true) and screen.setForcedToolTip() work together as expected in my maze game.

I notice that getEventNode() looks only at the first scene in the default viewport. Because my game has two overlapping viewports, I don’t think I can use the 3D scene support as it stands—which is fine with me because I already have my own picking routines in place.

I can update this to cycle through the available scenes. This shouldn’t be a problem at all.

EDIT: Most people will have their own method of doing this already. It’s more for convenience sake if starting a new project. It will take me a bit to get this to a point that it is simple plug-n-play, but I plan on updating the old MMO project I was working on to use this in place of current 3d scene event handling and see how it goes/update to accommodate/etc.

EDIT 2: It also doesn’t account for picking through the OSRViewPort control… something that should probably be added as well.

@sgold
One last question… can you give me a rundown of how these 2 scenes are handled? Are they offscreen and then a composite of the output? Or just 2 separated root nodes?

The second scene is an inset viewport in the main viewport, created using renderManager.createMainView().

By the way, I found some issues with the implementation of setForcedToolTip() and relaseForcedToolTip().

First, releasing a forced tool tip should not do
[java]
toolTip.setText("");
toolTip.hide();
[/java]
because that effectively disables ordinary tool tips.

Second, shouldn’t these methods update forcedToolTip and forcedToolTipText regardless of whether useToolTips is true? What if tool tips get enabled right after a mouse motion event?

1 Like

Another thing: I want the GUI tool tips to override 3-D tool tips, not vice versa. In other words, if the mouse cursor hovers over a button that happens to be obscuring a 3-D object, I want the button’s tool tip to be displayed, not the 3-D object’s tool tip. (The word “forced” implies the opposite priority.) So the decision which tool tip to display has to be made each time a mouse motion event is received.

1 Like
@sgold said: Another thing: I want the GUI tool tips to override 3-D tool tips, not vice versa. In other words, if the mouse cursor hovers over a button that happens to be obscuring a 3-D object, I want the button's tool tip to be displayed, not the 3-D object's tool tip. (The word "forced" implies the opposite priority.) So the decision which tool tip to display has to be made each time a mouse motion event is received.

The GUI element will take priority as onLoseFocus should call releaseForcedToolTip.

@sgold said: The second scene is an inset viewport in the main viewport, created using renderManager.createMainView().

By the way, I found some issues with the implementation of setForcedToolTip() and relaseForcedToolTip().

First, releasing a forced tool tip should not do
[java]
toolTip.setText(“”);
toolTip.hide();
[/java]
because that effectively disables ordinary tool tips.

Second, shouldn’t these methods update forcedToolTip and forcedToolTipText regardless of whether useToolTips is true? What if tool tips get enabled right after a mouse motion event?

The above removes the tooltip element from the rendered scene, however the next time the mouse moves over an element with ToolTipText set (or setForcedToolTip is called), it is displayed again. Clearing the BimapText is to ensure that there is no visible flicker when updating the text for the next tooltip… in case of lag, frame rate stutter, etc.

The tooltips element is only created when screen.setUseToolTips(true) is called, otherwise it doesn’t exist. This same thing applies to the BitmapText associated with an Element. If setText is never called, the BitmapText is never created. I believe if setText(null) is called after the fact, the BitmapText is removed and GC’d.

The reason I went with setForcedToolTip is it mirrors setForcedCursor if using custom cursors. Since this is the method you would use to apply custom cursors to your scene objects, I figured it would be far easier to remember both.

EDIT: To verify that the tooltip is not disabled when completely when setText(“”) and hide() are called, watch the second video I posted in the other thread on tool tips. As I mouse over the ui element from the event driven spatial, onLoseFocus triggers releaseForcedToolTip allowing the GUI to take back control of the tooltip window.

EDIT 2: Forget what I just said. I’ll fix the ui priority issue. My test case is flawed.

@sgold
I updated my test case to drop a button drirectly onto the screen with Tool Tip Text set. Moved the camera to have the spatial below the button and moved the mouse back and forth between the two object.

The tooltip updated properly.

EDIT: What didn’t update properly was the cursor >.<

@sgold
I also committed an update that will check through each scene within the main viewport and return the closest collision between all results. if you want to grab the latest copy of screen and let me know if it works for you, I’d greatly appreciate it :wink:

EDIT: Hold off on this for a few. I think I want to change how this works. I’ll keep you posted :wink:

EDIT 2: Ok, I updated this to (by default) add the rootNode of the default scene to the list of nodes to check within.

To edit this list: (These can reside in any ViewPort however, the distance check is against the default ViewPort’s camera atm)

[java]
screen.addScene(Node n);
screen.removeScene(Node n);
[/java]

The check will now find the closest collision within all nodes added to the list of scenes.

If you want to remove the default scene, just call: (Yes, obvious… but thought it worth mentioning anyways)

[java]
screen.removeScene(rootNode);
[/java]

Perhaps it’s an unimportant point, but I don’t want the closest collision within all scenes in my game. I want the scene object that’s visible on the screen. In the case of an inset viewport, these are quite different things.

@sgold said: Perhaps it's an unimportant point, but I don't want the closest collision within all scenes in my game. I want the scene object that's visible on the screen. In the case of an inset viewport, these are quite different things.

This can be limited to whatever you want it to be now. However, the distance check is against returns from raycasting… not every spatial in your scene. The point of the update was to allow you to limit what nodes are checked against, so if you have all collidables branched in your scene (or scenes depending on the number of viewports you are using), even if these are branched in unrelated places, you could:

[java]
screen.removeScene(rootNode);
screen.addScene(myClickableDohickies);
screen.addScene(myTraversableMeshes);
// etc
[/java]

And these are the only nodes that will be checked against when raycasting into your scene(s)

EDIT: There is one further update I’ll be adding today and that is to ensure that if the object obstructing your event driven Node resides in another node, these are taken into account. If they both reside in the same Node, this automatically negates the other collision… not so if you are checking multiple nodes. Anyways, this will be accounted for in just a bit here.

Here’s a three week-old screenshot:

The map inset covers about 10% of the screen, in the upper left. When the cursor is in the inset, I want collisions only with spatials in the inset’s scene graph. When the cursor is outside the inset, I want collisions with spatials in the main 3-D scene graph. In order to use your picking interface, I’d have to invoke removeScene() and addScene() every time the cursor crossed this rectangular boundary. To me it seems easier to decide on each event which scene graph to collide with.

1 Like
@sgold said: Here's a three week-old screenshot:

The map inset covers about 10% of the screen, in the upper left. When the cursor is in the inset, I want collisions only with spatials in the inset’s scene graph. When the cursor is outside the inset, I want collisions with spatials in the main 3-D scene graph. In order to use your picking interface, I’d have to invoke removeScene() and addScene() every time the cursor crossed this rectangular boundary. To me it seems easier to decide on each event which scene graph to collide with.

Ooooooh… I r teh get it now >.<

Ok… gimme a little bit to consider this. This was something I was going to handle (the comment above about the OSRViewPort control) but haven’t thought much past the initial “O.o wtf am I gonna do with that???”

Initial thought is:

Separate out checks against Viewports.
Still allow for limiting nodes within each.

To deal with this, the GUI will need access to all the “main” (onscreen) viewports, their cameras, and their scene nodes. And the order in which they’re rendered. And whether they’re transparent. Keep in mind that this info can change at runtime.

I don’t need/expect the GUI to juggle all this for me. I just need an interface which allows me to supply tooltips for areas of the screen not hidden by GUI controls. Provided the GUI consumes the right events, I think I can do this using raw input.

1 Like
@sgold said: To deal with this, the GUI will need access to all the "main" (onscreen) viewports, their cameras, and their scene nodes. And the order in which they're rendered. And whether they're transparent. Keep in mind that this info can change at runtime.

I don’t need/expect the GUI to juggle all this for me. I just need an interface which allows me to supply tooltips for areas of the screen not hidden by GUI controls. Provided the GUI consumes the right events, I think I can do this using raw input.

This is actually not an issue at all. I’d like it to function in a way that would allow you to comfortably opt for your own event handling or the gui’s support. So, every bit of input I can get in making this so is a HUGE help! Soooo… thank you!!

2 Likes