Using jMonkey in applications

Hi to everyone.

Few months ago I discovered jMonkeyEngine and I am pretty impressed with it.



Now I need your opinion about using jMonkeyEngine in applications (not in games).

This should be swing application that has some panels with some elements and one canvas (or more than one in tabs).

Currently I use java2d for drawing graphics on that canvas (some charts).

Now, I am thinking to replace java2d with jmonkey.



Do any of You have some advice how to start?

Is jMonkey good choice for that?



For the start I need:

  • canvas with 2d view
  • mouse cursor that can go outside canvas



    Later it will be needed some selection/picking abilities, and 3d views…



    Btw, I have some expirience with java/openGL ( I am the author of lavirinto3d game

    ( http://lavirinto3d.sourceforge.net ) ), but not so much in jMonkey.



    Thanks.

did you see RenParticleEditor / RenControlEditor ?

I did not, until now!



Thanks, this should be enough for start.



If I stuck somewhere I will post here :wink:


OK, it all make more sense now :slight_smile:



I have my class MonkeyPanel which contains one glCanvas.



My application has left panel with some components.

On the right side is a tabbedPane.

Initially there is one tab in tabbedPane wich holds one MonkeyPanel.

I also have a button for adding more tabs.

And, here is the problem:



When I try to create another tab with monkeyPanel application starts to throw exceptions:



Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
at com.jme.scene.state.lwjgl.LWJGLZBufferState.enableDepthTest(LWJGLZBufferState.java:109)
at com.jme.scene.state.lwjgl.LWJGLZBufferState.apply(LWJGLZBufferState.java:67)
at com.jme.renderer.lwjgl.LWJGLRenderer.clearBuffers(LWJGLRenderer.java:482)
at com.jmex.awt.SimpleCanvasImpl.doRender(SimpleCanvasImpl.java:142)
at com.jmex.awt.lwjgl.LWJGLCanvas.paintGL(LWJGLCanvas.java:110)
at org.lwjgl.opengl.AWTGLCanvas.paint(AWTGLCanvas.java:308)
at sun.awt.RepaintArea.paintComponent(Unknown Source)
at sun.awt.RepaintArea.paint(Unknown Source)
at sun.awt.windows.WComponentPeer.handleEvent(Unknown Source)
at java.awt.Component.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)




Is it possible to have more GLCanvases in application (in tabbedPane)?

If you need I can post here MonkeyPanel class...

Do you use jme1 or 2 ?

There were some improvements in that are in jme2.

I use jme 1.

Do you recommended me to switch to jme2?

Is it stable enough ?



Anyway I will download jme2 now…

Yeah, use jME 2.0 :slight_smile:

OK, I installed jme2…

there are many differences…

Example:

How to create glCanvas?

Function

DisplaySystem.getDisplaySystem().createCanvas(getWidth(), getHeight());



returns JMeCanvas and it can't be inserted in JPanel.



What is equivalent to this code from jme1:


canvas = DisplaySystem.getDisplaySystem().createCanvas(getWidth(),getHeight());
canvas.setMinimumSize(new Dimension(100, 100));
canvas.addComponentListener(new ComponentAdapter() {
public void componentResized(ComponentEvent ce)
{
doResize();
}
});

impl = new MyImplementor(getWidth(), getHeight());
((JMECanvas) canvas).setImplementor(impl);



How to add jmeCanvas in JPanel?

Or, please, give me some links for jme2 specific examples/tutorials.

the best examples are the examples in the jmetest package

Thanks, examples are great.



Now I have application with many glCanvases.



Next thing is to implement simple mouse pick.

I studied HelloMousePick, but I still have some troubles.



My code for the picking is:




@Override
public void simpleUpdate()
{
if (MouseInput.get().isButtonDown(0)) {
Vector2f screenPos = new Vector2f();
// Get the position that the mouse is pointing to

screenPos.set(MouseInput.get().getXAbsolute(), MouseInput.get()
.getYAbsolute());

Ray mouseRay = DisplaySystem.getDisplaySystem().getPickRay(screenPos, false, null);
pickResults.clear();
rootNode.findPick(mouseRay, pickResults);

System.out.println(pickResults.getNumber());
}
}



when I click on the canvas aplication throws many exceptions:


Exception in thread "AWT-EventQueue-0" java.lang.ArithmeticException: This matrix cannot be inverted
at com.jme.math.Matrix4f.invert(Matrix4f.java:1041)
at com.jme.renderer.AbstractCamera.getWorldCoordinates(AbstractCamera.java:1017)
at com.jme.system.DisplaySystem.getWorldCoordinates(DisplaySystem.java:828)
at com.jme.system.DisplaySystem.getPickRay(DisplaySystem.java:855)
at base.MonkeyPanel$MyImplementor.simpleUpdate(MonkeyPanel.java:219)
at com.jme.system.canvas.SimpleCanvasImpl.doUpdate(SimpleCanvasImpl.java:133)
at com.jmex.awt.lwjgl.LWJGLCanvas.paintGL(LWJGLCanvas.java:138)
at org.lwjgl.opengl.AWTGLCanvas.paint(AWTGLCanvas.java:290)
at org.lwjgl.opengl.AWTGLCanvas.update(AWTGLCanvas.java:321)
at sun.awt.RepaintArea.updateComponent(RepaintArea.java:239)
at sun.awt.RepaintArea.paint(RepaintArea.java:216)
at sun.awt.windows.WComponentPeer.handleEvent(WComponentPeer.java:306)
at java.awt.Component.dispatchEventImpl(Component.java:4577)
at java.awt.Component.dispatchEvent(Component.java:4331)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:599)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)


Exception occurred in the line:
Ray mouseRay = DisplaySystem.getDisplaySystem().getPickRay(screenPos, false, null);

Is this becose I use the parallelProjection?
Is there any article for jme2 mousePicking in the parallelProjection?

Thanks, and sorry for (eventually) stupid question :)

I replace:

Ray mouseRay = DisplaySystem.getDisplaySystem().getPickRay(screenPos, false, null);



with:

int x = MouseInput.get().getXAbsolute();

int y = MouseInput.get().getYAbsolute();

Ray ray = new Ray(new Vector3f(x,y, -1), new Vector3f(x,y,0));



And i think that it works OK (it returns 1 for pickResults.getNumber() when I click on the test Box).



Is this right approach? Do you have some advice form me?

I found that this post http://www.jmonkeyengine.com/jmeforum/index.php?topic=8041.0 can be very usefull later…

Well, I guess it all depends on how often you call that routine…



In your approach you are creating 2 ints, 2 Vector3fs, and 1 Ray every time you pick, with first method you can send in the Ray as the final argument (which if you keep it as global variable, nothing is created).  If you are calling the second approach very frequently the you are gonna end up with a lot of orphaned references, which in turn will cause the GC to run more often (which can kill your perfomance).



Here's how I like to do Canvas picking:


private final Point screenPos = new Point();    // Global Variable
private final Ray pickRay = new Ray();           // Global Variable

public void pick() {
    screenPos.set( MouseInfo.getPointerInfo().getLocation() );
    DisplaySystem.getDisplaySystem().getPickRay(screenPos, true, pickRay);
}


This creates NO variables when it is run :); also notice the 'true' argument when getting the Ray, it means the screen is flipped (openGL starts coordinates at bottom left corner while Java starts at top left).

(I don't know why your original picking post didn't work though, when I created a simple parallel projection pick test it worked fine...)

I still have problems :frowning:



(I don't know why your original picking post didn't work though, when I created a simple parallel projection pick test it worked fine...


Can you, please, post here that simple parallel projection test?

Sure…




import com.jme.app.SimpleGame;
import com.jme.bounding.BoundingBox;
import com.jme.input.FirstPersonHandler;
import com.jme.input.KeyBindingManager;
import com.jme.input.KeyInput;
import com.jme.input.MouseInput;
import com.jme.intersection.PickResults;
import com.jme.intersection.TrianglePickResults;
import com.jme.math.FastMath;
import com.jme.math.Quaternion;
import com.jme.math.Ray;
import com.jme.math.Triangle;
import com.jme.math.Vector2f;
import com.jme.math.Vector3f;
import com.jme.scene.Node;
import com.jme.scene.Spatial;
import com.jme.scene.TriMesh;
import com.jme.scene.shape.Sphere;
import com.jme.system.DisplaySystem;

import java.awt.Point;
import java.util.ArrayList;



public class PickTest extends SimpleGame {

    final Node pickNode = new Node();
    final float angle = 5 * FastMath.DEG_TO_RAD;
    final Quaternion upRotation = new Quaternion().fromAngleAxis( angle, Vector3f.UNIT_X );
    final Quaternion downRotation = new Quaternion().fromAngleAxis( -angle, Vector3f.UNIT_X );
    final Quaternion leftRotation = new Quaternion().fromAngleAxis( angle, Vector3f.UNIT_Y );
    final Quaternion rightRotation = new Quaternion().fromAngleAxis( -angle, Vector3f.UNIT_Y );
    final MousePicker picker = new MousePicker();
    //
    boolean isTriangleAccurate = true;

    public static void main( String[] args ) {
        new PickTest();
    }

    public PickTest() {
        this.setConfigShowMode( ConfigShowMode.AlwaysShow );
        this.start();
    }

    @Override
    protected void simpleUpdate() {

        boolean updateNode = false;

        if( KeyBindingManager.getKeyBindingManager().isValidCommand( "UP", false ) ){
            pickNode.getLocalRotation().multLocal( upRotation );
            updateNode = true;
        }

        if( KeyBindingManager.getKeyBindingManager().isValidCommand( "DOWN", false ) ){
            pickNode.getLocalRotation().multLocal( downRotation );
            updateNode = true;
        }

        if( KeyBindingManager.getKeyBindingManager().isValidCommand( "LEFT", false ) ){
            pickNode.getLocalRotation().multLocal( leftRotation );
            updateNode = true;
        }

        if( KeyBindingManager.getKeyBindingManager().isValidCommand( "RIGHT", false ) ){
            pickNode.getLocalRotation().multLocal( rightRotation );
            updateNode = true;
        }

        if( KeyBindingManager.getKeyBindingManager().isValidCommand( "TOGGLE_CAMERA", false ) ){
            if( cam.isParallelProjection() ){
                cameraPerspective();
            } else{
                cameraParallel();
            }
        }

        if( KeyBindingManager.getKeyBindingManager().isValidCommand( "TOGGLE_TRI_ACCURATE", false ) ){
            isTriangleAccurate = !isTriangleAccurate;
        }



        picker.setPickSpatial( rootNode );
        picker.checkMousePicks( isTriangleAccurate );

        if( picker.wasObjectPicked() ){
            System.out.print( "Picked" + picker.getPickedObjectName() );
            if( isTriangleAccurate ){
                System.out.print( ": " + picker.getWorldPickLocation() );
            }

            System.out.print( "n" );
        } else{
            System.out.println( "" );
        }
    }

    @Override
    protected void simpleInitGame() {

        KeyBindingManager.getKeyBindingManager().set( "TOGGLE_TRI_ACCURATE", KeyInput.KEY_RETURN );
        KeyBindingManager.getKeyBindingManager().set( "TOGGLE_CAMERA", KeyInput.KEY_SPACE );
        KeyBindingManager.getKeyBindingManager().set( "UP", KeyInput.KEY_NUMPAD8 );
        KeyBindingManager.getKeyBindingManager().set( "DOWN", KeyInput.KEY_NUMPAD2 );
        KeyBindingManager.getKeyBindingManager().set( "RIGHT", KeyInput.KEY_NUMPAD6 );
        KeyBindingManager.getKeyBindingManager().set( "LEFT", KeyInput.KEY_NUMPAD4 );

        MouseInput.get().setCursorVisible( true );
        ( (FirstPersonHandler) this.input ).setButtonPressRequired( true );


        cameraParallel();

        showBounds = true;

        pickNode.attachChild( new Sphere( "sphere", 12, 12, 5 ) );
        pickNode.setModelBound( new BoundingBox() );
        pickNode.updateModelBound();

        rootNode.attachChild( pickNode );
    }
}



class MousePicker {

    private String closestPickedName = null;
    private TriMesh savedMesh = null;
    private Spatial pickNode = null;      // Typically this will be the rootNode
    //
    private final PickResults pickResults = new TrianglePickResults();
    private final DisplaySystem display;
    //
    private final Ray ray = new Ray();
    private final Vector2f screenPos = new Vector2f();
    private final Vector2f cursorPosition = new Vector2f();
    private final Vector3f worldPickLocation = new Vector3f();
    private final Vector3f camLocation = new Vector3f();
    private final Vector3f tempWorldPick = new Vector3f();
    private final Vector3f[] tempTriVerticies = new Vector3f[ 3 ];
    private final Triangle pickedTriangle = new Triangle( new Vector3f(), new Vector3f(), new Vector3f() );
    private final Triangle tempTriangle = new Triangle( new Vector3f(), new Vector3f(), new Vector3f() );
    //
    private final ArrayList<String> pickedNames = new ArrayList();
    private final ArrayList<Integer> tempTriIndicies = new ArrayList();
    //
    private boolean objectWasPicked = false,  triangleWasPicked = false,  wasTriangleAccurate = false;
    private boolean useCanvas = false,  useSuppliedRay = false;
    //
    public MousePicker() {
        display = DisplaySystem.getDisplaySystem();
        useCanvas = false;
        pickResults.setCheckDistance( true );
    }

    public MousePicker( boolean useCanvas ) {
        this();

        this.useCanvas = useCanvas;
    }

    public MousePicker( boolean useCanvas, Spatial pickNode ) {
        this( useCanvas );

        setPickSpatial( pickNode );
    }

    public void setPickSpatial( Spatial spatial ) {
        pickNode = spatial;
    }

    public boolean checkMousePicks() {
        this.useSuppliedRay = false;
        return checkMousePicks( false );
    }

    public boolean checkMousePicks( boolean triangleAccurate ) {
        this.useSuppliedRay = false;
        return runPicker( triangleAccurate );
    }

    public boolean checkMousePicks( boolean triangleAccurate, Point cursorPosition ) {

        if( cursorPosition != null ){
            this.cursorPosition.set( cursorPosition.x, cursorPosition.y );
        }

        this.useSuppliedRay = false;
        return runPicker( triangleAccurate );
    }

    public boolean checkMousePicks( boolean triangleAccurate, final Ray suppliedRay ) {
        ray.set( suppliedRay );
        useSuppliedRay = true;

        return runPicker( triangleAccurate );
    }

    private boolean runPicker( boolean triangleAccurate ) {

        if( pickNode == null ){
            return false;
        }

        clearPickResults();
        wasTriangleAccurate = triangleAccurate;
        float shortestDist = Float.MAX_VALUE;

        if( !useSuppliedRay ){
            camLocation.set( DisplaySystem.getDisplaySystem().getRenderer().getCamera().getLocation() );

            if( useCanvas ){
                screenPos.set( cursorPosition );
            } else{
                screenPos.set( MouseInput.get().getXAbsolute(), MouseInput.get().getYAbsolute() );
            }

            try{
                display.getPickRay( screenPos, useCanvas, ray );
            } catch( Exception e ){
                return false;
            }
        } else{
            camLocation.set( ray.getOrigin() );
        }

        pickNode.findPick( ray, pickResults );

        if( pickResults.getNumber() <= 0 ){
            return false;
        }

        objectWasPicked = true;

        for( int i = 0; i < pickResults.getNumber(); i++ ){

            Node parent = pickResults.getPickData( i ).getTargetMesh().getParent();

            // itterate up through nodes while:
            // parent is not null AND parent has name AND parent does not equal pickNode
            while( parent.getParent() != null &&
                    parent.getParent().getName() != null &&
                    !parent.getParent().equals( pickNode ) ){
                parent = parent.getParent();
            }

            pickedNames.add( parent.getName() );

            if( triangleAccurate ){

                final TriMesh mesh = (TriMesh) pickResults.getPickData( i ).getTargetMesh();

                for( Integer triIndex : pickResults.getPickData( i ).getTargetTris() ){

                    // Get a single triangle and set the world transformations
                    mesh.getTriangle( triIndex, tempTriVerticies );
                    setTriVerticies( tempTriangle, tempTriVerticies, mesh );

                    if( ray.intersect( tempTriangle ) ){

                        // get the exact location (in World coordinates) of the pick
                        triangleWasPicked = true;
                        ray.intersectWhere( tempTriangle, tempWorldPick );
                        final float tempDist = tempWorldPick.distance( camLocation );

                        // check to make sure picked triangle is the closest possible triangle
                        if( tempDist < shortestDist ){

                            // Set pickedTriangle and then save target mesh
                            setPickedTriangle( tempTriVerticies );
                            worldPickLocation.set( tempWorldPick );
                            closestPickedName = pickedNames.get( pickedNames.size() - 1 );
                            shortestDist = tempDist;
                            savedMesh = mesh;
                        }
                    }

                }  // End Triangle Accurate


            }

        }


        if( triangleAccurate ){
            return triangleWasPicked;
        }

        return objectWasPicked;
    }

    public boolean wasObjectPicked() {
        if( wasTriangleAccurate ){
            return triangleWasPicked;
        }

        return objectWasPicked;
    }

    public String getPickedObjectName() {

        if( wasTriangleAccurate ){
            if( closestPickedName != null ){
                return closestPickedName;
            } else{
                return "";
            }
        }

        try{
            return pickedNames.get( 0 );
        } catch( Exception e ){
            return "";
        }
    }

    public ArrayList<String> getAllPickedObjects() {
        return pickedNames;
    }

    public Triangle getPickedTriangle() {

        if( triangleWasPicked ){
            return pickedTriangle;
        }
        return null;
    }

    public TriMesh getSavedMesh() {
        return savedMesh;
    }

    public Vector3f getWorldPickLocation() {
        return worldPickLocation;
    }

    public void clearPickResults() {
        triangleWasPicked = false;
        objectWasPicked = false;
        savedMesh = null;
        closestPickedName = null;

        pickResults.clear();
        tempTriIndicies.clear();
        pickedNames.clear();
        worldPickLocation.set( 0, 0, 0 );
        pickedTriangle.get( 0 ).set( 0, 0, 0 );
        pickedTriangle.get( 1 ).set( 0, 0, 0 );
        pickedTriangle.get( 2 ).set( 0, 0, 0 );
    }

    // Save the triangle in terms of its original mesh (no transformations)
    private void setPickedTriangle( Vector3f[] verts ) {

        for( int i = 0; i < verts.length; ++i ){
            pickedTriangle.get( i ).setX( verts[i].getX() );
            pickedTriangle.get( i ).setY( verts[i].getY() );
            pickedTriangle.get( i ).setZ( verts[i].getZ() );
        }

        pickedTriangle.calculateCenter();

    }

    private void setTriVerticies( Triangle triangle, Vector3f[] verts, TriMesh mesh ) {
        for( int i = 0; i < verts.length; ++i ){
            mesh.localToWorld( verts[i], triangle.get( i ) );
        }

        triangle.calculateCenter();

    }
}



(I have included the 'picker' class that I have developed, feel free to use it; you might have to change how the names are 'accumulated' though...)

Thanks - great example!  :wink:



I tried to implement it to the my application but (again) something is wrong.



Example:

I have a sphere attached to the root node.

When I move the sphere along x - axis picking is still correct, but when I move it in y direction it isn't.

It looks like when the sphere is moved in the x direction, the bounding box is also moved (which is ok), but when the sphere is moved in the positive y direction that the boundingBox is moved in negative y direction (sorry for my bad english :slight_smile: , I hope that you can understand me)

You can render the bounding volumes to confirm your diagnosis.



In your render() method you can use Debugger.drawBounds(rootNode, renderer);.


Debugger.drawBounds(rootNode, renderer);

That has no effect. Maybe, it is becose my application is no extended from SimpleGame (I am using glCanvas and MyImplementor).
zchira said:

When I move the sphere along x - axis picking is still correct, but when I move it in y direction it isn't.
It looks like when the sphere is moved in the x direction, the bounding box is also moved (which is ok), but when the sphere is moved in the positive y direction that the boundingBox is moved in negative y direction (sorry for my bad english :) , I hope that you can understand me)

Reading this again, it dawned on me that I think I know what your problem is. It could be attributed to the difference in screen coordinate systems between swing and jME. The mouse coordinates returned by swing differ from those used in jME only in that the origin for swing is the upper left corner and in jME it is the lower left corner.

This means that you will need to adjust the mouse position given by the canvas before using it in jME. It is a simple calculation of y = display.getHeight() - y to get it into jME terms.
zchira said:

That has no effect. Maybe, it is becose my application is no extended from SimpleGame (I am using glCanvas and MyImplementor).

It should still work. I guess it might depend on its order in the render method, usually you would want to render this last, over everything else. You may not need it anymore, but did you try putting it at the end of your render method?

OK, I found the solution.

The problem was in the wrong using of the MousePicker class.

Now picking works, but…

when I change frustum parameters on my camera, picking starts to work wrong.



Here is camera code when picking is working:


cam.setParallelProjection(true);
float aspect = (float)canvas.getWidth() / canvas.getHeight();
cam.setFrustum(-100, 1000, -50 * aspect, 50 * aspect, -50, 50);
cam.update();



and when I change it to:


float aspect = (float)canvas.getWidth() / canvas.getHeight();

int w = getWidth() / 2;
int h = getHeight() / 2;
cam.setParallelProjection(true);
cam.setFrustum(-200, 3000, -w * aspect, w * aspect , -h, h);
cam.update();


It doesn't work anymore.