AWT Canvas Resizing and Picking Problems (parallel projection)

Hey all,

I was wondering if somebody could help me out with some problems I'm having with awt canvas resizing and picking problems.

First, the code:

package test.pickprobs;

import java.awt.Canvas;
import java.awt.Container;
import java.awt.event.ComponentAdapter;
import java.awt.event.ComponentEvent;
import java.awt.event.MouseAdapter;
import java.awt.event.MouseEvent;
import java.util.concurrent.Callable;

import javax.swing.JFrame;

import com.jme.bounding.BoundingBox;
import com.jme.bounding.BoundingVolume;
import com.jme.image.Texture;
import com.jme.input.KeyInput;
import com.jme.intersection.BoundingPickResults;
import com.jme.intersection.PickResults;
import com.jme.math.FastMath;
import com.jme.math.Ray;
import com.jme.math.Vector2f;
import com.jme.math.Vector3f;
import com.jme.renderer.ColorRGBA;
import com.jme.scene.Geometry;
import com.jme.scene.Node;
import com.jme.scene.Text;
import com.jme.scene.shape.Box;
import com.jme.scene.shape.Quad;
import com.jme.scene.state.AlphaState;
import com.jme.scene.state.LightState;
import com.jme.scene.state.TextureState;
import com.jme.system.DisplaySystem;
import com.jme.util.GameTaskQueue;
import com.jme.util.GameTaskQueueManager;
import com.jme.util.TextureManager;
import com.jmex.awt.JMECanvas;
import com.jmex.awt.SimpleCanvasImpl;
import com.jmex.awt.input.AWTMouseInput;

public class TestPickProbs {
   public static void main(String[] args) {
      JFrame frame = new JFrame();
      frame.setSize(1024, 768);
      final TestPickProbs con = new TestPickProbs(frame.getContentPane());
      try {
      } catch (Exception e) {
   private Container parent;
   private Canvas canvas;
   private PickProbsImplementor implementor;
   private JMECanvas jmeCanvas;
   public TestPickProbs(Container par) {
      this.parent = par;
      int width = 1280;
      int height = 1024;
      canvas = DisplaySystem.getDisplaySystem("lwjgl").createCanvas(width, height);
      implementor = new PickProbsImplementor(width, height);
      AWTMouseInput.setup(canvas, false);
      jmeCanvas = (JMECanvas) canvas;
      canvas.setBounds(0,0, par.getWidth(),par.getHeight());
      new Thread() {

         public void run() {
            while (true) {
      Callable<?> exe = new Callable() {
         public Object call() {
            canvas.setSize(canvas.getWidth(), canvas.getHeight() + 1);
            canvas.setSize(canvas.getWidth(), canvas.getHeight() - 1);
            return null;

   private void addListeners() {
      parent.addComponentListener(new ComponentAdapter() {
         public void componentResized(ComponentEvent e) {

      canvas.addMouseMotionListener(new MouseAdapter() {
         public void mouseMoved(MouseEvent e) {
            implementor.hover(e.getX(), canvas.getHeight()-e.getY());
   public void createQuad() {
      Quad q1 = new Quad("quad", 70f, 70f);
      q1.setModelBound(new BoundingBox());

   public void updateCanvasSize() {
      canvas.setSize(parent.getWidth(), parent.getHeight());
      implementor.resizeCanvas(parent.getWidth(), parent.getHeight());
   class PickProbsImplementor extends SimpleCanvasImpl {
      public PickProbsImplementor(int width, int height) {
         super(width, height);
      Vector3f focus = new Vector3f();
      private Text pickText;
      private Text posText;
      public void simpleSetup() {
         cam.setLocation(new Vector3f(0f, 0f, 30f));
         cam.lookAt(new Vector3f(), Vector3f.UNIT_Y);
         Box b = new Box("box", new Vector3f(-1.3f, -1.3f, -1.3f), new Vector3f(1.3f, 1.3f, 1.3f));
         rootNode.setModelBound(new BoundingBox());
         Callable<?> exe = new Callable() {
            public Object call() {
               resizeCanvas(width, height);
               return null;
      private void setupText() {
         AlphaState as = renderer.createAlphaState();

           TextureState ts = renderer.createTextureState();
           posText = new Text("text", "Mouse Position: ");
         posText.setLocalTranslation(new Vector3f(1,60,0));
           pickText = new Text("text", "Pick: ");
         pickText.setLocalTranslation(new Vector3f(1,42,0));


           rootNode.updateGeometricState(0.0f, true);
      public void resizeCanvas(int width, int height) {
         super.resizeCanvas(width, height);
         this.width = width;
         this.height = height;

      public void centerCamera(Node node) {
         node.setModelBound(new BoundingBox());
         focus = node.getWorldBound().getCenter();//.add(node.getWorldTranslation());
         float diameter = 50f;
         if(node.getWorldBound().getType() == BoundingVolume.BOUNDING_BOX) {
            BoundingBox bb = (BoundingBox) node.getWorldBound();
            diameter = Math.abs(bb.xExtent*2) + 20f;
         System.out.println("diamter: " + diameter);
         Vector3f offset = new Vector3f(0,0,diameter);
         Vector3f campos = focus.add(offset);
         cam.lookAt(focus, Vector3f.UNIT_Y);
         node.attachChild(new Box("test", focus, 1f, 1f, 1f));
      public void updateCamera() {
         if(cam == null) {
         float dist = cam.getLocation().distance(focus);
         float mod = dist;
         float aspect = (float) width/height;

         cam.setFrustum(0f, 1000f, -mod * aspect, mod * aspect, -mod, mod);

      public PickResults hover(int x, int y) {
         float x_ = (float) x * 1.0f;
         float y_ = (float) y * 1.0f;
         Vector3f near = cam.getWorldCoordinates(new Vector2f(x_,y_), 0);
         Vector3f far = cam.getWorldCoordinates(new Vector2f(x_,y_), 1);
         posText.print("Screen["+x+","+y+"] ---> World[" + near.x + ", " + near.y + ", " + near.z+ "]");
         Ray ray = new Ray(near, far);
         PickResults results = new BoundingPickResults();
         rootNode.findPick(ray, results);
         String pstr = "Pick: ";
         for(int i=0; i < results.getNumber(); i++) {
            if(i != 0)
               pstr += ", ";
            pstr += results.getPickData(i).getTargetMesh().getParentGeom().getName();
         return results;
      private Geometry createGrid() {
         int GRID_LINES = 10;
         float GRID_SPACING = 10;
           Vector3f[] vertices = new Vector3f[GRID_LINES * 2 * 2];
           float edge = GRID_LINES / 2 * GRID_SPACING;
           for (int ii = 0, idx = 0; ii < GRID_LINES; ii++) {
               float coord = (ii - GRID_LINES / 2) * GRID_SPACING;
               vertices[idx++] = new Vector3f(-edge, 0f, coord);
               vertices[idx++] = new Vector3f(+edge, 0f, coord);
               vertices[idx++] = new Vector3f(coord, 0f, -edge);
               vertices[idx++] = new Vector3f(coord, 0f, +edge);
           Geometry grid = new com.jme.scene.Line("grid", vertices, null,
                   null, null);
           grid.setLocalRotation(grid.getLocalRotation().fromAngleAxis(90 * FastMath.DEG_TO_RAD, Vector3f.UNIT_X));
           return grid;
      public void doRender() {

I've put everything in a single class, so you should be able to just throw it in your favorite IDE and run.

And some screen shots (white 'x' is mouse position):

Screenshot 1: Mouse over grid and quad, but only grid is picked.

Screenshot 2: Mouse moved slightly left from screenshot 1, now both grid and quad are picked.

So, we've got a pretty basic scene.  The root node has 3 children: a small box in the center, a quad, and the line grid from Ren's particle editor.

1. On the initial load I've noticed that the world coordinates range from -1 to 1, for x and y.  After resizing the window they range (correctly?) from frustum left to frustum right along x, and frustum top to frustum bottom along  y.  Anybody know whats going on here?

2. My major problem is that it seems that picking is off by a bit.  As you can see in screenshot 1, the mouse is over the quad, however only the grid has been picked.  Screenshot 2 shows that if you move the mouse further towards the center the quad also gets picked.

Due to the use of parallel projection I'm doing some pretty funky stuff with the frustums (see updateCamera()).  This allows me to "zoom" when in parallel projection by changing the frustums to reflect the camera's distance to a focus point.

Also, I had to change FastMath.FLT_EPSILON from 1.1920928955078125E-7f to 1.1920928955078125E-8f to get around the "This matrix cannot be inverted." exception I was getting from Camera.getWorldCoordinates while in parallel projection mode.

I've spent about a week now revisiting these problems over and over and can't seem to get anywhere, so any help would be much appreciated.  And if I've left anything out or more info is needed please let me know.


if you add an cam.update() inside your hover() Method before reading cam.getWorldCoordinates() from the Camera, it worked pretty good for me.

Without cam.update() i was only getting WorldCoordinates between -1 and +1.

The Quad you created ranges from -35 to +35 units (X-Axis), but picking seems to work only till unit 30, the last 5 are ignored.

But if i change the far Frustum from 1000 to 7000 i can pick till 34,3, with a Far Frustum of 9000 i get the "This matrix cannot be inverted" Error.

I guess something with the camera setup is wrong, that you need to change FastMath.FLT_EPSILON is a bit strange too no?

Yeah, changing FLT_EPSILON seems pretty hackish. 

I'm still learning the codebase and it was the only way I could figure out how to get past the matrix inversion exception.  I feel like I'm missing some fundamental know-how about the cam/renderer/canvas and therefore missing the big underlying problem with my code.