Issues With Mouse Input on AWTPanel

I am running my JME application through an AWTPanel and I am attempting to create/edit objects using the mouse, but I was getting some weird behavior with mouse Y being reversed in some instances. After doing some investigating I noticed that only when using AWTPanels Mouse Y is reversed( compaired to normal JME mouse coordinates) on mouse click. For example if you print out the mouse x,y from the InputManager on both MouseButtonEvent and MouseMotionEvent, then in a basic SimpleApplication mouse 0,0 will be the lower left of the screen in both cases. But, on an AWTPanel, mouse motion will return the lower left as 0,0 but mouse button returns the upper left as 0,0. this can easily be reproduced by simply adding a printout in the methods onMouseMotionEvent and onMouseButtonEvent in the InputManager class, and running TestAwtPanels and TestMousePick to see the difference. Is there something I am doing wrong or is this a bug/feature.

1 Like

Uhm, what kind of component do you assign for the input? I don’t have this issue afaict. Do you use the latest version of jME using the SDK or nightly or some dubious other source? :wink:

I have been running off of a SVN checkout that was a few weeks old, and I also just now ran it off of the nightly build, both with the same results. Here is a simple test program that will show what I am seeing, hopefully you will see the same. If you run the program and click in the top left corner, on my machine this will print out ~0,0, if you click and start dragging in the top left corner it will print out ~0,300. Both printouts are printing the inputManager.getCursorPosition() the only difference is one is after an action event and the other is after an analog event.



[java]import java.awt.BorderLayout;

import java.awt.Dimension;

import java.awt.Toolkit;



import javax.swing.JFrame;



import com.jme3.app.SimpleApplication;

import com.jme3.input.MouseInput;

import com.jme3.input.controls.ActionListener;

import com.jme3.input.controls.AnalogListener;

import com.jme3.input.controls.MouseButtonTrigger;

import com.jme3.material.Material;

import com.jme3.math.Vector3f;

import com.jme3.scene.Geometry;

import com.jme3.scene.shape.Box;

import com.jme3.system.AppSettings;

import com.jme3.system.awt.AwtPanel;

import com.jme3.system.awt.AwtPanelsContext;

import com.jme3.system.awt.PaintMode;



public class AWTMouseInputTest extends SimpleApplication implements AnalogListener, ActionListener

{

private static AwtPanel panel;



@Override

public void simpleInitApp()

{

flyCam.setEnabled(false);

Box b = new Box(Vector3f.ZERO, 1, 1, 1);

Geometry geom = new Geometry("Box", b);

Material mat = new Material(assetManager, "Common/MatDefs/Misc/Unshaded.j3md");

mat.setTexture("ColorMap", assetManager.loadTexture("Interface/Logo/Monkey.jpg"));

geom.setMaterial(mat);

rootNode.attachChild(geom);

AwtPanelsContext context = (AwtPanelsContext) getContext();

panel = context.createPanel(PaintMode.Accelerated);

panel.setPreferredSize(new Dimension(400, 300));

panel.attachTo(false, viewPort);

context.setInputSource(panel);

createWindowForPanel(panel, 300);

setUpInputTest();

}



private void setUpInputTest()

{

inputManager.addMapping("MouseButton", new MouseButtonTrigger(MouseInput.BUTTON_LEFT));

inputManager.addListener(this, "MouseButton");

}



@Override

public void onAction(String name, boolean isPressed, float tpf)

{

System.out.println("onAction Mouse Location " + inputManager.getCursorPosition());

}



@Override

public void onAnalog(String name, float value, float tpf)

{

System.out.println("onAnalog Mouse Location " + inputManager.getCursorPosition());

}



private static void createWindowForPanel(AwtPanel panel, int location)

{

JFrame frame = new JFrame("Render Display " + location);

frame.getContentPane().setLayout(new BorderLayout());

frame.getContentPane().add(panel, BorderLayout.CENTER);

frame.setDefaultCloseOperation(JFrame.DISPOSE_ON_CLOSE);

frame.pack();

frame.setLocation(location, Toolkit.getDefaultToolkit().getScreenSize().height - 400);

frame.setVisible(true);

}



public static void main(String[] args)

{

AWTMouseInputTest app = new AWTMouseInputTest();

app.setShowSettings(false);

AppSettings settings = new AppSettings(true);

settings.setCustomRenderer(AwtPanelsContext.class);

settings.setFrameRate(60);

app.setSettings(settings);

app.start();

}



}[/java]

2 Likes

I believed i traced this issue down to AwtMouseInput.java line:168 in the update method.



[java]// invert DY

int actualX = lastKnownLocation.x;

int actualY = component.getHeight() - lastKnownLocation.y;[/java]



this inversion only appears to take place in AwtMouseInput and not in LwjglMouseInput, and only effects when the mouse has moved. If just a mouse button is pressed, the the raw mouse x and y are pushed into the MouseButtonEvent, but if the mouse moves, then the raw x and inverted y gets pushed into the MouseMotionEvent. is there a reason for this or is this a bug.

1 Like
@jcarff said:
I believed i traced this issue down to AwtMouseInput.java line:168 in the update method.

[java]// invert DY
int actualX = lastKnownLocation.x;
int actualY = component.getHeight() - lastKnownLocation.y;[/java]

this inversion only appears to take place in AwtMouseInput and not in LwjglMouseInput, and only effects when the mouse has moved. If just a mouse button is pressed, the the raw mouse x and y are pushed into the MouseButtonEvent, but if the mouse moves, then the raw x and inverted y gets pushed into the MouseMotionEvent. is there a reason for this or is this a bug.

Cool, thanks. Yeah, this should not be, as you say.

After looking into this some more, it appears that this inversion on mouse movement is correct as JME expects mouse 0,0 to be lower left, and awt is usually upper left. So inverting the y location from an awt mouse movements puts it in the correct frame. it appears that the issue lies in that mouse button events do not invert the y location. after inverting the mouse y location for mouse press and release in my own branch, everything seems to line up correctly and work as expected.

@jcarff said:
After looking into this some more, it appears that this inversion on mouse movement is correct as JME expects mouse 0,0 to be lower left, and awt is usually upper left. So inverting the y location from an awt mouse movements puts it in the correct frame. it appears that the issue lies in that mouse button events do not invert the y location. after inverting the mouse y location for mouse press and release in my own branch, everything seems to line up correctly and work as expected.

AWT puts it "upside down" in jme terms, so it should be reversed globally in all classes when awt is used, ideally at the source. I though thats what you meant.

What needs to be done to fix this issue in an upcoming JME3 build, currently in my own JME3 branch I simply needed to invert the mouse y on the mouse pressed and mouse released methods of AwtMouseIinput and all of my issues were solved. Is there more to it that this.

@jcarff: Can you please post a diff patch against AwtMouseInput? Thanks.

@Momoko_Fan here’s the diff showing the only changes we’ve made in regards to the mouse. I think @jcarff’s concern is that we’re not sure if this is the only thing that needs to be done in order for jME to play well with the AWT/Swing coordinate system in the general case, or if this is just good enough for our use case.



[java]

Index: AwtMouseInput.java

===================================================================

— AwtMouseInput.java (revision 10022)

+++ AwtMouseInput.java (working copy)

@@ -220,7 +220,8 @@

}



public void mousePressed(MouseEvent awtEvt) {

  •    MouseButtonEvent evt = new MouseButtonEvent(getJMEButtonIndex(awtEvt), true, awtEvt.getX(), awtEvt.getY());<br />
    
  •   int actualyY = component.getHeight() - awtEvt.getY();<br />
    
  •    MouseButtonEvent evt = new MouseButtonEvent(getJMEButtonIndex(awtEvt), true, awtEvt.getX(), actualyY);<br />
    

evt.setTime(awtEvt.getWhen());

synchronized (eventQueue) {

eventQueue.add(evt);

@@ -228,7 +229,8 @@

}



public void mouseReleased(MouseEvent awtEvt) {

  •    MouseButtonEvent evt = new MouseButtonEvent(getJMEButtonIndex(awtEvt), false, awtEvt.getX(), awtEvt.getY());<br />
    
  •   int actualyY = component.getHeight() - awtEvt.getY();<br />
    
  •    MouseButtonEvent evt = new MouseButtonEvent(getJMEButtonIndex(awtEvt), false, awtEvt.getX(), actualyY);<br />
    

evt.setTime(awtEvt.getWhen());

synchronized (eventQueue) {

eventQueue.add(evt);

[/java]



You can download the diff here: http://d.pr/f/igXV

One and half year later I bumped into exactly the same issue.

Is the fix for this ever going to reach main?