[Solved] Mouse input delay and zero DX/DY

Troubleshooting General Help
1

I have two separate problems here, but they’re closely related and the setup for both is the same so I thought I’d keep them together in a single post.

I’m building a windowing layer on top of Lemur. Windows can be stacked, dragged, minimized, etc. Right now I’m working on dragging and I’ve hit a couple snags. Windows are Lemur Container instances with a title label, minimize/close buttons, etc. and are made draggable by adding a MouseListener to the title label. I start a drag movement in mouseButtonEvent() when the left button is clicked, and I detect the mouse motion and re-position the window in mouseMoved().

My two problems are:

  1. When I try to use MouseMotionEvent.getDX() (or getDY()) in mouseMoved() the return value is always zero. I have to calculcate the window movement by tracking the cursor movement deltas manually. It’s not a big deal to work around, but it would be cleaner to use getDX()/getDY() and I’m also surprised that those are always zero. (I’ve confirmed that they are actually zero and that I’m not just observing a side effect of a bug in my code.)

  2. There’s a notable delay between the cursor moving and the window following it, creating a notable “rubber band effect.” I’d guess that this is a 1 frame difference, but I haven’t confirmed that so I’m not certain. When I move the cursor quickly, the difference between the cursor position and the window position can be as much as an inch in screen space. Small movements also demonstrate the same effect, albeit at a smaller magnitude. Enabling/disabling VSync makes no difference (although FPS is staying at 60 regardless of the VSync setting - it seems that the LWJGL 2 renderer is limiting the framerate even when VSync is disabled - I temporarily switched to the LWJGL 3 renderer, and even with a higher framerate the delay is still visible). I tried out a couple of other games/OpenGL applications with built-in UIs, and interestingly enough they exhibited the same effect - although not always to as high a degree. I thought that input would be processed before the update logic runs so that input and visuals would stay in sync, but that doesn’t seem to be the case here. If the framerate was lower than the monitor’s refresh rate I’d expect this since the cursor position would be updated faster than the application, but this is a 60 Hz monitor so they should be in sync unless they’re some double/triple buffering going on that’s inducing an apparent delay.

Any help/comments for either problem (1) or (2) would be greatly appreciated. (I don’t believe that this is a Lemur-specific issue so I’m posting here instead of in the Lemur category.)

2

As to (1), yes… they will always be 0 because Lemur isn’t using the mouse events from JME.

You shouldn’t rely on them anyway as they will possibly be totally bogus on touch screens.

As to (2), do the Lemur demos also exhibit this behavior?

I’ve personally never seen it. Mouse events should be processed before everything else and movement should be in sync. I rely on this a lot all the time, actually. So it may be a delay caused be something in your own tracking.

(Edit: even the default drag handler Lemur provides can be checked to see if it exhibits this behavior… it’s easy to add.)

3

Noted - I’ll refrain from using getDX()/getDY().

I tried running it on my main workstation with a few different configurations:

  1. No framerate limiter and vsync disabled (FPS > 1000): some minor rubber banding, but within a several pixel radius of the original click position.
  2. No framerate limiter, vsync enabled (FPS = 60): major rubber banding, easily exceeding an inch on screen.
  3. Framerate limit set to 60, vsync enabled: notable rubber banding, but not quite as bad as with 2.
  4. Framerate limit set to 60, vsync disabled: visible rubber banding, but not as bad as with 2 or 3.

I’ll check the Lemur demos next, and I still haven’t found my way around the Lemur DnD support so I haven’t tested that yet either. Here’s my drag handler, however (setActiveWindow() handles window stacking and viewport depth):

window.getTitle().addMouseListener(new MouseListener(){
			
   boolean dragging = false;
   int startX = 0;
   int startY = 0;

   @Override
   public void mouseButtonEvent(MouseButtonEvent event, Spatial target, Spatial capture) {
   if(event.getButtonIndex() == MouseInput.BUTTON_LEFT){
      dragging = event.isPressed();
      startX = event.getX();
      startY = event.getY();
      event.setConsumed();
					
      setActiveWindow(window);
    }
}

// empty moved/exited method bodies here

@Override
public void mouseMoved(MouseMotionEvent event, Spatial target, Spatial capture) {
   if(dragging){
       window.move(event.getX() - startX, event.getY() - startY, 0);
       startX = event.getX();
       startY = event.getY();
       event.setConsumed();
   }
}
});
4

I mean, if you want you can just attach Lemur’s standard drag handler (which has nothing to with DnD) and see if it has the same issue:

It handles dragging in 2D or 3D and I’ve never seen it lag before.

If nothing else you can compare to see if your code is different in some significant way.

5

Derp… that’ll teach me not to scan the Javadoc so fast…

DragHandler has identical behavior to mine. The only thing that improves the situation is if the framerate is significantly faster than 60 FPS (200+ FPS), and even then the difference is still noticeable.

Just to be sure we’re on the same page here, suppose I click with the cursor tip at (10, 15) pixels relative to the upper-left corner of the window and then start to drag. With every frame I’d expect the window to move such that the cursor always appears at (10, 15) (assuming the framerate is exactly 60 FPS). What actually happens is that the window follows behind the cursor and only after the cursor stops moving (and the window catches up) does the cursor appear to be at (10, 15) again. It follows quickly, but it’s still a noticeable difference and it does it both on nVidia and Intel hardware (nVidia proprietary drivers and Intel Mesa drivers) on Linux. VSync significantly worsens the effect. Am I just mistaken that the window should be able to move such that the cursor always appears fixed at (10, 15) relative to the window, provided the FPS remains sufficiently high?

6

Frame rate shouldn’t matter. Unless you have something else weird going on, mouse events are handled before rendering… so it will always be in sync.

I’ve never seen this issue, personally.

7

It occurs to me that maybe what you are seeing is a lag between where JME thinks the mouse is and where your OS is rendering the mouse.

8

Well this is really odd. I guess the ultimate test would be to go directly to simpleUpdate and get mouse coords from the input manager, then hardcode to set the window to them.

If that lags, it’s not anything anyone can solve.

9

The app states are actually updated BEFORE simpleUpdate()…

It goes:
-queued tasks
-timer update
-input manager
-audio manager
-app states update()
-simpleUpdate()
-default update logical state
-default update geometric state
-app states render() (which to me should have been called prerender()… but whatever)
-actual render
-simpleRender()
-app states postRender()

Lemur calls mouse listeners as part of app state update.

Which presuming you initialize it (GuiGlobals) early in simpleInit() and presuming you aren’t running nifty or something else weird that will grab things… then mouse listeners are already running before simpleUpdate().

10

My apologies for the delay responding - I’ve had a busier schedule than usual the last couple of days and wanted to thoroughly investigate this issue before posting back.

That’s what I was thinking - but in the case that the window is refreshing at the same rate as the monitor you know that if input is properly handled (and nothing else is adding latency) then there will be no apparent lag.

Just tried something very similar - in simpleInitApp() I added a RawInputListener and moved the window via that. It behaved identically.

At this point I think you’re right - I first observed this in the Cinnamon desktop on Linux Mint. Switching to a KDE session helped some, especially when I disabled the KDE compositor’s VSync - but I haven’t found a single combination that totally eliminates it. It’s the same situation on my desktop with a GTX 550 Ti and an up-to-date nVidia driver. Other OpenGL applications that provide their own UI display the exact same effect too - so I’m thinking this has got to be an anomaly caused by how X11 is synchronizing input events with windows. I don’t have the link handy, but I did see a post where someone was discussing apparent “input lag” on Linux that didn’t happen on Windows with the same application.

At this point I’m going to assume that you’re right and that there’s nothing I can do about it. Thanks both of you for your help - I appreciate it.

11

Well, there is “something” you can do about it but it’s also distasteful.

Hide the OS cursor and render your own above everything else.

12

Oh… I don’t think I ever would have thought of that. That may very well be enough to mask the apparent delay from the user. It’s not that big a deal, either - I had already planned to use fully customized cursors. I’ll just need to render them on a quad that tracks the mouse position instead of using the regular means of switching cursors. Thanks!