[Lemur] Weird Picker AssertionError when minimizing the game window

miimii1205 · June 24, 2018, 3:47pm

I’ve been trying to record the game using OBS.
Because I’m on windows 10, the OS comes with virtual desktops.

So I had OBS on one desktop and the game running on an other.

When I was ready to record, I “paused” the game (i.e. disabling everything but Lemurs’ app states) and switched to the other desktop.

I then get this AssertionError :

java.lang.AssertionError
	at com.jme3.math.Ray.setDirection(Ray.java:479)
	at com.jme3.math.Ray.<init>(Ray.java:84)
	at com.simsilica.lemur.event.PickEventSession.getPickRay(PickEventSession.java:370)
	at com.simsilica.lemur.event.PickEventSession.cursorMoved(PickEventSession.java:457)
	at com.simsilica.lemur.event.MouseAppState.dispatchMotion(MouseAppState.java:93)
	at com.simsilica.lemur.event.BasePickState.update(BasePickState.java:169)
	at com.jme3.app.state.AppStateManager.update(AppStateManager.java:287)
	at com.jme3.app.SimpleApplication.update(SimpleApplication.java:236)
	at com.jme3.system.lwjgl.LwjglWindow.runLoop(LwjglWindow.java:367)
	at com.jme3.system.lwjgl.LwjglWindow.run(LwjglWindow.java:450)
	at com.jme3.system.lwjgl.LwjglWindow.create(LwjglWindow.java:295)
	at com.jme3.app.LegacyApplication.start(LegacyApplication.java:463)
	at com.jme3.app.LegacyApplication.start(LegacyApplication.java:424)
	at com.jme3.app.SimpleApplication.start(SimpleApplication.java:125)
	at com.chevreuilgames.retroflashyrpg.Game_Main.main(Game_Main.java:115)

It also happens not just when we switch virtual desktop but also when the window is minimized…

After investigating a bit, I realized that this assertion only happens when the GUI is listening to mouse events with the picker. To me. It seems that somehow the picker isn’t deactivated when the window is minimized,

Any help is appreciated.

pspeed · June 24, 2018, 4:05pm

As I recall, JME doesn’t provide listeners for window state. So libraries like Lemur out out of luck if they want to adjust themselves based on this.

Unfortunately, this leaves it up to the application to deal with it as they can override the appropriate methods on Application/SimpleApplication and enable/disable things as the window is minimized/restored.

It’s long been on my todo list to add this capability to JME so that there is a proper window state listener so app contexts and other components can know when the window is focused/minimized, etc…

Benoit_Dubreuil · June 25, 2018, 7:13pm

setLostFocusBehavior(LostFocusBehavior.PauseOnLostFocus);

This works.

We can do that because our game is not multiplayer. It also does not need to run in background.

EvilSpirit · January 28, 2021, 11:11am

Hi @pspeed,
this issue is still relevant, so I have to ask,
do you have some success on window state listener?

pspeed · January 28, 2021, 11:49am

For my own needs, I ended up getting what I wanted by adding a scene processor to the main viewport.

github.com

Simsilica/SiO2/blob/master/src/main/java/com/simsilica/state/DebugHudState.java#L291


            if( ref.update() ) {            
                label.setText(ref.get());
            }
        }
    }

    /**
     *  The only way I've found in JME to listen for reshape events
     *  is to hook in a scene processor.
     */    
    private class ReshapeListener extends DefaultSceneProcessor {
        @Override
        public void reshape( ViewPort vp, int w, int h ) {
            if( log.isTraceEnabled() ) {
                log.trace("reshape(" + vp + ", " + w + ", " + h + ")");
            }
            sizeInvalid = true;
        }            
    }
}

I don’t know what exact issue causes the Ray to get weird in your case but maybe there is some method on SceneProcessor that can be used to detect that and turn off picking when in effect… then turn it back on again when the screen is good.

EvilSpirit · January 28, 2021, 11:54am

Thanks for your reply.
I will try to solve my issue and send a solution here, if find some

pspeed · January 28, 2021, 12:22pm

I’m procrastinating doing something else so I decided to look a little deeper…

In the original post, they were running an older version of JME because the stack trace does not line up with either the latest 3.2 or the latest 3.3. By my guess, may be 3.2.3.

I have no reason to believe that upgrading will fix the issue nor what version you are running… but make sure you run the latest possible and if you can, provide a more up-to-date stack trace.

The Lemur is also quite old from those stack traces… 1.9.1 or maybe earlier.

github.com

jMonkeyEngine-Contributions/Lemur/blob/v1.9.1/src/main/java/com/simsilica/lemur/event/PickEventSession.java#L370


        // Ortho and perspective can be handled the same exact way it turns out.
        // It's only the Gui bucket that is special because it overrides the normal
        // camera and viewport setup.
        if( viewContains(cam, cursor) ) {
            // Turns out these can be calculated the same as perspective... and
            // we should technically clip perspective also.
            Vector3f clickFar  = cam.getWorldCoordinates(cursor, 1);
            Vector3f clickNear = cam.getWorldCoordinates(cursor, 0);
            result = new Ray(clickNear, clickFar.subtractLocal(clickNear).normalizeLocal());
        } else {
            result = null;
        }
    }
    rayCache.put(cam, result);
    return result;
}
public boolean cursorMoved( int x, int y ) {

The getPickRay() method has had some changes since then but I don’t think they affect this issue.

It looks like something happens to the ViewPort’s camera that makes getting the near and far coordinates return weird values or most likely just (0,0,0) making the direction of length 0. (Which may be NaN after normalizeLocal() or 0… either way it’s not a unit vector and the assertion in Ray() will complain).

Given that it’s related to Camera, it seems quite likely that a scene processor could be used to detect this.

If your exception is the same as the original post, then the quick work around is just to turn off assertions if you don’t have a strong need for them while trying to run OBS.

EvilSpirit · January 28, 2021, 12:29pm

My issue similar, but not the same (with probably the same root cause).
Just adding some lemur GUI, start game and trying to minimize game window, then I got such exception and game crashes:

SEVERE: Uncaught exception thrown in Thread[main,5,main]
java.lang.AssertionError
at com.jme3.math.Ray.setDirection(Ray.java:480)
at com.jme3.math.Ray.(Ray.java:85)
at com.simsilica.lemur.event.PickEventSession.getPickRay(PickEventSession.java:452)
at com.simsilica.lemur.event.PickEventSession.cursorMoved(PickEventSession.java:557)
at com.simsilica.lemur.event.MouseAppState.dispatchMotion(MouseAppState.java:94)
at com.simsilica.lemur.event.BasePickState.update(BasePickState.java:239)
at com.jme3.app.state.AppStateManager.update(AppStateManager.java:356)
at com.jme3.app.SimpleApplication.update(SimpleApplication.java:255)
at com.jme3.system.lwjgl.LwjglWindow.runLoop(LwjglWindow.java:530)
at com.jme3.system.lwjgl.LwjglWindow.run(LwjglWindow.java:632)
at com.jme3.system.lwjgl.LwjglWindow.create(LwjglWindow.java:466)
at com.jme3.app.LegacyApplication.start(LegacyApplication.java:463)
at com.jme3.app.LegacyApplication.start(LegacyApplication.java:424)
at com.jme3.app.SimpleApplication.start(SimpleApplication.java:127)

Lemur version: 1.14.0
JME version: 3.3.2-stable

pspeed · January 28, 2021, 12:41pm

Yeah, you should definitely be able to catch this from a scene processor and just disable cursor events. My guess is that maybe camera width/height goes to zero or something.

Edit: note that I will/can fix this particular issue in Lemur but I’m not near the point where I will be cutting a release yet… so it may not help you if you won’t be building lemur from source.

EvilSpirit · January 28, 2021, 12:49pm

That would be great if you can fix this in Lemur.
Thanks a lot!

pspeed · January 28, 2021, 4:30pm

Interestingly, I cannot reproduce this locally… though I know what the fix is anyway.

Are you running lwjgl2 or lwjgl3? (I still run 2 and maybe this is another case where 3 is a little rough around the edges.)

pspeed · January 28, 2021, 4:36pm

Lemur master has the fix:

If you build from source then you can get it now. I don’t know when I will cut the 1.15 release.

EvilSpirit · January 28, 2021, 5:35pm

I use lwjgl3
Thanks a lot, will try the fix today and reply if this fix helps

EvilSpirit · January 28, 2021, 8:07pm

Thanks you @pspeed one more time, it works well now.