Behavior difference in TextArea between lwjgl 2 and 3

[xuan]

Hi I have a problem and I tracked it down to the change of lwjgl version.
This is the code:

public class LemurInputBugDemo extends SimpleApplication {

  public static void main(String[] args) {
    new LemurInputBugDemo().start();
  }

  @Override
  public void simpleInitApp() {
    GuiGlobals.initialize(this);
    BaseStyles.loadGlassStyle();
    GuiGlobals.getInstance().getStyles().setDefaultStyle("glass");

    Material material = new Material(assetManager, "Common/MatDefs/Misc/Unshaded.j3md");
    material.setColor("Color", Yellow);
    Geometry geo = new Geometry("box", new Box(1, 1, 1));
    geo.setMaterial(material);
    rootNode.attachChild(geo);

    Container container = new Container();
    container.setPreferredSize(new Vector3f(200, 50, 0));
    container.setLocalTranslation(cam.getWidth() / 2 - 100, cam.getHeight() / 2, 0);
    container.addChild(new TextField("type here"));
    guiNode.attachChild(container);
  }
}

When I build this code with LWJGL 2, if I type in the TextField using the wasd keys then the flycam state does not receive the input and stays still.
However when I build the exact same code with LWJGL 3 then the flycam moves (the text is also written in the field). Is as if the TextArea doesn’t consume the input and lets it go through.

I believe blocking the input is the expected behavior, or I’m missing something?

[pspeed]

@xuan

I believe blocking the input is the expected behavior, or I’m missing something?

Yes, blocking the input is the expected behavior.

Either something in JME’s lwjgl3 support is not paying attention to the “consumed” flag of events or there is some input listener ordering issue or something. I’m not sure what’s different about lwjgl3 in this case and I don’t run it myself yet.

Hopefully someone else running lwjgl3 with a little extra knowledge may have a hint.

[xuan]

I made a little debugging, I looked at this code in lemur:

    private class KeyHandler implements KeyListener {
        private boolean shift = false;
        private boolean control = false;

        @Override
        public void onKeyEvent( KeyInputEvent evt ) {
            ModifiedKeyInputEvent mEvt = (ModifiedKeyInputEvent)evt;
            if( mEvt.isPressed() || mEvt.isRepeating() ) {
                KeyAction key = mEvt.toKeyAction(); //new KeyAction(code, (control?KeyAction.CONTROL_DOWN:0), (shift?KeyAction.SHIFT_DOWN:0) );
                KeyActionListener handler = actionMap.get(key);
                if( handler != null ) {
                    handler.keyAction(TextEntryComponent.this, key);
                    evt.setConsumed();
                    return;
                }

                // Making sure that no matter what, certain
                // characters never make it directly to the
                // document
                if( evt.getKeyChar() >= 32 ) {
                    model.insert(evt.getKeyChar());
                    evt.setConsumed();
                    //resetText(); ...should be automatic now
                }
            }
        }
    }

With LWJGL 2 it is called only once when pressed (and once when released), with this event:

notice keyChar=a and keyCode=30

But with LWJGL 3 it seems the event is split, because it’s being called twice with different events:

the first one has keyChar=\0 and keyCode=a and the second one keyChar=a and keyCode=0, that seems off.

So the first event is ignored by the handler (if( evt.getKeyChar() >= 32 )) and I assume the flycam is looking at the keyCode so it accepts the input.

So it seems like a jme-jwjgl integration bug.
Could it be platform dependent? I ran Linux and tried with multiple jvms (8, 11 and 17).

[pspeed]

It’s not platform dependent. I do remember this issue with lwjgl3 now. (To me) they really messed up keyboard events.

Probably this key handler needs some extra checking for the redundant events.

Edit: if you build Lemur from source then maybe you can try some things out? If you find something then let me know and/or submit a PR. I’d be super grateful.

[xuan]

Looking where those events are created:

In lwjgl 2, they are created by LwjglKeyInput:

    @Override
    public void update() {
        if (!context.isRenderable())
            return;
        
        Keyboard.poll();
        while (Keyboard.next()){
            int keyCode = Keyboard.getEventKey();
            char keyChar = Keyboard.getEventCharacter();
            boolean pressed = Keyboard.getEventKeyState();
            boolean down = Keyboard.isRepeatEvent();
            long time = Keyboard.getEventNanoseconds();
            KeyInputEvent evt = new KeyInputEvent(keyCode, keyChar, pressed, down);
            evt.setTime(time);
            listener.onKeyEvent(evt);
        }
    }

with lwjgl3 it’s in GlfwKeyInput, creates the events in two callbacks:

    private void initCallbacks() {
        glfwSetKeyCallback(context.getWindowHandle(), keyCallback = new GLFWKeyCallback() {
            @Override
            public void invoke(final long window, final int key, final int scancode, final int action, final int mods) {

                if (key < 0 || key > GLFW_KEY_LAST) {
                    return;
                }

                int jmeKey = GlfwKeyMap.toJmeKeyCode(key);

                final KeyInputEvent event = new KeyInputEvent(jmeKey, '\0', GLFW_PRESS == action, GLFW_REPEAT == action);
                event.setTime(getInputTimeNanos());

                keyInputEvents.add(event);
            }
        });

        glfwSetCharCallback(context.getWindowHandle(), charCallback = new GLFWCharCallback() {
            @Override
            public void invoke(final long window, final int codepoint) {

                final char keyChar = (char) codepoint;

                final KeyInputEvent pressed = new KeyInputEvent(KeyInput.KEY_UNKNOWN, keyChar, true, false);
                pressed.setTime(getInputTimeNanos());

                keyInputEvents.add(pressed);

                final KeyInputEvent released = new KeyInputEvent(KeyInput.KEY_UNKNOWN, keyChar, false, false);
                released.setTime(getInputTimeNanos());

                keyInputEvents.add(released);
            }
        });
    }

both callbacks are called, so three events are created when I press:

new KeyInputEvent(jmeKey, '\0', GLFW_PRESS == action, GLFW_REPEAT == action);
new KeyInputEvent(KeyInput.KEY_UNKNOWN, keyChar, true, false);
new KeyInputEvent(KeyInput.KEY_UNKNOWN, keyChar, false, false);             

they match what I saw in the handler.

[xuan]

@pspeed

Edit: if you build Lemur from source then maybe you can try some things out? If you find something then let me know and/or submit a PR. I’d be super grateful.

Gladly, I made a super quick hack

                if( evt.getKeyChar() >= 32 ) {
                    model.insert(evt.getKeyChar());
                    evt.setConsumed();
                    //resetText(); ...should be automatic now
                } else if (  evt.getKeyChar() == '\0') {
                    evt.setConsumed();
                }

and it consumed the extra events, but it also consumed the ESCAPE key and I guess other stuff that shouldn’t have.
If you tell me what to change I can try it (tomorrow now is late here).

[pspeed]

It’s tricky. Especially if the non-char event comes first.

We may have to specifically check for the KeyInput values we want to let through. Which is fragile but I can’t think of another way at the moment.

For keys like escape, do you still see the character version of the event come through?

[Ali_RS]

@xuan

Gladly, I made a super quick hack

This was also bugging me in my editor for a while now. Applying your patch solved the problem. Thanks!

[xuan]

@Ali_RS

This was also bugging me in my editor for a while now. Applying your patch solved the problem. Thanks!

I think it would be good enough for my case too, but I’d like to get a proper fix to not depend on a custom build.

@pspeed

It’s tricky. Especially if the non-char event comes first.

We may have to specifically check for the KeyInput values we want to let through. Which is fragile but I can’t think of another way at the moment.

How would you go about that? Something like this? looks terrible :

                if( evt.getKeyChar() >= 32 ) {
                    model.insert(evt.getKeyChar());
                    evt.setConsumed();
                    //resetText(); ...should be automatic now
                  } else if ( evt.getKeyChar() == '\0'
                      && ( evt.getKeyCode() >= KeyInput.KEY_0 && evt.getKeyCode() <= KeyInput.KEY_RETURN
                          || evt.getKeyCode() >= KeyInput.KEY_A && evt.getKeyCode() <= KeyInput.KEY_MULTIPLY
                          || evt.getKeyCode() == KeyInput.KEY_SPACE
                          || evt.getKeyCode() >= KeyInput.KEY_NUMPAD7 && evt.getKeyCode() <= KeyInput.KEY_DECIMAL
                          || evt.getKeyCode() >= KeyInput.KEY_YEN && evt.getKeyCode() <= KeyInput.KEY_UNDERLINE
                          || evt.getKeyCode() >= KeyInput.KEY_HOME && evt.getKeyCode() <= KeyInput.KEY_DELETE) ) {
                    evt.setConsumed();
                }

I excluded the Ctrl and other keys and made guesses about the japanese keyboard codes.

@pspeed

For keys like escape, do you still see the character version of the event come through?

No, I only get one event from the first callback (the one with the keycode and not charcode).

[Ali_RS]

Sorry for being ignorant, but isn’t it enough to just check with evt.getKeyChar() == '\0'? Isn’t that supposed to block all non-char keys?

[xuan]

Yes, but you don’t want to block ESC, or at least when running with lwjgl2 it does not block it (so the simple app exists).

But I suppose any other key that’s not relevant to the text field should also not be blocked (F1, Print Screen, etc.). I’m not 100% sure which ones exactly, Ctrl? Alt?

[pspeed]

@pspeed

For keys like escape, do you still see the character version of the event come through?

Asking again because I didn’t see an answer.

[xuan]

@xuan

No, I only get one event from the first callback (the one with the keycode and not charcode).

[pspeed]

I like lwjgl3’s key handling less and less every day.

…does it at least send through character 8 for backspace?

[xuan]

@pspeed

…does it at least send through character 8 for backspace?

No, it only sends the keycode 14 with backspace.

[Ali_RS]

@xuan

Yes, but you don’t want to block ESC, or at least when running with lwjgl2 it does not block it (so the simple app exists).

Regarding the escape key, I have registered a release-focus action, mapped to the escape key in the TextEntryComponent (see my other post) so it won’t block the escape key. After the focus is released the rest of the keys should also work normally.

[xuan]

@pspeed

I like lwjgl3’s key handling less and less every day.

Any ideas how to proceed?

@Ali_RS

Regarding the escape key, I have registered a release-focus action, mapped to the escape key in the TextEntryComponent (see my other post) so it won’t block the escape key. After the focus is released the rest of the keys should also work normally.

I see, that’s a workaround I think, ideally lemur would behave the same with lwjgl2 and 3.
Unless Paul think it’s fine to consume all input even non-char keys.

[pspeed]

I have to look at it a little more and play with the lwjgl2 side to remind me how that works.

[pspeed]

If someone is looking for ways to help this along then one thing that would save my a BUNCH of time would be to submit a PR for a couple test cases… I’m thinking of some simple app with a UI with text field, a couple buttons (OK, Cancel). Like a project under Lemur/examples.

…even better if there is an lwjgl2 project and an lwjgl3 project so that I can easily run them side-by-side.

Else I will have to create this for myself and that pushes things out a little farther. (I’m trying to get a public Mythruna release out at the end of the month and that is monopolizing my limited free time these days.)

[xuan]

The code snippet I posted at the beginning is enough to see the problem easily, typing and the camera moves or not depending on the lwjgl version.
I can make a simple project that can be easily be built with one or the other versions (but I’ll make it with maven and somebody else port it to gradle I they want), otherwise I could add another Demo to the demo submodule in lemur.