How can I use lemur buttons with a touchscreen?

I finally got lemur to work with android. My problem is you can’t click anything from android. I assume this is because the touchscreen is not the same as a mouse. I have inputManager.setSimulateMouse(true); active in my main class. Is there any workaround to this?

Hi @TooMuchJava,

just a guess cause your problem description is lacking some information for good assumptions.
If you use the newest version of lemur (V.1.11) then in some cases the mouse ray is calculated incorrectly - thus not reaching any lemur element.
Check: Buttons and DragHandler without effect after upgrading to Lemur 1.11 - #17 by Aufricer - the last entries of the threat

Otherwise you may need to give more detailed explaination of your problem to get more advice.

Touch screen should work fine. It even supports multitouch.

So I have to go with Aufricer in that we’ll need more information. If you are running the newest version then you can try one version older to see if the problem persists.

Older version seems to do the same thing, except no setIconSize in this version(can’t get small enough with setIconScale) for icons. So they’re huge. However with the gigantic buttons, i caught something that i wasn’table to see before.

It only happens when i drag my fingers down the screen no taps, and i couldn’t do it with only one finger.
If you watch the vidio, you’ll see when i drag two fingers down, i can get a glimpse of my menu. The menu is set to disappear when clicked again, so the click event must be firing twice.
If you want to see my code(lemur 10)

public class MainMenu extends AbstractAppState {
    Geometry title, ttp, darken;
    Container lemur, smenu, name;
    float ttpScale = 1;
    boolean ttpDir = false, triggered = false, settingsOpen = false;
    
    TextField nameBlank;
    public final ActionListener listener = new ActionListener() {
        @Override
        public void onAction(String name, boolean isPressed, float tpf) {
            if (isPressed) return;
            if (settingsOpen) {
                settingsOpen = false;
                Main.guiNode_.detachChild(smenu);
                return;
            }
            if (triggered) return;
            triggered = true;
            Running.killAll = false;
            Main.stateManager_.attach(new Running());
            Main.stateManager_.detach(MainMenu.this);
        }
    };
    @Override
    public void initialize(AppStateManager stateManager, Application app) {
        super.initialize(stateManager, app);
        Main main = (Main) app;
        title = new Geometry("title", new Quad(Main.settings_.getWidth() / 2, (Main.settings_.getWidth() / 2) / 7));
        Material mat = new Material(Main.assets, "Common/MatDefs/Misc/Unshaded.j3md");
        mat.setTexture("ColorMap", Main.assets.loadTexture("Textures/Title.png"));
        mat.getAdditionalRenderState().setBlendMode(RenderState.BlendMode.Alpha);
        title.setMaterial(mat);
        title.setLocalTranslation((Main.settings_.getWidth() / 4), (Main.settings_.getHeight() / 2) + (Main.settings_.getHeight() / 5), 0);
        Main.guiNode_.attachChild(title);
        
        ttp = new Geometry("ttp", new Quad(Main.settings_.getWidth() / 4, (Main.settings_.getWidth() / 4) / 5));
        mat = new Material(Main.assets, "Common/MatDefs/Misc/Unshaded.j3md");
        mat.setTexture("ColorMap", Main.assets.loadTexture("Textures/tap_play.png"));
        mat.getAdditionalRenderState().setBlendMode(RenderState.BlendMode.Alpha);
        ttp.setMaterial(mat);
        ttp.setLocalTranslation((Main.settings_.getWidth() / 2) - (Main.settings_.getWidth() / 4 / 2.5f), (Main.settings_.getHeight() / 2) - (Main.settings_.getHeight() / 2.5f), 0);
        Main.guiNode_.attachChild(ttp);
        
        darken = new Geometry("darken", new Quad(Main.settings_.getWidth(), Main.settings_.getHeight()));
        mat = new Material(Main.assets, "Common/MatDefs/Misc/Unshaded.j3md");
        mat.setColor("Color", new ColorRGBA(0, 0, 0, 0.5f));
        mat.getAdditionalRenderState().setBlendMode(RenderState.BlendMode.Alpha);
        darken.setMaterial(mat);
        darken.setLocalTranslation(0, 0, -1);
        Main.guiNode_.attachChild(darken);
        
        new Thread(() -> {
            try {
                Thread.sleep(500);
            } catch (InterruptedException ex) {
                Logger.getLogger(MainMenu.class.getName()).log(Level.SEVERE, null, ex);
            }
            Main.inputManager_.addListener(listener, "Click");
        }).start();
        gl.player.killed = 0;
        gl.destroy();
        gl.genCrushers(30);
        gl.genNPCs(20);
        gl.setupPlayer();
        gl.attatchChildrenToNode(Main.rootNode_);
        gl.update();
        
        
        GuiGlobals.initialize(Main.instance);
        //GuiGlobals.getInstance().getStyles().setDefaultStyle("glass");
        lemur = new Container();
        Main.guiNode_.attachChild(lemur);
        lemur.setLocalTranslation(0, Main.settings_.getHeight(), 0);
        Button settings = lemur.addChild(new Button(""));
        IconComponent icon = new IconComponent("Textures/gear.png");
        icon.setIconScale(Main.settings_.getWidth() / 200);
        settings.setIcon(icon);
        Button ads = lemur.addChild(new Button(""));
        icon = new IconComponent("Textures/noads.png");
        icon.setIconScale(Main.settings_.getWidth() / 200);
        ads.setIcon(icon);
        
        //GuiGlobals.getInstance().getStyles().setDefaultStyle("glass");
        smenu = new Container();
        smenu.setLocalTranslation(0, Main.settings_.getHeight() - Main.settings_.getWidth() / 20, 0);
        
        Button lgm = smenu.addChild(new Button((Variables.instance.fastmode() ? "Normal" : "Low") + " Graphics Mode"));
        lgm.setFontSize(Main.settings_.getWidth() / 32);
        lgm.addClickCommands((Button source) -> {
            if (Variables.instance.fastmode()) {
                Variables.instance.setFastmode(false);
                Main.instance.reload();
            } else {
                Variables.instance.setFastmode(true);
                Main.instance.reload();
            }
            lgm.setText((Variables.instance.fastmode() ? "Normal" : "Low") + " Graphics Mode");
        });
        settings.addClickCommands((Button source) -> {
            if (settingsOpen) {
                settingsOpen = false;
                Main.guiNode_.detachChild(smenu);
            } else {
                settingsOpen = true;
                Main.guiNode_.attachChild(smenu);
            }
        });
        Button reload = smenu.addChild(new Button("Quick Reload"));
        reload.setFontSize(Main.settings_.getWidth() / 32);
        reload.addClickCommands((Button source) -> {
            Main.instance.reload();
        });
        
        name = new Container();
        Main.guiNode_.attachChild(name);
        Label l = new Label("          Username          ");
        l.setFontSize(Main.settings_.getWidth() / 32);
        name.addChild(l);
        
        nameBlank = name.addChild(new TextField(Variables.instance.getUsername()));
        nameBlank.setFontSize(Main.settings_.getWidth() / 32);
        name.setLocalTranslation(0, Main.settings_.getHeight() / 2, 0);
        Running.killAll = true;
    }
    
    @Override
    public void update(float tpf) {
        Variables.instance.setUsername(nameBlank.getText());
        /*if (ttpDir) {
            ttpScale+= 0.01f;
            if (ttpScale > 1.2f) {
                ttpDir = !ttpDir;
            }
        } else {
            ttpScale-= 0.01f;
            if (ttpScale < 1f) {
                ttpDir = !ttpDir;
            }
        }
        ttp.scale(ttpScale);*/
    }
    
    @Override
    public void cleanup() {
        super.cleanup();
        Main.guiNode_.detachChild(title);
        Main.guiNode_.detachChild(ttp);
        Main.guiNode_.detachChild(darken);
        Main.guiNode_.detachChild(lemur);
        Main.guiNode_.detachChild(smenu);
        Main.guiNode_.detachChild(name);
        Main.inputManager_.removeListener(listener);
    }
    
}

I hope I don’t have to go write the GUI in 100% JME
Turns out, it’s only the text that can be clicked on my phone, no icons.

I’m guessing that you don’t have a style defined because Android makes that tricky. (Though you can define them in code, too.) Your buttons will need a background even if it’s invisible else they are unclickable except on the text.

Also, instead of scaling each button a huge amount, just scale the main GUI elements, the containers that contain everything. (Or make your own parent node and scale that and add your stuff to that instead of the guiNode directly, ie: add the parent to the guiNode.)

There are lot of other strange things you do in your code but I’m not sure you want to hear about it.

A usual pattern is the GuiGlobals.initialize() in your main application in simpleInitApp() else you may have other problems if you ever want to use Lemur from another app state. (simpleInitApp() runs before anything else.)

The 500 ms delayed add to InputManager is also strange.

Also not sure why you have static fields for things like guiNode when you could just call main.getGuidNode().

I don’t recommend extending AbstractAppState. It makes you have to do everything yourself versus extending BaseAppState.

Generally, a GUI app state like this would extend BaseAppState. In it’s initialize(Application) it would create a parent for its containers and the children to that. In onEnable() it would add that parent to the guiNode. in onDisable() it would remove it. cleanup() doesn’t necessarily need to do anything in that case.

This approach could also let you just use the regular getters on your Main class and pass them to ActionButtons with a CallMethodAction.

For examples of a main menu app state, you can look at these sample apps:

This is good to
know. It still works on my computer though without a background.

This is what worked with lemur 11

Icon.setIconSize(new Vector2f(Main.settings_.getWidth() / 20, Main.settings_.getWidth() / 20));

I just want everything to scale a certain way, so I use the display size so it shows up the same.

It’s probably going to drive me nuts but I need to hear it…

Didn’t look any farther than “oh. That’s a private method. I know what I can do…”

I must’ve read the wrong thing, but somewhere on the wiki it said something about AbstractAppState on the app state page. I’m 98% sure…

In general thanks for the help Though. :smile:

Update: can’t click the button. It blocks the input event from reaching my handler and doesn’t start the game, but it still does nothing. I nene blacked out the background to check.

But it works on desktop?

If so then that’s really strange.

Edit: and Lemur 1.10.1 also fails?

Yes, but if you add everything in natural sizes to a node then you can scale the whole NODE.

Node myGuiNode = new Node(“My Gui”);
getApplication().getGuiNode().attachChild(myGuiNode);

myGuiNode.setLocalScale(xScale, yScale, 1);

…then add all of your elements in natural size to myGuiNode.

Scaling the icon is meant to scale it relative to the size of the text, etc… not to scale it to be the same size for different displays. You probably want the whole GUI to scale in that case.

See the examples already linked:

…only in this case they just scale the root window instead of a whole parent node.

Yes. The desktop has been 100% functional from the beginning.

I set the background like this.

Attributes a=
Bg = new QuadBackgroundComponent(ColorRGBA.Black);
A.set(“background”, bg);

The background appears black both on my desktop and my phone. However, as I said before, it’s only clickable on the desktop.

To answer everything you said about scale. What I’ve been doing is finding a good spot for each element group, and using fractions of the screen size to position and scale it. Currently I haven’t had any problems with my scaling.

And this also fails with Lemur 1.10.1?

Yes. Exactly the same problem

I can’t see any problems with my source code, can you?

Well, if it works on desktop then the code is right to a certain extent… so it must be something android specific. Either in the way the screen is setup or in the parts of your code that change because of the screen size or whatever.

I’m not sure and I’m not really that familiar with Android. I just know that the touch screen stuff has always worked in the past. And you said earlier that in some cases it was working.

…might require some deeper looking on your end. Sorry.

When you say “android” your effectively saying the equivalent of “linux” in that there are a lot of different flavours and each work slightly differently, so you need to be a little more specific.

Are you using an emulator to test this or directly on your phone? If not you should be doing both. Does it work as intended on the emulators? Does the version of android you are using in the emulator make any difference to the behaviour? And so on…

These are the things you have to go through with mobile development.

I made this game on android for fun using jmonkey and lemur. I never had such issues.

1 Like

The simplest thing to do is create a bare-bones test case. That way you remove any other potential. Create a test appstate with a single button that does something like change the color of it or remove the button or something when you press it.

Good idea, So far we know that:

  • Text Buttons Work
  • Buttons with only Icons don’t
  • Except they do on a desktop.

Edit: when i added text to the button, i could see it was recieving hover events. I could not click the text, but it turned green when i clicked the button or icon.

I mean, it certainly feels like you’ve squashed your z scale to 0 somewhere.

A simple test case would help eliminate a lot of confusion on everyone’s part, I think.

Effective demo copy:

import com.jme3.app.SimpleApplication;
import com.jme3.math.Vector2f;
import com.simsilica.lemur.Button;
import com.simsilica.lemur.Command;
import com.simsilica.lemur.Container;
import com.simsilica.lemur.GuiGlobals;
import com.simsilica.lemur.Label;
import com.simsilica.lemur.component.IconComponent;
import com.simsilica.lemur.style.BaseStyles;

public class Main extends SimpleApplication {

    public static void main( String... args ) {
        Main main = new Main();
        main.start();
    }           
    
    @Override
    public void simpleInitApp() {
            
        // Initialize the globals access so that the defualt
        // components can find what they need.
        GuiGlobals.initialize(this);
            
        // Load the 'glass' style
        //BaseStyles.loadGlassStyle();
            
        // Set 'glass' as the default style when not specified
        //GuiGlobals.getInstance().getStyles().setDefaultStyle("glass");
    
        // Create a simple container for our elements
        final Container myWindow = new Container();
        guiNode.attachChild(myWindow);
            
        // Put it somewhere that we will see it
        // Note: Lemur GUI elements grow down from the upper left corner.
        myWindow.setLocalTranslation(300, 300, 0);
    
        // Add some elements
        myWindow.addChild(new Label("Hello, World."));
        Button clickMe = myWindow.addChild(new Button("Click Me"));
        IconComponent icon = new IconComponent("Textures/gear.png");
        icon.setIconSize(new Vector2f(settings.getWidth() / 15, settings.getWidth() / 15));
        clickMe.setIcon(icon);
        clickMe.addClickCommands(new Command<Button>() {
                @Override
                public void execute( Button source ) {
                    myWindow.addChild(new Label("The world is yours"));
                }
            });            
    }    
}

It works :confused: Now i have to find out what I did wrong…
Edit: I added a 500ms delay between use to make sure the button wasn’t being double-clicked and the menu dissapearing again, but still othing happened. Like this demo, the text turned green, but I think it has something to do with my click handler.

settings.addClickCommands((Button source) -> {
            if (!canPress) return;
            if (settingsOpen) {
                settingsOpen = false;
                Main.guiNode_.detachChild(smenu);
            } else {
                settingsOpen = true;
                Main.guiNode_.attachChild(smenu);
            }
            canPress = false;
            new Thread(() -> {
                try {
                    Thread.sleep(500);
                    canPress = true;
                } catch (InterruptedException ex) {
                    Logger.getLogger(MainMenu.class.getName()).log(Level.SEVERE, null, ex);
                }
            }).start();
        });

If you have the ability to add logging, you should log the value of canPress at the start of your handler.

There could be a dozen things going wrong with this rube-goldberg way to avoid rapid clicks.

Starting a thread just to see if the right time has elapsed since the last click is almost the most resource-intensive way you can do this. (Short of starting a whole other application to count to 500 a ms at a time.)

If you want to debounce your clicks just save the time it was clicked and then don’t click if the new time - old time < 500.

I logged everything, and it seems to be working(click event firing every time the button is pressed)
I just need to figure out why it isn’t opening the menu though