JME canvas in a complex Swing app: I could use some advice

Hi. Apologies if these are noob questions, but I’ve read the tutorials, done some research and feel that I’m at the stage that I could use some guidance.

The short version is that I have a swing application with tree view on the left, tabbed content view on the right. Think Netbeans/IntelliJ etc. Hence I’d like to be able to:

  1. Have a JME canvas running as tabbed content. I’ve looked at the ‘JME Canvas in Swing GUI example’, and I’ve had some success here. However, the canvas is initially displayed over the top of the JSplitpane on the left, although it does display correctly when I manually resize the main frame. I think this is something to do with the JME canvas being a AWT heavy-weight component, but I’m not sure if there is an easy workaround. Anyone know of one?
  2. Have multiple tabs with different JME content. The Canvas won’t respond to the tab switching, for reasons above. Also it looks as though there can only be one instance of simple application, and hence only one canvas per Java app. I assume the best workaround is to use a method based on the ’canvas torture methods’ of the example to shift the canvas between tabs, and use appStates to switch in the correct content (eg robot designer to environment? Again, have I misunderstood anything?
  3. I’d like to be able to pull the tabs out as independent windows with different JME canvases running in them. I assume this can’t be done, since there can only be a single JME canvas. Is this understanding correct?
  4. At some point, I need to port the whole lot over to Android. I haven’t really looked into this at all at this stage. Will a JME component happily coexist with other android UI elements? Also I’m wondering whether I should just ditch everything and create a new GUI based around Nifty to use for both?

Ok, so the short version wasn’t short but, if anyone is still reading, the longer version is that I’ve invested a lot of time building a robotic simulator tool with drag and drop designers for building neural networks, designing walking gaits, interfacing with sensors etc. So I’d like to reuse as much as possible in terms of the UI. Porting all this over to Android should be manageable, since drawing all the flow chart etc to a Swing canvas and handling input isn’t much different to drawing on an Android one. However, I get the feeling that I’m swimming against the tide here.
Conversely, the little time I’ve spent looking at Nifty suggests that it’s an entirely different beast, and I’m not sure it would do what I need of it. Any thoughts? :grinning:

p.s Thanks to everyone involved in this project for giving up their time to make something awesome that shared for free. It looks amazing!

It’s only an aside based on what you mentioned. Be aware nifty isnt the only game in town for UI within JMonkey. You can also use lemur (which is philosophically similar to swing) or javaFX.

The jmonkey docs are a little misleading on this item and imply that nifty is what everyone is using

Thanks, that’s really helpful. To be honest, I did rather get the impression the nifty was ‘the only game in town’ with jMonkey :grinning:. Will check out both lemur and javaFx. The slightly perverse thing is that most of my UI elements are completely custom controls, like various editors, that I’ve built by implementing custom drawing routines and handling mouse input. Therefore if it had a tree view, a tab view and, most importantly, some kind of canvas that I could subclass and port all the drawing code to then I would be a happy camper!

You are right that it’s one application per application. JME was designed around games and so sometimes doesn’t support “writing your own IDE” style applications very well. So as you say, you’d have to do some fast switching to make it look like that… and you’ll never really have pull outs. (Furthermore, using Swing at all limits you to using the lwjgl2 backend… I’m stuck there for similar reasons.)

For more information on Lemur see:

It has a tabbed pane but no tree view yet.

Afaik, Jme Canvas is nothing but a Swing/AWT Canvas, so swing/awt canvas dimensions fit onto its parent, so you need to have the canvas placed inside a JPanel or AWTPanel & so the canvas fit the panel then control [the canvas dimensions, parent/child properties with other JComponents] through the panel :

Code example :

final TestSafeCanvas app = new TestSafeCanvas(); app.setPauseOnLostFocus(false);
app.setSettings(settings);
app.createCanvas();
app.startCanvas(true);

final JmeCanvasContext context = (JmeCanvasContext) app.getContext();
final Canvas canvas = context.getCanvas();
canvas.setSize(settings.getWidth(), settings.getHeight());

final JPanel container = new JPanel("Jme Window 1");
//control jme canvas through the parent (parent~Window control)
conatiner.setSize(canvas.getWidth(), canvas.getHeight());
conatiner.add(canvas);
jframe.getContentPane().add(conatiner);

But, beware jme-canvas only exists on lwjgl-2, so you cannot use swing beyond that.

From my point of view, you can try lemur its highly supported & javaFX is a second option but you may have to postponed jfx for now to when you have some exp with jme at first & jfx-jme stills lack some support, but anyway you can search the forums threads for that.

For Lemur, see :

Dependencies :

 implementation "com.simsilica:lemur:1.15.0"
 implementation "com.simsilica:lemur-proto:1.12.0"

 // needed for the style language
 runtime "org.codehaus.groovy:groovy-all:2.4.5"

// Standard utility stuff
implementation 'com.google.guava:guava:19.0'
implementation 'org.slf4j:slf4j-api:1.7.13'
runtime 'org.apache.logging.log4j:log4j-slf4j-impl:2.5'
runtime 'org.apache.logging.log4j:log4j-core:2.5'

Try to add the canvas to 2 JPanels in parallel & see whats the result, if you got it working then it works, you can detach the canvas from of those 2 panels when you finish using it, you can also attach the panel holding the canvas onto a JFrame that would create a separate floating window.

As for android stuff, for android apps, the design patterns will be different in case of UI/UX, but as for games, gui is similar to what you create on PC games, anyway if you want something crossplatform, then lemur works on android too, but if you want to build something special, specific, highly optimized for android, then try Custom Android Views (similar to jfx, but platform specific), check my lib for custom android views :

Thanks for the clarification, it makes perfect sense. I guess I could just keep the rest of the app as is and go into full-screen simulator mode when the user hits the start simulation button. At this stage getting a robust 3D rendering framework and a physics engine is worth just about any compromise I need to make! :grinning:

1 Like

@Pavi_G. Thanks for taking the time to provide such a comprehensive response. I’m still struggling with the placement on the SplitPane, but I’ll strip my code down and test it again later. Similarly I’ll look again at using multiple canvases just to see what happens. Since there can only be one instance of the JME canvas, I’m guessing this will end badly, but you’re right - it does no harm to experiment. Good point about lwjgl-2 - hadn’t realised that. Finally, thanks for the Lemur info and Custom Android Views example. I think Custom Android Views might be what I’m looking for: something that I can draw my own custom graphics on to and do basic hit testing. Lots to look at and so much to learn, but this kind of advice is really useful!

3 Likes

You are welcome :slightly_smiling_face:

For Android examples check this :

For Custom views simple examples using android views & android core ui directly check the helloandroidui android module inside this repo.

For a full example using my lib that uses android core ui, check these twos :

Have a good day :slightly_smiling_face: !

Quick update: I set up a minimal test with JSplitPane based on @Pavi_G’s previous post. The behaviour I get is that even though the canvas is added to the right panel, it is initially placed at the far left (at least until the frame is resized when it is positioned correctly). This is the problem I was having previously. The image below hopefully makes this clearer.

And here’s the code.

        AppSettings settings = new AppSettings(true);
        settings.setWidth(640);
        settings.setHeight(480);
        settings.setVSync(true);
        settings.setFrameRate(60);

        final TestSafeCanvas app = new TestSafeCanvas();
        app.setPauseOnLostFocus(false);
        app.setSettings(settings);
        app.createCanvas();
        app.startCanvas(true);

        JmeCanvasContext context = (JmeCanvasContext) app.getContext();
        Canvas canvas = context.getCanvas();
        canvas.setSize(settings.getWidth(), settings.getHeight());

        final JPanel leftPane= new JPanel();
        leftPane.setBackground(Color.GREEN);
        leftPane.setMaximumSize(new Dimension(400,0));
        leftPane.setPreferredSize(new Dimension(200,0));


        final JPanel rightPane = new JPanel();
        rightPane.setBackground(Color.BLUE);
        rightPane.setSize(canvas.getWidth(), canvas.getHeight());
        rightPane.add(canvas);

        //If enabled, canvas jumps from right pane to left.
        //leftPane.add(canvas);

        JFrame frame = new JFrame("Test");
        final JSplitPane splitPane = new JSplitPane(JSplitPane.HORIZONTAL_SPLIT);
        frame.getContentPane().add(splitPane,BorderLayout.CENTER);
        splitPane.setLeftComponent(leftPane);
        splitPane.setRightComponent(rightPane);

        frame.pack();
        frame.setLocationRelativeTo(null);
        frame.setVisible(true);
    }

I also tried adding a canvas to both panels as an experiment (as suggested above). The canvas will jump to whichever container it was last added to, which makes sense. I think I probably need to rethink my design a bit. :grinning:

1 Like

It seems to be a layouting problem, usually when you use NULL layouting or absolute layouts you will have a problem similar to this(you are cancelling the layout manager calculations), in swing, UI layouts(or layout manager) in swing updates your child components onDraging, onResizing, & onAddingNewComponent that’s the core idea, so may be you can try this without the split pane in order to know the problem, also don’t use setMaximumSize(), setMinimumSize() beacuse they would restrict the max & min size of the JComponents to a fixed number…use setSize() only if you want something dynamically updated by the LayoutManager,

btw JComponents get their location & size updated using the LayoutManager super class using setBounds(int x, int y, int w, int h); with each addition of new component, resizing, dragging~dropping…etc, there are custom layout managers as well…

https://docs.oracle.com/javase/7/docs/api/java/awt/LayoutManager.html

In my unreleased MOSS stuff, I have a Swing app that has a JSplitPane with a JSplitPane in it because I wanted a center JME window with Swing controls to the left and right. I remember something about the event updating and doing things a frame later or something. Anyway, here is the source code for the relevant parts.

Hopefully you can read around the app-specific parts and see something different that I’m doing.

Main.java:

/*
 * $Id$
 * 
 * Copyright (c) 2020, Simsilica, LLC
 * All rights reserved.
 * 
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions 
 * are met:
 * 
 * 1. Redistributions of source code must retain the above copyright 
 *    notice, this list of conditions and the following disclaimer.
 * 
 * 2. Redistributions in binary form must reproduce the above copyright 
 *    notice, this list of conditions and the following disclaimer in 
 *    the documentation and/or other materials provided with the 
 *    distribution.
 * 
 * 3. Neither the name of the copyright holder nor the names of its 
 *    contributors may be used to endorse or promote products derived 
 *    from this software without specific prior written permission.
 * 
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 
 * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 
 * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, 
 * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 
 * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 
 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, 
 * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 
 * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED 
 * OF THE POSSIBILITY OF SUCH DAMAGE.
 */

package com.simsilica.mapper;

import java.awt.*;
import java.awt.event.*;
import javax.swing.*;

import org.slf4j.*;

import org.pushingpixels.substance.api.skin.SubstanceGraphiteGlassLookAndFeel;

import com.jme3.app.*;
import com.jme3.app.state.ScreenshotAppState;
import com.jme3.input.*;
import com.jme3.light.*;
import com.jme3.material.*;
import com.jme3.math.*;
import com.jme3.scene.*;
import com.jme3.scene.shape.Box;
import com.jme3.system.AppSettings;
import com.jme3.system.awt.AwtPanelsContext;

import com.simsilica.fx.LightingState;
import com.simsilica.fx.sky.SkyState;
import com.simsilica.fx.sky.SkySettingsState;

import com.simsilica.lemur.*;
import com.simsilica.lemur.input.*;

import com.simsilica.input.*;
import com.simsilica.state.*;
import com.simsilica.thread.*;


/**
 *
 *
 *  @author    Paul Speed
 */
public class Main extends SimpleApplication {
    static Logger log = LoggerFactory.getLogger(Main.class);

    private volatile JFrame mainFrame;
       
    public static void main( String... args ) throws Exception {
        JFrame.setDefaultLookAndFeelDecorated(true);
        JPopupMenu.setDefaultLightWeightPopupEnabled(false);
        UIManager.setLookAndFeel(new SubstanceGraphiteGlassLookAndFeel());

        final Main app = new Main();
        app.setShowSettings(false);

        AppSettings settings = new AppSettings(true);
        settings.setCustomRenderer(AwtPanelsContext.class);
        settings.setFrameRate(30);
        settings.setGammaCorrection(false);
        app.setSettings(settings);
        app.start();
    }
    
    public Main() throws Exception {
        super(new StatsAppState(), new DebugKeysAppState(), new BasicProfilerState(false),
              new DebugHudState(),      // from SiO2
              new MemoryDebugState(),   // from SiO2
              new JobState(null, 4, -1),           // from SiO2
              new CameraState(70, 0.1f, 8000),  // from SiO2
              //new MovementState(),    // from SiO2
              new CameraControlState(),
              new LightingState(),      // from SimFX
              new SkyState(true),       // from SimFX
              new SkySettingsState(),   // from SimFX
              new HudState(),
              new PostProcessingState(),
              new CubeSceneState(ColorRGBA.Blue, false),  // from SiO2
              new MapState(),
              new MapViewState(),
              new AvatarState(),
              new ModelState(),
              new MarkerState(),
              new ScriptState()
              ); 
 
        stateManager.attach(new ScreenshotAppState("", System.currentTimeMillis()) {
            @Override
            protected void writeImageFile( final java.io.File file ) throws java.io.IOException {
                super.writeImageFile(file);
                System.out.println("Wrote file:" + file);
            }
        });
 
        //float masterScale = 1024f/32768;
        //stateManager.getState(CubeSceneState.class).getScene().setLocalScale(0.5f * masterScale, masterScale, 0.5f * masterScale);
        //stateManager.getState(CubeSceneState.class).getScene().setLocalScale(20);
        stateManager.getState(CubeSceneState.class).getScene().setLocalScale(0);
 
        // Have to create the frame on the AWT EDT.
        SwingUtilities.invokeAndWait(new Runnable() {
            public void run() {
                mainFrame = new JFrame("Mapper v1.0.0");
                mainFrame.setDefaultCloseOperation(JFrame.DISPOSE_ON_CLOSE);
                mainFrame.addWindowListener(new WindowAdapter() {
                    @Override
                    public void windowClosed(WindowEvent e) {
                        stop();
                    }
                });

                mainFrame.setJMenuBar(createMainMenu());

                JSplitPane split = new JSplitPane();
                split.setContinuousLayout(false);
                split.setBackground(Color.black);

                JPanel left = new JPanel();
                JLabel testLabel = new JLabel("Testing");
                JLabel testLabel2 = new JLabel("Testing2");
                left.add(testLabel);
                //left.add(testLabel2);
                split.add(left, JSplitPane.LEFT);
                mainFrame.getContentPane().add(split, BorderLayout.CENTER);

                // For the right panel, we'll split it again so we can have a property
                // pane on the right-right.
                JSplitPane rightSplit = new JSplitPane();
                split.add(rightSplit, JSplitPane.RIGHT);
                rightSplit.setContinuousLayout(false);
                rightSplit.setBackground(Color.black);

                //PropertyEditorPanel objectEditor = new PropertyEditorPanel(gui, "ui.editor");
                //objectEditor.setPreferredSize(new Dimension(250, 100));                
                rightSplit.add(testLabel2, JSplitPane.RIGHT);

                stateManager.attach(new AwtPanelState(rightSplit, JSplitPane.LEFT));
                
                //stateManager.attach(new MovementState());
            }
        });

        stateManager.getState(AwtPanelState.class).addEnabledCommand(new Runnable() {
            public void run() {
                if( !mainFrame.isVisible() ) {
                    // By now we should have the panel inside
                    mainFrame.pack();
                    mainFrame.setLocationRelativeTo(null);
                    mainFrame.setVisible(true);
 
                    //  Just in case
                    GuiGlobals.getInstance().setCursorEventsEnabled(true);
                    inputManager.setCursorVisible(true);
                }
            }
        });
    }
    
    @Override
    public void simpleInitApp() {
        log.info("simpleInitapp()");

        // Because we will use Lemur for some things... go ahead and setup
        // the very basics
        GuiGlobals.initialize(this);
        //GuiGlobals.getInstance().setCursorEventsEnabled(true);

        MapperConfig.getInstance();

        cam.setLocation(new Vector3f(0, 1000, 0));
        cam.lookAt(new Vector3f(), Vector3f.UNIT_Z.mult(-1));
 
        stateManager.getState(LightingState.class).setTimeOfDay(0.2f);
        stateManager.getState(LightingState.class).setOrientation(0.5f);
        LightingState lighting = stateManager.getState(LightingState.class);
        lighting.setSunColor(ColorRGBA.White.clone());
        lighting.setAmbient(ColorRGBA.White.clone().mult(0.75f));
         
        // The swing->JME interface swallows the print screen.
        // (in fact, even through AWT we only get the release event)
        KeyboardFocusManager.getCurrentKeyboardFocusManager()
            .addKeyEventDispatcher(new KeyEventDispatcher() {
                @Override
                public boolean dispatchKeyEvent( KeyEvent e ) {
                    if( e.getKeyCode() == KeyEvent.VK_PRINTSCREEN ) {
                        // We only get the release event but we'll check
                        // for it anyway
                        if( e.getID() == KeyEvent.KEY_RELEASED ) {
                            takeScreenShot();
                        }
                    }
                    return false;
                }
            });
            
    }
    
    public void takeScreenShot() {
        stateManager.getState(ScreenshotAppState.class).takeScreenshot();
    }

    @Override
    public void simpleUpdate( float tpf ) {
    }
        
    private JMenuBar createMainMenu() {
        return new JMenuBar();
    }    
}

AwtPanelState.java

/*
 * $Id$
 * 
 * Copyright (c) 2016, Simsilica, LLC
 * All rights reserved.
 * 
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions 
 * are met:
 * 
 * 1. Redistributions of source code must retain the above copyright 
 *    notice, this list of conditions and the following disclaimer.
 * 
 * 2. Redistributions in binary form must reproduce the above copyright 
 *    notice, this list of conditions and the following disclaimer in 
 *    the documentation and/or other materials provided with the 
 *    distribution.
 * 
 * 3. Neither the name of the copyright holder nor the names of its 
 *    contributors may be used to endorse or promote products derived 
 *    from this software without specific prior written permission.
 * 
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 
 * FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 
 * COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, 
 * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 
 * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 
 * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 
 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, 
 * STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 
 * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED 
 * OF THE POSSIBILITY OF SUCH DAMAGE.
 */

package com.simsilica.mapper;

import java.awt.*;
import java.lang.reflect.InvocationTargetException;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
import javax.swing.*;

import com.jme3.app.Application;
import com.jme3.app.state.BaseAppState;
import com.jme3.system.awt.AwtPanel;
import com.jme3.system.awt.AwtPanelsContext;
import com.jme3.system.awt.PaintMode;



/**
 *
 *
 *  @author    Paul Speed
 */
public class AwtPanelState extends BaseAppState {
 
    // Other than initial setup these fields are _only_ accessed
    // from the swing thread except Viewport attachment
    private final Container container;
    private final Object constraints;   
    private volatile AwtPanel panel;
 
    private List<Runnable> enabledCommands = new CopyOnWriteArrayList<>(); 
    
    public AwtPanelState( Container container, Object constraints ) {
        this.container = container;
        this.constraints = constraints;               
    }
 
    public void addEnabledCommand( Runnable cmd ) {
        enabledCommands.add(cmd);
    }
    
    protected void initialize( Application app ) {
        try {        
            SwingUtilities.invokeAndWait(new InitializeCommand());
        } catch( InterruptedException | InvocationTargetException e ) {
            throw new RuntimeException("Error creating panel on swing thread", e);
        }

        // Can't unattach them so we might as well do it on init
        panel.attachTo(true, app.getViewPort(), app.getGuiViewPort());        
    }
    
    protected void onEnable() {
        SwingUtilities.invokeLater(new AttachPanelCommand());
    }
    
    protected void onDisable() {
        SwingUtilities.invokeLater(new DetachPanelCommand());
    }
    
    protected void cleanup( Application app ) {
    }
 
    private class InitializeCommand implements Runnable {
        public void run() {
 
            AwtPanelsContext ctx = (AwtPanelsContext)getApplication().getContext();
            panel = ctx.createPanel(PaintMode.Accelerated);
            panel.setPreferredSize(new Dimension(1280, 720));
            panel.setMinimumSize(new Dimension(400, 300));
            panel.setBackground(Color.black);
            ctx.setInputSource(panel);
        }
    }
 
    private class AttachPanelCommand implements Runnable {
        public void run() {
            // Add it to the container provided
            container.add(panel, constraints);
            
            for( Runnable r : enabledCommands ) {
                r.run();
            }
        }
    }
    
    private class DetachPanelCommand implements Runnable {
        public void run() {
            container.remove(panel);
        }
    }
}
1 Like

@pspeed Thanks. I’ve managed to get the layout working property now, based on your code, which is a good start. I’ll take another look at this tomorrow and try to create a minimum code example for posterity as soon as I understand it properly. :grinning: Thanks again for your help.

2 Likes

As promised here’s my cut down code:


import com.jme3.app.SimpleApplication;
import com.jme3.material.Material;
import com.jme3.scene.Geometry;
import com.jme3.scene.shape.Box;
import com.jme3.system.AppSettings;
import com.jme3.system.awt.AwtPanel;
import com.jme3.system.awt.AwtPanelsContext;
import com.jme3.system.awt.PaintMode;

import java.awt.*;
import javax.swing.*;

public class TestSplitPane extends SimpleApplication {

    AwtPanel jmePanel;

    public static void main(String[] args)
    {
        AppSettings settings = new AppSettings(true);
        settings.setWidth(640);
        settings.setHeight(480);
        //This must be included to allow the context to be cast as an AwtPanelsContext
        settings.setCustomRenderer(AwtPanelsContext.class);

        final TestSplitPane app = new TestSplitPane();
        app.setSettings(settings);
        app.createCanvas();
        app.startCanvas(true);

        JFrame mainFrame = new JFrame("Test Frame");

        JSplitPane split = new JSplitPane();
        split.setContinuousLayout(false);

        JPanel left = new JPanel();
        left.add(new JLabel("Left Panel"));
        split.add(left, JSplitPane.LEFT);

        /*
        //Option 1: Use an AppState to configure the panel when
        //its initialise method is called as part of JME initialisation.
        AwtPanelState state= new AwtPanelState(split, JSplitPane.RIGHT);
        app.stateManager.attach(state);
        */

        //Option 2: Set up the panel, but defer attaching it
        //until after the engine has been initialised.
        AwtPanelsContext ctx = (AwtPanelsContext)app.getContext();
        AwtPanel panel = ctx.createPanel(PaintMode.Accelerated);
        panel.setPreferredSize(new Dimension(1280, 720));
        panel.setMinimumSize(new Dimension(400, 300));
        panel.setBackground(Color.black);
        ctx.setInputSource(panel);
        app.jmePanel = panel;
        split.add(panel, JSplitPane.RIGHT);

        mainFrame.getContentPane().add(split, BorderLayout.CENTER);
        mainFrame.setSize(640,480);
        mainFrame.setVisible(true);
    }

    @Override
    public void simpleInitApp() {
        flyCam.setDragToRotate(true);
        Box b = new Box(1, 1, 1);
        Geometry geom = new Geometry("Box", b);
        Material mat = new Material(assetManager, "Common/MatDefs/Misc/Unshaded.j3md");
        mat.setTexture("ColorMap", assetManager.loadTexture("Interface/Logo/Monkey.jpg"));
        geom.setMaterial(mat);
        rootNode.attachChild(geom);

        //Option 2: Can now safely attach the panel after JME application initialisation.
        jmePanel.attachTo(true, getViewPort(), getGuiViewPort());
    }
}

Couple of interesting things here. Firstly, setCustomRenderer needs to be included in the app settings to allow the context to be recast as a AwtPanelsContext (which is subsequently required to set the InputSource to the panel being rendered to). Secondly, the panel can only be safely attached after initialisation of the engine has taken place (at least this is my understanding).

@pspeed Your approach subclassed an AppState to do the setup of the panel. Is that generally the preferred way of doing this, rather than having post initialise stuff in simpleInitApp?

Also, I noticed that the frame rate really took a dive when I hosted the app in a panel (from around 300+ fullscreen to 40 fps when running in maximised window). Is this normal, or am I still doing something daft?

@Pavl_G Thanks for the suggestions. I could probably use a refresher on layout logic, since I don’t always find it massively intuitive. :grinning:

2 Likes

Glad you managed to do it ! btw, I have worked on Null Layout before on my project HybridJme which involves you to update the layout constraints regularly, anyway this is the code, it stills under alpha flags, anyway they may give some insights on your logic of refreshing things out :

package core.uiStates;

import core.window.HybridWindow;
import javafx.embed.swing.JFXPanel;
import javafx.fxml.FXMLLoader;
import javafx.scene.Parent;
import javax.swing.*;
import java.awt.*;
import java.awt.event.FocusEvent;
import java.awt.event.FocusListener;
import java.io.IOException;
import java.net.URL;
import java.util.ConcurrentModificationException;
import java.util.HashMap;
import java.util.concurrent.ExecutionException;
import static core.window.HybridWindow.JME_CANVAS;

/**
 * A Class holds UiStates on a GamePanel, the GamePanel holds the UiStateManager & the Game Canvas
 * @author pavl_g
 */
public final class UiStateManager extends JPanel implements FocusListener {
    private final Container gamePanel;
    private final HashMap<Integer, Component> uiStates = new HashMap<>();
    private volatile int uiStateIndex = 0;
    /**
     * Create a UiStateManger Swing Component to be attached to the game panel
     */
    public UiStateManager(final HybridWindow gameWindow) {
            setEnabled(true);
            setLayout(null);
            gameWindow.add(this);
            addFocusListener(this);
        this.gamePanel = gameWindow;
    }
    /**
     * detaches the UI-Manager from the parent Context.
     * @return true if detached successfully , false otherwise.
     */
    public boolean detachUiManager(){
        try {
            gamePanel.remove(this);
            return true;
        }catch (Exception e){
            return false;
        }
    }

    public Component attachUiState(Component uiState, int id) throws ExecutionException, InterruptedException {
        SwingWorker<Component, Void> worker = new SwingWorker() {
            @Override
            protected Component doInBackground() {
                add(uiState, uiStateIndex++).setName(String.valueOf(id));
                revalidate();
                repaint();
                updateUI();
                return uiState;
            }

            @Override
            protected void done() {

                /*update bounds of the gamePanel children*/
                updateUiStateBounds(UiStateManager.this);
            }
        };
        worker.execute();
        return worker.get();
    }

    public void updateUiStateBounds(Container gamePanel)  {
        SwingUtilities.invokeLater(()-> {
            synchronized(gamePanel) {
                for (Component child : gamePanel.getComponents()) {
                    if (child != null) {
                        if (!child.getName().equals(JME_CANVAS)) {
                            child.setBounds(child.getBounds().x, child.getBounds().y, child.getBounds().width, child.getBounds().height);
                        } else if (child.getName().equals(JME_CANVAS)) {
                            /*Handle the screen margins*/
                            child.setBounds(0, 0, getWidth()-getInsets().left, getHeight()-getInsets().top);
                        }
                        //applying recursive calling
                        if (child instanceof Container) {
                            updateUiStateBounds((Container) child);
                        }
                    }
                }
            }
        });
    }

    /**
     * gets the index of the Last UI-State attached to the UI-State-Manager.
     * @return the index of the last UI state.
     */
    public int getLastStateIndex() {
        return uiStateIndex;
    }
    /**
     * Detaches all game Ui states from the UI-Manager.
     */
    public void detachAllUiStates(){
        for(Component uiState : getComponents()){
            if(!uiState.getName().equals(JME_CANVAS)) {
                remove(uiState);
            }
        }
        if(!uiStates.values().isEmpty()){
            uiStates.clear();
        }
        uiStateIndex = 0;
    }

    public void detachUiState(Component uiState){
        new SwingWorker<Void, Void>() {
            @Override
            protected Void doInBackground() {
                remove(uiState);
                revalidate();
                repaint();
                return null;
            }

            @Override
            protected void done() {
                uiStates.values().remove(uiState);
                revalidate();
                repaint();
                /*update bounds of the gamePanel children*/
                updateUiStateBounds(UiStateManager.this);
            }
        }.execute();
    }
    /**
     * detaches the specified UI-State by index.
     * @param index the index of the UI-State by stacks convention.
     * @return the removed view
     */
    public <T extends Component> T detachUiStateByIndex(int index){
        Component uiState = getChildUiStateByIndex(index);
        SwingUtilities.invokeLater(() ->{
            if(hasUiStateById(index)){
                remove(uiState);
                updateUiStateBounds(UiStateManager.this);
            }
            if(uiStates.containsValue(uiState)){
                uiStates.values().remove(uiState);
            }
        });
        return (T) uiState;
    }
    /**
     * detach a UI-State from the State-Manager by id.
     * @param id the UI-State id.
     * @return the removed view
     */
    public <T extends Component> T detachUiStateById(int id){
        Component uiState = getChildUiStateById(id);
            if(hasUiStateById(id)){
                if(uiState instanceof JFXPanel){
                    ((JFXPanel)uiState).setScene(null);
                }
                detachUiState(uiState);
            }
            if(uiStates.containsValue(uiState)){
                uiStates.values().remove(uiState);
            }
        return (T) uiState;
    }
    /**
     * Checks if the current Ui-Pager has got Ui-States
     * @return true if there are Ui-States inside
     */
    public boolean hasUiStates(){
        return uiStates.size() > 0;
    }

    /**
     * Checks for the existence of a current Ui-State by an id
     * @param resId the id of the proposed Ui-State to check for
     * @return true if the proposed Ui-State exists
     */
    public boolean hasUiStateById(int resId){
        return getChildUiStateById(resId) != null;
    }

    /**
     * Checks for the existence of a current Ui-State by an index in the UiPager Stack
     * @param index the index of the proposed Ui-State to check for
     * @return true if the proposed Ui-State exists
     */
    public boolean hasUiStateByIndex(int index){
        return getChildUiStateByIndex(index) != null;
    }
    /**
     * get the child UI-state of the #{@link UiStateManager} by id.
     * @param resId the id of the child UI-State to get.
     * @return an android view representing the specified view for this id.
     */
    public <T extends Component> T getChildUiStateById(int resId){
        return (T) uiStates.get(resId);
    }
    /**
     * get a child Ui-State of the #{@link UiStateManager} by index.
     * @param index the index of the UI-State by stacks convention.
     * @return an android view representing the specified view for this index,
     */
    public <T extends Component> T getChildUiStateByIndex(int index){
        return (T) getComponent(index);
    }
    /**
     * Loop over UI-States & do things , w/o modifying #{@link UiStateManager#uiStates} stack size.
     * @param uiStatesLooper a non-modifiable annotated interface to include the piece of code , that would be executed for each UI-State.
     */
    @UiStatesLooper.NonModifiable
    public void forEachUiState(UiStatesLooper.NonModifiable.Looper uiStatesLooper){
        final int modCount=uiStates.size();
        for(int position=0;position<uiStates.size() && modCount==uiStates.size(); position++) {
            uiStatesLooper.applyUpdate(getChildUiStateByIndex(position),position);
        }
        if(modCount!=uiStates.size()){
            throw new ConcurrentModificationException("Modification of UIStates Stack positions or size isn't allowed during looping over !");
        }
    }
    /**
     * Loop over UI-States & do things , w/ or w/o modifying #{@link UiStateManager#uiStates} stack size.
     * @param uiStatesLooper a modifiable annotated interface to include the piece of code , that would be executed for each UI-State.
     */
    @UiStatesLooper.Modifiable
    public void forEachUiState(UiStatesLooper.Modifiable.Looper uiStatesLooper){
        for(int position=0;position<uiStates.size(); position++) {
            uiStatesLooper.applyUpdate(getChildUiStateByIndex(position),position);
        }
    }
    /**
     * gets the parent Context of which the UiStateManager is attached to.
     * @return a viewGroup instance representing that parent context.
     */
    public Container getUiStackParentContext() {
        return this;
    }

    @Override
    public void focusGained(FocusEvent e) {
        updateUiStateBounds(this);
    }

    @Override
    public void focusLost(FocusEvent e) {

    }
    public Parent fromXML(URL url) throws IOException {
        return FXMLLoader.load(url);
    }
}

HybridWindow.java :

package core.window;

import com.jme3.app.LegacyApplication;
import com.jme3.system.AppSettings;
import com.jme3.system.JmeCanvasContext;
import com.sun.istack.internal.NotNull;
import javax.swing.*;
import java.awt.*;
import java.awt.event.WindowEvent;
import java.awt.event.WindowFocusListener;
import java.awt.event.WindowListener;
import java.awt.event.WindowStateListener;
import java.util.logging.Level;
import java.util.logging.Logger;

public class HybridWindow extends JFrame implements WindowStateListener, WindowFocusListener, WindowListener {
    public static final String JME_CANVAS = "JmeCanvas";
    private final LegacyApplication legacyApplication;
    private Canvas jmeCanvas;
    private Container uiStateManager;
    private int height = 0;
    private int width = 0 ;
    private final Logger hybridJmeLogger = Logger.getLogger(getClass().getName());

    public HybridWindow(@NotNull LegacyApplication legacyApplication){
        this.legacyApplication = legacyApplication;
    }
    public JFrame initWindow(int windowWidth, int windowHeight, Container uiStateManager){
            this.width = windowWidth;
            this.height = windowHeight;
            this.uiStateManager = uiStateManager;
            addWindowStateListener(this);
            addWindowFocusListener(this);
            addWindowListener(this);
            initializeSwing(windowWidth, windowHeight, uiStateManager);
        return this;
    }
    protected void initializeSwing(int windowWidth, int windowHeight, Container uiStateManager){
            setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
            setBounds(0, 0,windowWidth,windowHeight);
            setLayout(new CardLayout());
            setDefaultLookAndFeelDecorated(false);
            setResizable(false);
            setLocationRelativeTo(null);
            setVisible(true);
            hybridJmeLogger.log(Level.FINE, "Hybrid window is initialized successfully.");
    }
    public void buildGame() {
            AppSettings settings = new AppSettings(true);
            settings.setWidth(getWidth());
            settings.setHeight(getHeight());

            legacyApplication.setPauseOnLostFocus(false);
            legacyApplication.setSettings(settings);
            legacyApplication.createCanvas();
            legacyApplication.startCanvas(true);

            jmeCanvas = ((JmeCanvasContext)legacyApplication.getContext()).getCanvas();
            jmeCanvas.setName(JME_CANVAS);
            uiStateManager.add(jmeCanvas);
            jmeCanvas.setBounds(0, 0, getWidth(), getHeight());
            updateUiStateBounds(uiStateManager);
    }
    public GraphicsDevice setImmersiveMode(int screen){
            setResizable(true);
            GraphicsDevice device = GraphicsEnvironment.getLocalGraphicsEnvironment().getScreenDevices()[screen];
            device.setFullScreenWindow(this);
        return device;
    }
    public LegacyApplication getLegacyApplication() {
        return legacyApplication;
    }

    public void updateUiStateBounds(Container gamePanel)  {
        SwingUtilities.invokeLater(()-> {
            for (Component child : gamePanel.getComponents()) {
                if (child != null) {
                    if (!child.getName().equals(JME_CANVAS)) {
                        child.setBounds(child.getBounds().x, child.getBounds().y, child.getBounds().width, child.getBounds().height);
                    } else if(child.getName().equals(JME_CANVAS)){
                        /*Handle the screen margins*/
                        child.setBounds(0, 0, getWidth()-getInsets().left, getHeight()-getInsets().top);
                    }
                    //applying recursive calling
                    if(child instanceof Container){
                        updateUiStateBounds((Container)child);
                    }
                }
            }
        });
    }


    @Override
    public void windowStateChanged(WindowEvent e) {
        updateUiStateBounds(uiStateManager);
    }

    @Override
    public void windowGainedFocus(WindowEvent e) {
        updateUiStateBounds(uiStateManager);
    }

    @Override
    public void windowLostFocus(WindowEvent e) {
        updateUiStateBounds(uiStateManager);
    }

    @Override
    public void windowOpened(WindowEvent e) {

    }

    @Override
    public void windowClosing(WindowEvent e) {

    }

    @Override
    public void windowClosed(WindowEvent e) {
        legacyApplication.stop();
    }

    @Override
    public void windowIconified(WindowEvent e) {
        updateUiStateBounds(uiStateManager);
    }

    @Override
    public void windowDeiconified(WindowEvent e) {
        updateUiStateBounds(uiStateManager);
    }

    @Override
    public void windowActivated(WindowEvent e) {
        updateUiStateBounds(uiStateManager);
    }

    @Override
    public void windowDeactivated(WindowEvent e) {
        updateUiStateBounds(uiStateManager);
    }
}

[The UiStateManager manages UiStates on a jme game screen with absolute or null layouting, ie you can specify the x & y & height & width of the UiStates, jfx can be used too within swing using JFXPanel JComponent].

1 Like

AppStates are nicer because they are modular and keep the app clean… else you end up with a 10000 line main class.

It’s going to be slower. There is more overhead and getting the buffer data to Swing’s canvas.

Note that “FPS” is not a good measure because we want to see it as linear but it’s not. Better to measure ‘tpf’. But either way, anything faster than the refresh of the monitor is wasted time. If you are properly running with vsync (which may actually be forced in the case of a canvas, I don’t remember) then you will end up with multiples of 60 hz.

In my code, I limit update to 30 FPS because an interactive swing-based editor didn’t really need more than that.

Thanks, that’s a big help. :grinning:

2 Likes

Makes sense.

That also makes sense, and probably isn’t too much of a problem for me either. Just wanted to check I wasn’t doing anything daft. I think 30 FPS will be fine.
Thanks for all the help everyone - it’s very much appreciated! Now time to get onto all the fun stuff ! :grinning:

1 Like

In my code, that’s what this line does:

settings.setFrameRate(30);

…in case it wasn’t clear.

1 Like

Actually, that was one of the first things I looked up how to do when I first started experimenting with JME. Without throttling the frame rate, the GPU on my Macbook goes into meltdown! :wink:

2 Likes

For non-canvas apps, it’s better to just turn vsync on. It lets the next frame start while the previous frame is streaming to the GPU (effectively) so you get a little boost in your CPU-based stuff.

2 Likes