Plugin-Based JME DevKit

A lot of the time when users create games they often create their own “editors” to suit their needs. These can range from ParticleEmitter editors, Terrain editors, Debuggers, Vehicle creators, physics editors (e.g. minie has one or two), Model importers, Material editors, etc.

My idea is based on a plugin system that runs from within a JME application. Users can create plugins that are loaded on startup - and they provide the various tools. This way - instead of having multiple different apps - we have multiple different plugins in a single app. In addition we can provide a central “repository” of sorts to automatically download these plugins.

I have just recently finished the primary functions of the plugin system. Plugins can have dependencies to other plugins, or soft depend on plugins (if exists do this, else this function is not available, or whatever). Plugins can also have the “visible” flag set as false so they don’t appear in the list (e.g. just a dependency plugin). Although I’m sure contributions would be very welcome in this area - this is not the issue. I’m fine developing the plugins system right now.

My issue is how to implement the plugin system effectively. Let me first say that my ideas hereafter are more “thinking out loud” than “this is how it’s going to happen”. I am not a UI/UX guru.

image

So the first port of discussion is how this is going to work. My thoughts are that each plugin is “activated” individually. Only one plugin can be active at a time (although as previously mentioned - you can depend on other plugins to use their functionality). My thoughts on this are that activating more than one plugin at a time will cause serious potential issues. If one plugin is enabled and displays a GUI - another plugin may overlay that GUI and make a big mess. Each plugin should/can also contain its state - so when it’s selected again it can resume that state.

I’d also like to implement some sort of command console system where you could “slash command” things in your plugin - or even select a plugin with a slash command, issue debug commands, etc. I think it sounds like a nice option to have. You could even have a plugin that has the “visible” flag as false that solely just provides some cool commands like move "mySpatial" x,y,z or whatever.

If we look at the picture above, there is a plugins ListBox<Plugin>. I chose the Lemur GUI framework solely because java-fx requires specific java versions - JME and Lemur do not, so we can run on any java version supported by the engine and avoid the whole JFX8/11/12/13/14/etc issue - as well as any cross-platform problems that may already exist when deploying java-fx.

The ListBox takes up all of the left side of the scene. I don’t like that. Maybe a menu would serve better? But this means 2 clicks to change select another plugin. And maybe even some scrolling if you have a lot of plugins. Maybe a “toolbox” type setup like photoshop where there’s a load of buttons you can just click would be better? I really don’t know. This is the part I’d really like some help with. You’d still have to scroll if your plugin count was very large, but it still seems like a better idea, although it still occupies the left side of the scene exclusively.

How do you feel the UI should be implemented? What are your feelings about the plugin system? Do you have any opinions on any additional functionalities?

12 Likes

Toolbox like in photoshop if want to stick “one plugin active at a time”

Lemur for toolbox is ok, but some better theme would be nice.

Also in my opinion, i could say that camera movement should be plugin itself(but idk how you would solve it)

1 Like

You can use TabbedPanel and RollupPanel as well as DragHandler to make things compact and movable.

As you already know you can use the AppStates attach/detach enable/disable with a ActionListener to hide it altogether.

1 Like

It can be an option to select if we wanted.

1 Like

Yeah… I quite like the idea of moveable with the option of collabsible.I like that idea.

1 Like

It seems your goal is to create something to improve workflow in jME. :slight_smile:
And/or to lessen the need to create a niche editor
And/or to make it easier to create a niche editor.

UI… let’s see.

  • a toggle button to hide/show the plugin UI - after all, one does not use the plugin all the time
  • i prefer separate window (eg. swing), especially for showing images, graphs and stuff that takes a lot of space that does not work well with embedded editor

Feelings…

  • it seems like a big project, and I think the determining factor whether it will succeed or fail will not be
    whether you can code it or not,
    but whether what you code is useful or not.
    Create what you consider useful.
    When you end up regularly using it, then you find out if it was actually useful.
  • I think to make it useful it could have some default plugins that are generally useful… however, you will have to figure out, or do some research on what ppl would actually use
2 Likes

For the gui, why not something like blender where there is multiple viewports and one plugin for each viewport, and each viewport with a toggle button to choose the plugin, or maybe im asking to much, lol.

2 Likes

Here is how I am doing for my Gui app states, I hope you find the idea useful. :slightly_smiling_face:

It’s very simple, I have a “ControlPanelState” which I use to register my Gui app states into it. Everything is extended from JME’s “BaseAppState”.

Here is the entry point into the control panel, a button on the top center of the screen :upside_down_face:

The control panel open with bunch of registered Gui app states:

at some point, I will switch it to use a multi-column ListBox to display plugins.

Here is how the code looks like:

/**
 * Main entry point to all GUI app states. AppStates can register themselves
 * so their enabling/disabling can be controlled from within control panel.
 *
 * @author Ali-RS
 */
public class ControlPanelState extends BaseAppState {

    // Keep track of registered app state id's to be displayed in settings panel
    private final SafeArrayList<String> ids = new SafeArrayList<>(String.class);

    // Contains registered Guis
    private Container controlPanel;
    // A closable/minimizable window for displaying contents of enabled Gui
    private Container mainWindow;
    // Contains widgets on right side of the screen
    private Container widgetContainer;
    // Current active Gui, only one at a time
    private AppState enabledState;

    private Panel contents;
    private HAlignment hAlignment;
    private VAlignment vAlignment;

    private boolean minimized = false;

    public ControlPanelState() {
        super("Control Panel");
    }

    @Override
    protected void initialize(Application app) {
        // FIXME: Use a proper icon for control panel button
        Button controlPanelToggleBtn = new Button("         ");
        controlPanelToggleBtn.addClickCommands(src -> {
            if (minimized) {
                setMinimised(false);
            } else if (enabledState == null) {
                toggleControlPanel();
            }
        });

        getState(SceneGraphState.class).getGuiNode().attachChild(controlPanelToggleBtn);
        alignPanel(controlPanelToggleBtn, HAlignment.Center, VAlignment.Top);

        controlPanel = new Container();
        mainWindow = new Container(new BorderLayout());
        widgetContainer = new Container(new BoxLayout());

        Container buttons = new Container(new BoxLayout(Axis.X, FillMode.Even));
        Button minimizeBtn = buttons.addChild(new Button("_"));
        minimizeBtn.setTextHAlignment(HAlignment.Center);
        minimizeBtn.addClickCommands(src -> {
            setMinimised(true);
        });
        Button closeBtn = buttons.addChild(new Button("x"));
        closeBtn.setTextHAlignment(HAlignment.Center);
        closeBtn.addClickCommands(src -> {
            enabledState.setEnabled(false);
            enabledState = null;
        });

        mainWindow.addChild(buttons, BorderLayout.Position.North);
        refresh();
    }

    @Override
    protected void cleanup(Application app) {
    }

    @Override
    protected void onEnable() {
        getState(SceneGraphState.class).getGuiNode().attachChild(widgetContainer);
    }

    @Override
    protected void onDisable() {
        widgetContainer.removeFromParent();
    }

    @Override
    public void update(float tpf) {
        if (controlPanel.getParent() != null) {
            alignPanel(controlPanel, HAlignment.Center, VAlignment.Center);
        }

        if (mainWindow.getParent() != null) {
            alignPanel(mainWindow, hAlignment, vAlignment);
        }

        alignPanel(widgetContainer, HAlignment.Right, VAlignment.Center);

        if (enabledState != null && !enabledState.isEnabled()) {
            enabledState = null;
        }
    }

    public float getStandardScale() {
        return getApplication().getCamera().getHeight() / 720f;
    }

    /**
     * Apply standard scale and align panel
     */
    public void alignPanel(Panel panel, HAlignment ha, VAlignment va) {
        Camera cam = getApplication().getCamera();

        // Apply standard scale
        float scale = getStandardScale() * 1.2f;
        panel.setLocalScale(scale);

        int width = cam.getWidth();
        int height = cam.getHeight();

        Vector3f prefSize = new Vector3f(panel.getPreferredSize());
        prefSize.multLocal(scale);

        float x = 0;
        float y = 0;
        // Align panel
        switch (ha) {
            case Center:
                x = (width - prefSize.x) / 2;
                break;
            case Left:
                x = 0;
                break;
            case Right:
                x = (width - prefSize.x);
                break;
        }

        switch (va) {
            case Center:
                y = (height + prefSize.y) / 2;
                break;
            case Bottom:
                y = prefSize.y;
                break;
            case Top:
                y = height;
                break;
        }

        Vector3f translation = panel.getLocalTranslation();
        if (translation.x != x || translation.y != y) {
            panel.setLocalTranslation(x, y, translation.z);
        }
    }

    /**
     * Registers an app state into the control panel. The state id will be used for panel name.
     *
     * @throws IllegalArgumentException if state has no id specified to it.
     */
    public <T extends AppState> void register(T state) {
        if (state.getId() == null) {
            throw new IllegalArgumentException("AppState has no id.");
        }
        ids.add(state.getId());
        refresh();
    }

    public boolean remove(AppState state) {
        if (state.getId() == null) {
            throw new IllegalArgumentException("AppState has no id.");
        }

        boolean removed = ids.remove(state.getId());
        if (removed) {
            refresh();
        }
        return removed;
    }

    public void show(Panel window, HAlignment hAlignment, VAlignment vAlignment, Panel... widgets) {
        showWindow(window, hAlignment, vAlignment);
        showWidgets(widgets);
    }

    public void close(Panel window, Panel... widgets) {
        closeWindow(window);
        closeWidgets(widgets);
    }

    public void showWindow(Panel contents, HAlignment hAlignment, VAlignment vAlignment) {
        if (this.contents != null) {
            closeWindow(this.contents);
        }

        if (contents != null) {
            this.contents = contents;
            this.hAlignment = hAlignment;
            this.vAlignment = vAlignment;
            mainWindow.addChild(contents, BorderLayout.Position.Center);
            getState(SceneGraphState.class).getGuiNode().attachChild(mainWindow);
            contents.runEffect(Panel.EFFECT_OPEN);
            GuiGlobals.getInstance().requestFocus(mainWindow);
        }
    }

    public void closeWindow(Panel contents) {
        if (Objects.equals(this.contents, contents)) {
            this.contents.runEffect(Panel.EFFECT_CLOSE);
            mainWindow.removeFromParent();
            mainWindow.removeChild(contents);
            GuiGlobals.getInstance().releaseFocus(mainWindow);
            this.contents = null;
            hAlignment = null;
            vAlignment = null;
        }
    }

    public void showWidgets(Panel... widgets) {
        for (Panel widget : widgets) {
            widgetContainer.addChild(widget);
        }
    }

    public void closeWidgets(Panel... widgets) {
        for (Panel widget : widgets) {
            widgetContainer.removeChild(widget);
        }
    }

    private void toggleControlPanel() {
        if (controlPanel.getParent() == null) {
            getState(SceneGraphState.class).getGuiNode().attachChild(controlPanel);
        } else {
            controlPanel.removeFromParent();
        }
    }

    private void setMinimised(boolean minimized) {
        this.minimized = minimized;
        if (minimized) {
            mainWindow.removeFromParent();
            contents.runEffect(Panel.EFFECT_CLOSE);
        } else {
            getState(SceneGraphState.class).getGuiNode().attachChild(mainWindow);
            contents.runEffect(Panel.EFFECT_OPEN);
        }
    }

    private void refresh() {
        if (isInitialized()) {
            controlPanel.clearChildren();
            int index = 0;
            for (String id : ids) {
                // FIXME: No hardcoded values
                int x = index / 3;
                int y = index % 3;
                Button toggleBtn = controlPanel.addChild(new Button(id), x, y);
                toggleBtn.setTextHAlignment(HAlignment.Center);
                toggleBtn.addClickCommands(source -> {
                    // Close ControlPanel main window
                    toggleControlPanel();
                    // Enable gui app state
                    enabledState = getStateManager().stateForId(id, AppState.class);
                    enabledState.setEnabled(true);
                });

                index++;
            }
        }
    }
}

So basically it gives me two panels for displaying stuff. A main window with close/minimize action and panel for adding widgets.

Only one window can be enabled at a time but there can be multiple widgets.

Let me show it in action. For example here is how my “Scene Editor” looks:

You can see the main window on the left side (it can be aligned by providing a HAlignment and a VAlignment) and widgets on the right side.

For the main window, I mostly use Lemur “TabbedPanel” and for widgets, I use Lemur “RollupPanel”.

So for a Gui app state to be listed on control panel I need to register it into the control panel. I do this from the initialize() method of the Gui app state:

    @Override
    protected void initialize(Application app) {
        

        ...
        

        getState(ControlPanelState.class).register(this);
    } 

    @Override
    protected void cleanup(Application app) {
        getState(ControlPanelState.class).remove(this);
    }

Now it will be listed on the control panel. When toggled, control panel is going to call it’s enable/disable method.

now on enable, a Gui app state can call the show method from “ControlPanelState” to display it’s stuff:

protected void onEnable() {
        getState(ControlPanelState.class).show(tabs, HAlignment.Left, VAlignment.Center, widgets);

        ...

}
@Override
    protected void onDisable() {
        getState(ControlPanelState.class).close(tabs, widgets);

        ... 

       
    }

now say I also want to use an external widget, for example I want to use “TransformWidget” from my Scene Editor, I just need to call this from the Scene Editor:

getState(TransformState.class).setEnabled(true);

now when the “TransformState” gets enable it can add it’s widgets into the control panel:

    @Override
    protected void onEnable() {
         ....
         
         getState(ControlPanelState.class).showWidgets(transformWidget);
    }
    @Override
    protected void onDisable() {
        ...

        getState(ControlPanelState.class).closeWidgets(transformWidget);
    }

This is how it looks like in action:

Sorry that it got abit long, hope you find it useful :slightly_smiling_face:

3 Likes

I’m not a UI Guru either, but I do have some experience working with a plugin-based editor. (Art of Illusion, artofillusion.org, if you want to see what I’m talking about.)

The overall arrangement of the UI would, I’d think, be partly dictated by the logical arrangement of the data that you are trying to edit.

A lot of the types of plugins that you have mentioned seem that they would probably manage data on a per-game-object basis. For example: Physics.

While I would expect some global/per scene/per physics-space options, Most of the interesting things for such a plugin would only make sense applied to an object or group of objects. I’d expect to edit these by:

  • Select my game object (Domino1|6 for example) In the main/home UI.
    • This gets me some sort of context menu, with options to edit (just a few that I think would be common):
      • Texture/Material
      • Edit Display Mesh (The com.jme.scene.Mesh that gets sent to the GPU)
      • Physics Properties
      • &tc, &tc. Whatever other plugins provide
  • Edit Physics Properties
    • This replaces the ‘Home’ UI with one that can edit Physics stuff, such as:
      • Generate of modify collision shapes
      • Mass/Center of Mass/Mass distribution
      • Deformation modes (Hard/soft/firm body settings, ragdoll or modified ragdoll rig-sets)
      • Surface Coefficient of Friction, Coefficient of Restitution, &tc.
  • Editing would be Modal. I cannot go back to the ‘home’ UI Unless I either accept or cancel my physics changes. This prevents multiple plugins from accidentally stepping on each other’s toes if they need to look at the same data.

Regarding the actual Plugin API, two thoughts:

  • Make sure that you know how to handle conflicts between third-party dependencies. EG: PluginA requires foo 1.8 and PluginB requires foo 2.1, which is not API compatible with 1.8. This can be done. Glad to explain how AOI approaches it, if you are interested.
  • Dogfood the plugin API. Build at least one or two plugins yourself. One way to do this is to structure the core tools (Stuff that needs to be available no matter what) as plugins.
2 Likes

Thanks for the input ladies/gents - I’ll try to digest it as best I can. I appreciate you all spending the time to write such exhaustive details.

2 Likes

+1
Separate window on second monitor. This would also integrate easier with VR apps.

So here’s a little view of it in action. Nothing special really, but shows 3 plugins (two of which just show labels) and them being selected.

Each plugin contains its own “state” which is resumed when you click on it again. Creating a plugin is super simple. It requires a plugin.json and a class that extends DesktopPlugin like below. There are a few utilities that will be provided - such as a EditorCameraState and EditorCameraGUI (to adjust the camera speeds, etc) - and I guess as time goes by more will be added - like a transform tool and all the other stuff.

So from the perspective of “I want x plugin to display images in a window or whatever” - it’s entirely up to the developer. Which is great in a lot of respects - but combined with some “built in tools” like a project assets treeview - it should be really easy to make it however you want it. You can literally do anything that you could do in JME.

Some cool config settings so users can define the camera frustum, background color, and all the other things would be nice, The “slash command” console, too…

It seems pretty cool thus far :slight_smile:

plugin.json

{
  "main": "com.jayfella.testplugin.TestPlugin",
  "name" : "TestPlugin",
  "version" : "1.0",
  "description" : "A simple test plugin",
  "website" : "https://jmonkeyengine.org",
  "prefix" : "TEST",
  "apiVersion" : "1.0",
  "authors": [ "jayfella", "a monkey", "another monkey" ],
  "softDepend" : [ "Something" ]
}

TestPlugin.java

package com.jayfella.testplugin;

import com.jayfella.pluginsystem.plugin.desktop.DesktopPlugin;
import com.jme3.app.SimpleApplication;
import com.simsilica.lemur.Label;

public class TestPlugin extends DesktopPlugin {

    public TestPlugin() {
        super();

        // getLogger().info("I have been constructed.");
    }

    @Override
    public void onLoad() {
        // getLogger().info("I have been loaded.");
    }

    @Override
    public void onEnable() {
        // getLogger().info("I have been enabled.");
    }

    @Override
    public void onDisable() {
        // getLogger().info("I have been disabled.");
    }

    Label label;

    @Override
    public void onSelected() {

        if (label == null) {
            label = new Label("I am active: " + getName());
            label.setLocalTranslation(200, 300, 1);
        }

        SimpleApplication simpleApplication = (SimpleApplication) getJmeApplication();
        simpleApplication.getGuiNode().attachChild(label);

    }

    @Override
    public void onUnselected() {

        if (label != null) {
            label.removeFromParent();
        }

    }

}

9 Likes

I am really excited about the idea of this!

Just a small note to mention I’m still working away. I’ve written a horizontal menu system and draggable window system, and want to also add a context menu system, too.

A lot of the work right now is purely API based.

To test it all I’m writing a model importer plugin, skybox creator plugin and a lightprobe generator.

I’ve also figured out how to live debug plugins - which is obviously very important.

Hopefully I’ll post a video of it all in action over the next few days.

The API is still pretty volatile right now, so I won’t bother releasing anything yet, but I should think by time I’ve finished a few plugins it should mostly be where I want it to be.

Everything is going pretty well. It’s more a case of just writing code than anything else. I’m looking forward to seeing others come up with some great plugins in the future!

5 Likes

As promised a video of the current progress. I had to create everything from scratch so it just takes time to make things ready for public use. It becomes apparent that the more I use it, the more I want to make more plugins - for example I’d really like a scenebuilder of sorts so I can load models and position them wherever I want. I’d like to make a Lemur GUI builder, too. Some kind of physics tools would also be nice.

I’ve started to create a dark theme. Not my strongest area but nevertheless it’s going to allow for customization, so there’s that.

I added a splash/loading screen just because it’s better than waiting for what might be quite a while once plugins become numerous.

I quite like the plugin slider on the side. It doesn’t take up any room at all once it’s collapsed. Plugins can create draggable windows and interact with the menu system - meaning you can add menu items for your plugin.

I’m still missing the slash command console. I’ll probably add that soon.

Any opinions are as always very welcome :slight_smile:

4 Likes

@jayfella, this is awesome.
Not priority, but maybe the gui could be spiced up, maybe something such as this

And another thing, this is like an editor rather than an IDE, but still maybe you should do like Godot engine and still have some support for scripting, maybe Groovy or Kotlin, that way people can still spice up smaller details like Lemur GUI. The more important and core Java code will be edited in the user’s IDE of choice of course.

Just to check in, even in two videos, you’ve shown how far the project has come, so how much more work do you think needs done before maybe uploading to a gh repo?

How easy do you expect it to be to maintain?

What I mean is the SDK is pretty complicated to contribute to, very advanced coding skills are required imo.

I would hope since it is all native jME related code it wouldn’t be any harder to contribute to then using jME would be.

The issue with uploading it to github too early is that if people start using it and the API changes, all plugins won’t work. That will probably alienate a lot of talent from the whole thing. It’s still too volatile right now. I’m writing plugins myself, too, so I guess once I’m satisfied the API is stable.

As for maintenance, Its all JME. All GUI is lemur. What I mean by that is that virtually all the code other than the plugin manager should be familiar to the eye. It’s been split into modules, too (menu, window, theme, plugins, etc) rather than one huge single project.

If I said maybe a month to six weeks I think we’d be somewhere in the ballpark.

2 Likes