JME3 Swing3d GUI

User Code & Projects
1

After searching around for a gui system that fits my needs, I came to the conclusion that a swing jme bridge is the best solution for the moment.



Performance is not the greatest as everyone would guess, however it is with a 512x512 panel renderd onto a quad with a gforce 750 (integratet shit card), In a grayscale mode it works faster, having with 1024x1024 pixels still over 100fps.

around 100 fps.



Left frame renderd into right bufferdimage, displayed in frame for debugging, renderd onto jme texture on the 2x2 quad.



(Due to the fact, that the texture automatic antialiases the pixel amount is not that important as you see)





Next thing, is a inputsystem, that simulates a mouse by using the aimvector, as well as keyboard input (and maybee a better look and feel)

Also planned is switchig to passive rendering, so the slow update only is called when it is really needed.



package de.empirephoenix.client.gui;

import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.GraphicsConfiguration;
import java.awt.GraphicsDevice;
import java.awt.GraphicsEnvironment;
import java.awt.image.BufferedImage;
import java.awt.image.DataBuffer;
import java.awt.image.DataBufferByte;
import java.awt.image.DataBufferInt;
import java.awt.image.WritableRaster;
import java.nio.ByteBuffer;

import javax.swing.CellRendererPane;
import javax.swing.ImageIcon;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.SwingUtilities;

import com.jme3.material.Material;
import com.jme3.scene.Geometry;
import com.jme3.scene.VertexBuffer.Type;
import com.jme3.scene.shape.Quad;
import com.jme3.texture.Image;
import com.jme3.texture.Texture2D;
import com.jme3.texture.Image.Format;

import de.empirephoenix.client.Core;

public class GuiMaster extends Geometry{
   final public static Format jmeformat = Format.RGBA8;
   final public static int awtformat = BufferedImage.TYPE_3BYTE_BGR;
   
   private GuiMaster(String string, Quad q) {
      super(string, q);
   }
   
   private JPanel child = new JPanel();
   private Texture2D rendertex;
   private int xpsize;
   private int ypsize;
   private Material mat;
   private BufferedImage image;
   private Graphics2D g2d;
   private CellRendererPane sr;
   private ImagePanel ip;
   private ByteBuffer buffer;
   private Image jimage;
   
   public JPanel getContainer(){
      return child;
   }
   
   public static GuiMaster getGui(float x,float y,int scale){
      Quad q = new Quad(x,y,true);

      
      GuiMaster gui = new GuiMaster("Gui ", q);
      //per meter use scale pixels
      gui.xpsize = (int)x*scale;
      gui.ypsize = (int)y*scale;
      System.out.println("Size " + gui.xpsize + "X" + gui.ypsize);
      gui.child.setSize(gui.xpsize,gui.ypsize);
      gui.rendertex = new Texture2D(gui.xpsize, gui.ypsize, jmeformat);
      gui.buffer = ByteBuffer.allocateDirect(gui.xpsize*gui.ypsize*4);
      gui.jimage = new Image();
      gui.jimage.setFormat(jmeformat);
      gui.jimage.setWidth(gui.xpsize);
      gui.jimage.setHeight(gui.ypsize);
      gui.rendertex.setImage(gui.jimage);
      
      gui.image =   new BufferedImage(gui.xpsize,gui.ypsize,awtformat);
      gui.g2d = gui.image.createGraphics();   
      gui.sr =new CellRendererPane();
      gui.sr.add(gui.child);
      
      gui.mat = new Material(Core.getManager(), "Common/MatDefs/Misc/SimpleTextured.j3md");
      gui.mat.setTexture("m_ColorMap",gui.rendertex);
      gui.setMaterial(gui.mat);
      
       JFrame f = new JFrame();
       f.add(gui.child);
       f.setVisible(true);
       f.setSize(gui.xpsize,gui.ypsize);
       f.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
       
       JFrame f2 = new JFrame();
       gui.ip = new ImagePanel();
       gui.ip.setImage(gui.image);
       f2.setLocation(gui.xpsize+50,0);
       f2.add(gui.ip);
       f2.setVisible(true);
       f2.setSize(gui.xpsize,gui.ypsize);
       f2.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
      return gui;
   }
   
   public void update(){
      this.child.paintAll(g2d);
      this.ip.setImage(image);
      
      DataBufferByte buf =  (DataBufferByte) image.getData().getDataBuffer();
      this.buffer.rewind();
      byte[] data = buf.getData();
      for(int i = 0; i<data.length;i+:3){
         byte b = data[i];
         byte g = data[i+1];
         byte r = data[i+2];
         
         this.buffer.put(r);
         this.buffer.put(g);
         this.buffer.put(b);
         this.buffer.put(Byte.MAX_VALUE);
      }
      
      
      this.jimage.setData(this.buffer);
      this.rendertex.setUpdateNeeded();
   }
}


2

This is pretty cool. I wonder if this can be made more efficent/powerful, maybe by somehow making a jME implementation of the Graphics2D interface?

3

Depends, I workd a bit more on this, and the non done version, supports also a monochrome mode, wich has the advantage, that the imagebuffer raster data byte array, can be directly copyed as a depth format texture data, really doing wonders on higher texture resolution, (100fps with 4k texture should be good enough)



The conversion however allows me to do some specials, I plan for the next version, to alllow a collor filter, so you can for example set te background colors of your frames as transparent and let jme render trough.



Also first test with switching to passive rendering show great results, but of course are only helping with unanimated stuff.



Also planned a cursor, that is render ontop of the quad, steered by raycollision checks from the cam and simulates events on the swing components.

4

Have you got some good links to swing gui if someone (like me) would like to read up on it? My google results returned very different things based on only slightly different criterias.

5
erlend_sh said:

Have you got some good links to swing gui if someone (like me) would like to read up on it? My google results returned very different things based on only slightly different criterias.


Swing is one of the Java windowing toolkits..  What Empire is doing is similar to the old JMEDesktop functionality.

http://java.sun.com/docs/books/tutorial/ui/overview/intro.html
6

yeah , except this one works on nearly every configuration, and shouldn't have strange bugs, as it paints to a bufferdimage wich is java standart, while the old one when I remind right uses some tricks to get it to a jme texture.



For input:

Modes i think should be usefull are:

-Full3d, where you aim is where a mouse is simulated, if you click a textfield it catches the key input untill enter is pressed(or right mouse button)

-Zoom3d, when you click it, your camera zooms to a fullscreen mode, that behaves identically to a normal 2d gui, gui has to take care when to release the player back into the game.

-2d Gui mode, used for gui node, mouse can be toggled on over a specifiyable key (or if not specified only display data), if mouse is on behaves like a normal swing gui, else it just displays stuff.

-Custom, you tell it over a method where in (3d) you want to fire wich event, it then calcualtes that data to a 2d plane, and uses the next element (with a few pixels defference at max) and fires that event.  (For example if you want to have a gui that reacts to physic node collision (why ever)

7

Swing’s shitty browser on a grayscale gui panel using 2000X2000 pixels, showing the jme blog with halfpassive rendering

fps is around 50, level is using normalmapping + parralaxmapping (both really kill these internal gc)

8

Does this mean you've gotten input working and clicked to the page or was the URL hardwired?



Check out this: The DJ project  I've heard some good things from people

9

yeah, it si great, and can be used in combination with the swing jme bridge.

Actually I just don't care about because it is lgpl I prefer licenses wich does not restrict me to much, and decide by myself what I want to share with all others, and what not.

10

Performance might be really good with java 7, they are allowing the mixing of hevyweights and lightweights

11

Well the largest performance loss is actually the converting to a jme texture, however while probably being the slowest gui system, it is still fst enough that way to be practically useable.

12

I know that Swing/AWT uses dirty rectangles, so it's possible to update only a relevant part of the texture?

13

Yes, I still search for a way to do that, but then I have to manully copy onlyparts into the data ByteArray, wich may be even slower thatn jsut dumping the whole array.(in color mode however it might be faster again^, well i still have much to do before this is finished.

14

I'm also researching on this topic and i think the best way to do this, is to implement a headless graphics environment like pja does(eTeks : PJA Toolkit). All other solution would be mean either to extend all swing classes and override all paint methods or to draw the  complete root swing component and lose performance.

15
Lutherion said:

I'm also researching on this topic and i think the best way to do this, is to implement a headless graphics environment like pja does(http://www.eteks.com/pja/en/). All other solution would be mean either to extend all swing classes and override all paint methods or to draw the  complete root swing component and lose performance.


If only swing were performant
16
theprism said:

If only swing were performant

Yeah, personally I really dont see why I would want to have Swing or SWT *in* my openGL window. They won't look, handle or behave any better when rendered by a GPU ;)
17

Cause the are widged based and have a real documentation also it's basics are easy to understand.

(something I really miss in nifty, and probably others too if I look on the count of simple problems regarding it)

18

I think project wonderland uses a swing bridge too,don't they? Perhaps there's a way to get open sources there. I will search for sources there. http://www.openwonderland.org/

19
Lutherion said:

I think project wonderland uses a swing bridge too,don't they? Perhaps there's a way to get open sources there. I will search for sources there. http://www.openwonderland.org/
Good point. They're a friendly bunch, I'm sure you could get some pointers by just inquiring on their open mailing list as well:
http://groups.google.com/group/openwonderland
20

Great work! Would be also interrested in that solution for the same reasons + WYSIWYG Editors on top of this.

Atm. its the main thing keeping me on jme2…