JavaFX embedded in jme3

tuffe · October 29, 2014, 2:20pm

@Empire Phoenix said: I would suggest to make it an interface, and allow users to push via guimanager their own custom logic. -> special thinking here is required as for usefull checks they would need access to jfx, but events are in jme thread.

Yeah but I’m mostly thinking about it for my own sake. What is the best custom logic to use?

Btw Empire Phoenix, I like how your avatar looks terrified or sleep-deprived (is that what you look like after year of working on your game?) while mine looks like a troll face.

Empire_Phoenix · October 29, 2014, 5:20pm

Hm the best approach for all cases (and yours)
would be

To query the jfx components actually for the position, and check using their mousetransparent & disabled property if there is something that wants input.

abies · October 29, 2014, 6:30pm

Problem is that I don’t think you can query jfx components from jme3 thread - but you need to know if event should be consumed or not immediately in jme3 thread. Otherwise, you will introduce few frames of input lag (jfx is running at 30fps internally I think, so we are talking about 30-50ms latency).

Empire_Phoenix · October 29, 2014, 11:12pm

Hm what might be possible, to generate/update actually a kinda like boolean bitmap in the jfx thread, and use it to fast query from jme thread.
Question is how fast it could be generated.

tuffe · October 31, 2014, 9:32am

I’m trying to figure out why MOUSE_DRAGGED events don’t quite work. What is the scenePeer? Some mysterious object.

Edil_Fernando · November 28, 2014, 9:00am

Hi folks! First off, the work you’ve done with JME-JFX is awesome!! I setup the sources in Eclipse and ran the TestMovie app. It works flawlessly with the FLV file from Oracle but with a MP4 [http://techslides.com/demos/sample-videos/small.mp4] the colours in the video [https://drive.google.com/file/d/0B1dPkfPAZHNFa3pvVVVSNUxPRGs/view?usp=sharing] are messed up. Also, after a few seconds the app crashes with a java.lang.IllegalArgumentException. The same is true for all MP4 files I’ve thrown at it.

INFO: Audio effect extension version: 1.0
Nov 27, 2014 6:33:13 PM com.jme3.audio.lwjgl.LwjglAudioRenderer initInThread
INFO: Audio max auxilary sends: 4
Nov 27, 2014 6:33:34 PM com.jme3.app.Application handleError
SEVERE: Uncaught exception thrown in Thread[LWJGL Renderer Thread,5,main]
java.lang.IllegalArgumentException: Number of remaining buffer elements is 6627840, must    be at least 14745600. Because at most 14745600 elements can be returned, a buffer with at least 14745600 elements is required, regardless of actual returned element count
at org.lwjgl.BufferChecks.throwBufferSizeException(BufferChecks.java:162)
at org.lwjgl.BufferChecks.checkBufferSize(BufferChecks.java:189)
at org.lwjgl.BufferChecks.checkBuffer(BufferChecks.java:230)
at org.lwjgl.opengl.GL11.glTexImage2D(GL11.java:2845)
at com.jme3.renderer.lwjgl.TextureUtil.uploadTexture(TextureUtil.java:352)
at com.jme3.renderer.lwjgl.LwjglRenderer.updateTexImageData(LwjglRenderer.java:1913)
at com.jme3.renderer.lwjgl.LwjglRenderer.setTexture(LwjglRenderer.java:1936)
at com.jme3.material.MatParamTexture.apply(MatParamTexture.java:86)
at com.jme3.material.Material.render(Material.java:1088)
at com.jme3.renderer.RenderManager.renderGeometry(RenderManager.java:523)
at com.jme3.renderer.queue.RenderQueue.renderGeometryList(RenderQueue.java:322)
at com.jme3.renderer.queue.RenderQueue.renderQueue(RenderQueue.java:374)
at com.jme3.renderer.RenderManager.renderViewPortQueues(RenderManager.java:763)
at com.jme3.renderer.RenderManager.flushQueue(RenderManager.java:719)
at com.jme3.renderer.RenderManager.renderViewPort(RenderManager.java:983)
at com.jme3.renderer.RenderManager.render(RenderManager.java:1029)
at com.jme3.app.SimpleApplication.update(SimpleApplication.java:252)
at com.jme3.system.lwjgl.LwjglAbstractDisplay.runLoop(LwjglAbstractDisplay.java:151)
at com.jme3.system.lwjgl.LwjglDisplay.runLoop(LwjglDisplay.java:185)
at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:228)
at java.lang.Thread.run(Thread.java:745)

The video playback is very critical to this 3D application I’m working on. I’m unable to make sense of these errors as I’m a n00b when it comes to graphics development. Any and all help will be appreciated mateys!

david_bernard_31 · November 28, 2014, 9:51am

Hi, I can reproduce it randomly. I should push a fix today or this WE. If you can open a issue on github (it will easy the tracking/history).

Edil_Fernando · November 28, 2014, 10:41am

Opened an issue. Thanks for the help buddy. If it means anything I really wanna stick with Java and JME3. It is beautiful, easy and productive enough for me to have already finished most of the 3D aspects of my work. The video is the content part of it and I really don’t wanna consider alternatives just for video playback issues. Also, found a pretty old link about WebM support in JavaFX, not sure if it is still relevant today and won’t be surprised if Oracle ignores open formats but with JavaFX beginning to be open sourced, maybe we’ll get to see WebM playback in JME3 someday?

Empire_Phoenix · November 28, 2014, 12:39pm

Hm I cannot reproduce the crash on my systems, hopefulle David can figure out the reason.

My wild guess would be some issue with how the image gets converted.

Edil_Fernando · November 29, 2014, 7:03am

The fix provided has resolved both issues. David is a superhero deserving of his own line of action figures!

david_bernard_31 · November 29, 2014, 10:03am

Thanks, but the color issue was introduced by my path for jme 3.0 compatibility.

Empire_Phoenix · December 16, 2014, 11:03am

So,

I finally got some time to implement a very nice feature that is only possible thanks to the underlying jfx.
Basically you can now externalize every AbstractWindow based ui.

Uses:
Multimonitor useage
Chatlog
GameConsole
SpaceShipcomputer IDE

How it works:
-> setExternalizable(true)
adds a new button to the WindowIcons that allow externalizing and internalizing.
-> setExternalized(boolean)
directly externalzied a window. does also work for non visible ones, upon attaching they will be directly external then.

Current Issues
-> Styling for the external window is hardcoded currently
If anyone has ideas how to do a unified styling for internal and external windows, so they look alike but are css adjustable suggestions would be nice
-> External window is not resizable currently, i guess all current operation systems use resizing on border drag, I will try to implement this over the holidays.

It is probably a bit rough on the edges currently, but feedback would be appreciated.

Edil_Fernando · December 17, 2014, 3:39pm

Hi people, I’m back. Post video playback (intro video), I tried to do a rootNode.detachAllChildren() and attach a new Spatial, I got to stare at a blank screen. Minus the video playback am able to attach the Spatial just fine. There are no abnormal error messages or exceptions raised. Am I missing anything super simple?

david_bernard_31 · December 17, 2014, 3:58pm

Do you have a sample test to reproduce ?

Edil_Fernando · December 18, 2014, 8:54am

I apologize for the delay. Here, is the test to reproduce :

package mygame;

import java.io.File;

import javafx.beans.value.ChangeListener;
import javafx.beans.value.ObservableValue;
import javafx.scene.media.Media;
import javafx.scene.media.MediaException;
import javafx.scene.media.MediaPlayer;

import com.jme3.app.SimpleApplication;

import com.jme3.light.DirectionalLight;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Vector3f;
import com.jme3.niftygui.NiftyJmeDisplay;
import com.jme3.renderer.RenderManager;
import com.jme3.scene.Geometry;
import com.jme3.scene.Node;
import com.jme3.scene.shape.Quad;
import com.jme3x.jfx.media.TextureMovie;
import com.jme3x.jfx.media.TextureMovie.LetterboxMode;
import com.sun.javafx.application.PlatformImpl;

import de.lessvoid.nifty.Nifty;
import de.lessvoid.nifty.screen.Screen;
import de.lessvoid.nifty.screen.ScreenController;

public class Test extends SimpleApplication implements ScreenController {  
       
    private Node brain;  
    private Nifty nifty;    
    private TextureMovie textureMovie;
    private MediaPlayer mp;

    
    public static void main(String[] args) {
    	
    	PlatformImpl.startup(() -> {
		});        
        
        Test app = new Test();
        app.start();
    }      

    @Override
    public void simpleInitApp() {
    	
    	flyCam.setEnabled(false);
        inputManager.setCursorVisible(true);
        setDisplayStatView(false); 
    	  
    	this.setVideoPlayer("/Videos/intro.mp4");
           
    	this.mp.setOnEndOfMedia(new Runnable() {
    	    @Override
    	    public void run()
    	    {
    	    	NiftyJmeDisplay niftyDisplay = new NiftyJmeDisplay(assetManager,
                        inputManager,
                        audioRenderer,
                        guiViewPort);
    	    	nifty = niftyDisplay.getNifty();    	    	
    	    	nifty.fromXml("Interface/dummy.xml", "start", Test.this);    	    

    	    	// attach the nifty display to the gui view port as a processor
    	    	guiViewPort.addProcessor(niftyDisplay);   	    	
    	   }
    	});
    }

    
    @Override
    public void simpleUpdate(float tpf) {
        //TODO: add update code       
    }

    @Override
    public void simpleRender(RenderManager rm) {
        //TODO: add render code
    }  
    
    
    
    private void setupLights() {       
        
        DirectionalLight front = new DirectionalLight();
        front.setDirection(new Vector3f(0f, 20f, 10f));
        //front.setColor(new ColorRGBA(0.75f,0.75f,0.75f,0));
        front.setColor(ColorRGBA.White);
        rootNode.addLight(front);
           
        DirectionalLight bottom = new DirectionalLight();
        bottom.setDirection(new Vector3f(0f, 40f, 0f));
        bottom.setColor(new ColorRGBA(0.5f,0.5f,0.5f,0)); 
        //bottom.setColor(ColorRGBA.White);
        rootNode.addLight(bottom);
        
        DirectionalLight top = new DirectionalLight();
        top.setDirection(new Vector3f(0f, -40f, 0f));
        top.setColor(ColorRGBA.White);
        rootNode.addLight(top);
       
    }  
  
  private void loadBrain() {	  
      
      brain = (Node) assetManager.loadModel("Models/brainupdate.j3o");     
      
      brain.setLocalScale(10f);       
      brain.move(0f,-1.5f,0f);    
      
      this.setupLights();  
     
      rootNode.detachAllChildren();
      rootNode.attachChild(brain);     
  }  
  
  @Override
  public void bind(Nifty nifty, Screen screen) {
	  System.out.println("bind( " + screen.getScreenId() + ")");
  }

  @Override
  public void onStartScreen() {
      System.out.println("onStartScreen");
  }

  @Override
  public void onEndScreen() {
      System.out.println("onEndScreen");
  }

  public void quit(){
      nifty.gotoScreen("end");
  }
     
  private void setVideoPlayer(String videoPath) {
   	  
  	  final Media media = new Media(new File(this.getClass().getResource(videoPath).getFile()).toURI().toASCIIString());
         
  	  media.errorProperty().addListener(new ChangeListener<MediaException>() {

   			@Override
   			public void changed(final ObservableValue<? extends MediaException> observable, final MediaException oldValue, final MediaException newValue) {
   				newValue.printStackTrace();
   			}
  	  });
  	  this.mp = new MediaPlayer(media);		
  	  this.mp.play();	

  	  this.textureMovie = new TextureMovie(this, this.mp, LetterboxMode.VALID_LETTERBOX);
  	  this.textureMovie.setLetterboxColor(ColorRGBA.Black);

  	  final Geometry screen1 = new Geometry("Screen1", new Quad(20,20));
  	  final Material s1mat = new Material(this.assetManager, "Common/MatDefs/Misc/Unshaded.j3md");
  	  s1mat.setTexture("ColorMap", this.textureMovie.getTexture());
  	  screen1.setMaterial(s1mat);
  	  this.rootNode.attachChild(screen1);			

  	  this.cam.setLocation(new Vector3f(10, 10, 13.5f));
   	  
  }    
     
  public void test() {     
      this.loadBrain();        
      nifty.gotoScreen("end");
  }   
  
  @Override
  public void destroy(){
      super.destroy();          
      this.mp.stop();
      PlatformImpl.exit();
  }
}

…and the corresponding screen layout :

[xml] <?xml version="1.0" encoding="UTF-8"?>

<nifty xmlns="http://nifty-gui.sourceforge.net/nifty-1.3.xsd">
	<useStyles filename="nifty-default-styles.xml" />
  	<useControls filename="nifty-default-controls.xml" />
	<screen id="start" controller="mygame.Test">
    	<layer id="layer" backgroundColor="#0008" childLayout="center">
      <panel id="panel" height="25%" width="35%" align="center" valign="center" backgroundColor="#1bbc9d" childLayout="center" visibleToMouse="true">
        <interact onClick="quit()"/>
        <effect>
          <onStartScreen name="move" mode="in" direction="top" length="300" startDelay="0" inherit="true"/>
          <onEndScreen name="move" mode="out" direction="bottom" length="300" startDelay="0" inherit="true"/>
          <onHover name="pulsate" scaleFactor="0.008" startColor="#1bbc9d" endColor="#ffff" post="true"/>
        </effect>
        <text id="text" font="aurulent-sans-16.fnt" color="#000f" text="Click here..." align="center" valign="center" />
      </panel>
    </layer>
  	</screen>
  	<screen id="end">
  </screen>  	
</nifty> [/xml]

david_bernard_31 · December 18, 2014, 9:08am

Hi,

Thanks for the sample, but a standalone sample packaged archive (zip, tar.gz) would be better. No need to change the code, and to provide place-holder for missing asset (j3o, video,…).

I never used nifty (the only one GUI for jME, I never tried ). IMHO, It’s strange to combine nifty and javafx.
I’ll give a try asap (my coding desktop crash this morning, no ETA).

A quick remark :

new Media(new File(this.getClass().getResource(videoPath).getFile()).toURI().toASCIIString());

should be replaced by (resource could be inside a jar,…) :

new Media(this.getClass().getResource(videoPath).toExternalForm());

Edil_Fernando · December 18, 2014, 9:31am

Well, I’m a beginner to all of this Is it possible to make the UI entirely outta JFX? I changed the code as suggested. Shall, provide a tar.gz bundle asap. Thanks mate!!

david_bernard_31 · December 18, 2014, 9:43am

Yes you can do the HUD (UI) in javafx. I use SceneBuilder 2 (visual editor to create fxml) + css.
you can take a look at samples into jme3-jfx, or 2 of my projects :

a game I made for ld31 : GitHub - davidB/ld31_p0cm0n: my game for ludumdare 31
a skeleton project I used : GitHub - davidB/jme3_skel: my project skeleton for jme 3.0 projects (gradle based)

abies · December 18, 2014, 11:39am

I doubt it is a real problem in this case, but for future - you need to dispatch code inside setOnEndOfMedia to jme thread, instead of modifying jme3 viewport from undefined jfx thread.

Edil_Fernando · December 18, 2014, 11:47am

Here is the link to the tar.gz archive : https://drive.google.com/open?id=0B1dPkfPAZHNFY2NGUWNzRTZxbVE&authuser=0

@ David : I would definitely want to try FXML, coming from the Android world myself. I’ll check your work out when time permits

@ Artur: Oh okay. I’ll try that out. Will probably need help (suck at multithreading )