I've made a MovieAppState so you don't have to

@Empire_Phoenix and @abies made an excellent job at making a powerful solution for putting movies on Texture.
I’ve made an AppState for the simplest use case: showing a movie full screen.
You attach the appstate and the movie plays, and detaches itself when finished.
Usage:

stateManager.attach(new MovieAppState("Movies/powered-by-jme3.mp4"));

Requires this lib:

Also requires Java8 (Java7 users was recently forced to upgrade to Java8 anyway). This also means that this solution doesn’t work on Android (but I believe that you can use an Activity to do the same thing).

The JME3-JFX uses internal oracle classes that may break even between minor revision. You need to bundle a JVM with your application.

Also requires the appropriate codec to be installed. More info here: http://www.oracle.com/technetwork/java/javafx/downloads/supportedconfigurations-1506746.html

Requires your application to extend SimpleApplication, and put this code, or else! :imp:

    @Override
    public void simpleInitApp() {
                PlatformImpl.startup(new Runnable() {
   @Override public void run() {
   }
});
	@Override
	public void destroy() {
		super.destroy();
		PlatformImpl.exit();
	}

The code assumes that the movie is in 16/9 aspect ratio. If the user is running on 4/3 then black bands are showed (the result isn’t perfect, but still better that full stretch).
If running crazy aspect ratio you get even less optimal results :weary:

Do I forget something? Ah yes, the MovieAppState! :wink:

import com.jme3.app.Application;
import javafx.beans.value.ChangeListener;
import javafx.beans.value.ObservableValue;
import javafx.scene.media.Media;
import javafx.scene.media.MediaException;
import javafx.scene.media.MediaPlayer;

import com.jme3.app.SimpleApplication;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.scene.Geometry;
import com.jme3.scene.shape.Quad;
import com.jme3x.jfx.media.TextureMovie;
import com.jme3x.jfx.media.TextureMovie.LetterboxMode;
import com.simsilica.lemur.event.BaseAppState;
import org.lwjgl.opengl.Display;

/**
 *
 */
public class MovieAppState extends BaseAppState {

    private TextureMovie textureMovie;
    private MediaPlayer mp;
    Geometry screen1;
    String movie;

    public MovieAppState(String movie) {
        this.movie = movie;
    }

    @Override
    protected void initialize(Application app) {
        Media media = new Media(Thread.currentThread().getContextClassLoader().getResource(movie).toString());
        media.errorProperty().addListener(new ChangeListener<MediaException>() {
            @Override
            public void changed(final ObservableValue<? extends MediaException> observable, final MediaException oldValue, final MediaException newValue) {
                newValue.printStackTrace();
            }
        });
        this.mp = new MediaPlayer(media);
        this.mp.setOnEndOfMedia(new Runnable() {
            public void run() {
                stopping = true;
            }
        });
        this.mp.play();

        boolean squareScreen = ((0f + Display.getWidth()) / Display.getHeight()) < 1.6f;
        this.textureMovie = new TextureMovie(app, this.mp, squareScreen ? LetterboxMode.VALID_LETTERBOX : LetterboxMode.VALID_SQUARE);
        this.textureMovie.setLetterboxColor(ColorRGBA.Black);

        screen1 = new Geometry("Screen1", new Quad(Display.getWidth(), Display.getHeight()));

        final Material s1mat = new Material(app.getAssetManager(), "com/jme3x/jfx/media/MovieShader.j3md");
        s1mat.setTexture("ColorMap", this.textureMovie.getTexture());
        s1mat.setInt("SwizzleMode", textureMovie.useShaderSwizzle());
        screen1.setMaterial(s1mat);
        ((SimpleApplication) app).getGuiNode().attachChild(screen1);

    }
    boolean stopping = false;

    @Override
    public void update(float tpf) {
        if (stopping) {
            ((SimpleApplication) getApplication()).getGuiNode().detachChild(screen1);
            cleanup();
            setEnabled(false);
        }
    }

    @Override
    protected void cleanup(Application app) {
        this.mp.stop();
    }

    @Override
    protected void enable() {
    }

    @Override
    protected void disable() {
        stopping = true;
    }
}
10 Likes

I’ve updated the wiki with links to this and another thread. Feel free to share suggestions and improvements to my solution! :wink:

Bonus points if somebody actually produces the official powered-by-jme3.mp4 :sunglasses:

While I like the fame, i actually had nothing to do with the video to texture part :smile:

About the bundled JVM, I try to keep the head compatible with the latest official 8 JRE, and try to circumvent changes with reflection. Currently the Lib works on all not totally outdated java8 vms without problems. I personally dont think there will be many breaking changes in the api for a longer time as it has mostly stabilized, so as long as you don’t use it productive but for hobby projects you are fine without bundling it.

Also having a real jme showcase video or something would be nice to have, yes :slight_smile:

Hi,

You forgot lemur in the dependencies list.

[AUTO-PROMO]
FYI, For gradle users :

  • to bundle with jre, you can use gradle-getdown-plugin
  • lemur, jme-jfx are available in the repository contrib - Maven - Bintray

@Pesegato, if you whish you can publish your jar with the class in this repository (I can help you).

Note: in JME trunk there is a BaseAppState class now… which is essentially a cut-paste of Lemur’s.

Doesn’t the trunk also have sdk support for bundling a jre while compiling?

Well, for now I’d like to receive feedback on how to improve it. It’s usable but still rough for my tastes :wink:
And since BaseAppState+jre bundling is on 3.1 maybe is more suitable as a 3.1 plugin.

On my pc there are performance problems with full HD video: I get stuttering video.
The same video works smoothly on VideoLan.

Core i5 sandy bridge
Intel HD 2000

Also 720p video has some stuttering, but is less noticeable.

I notice that the TextureMovie class performs the YUV to RGB conversion on the CPU. This is not as efficient as uploading the YUV planes ByteBuffers as jME3 Luminance textures and then decoding that in the shader. Also the letter boxing effect is performed by modifying the RGB buffer … letter boxing is much more easily performed in a shader.

This is true - as only saving grace I can mention that this conversion is done by optimized native code bit (so probably using all the vector/SIMD/whatever goodness). I was looking into uploading YUV, but there are few different implementations on native side (pages with stride versus separate pages) so I haven’t looked into that.

Yes - this is why there are RAW_SQUARE and VALID_SQUARE options in the constructor. If you specify any of them, no letterboxing will be performed on CPU, but you will need to provide a proper shader (depending on what kind of object you want to texture with the movie). I suppose we could modify example shader to support this option as well, but I was trying to keep it shader independent as much as possible (this is TextureMovie, not TextureAndShaderMovie :wink: - thus the option to have letterboxing done automatically on texture side.

EDIT:
Looking at the code again, I have realized this part is not using image swap technique used in the rest of the jfx integration code, so copying is happening on jme thread. This is quite bad, I will try to make it use off-main thread processing (but YUV->ARGB conversion is happening outside of jme thread)

The most common video format is NV12 (I assume that’s what JFX uses), which is the luminance plane in full resolution, followed by chroma plane containing interleaved Cr/Cb samples at half resolution.
The luma plane is easy, its just a full resolution Luminance8 texture, so you should get that to work first. The chroma plane has both samples … you will probably need to use something like Luminance8Alpha8 format - turn on linear mag filter on the chroma texture so UV samples are properly interpolated. Sample both textures in the shader and then do regular YUV → RGB conversion, that should do it.

I already have zero-copy version working on my side, but I’m fighting with proper letterboxing - I have to wrap my head around differences between frame width, frame encoded width and frame buffer stride (which are all different). I should commit working version later this evening.

Latest version in git has working zero-copy playback for YCrCb movies. You need to use MovieMaterial instead of MovieTexture, as it is very tightly coupled with specific shader. There is also a need to call update() on it each frame from jme thread manually - I will need to make some kind of app state to manage it globally later.

Thanks to @Pesegato for testing it on Intel GPU.

1 Like

When the maven experts ( @Empire_Phoenix ?) rebuild the jar I’ll post updated code! :banana: :chimpanzee_amused:

Running this on 3.1 shows serious issues… @Momoko_Fan is this to be expected?

It should already be rebuilded (since 11 hours), the build process is automated.
However the bintray repository apparently needs some time to deliver the new builds.

Not sure. Would help to know what those issues are…

Issue is with luminance8 textures. With 3.0, they work as expected. With 3.1 head, they are turning into blurry mess (possibly high level minmap?). On top of that, it looks like uploading them takes more cpu time, but it might be a side effect of primary issue.

It seems mipmap generation is now occurring for textures that aren’t using mipmap filtering… That would explain both of your symptoms.
I have a fix locally but I still need to test it.

Any update on this? Thanks! :yum:

Any update? :monkey_face: