News about playing videos on jme?

Hi,
I was wondering how to play a video on my game, but the old video API is deprecated… :frowning:
But looking around at the jme googleCode page, i’ve found this: https://code.google.com/p/jmonkeyengine/source/browse/trunk/engine/src/jheora/com/jme3#jme3%2Fnewvideo

So, i tried to run it but i miss the com.fluendo library.
And the question is, how can i get this fluendo library and how can i put this example to work?

Thanks

this was talked about heavily a short while ago.

your two options are this:

  1. use java fx

empire phoenix has beeing expirementing with this idea and has been working on implementing jfx with jme which would then allow you to play videos. afaik right now this is more of a work in progress: http://hub.jmonkeyengine.org/forum/topic/jme-jfx-integration/

  1. use javacv

http://hub.jmonkeyengine.org/forum/topic/video-playback-in-jme3/page/2/ , it works, however the code is GPL so you’re game would have to also be GPL to use it.

1 Like

Ok, thanks bro.

i’ll look into those topics.

Currently it should be possible to display movie through jfx integration, but inside jfx window and with probably not very impressive performance. If you are looking for more powerful solution, which would for example allow rendering to jme/opengl textures, I think there is a way to do it in high performance way, but it would take few days of hardcore hacking of jfx. They have very proper interfaces for video rendering, but we are talking about a lot of fun to get it integrated.

Edit:
If somebody likes to tries his hacking skills, key class seems to be com.sun.javafx.media.PrismMediaFrameHandler. Methods getTexture/updateTexture are doing the rendering and with a bit of luck it might be possible to convince them to render directly to opengl texture. Otherwise, it might be needed to do blitting from generated texture into opengl buffer, it this would be a horrible waste - getting something which is already on GPU back to CPU land, doing memory copy and then uploading back to GPU.
Further edit:
Might be even simplier. Seems that MediaPlayer is exposing impl_getLatestFrame, which returns VideoDataBuffer, which can be asked to convertToFormat for proper format and then blitted into byte buffer with getBuffer and a bit of stride counting magic (look into com.sun.prism.j2d.J2DTexture for implementation).

And here are the results of playing around with that. On the left, there is plain quad with movie texture. Middle - rotating sphere with movie texture. Right - Spot light projecting movie cinema-like, with two annoying cubes floating in the way.

Only 87 fps because I have limited them to 90 and enabled Fraps. Without any limitations, it runs at around 1000fps (movies are 30fps anyway, so it doesn’t really mean much except that it is not that heavy to run 3 mp4 movies in parallel).

Code for using textures is something like

media = new Media("file:///link/to/movie.mp4");
mp = new MediaPlayer(media);
mp.volume = 0;
mp.play();
tex = new TextureMovie(mp, LetterboxMode.VALID_LETTERBOX);
material.setTexture("ColorMap",tex.getTexture());

MediaPlayer is jfx class, so it has all fancy controls for looping, audio control, skipping to specific part of the movie etc.

I plan to check in code into jfx-jme3 soon. It is not dependent on the rest of jfx integration, but of course requires java 8 installation. Also, media support is not that great
http://docs.oracle.com/javafx/2/api/javafx/scene/media/package-summary.html#SupportedMediaTypes
Basically only MP4(H.264) and FLV(VP6).

14 Likes

Dude that’s brilliant!

Committed code into


Please take a look at TestMovie class for example.

There will be some improvements later - at the moment I’m doing quite dumb way of copying data, it can be done by scanlines instead of pixel by pixel and in case of RAW_SQUARE could be possibly done as direct buffer replace. Additionally, there is a chance of tearing happening, as I’m not synchronizing frame updates (should be a problem only if you are really unlucky), again something to fix later. Still, it is usable and API should not change too much.

Java 8 is required and as entire jfx integration, it is quite brittle - depends on internal Oracle API. Can break between minor patch releases, so if you plan to use it in your game, make sure you bundle/require very specific jdk version on which it was tested.

Edit:
Changed copy to per-scanline and got order of magnitude speedup (previously copying frame was taking 1.5-6ms, now it is around 0.3ms). It is on dedicated non-jme3, non-jfx thread anyway so it doesn’t affect framerate directly, but faster is always better :wink:

7 Likes

Very great :slight_smile:

I just added

		final Media media = new Media("http://download.oracle.com/otndocs/products/javafx/oow2010-2.flv");
		media.errorProperty().addListener(new ChangeListener<MediaException>() {

			@Override
			public void changed(final ObservableValue<? extends MediaException> observable, final MediaException oldValue, final MediaException newValue) {
				newValue.printStackTrace();
			}
		});

to the testcase, to ensure that others will have proper error logging in it :slight_smile:

(As on my machine some codec was missing for the audio format of that flv^^, and errors are reported async on that property if using a network uri)

Sorry for my silliness, but i was not able to run this project.
If you can explain how can i create a project using jfx at jme, i’ll be extremely happy! XD
Can you explain how to run the project https://github.com/empirephoenix/JME3-JFX ?
thanks!

Well clone it using git,
then compile it using

->you need gradle installed
gradle build
-> if you use eclipse you can do … to create the project files and allow simple import of it.
gradle eclipse

Else you can manually add it.
Note the librarys bín the providedlibs folder are NOT meant to be included in the ide, instead you should let your own project inherit the build result from JME-JFX and from JME3(the second should be default)

For easier use in the build libs subfolder there will be a jar after compiling, that you can use.

Hi folks, seeing some video playback is wonderful stuff. I have setup the gradle sources in eclipse and ran the TestMovie class with a custom MP4 file. The video is rendered with the colours in it inverted, as in Red is shown as Blue and vice-versa. A few seconds into playback the application exits with an exception:

java.lang.IllegalArgumentException: Number of remaining buffer elements is 6120960, must be at least 14745600.

`Nov 24, 2014 7:48:48 PM com.jme3.system.JmeDesktopSystem initialize
INFO: Running on jMonkeyEngine 3.0.10
Nov 24, 2014 7:48:48 PM com.jme3.system.Natives extractNativeLibs
INFO: Extraction Directory: C:\Users\Lokesh\Development\Projects\JME3\Examples\Eclipse\JME3-JFX\JME3-JFX
Nov 24, 2014 7:48:49 PM com.jme3.system.lwjgl.LwjglContext printContextInitInfo
INFO: Lwjgl 2.9.0 context running on thread LWJGL Renderer Thread
Nov 24, 2014 7:48:49 PM com.jme3.system.lwjgl.LwjglContext printContextInitInfo
INFO: Adapter: igdumdim64
Nov 24, 2014 7:48:49 PM com.jme3.system.lwjgl.LwjglContext printContextInitInfo
INFO: Driver Version: 10.18.10.3960
Nov 24, 2014 7:48:49 PM com.jme3.system.lwjgl.LwjglContext printContextInitInfo
INFO: Vendor: Intel
Nov 24, 2014 7:48:49 PM com.jme3.system.lwjgl.LwjglContext printContextInitInfo
INFO: OpenGL Version: 4.3.0 - Build 10.18.10.3960
Nov 24, 2014 7:48:49 PM com.jme3.system.lwjgl.LwjglContext printContextInitInfo
INFO: Renderer: Intel® HD Graphics 5000
Nov 24, 2014 7:48:49 PM com.jme3.system.lwjgl.LwjglContext printContextInitInfo
INFO: GLSL Ver: 4.30 - Build 10.18.10.3960
Nov 24, 2014 7:48:49 PM com.jme3.asset.AssetConfig loadText
WARNING: Cannot find loader com.jme3.audio.plugins.OGGLoader
Nov 24, 2014 7:48:49 PM com.jme3.asset.AssetConfig loadText
WARNING: Cannot find loader com.jme3.scene.plugins.ogre.MeshLoader
Nov 24, 2014 7:48:49 PM com.jme3.asset.AssetConfig loadText
WARNING: Cannot find loader com.jme3.scene.plugins.ogre.SkeletonLoader
Nov 24, 2014 7:48:49 PM com.jme3.asset.AssetConfig loadText
WARNING: Cannot find loader com.jme3.scene.plugins.ogre.MaterialLoader
Nov 24, 2014 7:48:49 PM com.jme3.asset.AssetConfig loadText
WARNING: Cannot find loader com.jme3.scene.plugins.ogre.SceneLoader
Nov 24, 2014 7:48:49 PM com.jme3.asset.AssetConfig loadText
WARNING: Cannot find loader com.jme3.scene.plugins.blender.BlenderModelLoader
Nov 24, 2014 7:48:50 PM com.jme3.audio.lwjgl.LwjglAudioRenderer initInThread
INFO: Audio Device: OpenAL Soft
Nov 24, 2014 7:48:50 PM com.jme3.audio.lwjgl.LwjglAudioRenderer initInThread
INFO: Audio Vendor: OpenAL Community
Nov 24, 2014 7:48:50 PM com.jme3.audio.lwjgl.LwjglAudioRenderer initInThread
INFO: Audio Renderer: OpenAL Soft
Nov 24, 2014 7:48:50 PM com.jme3.audio.lwjgl.LwjglAudioRenderer initInThread
INFO: Audio Version: 1.1 ALSOFT 1.15.1
Nov 24, 2014 7:48:50 PM com.jme3.audio.lwjgl.LwjglAudioRenderer initInThread
INFO: AudioRenderer supports 64 channels
Nov 24, 2014 7:48:50 PM com.jme3.audio.lwjgl.LwjglAudioRenderer initInThread
INFO: Audio effect extension version: 1.0
Nov 24, 2014 7:48:50 PM com.jme3.audio.lwjgl.LwjglAudioRenderer initInThread
INFO: Audio max auxilary sends: 4
Nov 24, 2014 7:48:53 PM com.jme3.app.Application handleError
SEVERE: Uncaught exception thrown in Thread[LWJGL Renderer Thread,5,main]
java.lang.IllegalArgumentException: Number of remaining buffer elements is 6120960, must be at least 14745600. Because at most 14745600 elements can be returned, a buffer with at least 14745600 elements is required, regardless of actual returned element count
at org.lwjgl.BufferChecks.throwBufferSizeException(BufferChecks.java:162)
at org.lwjgl.BufferChecks.checkBufferSize(BufferChecks.java:189)
at org.lwjgl.BufferChecks.checkBuffer(BufferChecks.java:230)
at org.lwjgl.opengl.GL11.glTexImage2D(GL11.java:2845)
at com.jme3.renderer.lwjgl.TextureUtil.uploadTexture(TextureUtil.java:352)
at com.jme3.renderer.lwjgl.LwjglRenderer.updateTexImageData(LwjglRenderer.java:1913)
at com.jme3.renderer.lwjgl.LwjglRenderer.setTexture(LwjglRenderer.java:1936)
at com.jme3.material.MatParamTexture.apply(MatParamTexture.java:86)
at com.jme3.material.Material.render(Material.java:1088)
at com.jme3.renderer.RenderManager.renderGeometry(RenderManager.java:523)
at com.jme3.renderer.queue.RenderQueue.renderGeometryList(RenderQueue.java:322)
at com.jme3.renderer.queue.RenderQueue.renderQueue(RenderQueue.java:374)
at com.jme3.renderer.RenderManager.renderViewPortQueues(RenderManager.java:763)
at com.jme3.renderer.RenderManager.flushQueue(RenderManager.java:719)
at com.jme3.renderer.RenderManager.renderViewPort(RenderManager.java:983)
at com.jme3.renderer.RenderManager.render(RenderManager.java:1029)
at com.jme3.app.SimpleApplication.update(SimpleApplication.java:252)
at com.jme3.system.lwjgl.LwjglAbstractDisplay.runLoop(LwjglAbstractDisplay.java:151)
at com.jme3.system.lwjgl.LwjglDisplay.runLoop(LwjglDisplay.java:185)
at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:228)
at java.lang.Thread.run(Thread.java:745)

I’m a total n00b when it comes to anything 3D. We’re a firm who specialize in edutainment and as such a good chunk of our content is in the form of animated videos. I’ve been pushing JME3 personally at work. Any help will be appreciated :slight_smile:

This solution looks very powerful for putting movie on a texture, but what if I just want to play a video on screen (2d)?

Can I just detach all JME appstates, run JFX fullscreen and when is finished reattach the JME appstates?

Advantages/disadvantages? Thanks!

You switch from the fullscreen OpenGL to a fullscreen AWT display this way. Sure its possible but idk if it makes sense and how the display system of the computer/OS reacts. Having JFX in jME in full screen is basically the same thing, you copy the JFX output to a texture… I guess if you do that just having the video on a texture makes more sense. Btw, most modern OS windowing systems do the same, they also basically use OGL / D3D textures to display stuff on screen.

In this case, can/should I attach the texture to the GUI node?
Assuming that I don’t want the video to be scaled, (so that is reproduced 1:1 with the monitor’ pixels)

Sure you can, you can also replace the background if you like (search forum for background image).

I’ve searched for a solution for loading a relative path, and apparently the alternative is putting the video in the sources folder :rage:
The AssetManager is suitable for this, by registering a custom loader?

Actually if you have a normal SDK project the assets jar is part of the classpath. Idk if the player can play videos from the classpath (and/or form a jar) but if it can it might work out of the box. Make sure you try this with a deployed application as well because when you run the project from the SDK the assets folder is not yet packaged into a jar file.

relative to what ?

Media take an absolute url (as String) so you can retreive the URL

  • from Context 's classpath :
    String url = Thread.currentThread().getContextClassLoader().getResource("/movies/MyMovies.mp4").toExternalForm()
  • Or search across every AssetManager.getClassLoaders()
  • Or use remote http url for remote video
2 Likes

Is there any .jar now that I can use without downloading and compiling the whole project?

I found https://bintray.com/empirephoenix/VGS-OSS-Releases/jfx/view/files/com/jme3x/jfx and from 1.147.2015-02-25_073431-de3711b folder I downloaded the jar. But classes like Media and TextureMovie are not present there. Eclipse wants to import LetterboxMode from com.jme3x.jfx.media.TextureMovie but right after that I’m getting an error with this import.

Well you can use gradle/maven for those dependencies.
Also you are useing a outdated version already, the one you mentions is two month old.
Then last but not least you use java 1.8?