Rendering Video to a texture with JMF

We had a nice thread where ranewc asked… “is it possible…?” to render video to a texture.



Then Camael came by and gave this link:



Chapter 28.3. Playing Movies in a Java 3D World (Part 1)



A thorough bashing of JMF ensued (led as usual by darfrog ), despite that however I came up with these little snippets.



JMFGrabber.java:


package org.llama.jmf;

/**
 * @author Tijl Houtbeckers
 *
 * Mostly Based on JMFSnapper.java by:
 * Andrew Davison, May 2005, ad@fivedots.coe.psu.ac.th
 *
 *

For those intrested: JMF seems to be capable of playing XVID Avi files (not however, DivX). But even with the MP3 plugin on the JMF page, no MP3 in Avi. This all without trying JFFMPEG yet though.



If anyone want to try this on linux and especially osx, please let me know if it works.



As for Sun slowly letting JMF die, I can see why, the lack of NIO in the design of the API makes it seem very outdated. <mini-rant> But you wonder why they don’t just release the source for it under a useable license right away…</mini-rant>

Cool! 8)



But can it be fast enough for fullscreen videos with that intermediate image? How much work would it be to render straight to a buffer? Meaning - does JMF support the required output format?

BRILLIANT! ACE! Thank you, thank you, thank you.

Hey, I have to remember that trick:

Just posting some random information and get something very useful back for it.

<wondering>Maybe I should just give a link to a list of my project ideas and you finish them for me?</wondering> :wink:



Anyway, very nice work, llama.



Oh, btw. there is a video player for MPEG 1 available, that might provide an alternative to the JMF solution: http://sureplayer.sourceforge.net/

Just in case you want to try to adapt it to JME as well, llama. :wink:

Llama, I'm just giving people the straight dope. :-p



Great job, I can't wait to try this out.  SurePlayer seems like a really nice potential option as well.  It sure would be nice if we could have video with sound in our games, but MPEG1s are HUGE.  MPEG2 wouldn't be too bad…Camael, in your plethora of links to useful information can you give us a link to an entirely Java MPEG2 player? :-p



darkfrog

@darkfrog

Believe me I searched for one… Didn't find anything open source/free, though. :frowning:

If I do, I will of course gladly provide a link to it.

But Camael, you're the keeper of the links.  :-o



Oh well, I have been searching periodically over the past year or so and haven't found anything really useful myself so maybe it doesn't exist yet.



darkfrog

Sure, I say that and then I read a post immediately afterwards on the JavaGaming forums that posted these links:



http://mediaframe.org/

http://fobs.sourceforge.net/



The first one looks very impressive.



EDIT: I’ve done some more examining of the MediaFrame API, though difficult to find I now have the source code and it looks as though it was designed to run through an applet.  However, I think it should be pretty trivial to modify it for our purposes.  I’ve been looking at the MPEG4 stuff but I’m assuming the MPEG1 support is pretty similar.



darkfrog

The source is just on the download page

I've made a renderer plugin for JMF, that renders frames directly to a texture.



If FOBS is present on the classpath (it should still without), it can also attempt to use FOBS, which will use FFMPEG to render directly into the native buffer, that is then used for updating the texture. However, this requires a small (1 line) change to FOBS (I'll take that to their list). It'll still work without this change, but not as efficient.



This gives excellent performance for video… the CPU usage (when using FOBS) can be compared to normal media player (no wonder since mplayer, VLC, directshow + ffdshow all use ffmpeg too), and will improve a bit more after I make some small changes to jME (support for better pixel packing and such). If you use FOBS you can load practically every format there is too (except WMV9, but who cares).



The only problem is audio. JavaSound is unreliable as ever, has incompatabilites between 1.4 and 1.5, and doesn't always seem to work with JMF. JMF for windows ships with a different audio renderer (directsound), maybe the linux pack has one for linux? However, DirectSound also isn't perfect, either the sound is broken up a bit (because jME takes all CPU), or it somehow drags down the framerate (when you limit jME to a certain FPS to reduce it's CPU usage) without actually using up CPU time! (so the processor will be largely idle). Maybe a jME audio renderer would solve this. Of course, you can make movies with no audio, or audio in a seperate file (or maybe use a null renderer).



The install is horribly complex right now, I'll try to make it a bit more simple, and then post the files tommorow.

That's awesome.  :-o



I'm gonna head to the zoo and feed my local llama as a thank you!



darkfrog

Wow! Really sounds awesome!!! :-o

Can't wait to see a movie running on a spinning cube :smiley:

Since it's a huge post, and I'll link it from an external site, I made a new topic:



http://www.jmonkeyengine.com/jmeforum/index.php?topic=2908.0