Movie in 3d object

Hi, don't want to open up a big debate or get flamed. Just searched this forum to look for approaches for playing movies (avi, mpeg or xvid/divx) within a rotating cube etc. I noticed there isn't a lot to work with. There is an article using JMF and a mention of some other possible library candidates (ie quicktime etc). I want to create a frontend for my new arcade cabinet I'm building because everything I've found so far doesn't really meet what I'm looking for. I am going to implement solution on windose, therefore I'm happy (NOT) to fall into the Microsoft mentality of complying with their designs. I'm tempted to switch to VB or .Net, but I love Java and JMonkeyEngine (already got some menu's whizzing around in 3d space etc)…



So, I'm now debating whether to switch away or not. I stumbled on "dsj - DirectShow <> Java wrapper" the other day:



http://www.humatic.de/htools/dsj.htm



I downloaded their sample and got it working with all the different avi clips etc I had with no problem (didn't try source yet). First of all, anyone tried integrating this into JMonkeyEngine? It's going to be Windose specific because its relying on DirectX. I'm now trying to work out whether its worth me spending time investigating it or not…



Any coments???

Renanse did some work quite a while back on movie loading support in jME but I don't think it ever got checked in because he hadn't had a chance to add sound support yet.  I doubt he's had time to do anymore work on it but that would be your best bet.



Renanse, what would you think about going ahead and checking that code in and maybe someone else might contribute the missing features?

The JMF/FOBS stuff that I wrote (it's posted in the forum), will handle pretty much any format you throw at it. It's not very polished yet though (more proof of concept).



Afaik Renanse's unreleased code will not play avi/xvid or mpeg. I also seem to remember it lacks sound support. JMF/FOBS sound support is ok on Windows.

llama said:

The JMF/FOBS stuff that I wrote (it's posted in the forum), will handle pretty much any format you throw at it. It's not very polished yet though (more proof of concept).


I thought JMF wouldn't handle MPEG-2 format?  (perhaps I'm misremembering though)

If you use FOBS you can handle pretty much every format that FFMPEG/Libavcodec supports:



http://en.wikipedia.org/wiki/Libavcodec (lists some of the codecs supported).



I think MPEG2 was actually one of the first supported formats of that project (hence the name).

llama said:

Afaik Renanse's unreleased code will not play avi/xvid or mpeg. I also seem to remember it lacks sound support. JMF/FOBS sound support is ok on Windows.


That's right, it's RoQ only.  Sound support though should not be a big deal to add as it already parses it.  That said, if you want a more traditional movie format, the suggestions above are great.

Okay, so when are you going to make that code available to us?  :smiley:

Probably never at this rate.  But hey, you did get the sound stuff…

Thanks for all the responses. I managed to miss the Jmf/FOBS article. I intend to give that a bash this afternoon at lunch time at work (honest, it'll be my own time  ;)). I like the idea of using any formats rather than one specific type because for me its a need to display a sample of a game, intro, ending etc for a game. I don't want to have to convert to a specific format, although if I was writing a game I probably would to ensure I reduce overheads in size and performance!!!



Thanks again, I'll update later when I've tried the above…



Regards



Sp33dy

renanse said:

Probably never at this rate.  But hey, you did get the sound stuff...


That's true and I haven't even gotten around to using it yet....I'm grateful for sound though. :)

Wow,

I posted this message over 4 years ago!!! I can’t believe it.



I evaluated jMonkeyEngine at the time to write an emulator front end, but Hyperspin came out around the same time. I was convinced at the time that a 3d engine would really make their 2d flash engine look dull. However, as always, real life caught me up.



Anyway’s, over the past few months I’ve had an urge and a need to see how jMonkeyEngine is going. It’s really impressive!

To get back to this subject. I’ve been using Xuggler for the past few months and it dawned on me that this would mesh really well with this engine for Movies on a surface. Blow me down, last night I easily hooked together the two and am getting great results. Even on my laptop that has a duo core 2 2ghz chip, I’m seeing 30 fps without any form of optimisation. I’m able to put FLV’s, Movs, Vobs, mpg4’s, Divxs, Avi’s and a whole bunch of other formats through it.



Having searched the forums, I can’t find anyone else talking about this or requesting for such a feature. Is there any call for me to write up how I stitched it together??? There might not be a call. There are also a few limiting factors (i.e. only 32bit jvm supported, even in win7. Although Linux and OSX is supported).



Regards

Sp33dy

Great news, Sp33dy. I’m very interested in hearing how you did this and the challenges you faced in making it possible.

I would like to create a wiki page, but not sure if I have authorisation right now. Can you try these for me:



XugglerVideoPlayer class:



[java]

import com.jme3.app.SimpleApplication;

import com.jme3.system.AppSettings;

import com.jme3.texture.Image;

import com.jme3.texture.Texture2D;

import com.jme3.ui.Picture;

import com.xuggle.xuggler.IVideoResampler;



/*

  • Copyright © 2009-2011 jMonkeyEngine
  • All rights reserved.

    *
  • Redistribution and use in source and binary forms, with or without
  • modification, are permitted provided that the following conditions are
  • met:

    *
    • Redistributions of source code must retain the above copyright
  • notice, this list of conditions and the following disclaimer.

    *
    • Redistributions in binary form must reproduce the above copyright
  • notice, this list of conditions and the following disclaimer in the
  • documentation and/or other materials provided with the distribution.

    *
    • Neither the name of ‘jMonkeyEngine’ nor the names of its contributors
  • may be used to endorse or promote products derived from this software
  • without specific prior written permission.

    *
  • THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
  • "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
  • TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
  • PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
  • CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
  • EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
  • PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
  • PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
  • LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
  • NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
  • SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

    */



    public class XugglerVideoPlayer extends SimpleApplication {



    private final String videoFile = “?.Your file, which can be *.flv, *.mpg, *.avi, etc, etc”;



    private Picture picture;



    private XugglerVideo video;



    public static void main(String[] args){

    // Check that the Xuggler Feature is available before starting

    if (!IVideoResampler.isSupported(IVideoResampler.Feature.FEATURE_COLORSPACECONVERSION))

    throw new RuntimeException(“Please install the GPL version of Xuggler for this demo to work”);



    //Xuggler available so continue

    XugglerVideoPlayer app = new XugglerVideoPlayer();

    AppSettings settings = new AppSettings(true);

    settings.setFrameRate(60);

    app.setSettings(settings);

    app.start();

    }



    public XugglerVideoPlayer() {

    createVideo();

    }



    private void createVideo() {

    video = new XugglerVideo(videoFile);

    }



    @Override

    public void simpleInitApp() {

    picture = new Picture(“VideoPicture”, true);

    picture.setPosition(0, 0);

    picture.setWidth(settings.getWidth());

    picture.setHeight(settings.getHeight());

    picture.setImage(assetManager, “Interface/Logo/Monkey.jpg”, false);



    // attach geometry to orthoNode

    rootNode.attachChild(picture);



    // start the video processing;

    video.start();

    }



    private Image lastVideoFrameImage = null;



    @Override

    public void simpleUpdate(float _float) {

    if (video.getFrameImage()!=null) {

    if (lastVideoFrameImage != video.getFrameImage()) {

    lastVideoFrameImage = video.getFrameImage();

    System.out.println(video.getFrameImage());

    Texture2D texture = new Texture2D(lastVideoFrameImage);

    picture.setTexture(assetManager, texture, false);

    }

    }

    }



    @Override

    public void destroy() {

    super.destroy();



    //Cleanly close the video stream

    video.close();

    }



    }

    [/java]



    XugglerVideo class:



    [java]

    import java.awt.image.BufferedImage;



    import javax.sound.sampled.AudioFormat;

    import javax.sound.sampled.AudioSystem;

    import javax.sound.sampled.DataLine;

    import javax.sound.sampled.LineUnavailableException;

    import javax.sound.sampled.SourceDataLine;



    import com.jme3.audio.AudioNode;

    import com.jme3.texture.Image;

    import com.jme3.texture.plugins.AWTLoader;

    import com.xuggle.xuggler.Global;

    import com.xuggle.xuggler.IAudioSamples;

    import com.xuggle.xuggler.ICodec;

    import com.xuggle.xuggler.IContainer;

    import com.xuggle.xuggler.IPacket;

    import com.xuggle.xuggler.IStream;

    import com.xuggle.xuggler.IStreamCoder;

    import com.xuggle.xuggler.IVideoPicture;

    import com.xuggle.xuggler.Utils;



    public class XugglerVideo extends Thread {



    private IContainer videoContainer;



    private int videoStreamId = -1;

    private IStreamCoder videoCoder = null;



    private int audioStreamId = -1;

    private IStreamCoder audioCoder = null;



    private Image frameImage = null;



    public XugglerVideo(String _videoFilename) {

    videoContainer = IContainer.make();



    // Open up the container

    if (videoContainer.open(_videoFilename, IContainer.Type.READ, null) < 0)

    throw new IllegalArgumentException("could not open file: " + _videoFilename);



    determineVideoAndAudioStreams();



    if (videoStreamId == -1)

    throw new RuntimeException(“Could not find a video stream in video file”);



    }



    private void determineVideoAndAudioStreams() {

    // query how many streams container has

    int numStreams = videoContainer.getNumStreams();



    // and iterate through the streams to find the first video and audio streams

    for(int i = 0; i < numStreams; i++) {

    // next stream object

    IStream stream = videoContainer.getStream(i);



    // Get the pre-configured decoder that can decode this stream;

    IStreamCoder coder = stream.getStreamCoder();



    if (coder.getCodecType() == ICodec.Type.CODEC_TYPE_VIDEO && videoStreamId == -1) {

    videoStreamId = i;

    videoCoder = coder;

    if (videoCoder.open() < 0)

    throw new RuntimeException(“Could not open the video stream”);

    } else if (coder.getCodecType() == ICodec.Type.CODEC_TYPE_AUDIO && audioStreamId == -1) {

    audioStreamId = i;

    audioCoder = coder;

    if (audioCoder.open() < 0)

    throw new RuntimeException(“Could not open the video stream”);

    openJavaSound(audioCoder);

    }

    }

    }



    @Override

    public void run() {

    IPacket packet = IPacket.make();



    // Loop for all packets in the file and process their type

    while (videoContainer.readNextPacket(packet) >= 0) {



    if (packet.getStreamIndex() == videoStreamId) {

    processVideoPacket(packet);

    } else if (packet.getStreamIndex() == audioStreamId) {

    processAudioPacket(packet);

    }



    }

    }



    private void processVideoPacket(IPacket _packet) {

    // Allocate Xuggler picture object to fill

    IVideoPicture picture = IVideoPicture.make(videoCoder.getPixelType(),

    videoCoder.getWidth(),

    videoCoder.getHeight());



    int offset = 0;



    while (offset < _packet.getSize()) {



    // Decode the video

    int bytesDecoded = videoCoder.decodeVideo(picture, _packet, offset);



    // Check for errors

    if (bytesDecoded <0)

    throw new RuntimeException(“error decoding video stream for picture”);



    offset += bytesDecoded;



    // Have we got a complete picture from stream?

    if (picture.isComplete()) {

    synchVideo(picture);



    // VideoPictureToImage has been deprecated

    // replaced with ConverterFactory

    // but I’m unfamiliar with this at the moment, so sticking with this call

    BufferedImage image = Utils.videoPictureToImage(picture);

    frameImage = new AWTLoader().load(image, false);

    }



    }

    }



    private void processAudioPacket(IPacket _packet) {

    IAudioSamples samples = IAudioSamples.make(1024, audioCoder.getChannels());



    int offset = 0;



    while (offset < _packet.getSize()) {

    int bytesDecoded = audioCoder.decodeAudio(samples, _packet, offset);

    if (bytesDecoded<0) {

    System.out.println(“Failed to decode audio”);

    break;

    }

    offset += bytesDecoded;



    if (samples.isComplete()) {

    byte[] bytes = samples.getData().getByteArray(0, samples.getSize());

    soundLine.write(bytes, 0, samples.getSize());

    }

    }

    }



    private long firstTimestampInStream = Global.NO_PTS;

    private long systemClockStartTime = 0;



    // Ensure video waits for the next synchronised time position.

    // This is a rehash of an example provided by Xuggler.

    private void synchVideo(IVideoPicture _picture) {

    if (firstTimestampInStream == Global.NO_PTS) {

    firstTimestampInStream = _picture.getTimeStamp();

    systemClockStartTime = System.currentTimeMillis();

    } else {

    long systemClockCurrentTime = System.currentTimeMillis();

    long millisecondsClockTimeSinceStartofVideo = systemClockCurrentTime - systemClockStartTime;

    long millisecondsStreamTimeSinceStartOfVideo = ( _picture.getTimeStamp()
  • firstTimestampInStream) / 1000;

    final long millisecondsTolerance = 0;

    final long millisecondsToSleep = ( millisecondsStreamTimeSinceStartOfVideo
  • ( millisecondsClockTimeSinceStartofVideo
  • millisecondsTolerance)

    );

    if (millisecondsToSleep > 0) {

    try {

    Thread.sleep(millisecondsToSleep);

    } catch (InterruptedException e) {

    return;

    }

    }

    }

    }



    private static SourceDataLine soundLine;



    private static void openJavaSound(IStreamCoder _coder) {

    AudioFormat af = new AudioFormat(_coder.getSampleRate()

    ,(int)IAudioSamples.findSampleBitDepth(_coder.getSampleFormat())

    ,_coder.getChannels()

    ,true

    ,false);

    DataLine.Info info = new DataLine.Info(SourceDataLine.class, af);

    try {

    soundLine = (SourceDataLine) AudioSystem.getLine(info);

    soundLine.open();

    soundLine.start();

    } catch (LineUnavailableException e) {

    System.out.println("Unabled to open line! "+e);

    }

    }



    private static void closeJavaSound() {

    if (soundLine != null) {

    soundLine.drain();

    soundLine.close();

    soundLine = null;

    }

    }



    public Image getFrameImage() {

    return frameImage;

    }



    public void close() {

    // Cleanly close streams and container

    if (videoCoder != null && videoCoder.isOpen()) videoCoder.close();

    if (audioCoder != null && audioCoder.isOpen()) {

    closeJavaSound();

    audioCoder.close();

    }



    videoContainer.close();

    }



    }

    [/java]



    You’ll need to download the Xuggler 3.4 librarys here [link]http://www.xuggle.com/xuggler/downloads/[/link]



    Please remember that 64Bit Windows is not supported, so you will need a 32Bit JVM if on Win7 64Bit (but this works fine for me!).



    I see Eclipse is not the main tool anymore, however. Which ever development tool you use, you’ll need to included the external jar’s:



    sharejavajars



    Everything is working really well for me. I’ve change a little of the coding and now getting 50+ fps on same said 2ghz laptop!



    Kind regards



    Sp33dy



    P.S. Let me know what you think and whether it’s worth me writing up a Wiki entry.
2 Likes

I forgot to add that I’m using Java sound for now for the video sound, purely as I’ve not looking into moneky audio… yet

This works beautifully. I’ve even tested it on 3D meshes and it still works. Thanks

Ah, that’s great to hear. It dawned on me after posting that it would make sense to add video.stop() in the destroy method of XugglerVideoPlayer class before the close; thereby ensuring the thread is stopped! In my eclipse image, I noticed that it didn’t always stop correctly.



Given this, is it worth me putting this onto a wiki page?



Secondly, do post up a video of anything interesting you do with it… Always keen to see how people use it.



Kind regards



Sp33dy

As soon as I have something to show, I’ll post it here.

I have never ever used this before it. Because i am currently using other. I would like to try it for my better experience and good result.