JFjr - the JMF/FOBS/jME video renderer

sbook · May 25, 2009, 8:53pm

For anyone still working on/using this, and for llama as well, I got the "Error rendering video to texture. No glTexSubImage2D/OpenGL 1.2 support?"

I'm running a first-gen Core Duo MacBook Pro with a Mobility x1600 (an ATI card)… The card supports OpenGL 2.0, I'm not entirely sure what the apple driver supports for it though… I'm about to boot into Linux to see what I get…

Strange thing is that the video does play in JMstudio, but once I try running it in jME, i get audio and a blank white texture, with that OpenGL related error… spooky

Thanks, llama, for all the work you've put into this though… cool stuff!

EDIT: Booted into a different Laptop with a 9800m GTS… no OpenGL error, but no imagae textures either… just sound.

another edit:

The audio definitely works, no video though, and I see nothing in the console output indicating why it shouldn't display video… I killed it before the number of frames got completely ridiculous, but here's an idea of what we're getting:

May 25, 2009 7:49:39 PM com.jme.app.BaseGame start
INFO: Application started.
May 25, 2009 7:49:39 PM com.jme.system.PropertiesGameSettings <init>
INFO: PropertiesGameSettings created
May 25, 2009 7:49:39 PM com.jme.system.PropertiesGameSettings load
INFO: Read properties
May 25, 2009 7:49:41 PM com.jme.system.PropertiesGameSettings save
INFO: Saved properties
May 25, 2009 7:49:41 PM com.jme.app.BaseSimpleGame initSystem
INFO: jME version 2.0 Stable (r4093)
May 25, 2009 7:49:41 PM com.jme.input.joystick.DummyJoystickInput <init>
INFO: Joystick support is disabled
May 25, 2009 7:49:41 PM com.jme.system.lwjgl.LWJGLDisplaySystem <init>
INFO: LWJGL Display System created.
May 25, 2009 7:49:42 PM com.jme.renderer.lwjgl.LWJGLRenderer <init>
INFO: LWJGLRenderer created. W: 640H: 480 Version: 2.0.1
May 25, 2009 7:49:42 PM com.jme.app.BaseSimpleGame initSystem
INFO: Running on: vga
Driver version: null
NVIDIA Corporation - GeForce 9800M GTS/PCI/SSE2 - 2.1.2
May 25, 2009 7:49:42 PM com.jme.renderer.AbstractCamera <init>
INFO: Camera created.
May 25, 2009 7:49:42 PM com.jme.util.lwjgl.LWJGLTimer <init>
INFO: Timer resolution: 1000 ticks per second
May 25, 2009 7:49:42 PM com.jme.scene.Node <init>
INFO: Node created.
May 25, 2009 7:49:42 PM com.jme.scene.Node <init>
INFO: Node created.
May 25, 2009 7:49:42 PM com.jme.scene.Node <init>
INFO: Node created.
May 25, 2009 7:49:42 PM com.jme.scene.Node attachChild
INFO: Child (Graph node) attached to this node (Stats node)
May 25, 2009 7:49:42 PM com.jme.renderer.lwjgl.LWJGLTextureRenderer <init>
INFO: FBO support detected.
May 25, 2009 7:49:42 PM com.jme.renderer.lwjgl.LWJGLTextureRenderer initCamera
INFO: Init RTT camera
May 25, 2009 7:49:42 PM com.jme.renderer.AbstractCamera <init>
INFO: Camera created.
May 25, 2009 7:49:42 PM com.jme.scene.Node <init>
INFO: Node created.
May 25, 2009 7:49:42 PM com.jme.scene.Line <init>
INFO: Line created.
May 25, 2009 7:49:42 PM com.jme.renderer.lwjgl.LWJGLTextureRenderer setupTexture
INFO: setup fbo tex with id 2: 640,360
May 25, 2009 7:49:42 PM com.jme.scene.Node attachChild
INFO: Child (lineGraph) attached to this node (Graph node)
May 25, 2009 7:49:42 PM com.jme.scene.Node attachChild
INFO: Child (f4) attached to this node (Graph node)
May 25, 2009 7:49:42 PM com.jme.renderer.lwjgl.LWJGLTextureRenderer <init>
INFO: FBO support detected.
May 25, 2009 7:49:42 PM com.jme.renderer.lwjgl.LWJGLTextureRenderer initCamera
INFO: Init RTT camera
May 25, 2009 7:49:42 PM com.jme.renderer.AbstractCamera <init>
INFO: Camera created.
May 25, 2009 7:49:42 PM com.jme.scene.Node <init>
INFO: Node created.
May 25, 2009 7:49:42 PM com.jme.renderer.lwjgl.LWJGLTextureRenderer setupTexture
INFO: setup fbo tex with id 3: 640,120
May 25, 2009 7:49:42 PM com.jme.scene.Node attachChild
INFO: Child (labelGraph) attached to this node (Graph node)
Fobs4JMF - Native shared library found
60.0184First Position: 0, 41 Duration: 60018
Frame Rate: 23.976
Number of channels: 2
Opening Thread[JMF thread: com.sun.media.PlaybackEngine@f1bb78[ com.sun.media.PlaybackEngine@f1bb78 ] ( configureThread),9,system]
ByteBufferRenderer: setInputFormat
Format (RGB masks, pixelstride, linestride, bbp, flipped
ff0000
ff
ff00
1
624
32
false
set Size: 624 352
texture format: GL_UNSIGNED_INT_8_8_8_8_REV
FOBS found
Frame buffer size (bytes): 878592
ByteBufferRenderer: setInputFormat
Format (RGB masks, pixelstride, linestride, bbp, flipped
ff0000
ff
ff00
1
624
32
false
set Size: 624 352
FOBS found
Frame buffer size (bytes): 878592
Created player for: file:src/vids/xvidmp3.avi
using flipped quad
May 25, 2009 7:49:42 PM com.jme.scene.Node attachChild
INFO: Child (quad) attached to this node (rootNode)
frame data:219648 frame: 1 time: 0 fps:Infinity type: int[]
rescaling texture
frame data:219648 frame: 2 time: 1426 fps:1.4025246 type: int[]
frame data:219648 frame: 3 time: 1468 fps:2.0435967 type: int[]
frame data:219648 frame: 4 time: 1505 fps:2.6578074 type: int[]
frame data:219648 frame: 5 time: 1553 fps:3.2195752 type: int[]
frame data:219648 frame: 6 time: 1626 fps:3.6900368 type: int[]
frame data:219648 frame: 7 time: 1629 fps:4.297115 type: int[]
frame data:219648 frame: 8 time: 1680 fps:4.7619047 type: int[]
frame data:219648 frame: 9 time: 1718 fps:5.2386494 type: int[]
frame data:219648 frame: 10 time: 1761 fps:5.6785917 type: int[]
frame data:219648 frame: 11 time: 1801 fps:6.107718 type: int[]
frame data:219648 frame: 12 time: 1844 fps:6.507592 type: int[]
frame data:219648 frame: 13 time: 1890 fps:6.878307 type: int[]
frame data:219648 frame: 14 time: 1928 fps:7.2614107 type: int[]
frame data:219648 frame: 15 time: 1969 fps:7.61808 type: int[]
frame data:219648 frame: 16 time: 2008 fps:7.9681277 type: int[]
frame data:219648 frame: 17 time: 2048 fps:8.300781 type: int[]
frame data:219648 frame: 18 time: 2090 fps:8.61244 type: int[]
frame data:219648 frame: 19 time: 2135 fps:8.899298 type: int[]
frame data:219648 frame: 20 time: 2175 fps:9.195402 type: int[]
frame data:219648 frame: 21 time: 2216 fps:9.476534 type: int[]
frame data:219648 frame: 22 time: 2262 fps:9.725906 type: int[]
frame data:219648 frame: 23 time: 2303 fps:9.986974 type: int[]
frame data:219648 frame: 24 time: 2355 fps:10.191083 type: int[]
frame data:219648 frame: 25 time: 2389 fps:10.46463 type: int[]
frame data:219648 frame: 26 time: 2428 fps:10.708402 type: int[]
frame data:219648 frame: 27 time: 2475 fps:10.909091 type: int[]
frame data:219648 frame: 28 time: 2506 fps:11.173184 type: int[]
frame data:219648 frame: 29 time: 2546 fps:11.390416 type: int[]
frame data:219648 frame: 30 time: 2596 fps:11.556241 type: int[]
frame data:219648 frame: 31 time: 2640 fps:11.742424 type: int[]
frame data:219648 frame: 32 time: 2675 fps:11.962617 type: int[]
frame data:219648 frame: 33 time: 2722 fps:12.123439 type: int[]
frame data:219648 frame: 34 time: 2755 fps:12.341197 type: int[]
frame data:219648 frame: 35 time: 2803 fps:12.486622 type: int[]
frame data:219648 frame: 36 time: 2846 fps:12.649333 type: int[]
frame data:219648 frame: 37 time: 2887 fps:12.8160715 type: int[]
frame data:219648 frame: 38 time: 2971 fps:12.790306 type: int[]
frame data:219648 frame: 39 time: 2973 fps:13.118062 type: int[]
May 25, 2009 7:49:45 PM com.jme.app.BaseSimpleGame cleanup
INFO: Cleaning up resources.
May 25, 2009 7:49:45 PM com.jme.app.BaseGame start
INFO: Application ending.
frame data:219648 frame: 40 time: 3006 fps:13.30672 type: int[]
mdb:43, lastbuf:0 skipping granule 0

llama · May 26, 2009, 2:24am

Did you follow the installation instructions? If you follow them correctly and the bytebufferrenderer is properly installed you should not see any video playing in JMStudio.

sbook · May 27, 2009, 12:08am

llama said:

Did you follow the installation instructions? If you follow them correctly and the bytebufferrenderer is properly installed you should not see any video playing in JMStudio.

Thanks for the reply, llama

followed the instructions using method 4. Where i was previously able to see video and hear sound in jmStudio, I now only hear sound. I also removed JavaSound for good measure.

I've tried: AVI(xVid/mp3), MOV(photo-jpg/uncompressed pcm), MOV(jpeg2000/uncompressed pcm)

mulova · June 5, 2009, 6:40am

I've got it worked on jme2.

Thanks llama for this wonderful plugin.

below is the patch for jme2 version

I hope it helps.

Index: testJFjr/org/llama/jmf/TestJMFVideoImage.java =================================================================== --- testJFjr/org/llama/jmf/TestJMFVideoImage.java (revision 20168) +++ testJFjr/org/llama/jmf/TestJMFVideoImage.java (working copy) @@ -32,9 +32,22 @@ */ +import java.util.Vector; + +import javax.media.Format; +import javax.media.PackageManager; +import javax.media.PlugInManager; +import javax.media.format.AudioFormat; +import javax.media.format.RGBFormat; +import javax.media.format.VideoFormat; +import javax.media.protocol.ContentDescriptor; + import com.jme.animation.SpatialTransformer; import com.jme.app.SimpleGame; +import com.jme.image.Image; import com.jme.image.Texture; +import com.jme.image.Texture2D; +import com.jme.image.Texture.MinificationFilter; import com.jme.math.Quaternion; import com.jme.math.Vector3f; import com.jme.scene.Spatial; @@ -56,6 +69,7 @@ Spatial spatial; protected void simpleInitGame() { + initFobs(); // enable for some extra debug data ByteBufferRenderer.printframes = true; @@ -70,7 +84,7 @@ // //////////////////////////// /* change to your file path */ // //////////////////////////// - String file = "c:\my\media.mpg"; + String file = "c:/my/media.mpg"; // ////////////////////////////////////////////////////////// /* change to switch between a spinning box and a quad test */ @@ -98,6 +112,7 @@ else { try { image = new JMFVideoImage(file, true, JMFVideoImage.SCALE_FIT); + image.setFormat(Image.Format.RGBA8); } catch (Exception e) { e.printStackTrace(); } @@ -113,8 +128,8 @@ } rootNode.attachChild(spatial); - tex = new Texture(); - tex.setMipmapState(Texture.MM_LINEAR); + tex = new Texture2D(); + tex.setMinificationFilter(MinificationFilter.BilinearNoMipMaps); tex.setImage(image); TextureState ts = display.getRenderer().createTextureState(); ts.setEnabled(true); @@ -183,8 +198,178 @@ public static void main(String[] arg) { TestJMFVideoImage app = new TestJMFVideoImage(); - app.setDialogBehaviour(SimpleGame.ALWAYS_SHOW_PROPS_DIALOG); + app.setConfigShowMode(ConfigShowMode.AlwaysShow); app.start(); } + + private boolean isFobsInited(Vector rendList){ + String rname = null; + for (int i = 0; i < rendList.size(); i++) { + rname = (String) (rendList.elementAt(i)); + if (rname.equals("org.llama.jmf.ByteBufferRenderer")) { + return true; + } + } + return false; + } + + private void initFobs(){ + Vector rendererlist=PlugInManager.getPlugInList(null,null,PlugInManager.RENDERER); + if(isFobsInited(rendererlist))return; + Format[] ffmpegformat = new VideoFormat[] { new VideoFormat("FFMPEG_VIDEO") }; + Format[] ffmpegdesc = new ContentDescriptor[] {new ContentDescriptor("video.ffmpeg")}; + Format[] ffmpegaudio = new Format[] { new AudioFormat("FFMPEG_AUDIO") }; + Format[] supportedInputFormats = new VideoFormat[] { + + new VideoFormat("iv31"), //CODEC_ID_INDEO3 + new VideoFormat("iv32"), + + new VideoFormat("msvc"), //CODEC_ID_MSVIDEO1 + new VideoFormat("cram"), + new VideoFormat("wham"), + + new VideoFormat("wmv1"), //CODEC_ID_WMV1 + + new VideoFormat("wmv2"), //CODEC_ID_WMV2 + + new VideoFormat("mpeg"), //CODEC_ID_MPEG1VIDEO + new VideoFormat("mpg1"), + new VideoFormat("mpg2"), + new VideoFormat("pim1"), + new VideoFormat("vcr2"), + + new VideoFormat("mjpa"), //CODEC_ID_MJPEG + new VideoFormat("mjpb"), + new VideoFormat("mjpg"), + new VideoFormat("ljpg"), + new VideoFormat("jpgl"), + new VideoFormat("avdj"), + + new VideoFormat("svq1"), //CODEC_ID_SVQ1 + new VideoFormat("svqi"), + + new VideoFormat("svq3"), //CODEC_ID_SVQ3 + + new VideoFormat("mp4v"), //CODEC_ID_MPEG4 + new VideoFormat("divx"), + new VideoFormat("dx50"), + new VideoFormat("xvid"), + new VideoFormat("mp4s"), + new VideoFormat("m4s2"), + new VideoFormat("div1"), + new VideoFormat("blz0"), + new VideoFormat("ump4"), + + + new VideoFormat("h264"), //CODEC_ID_H264 + + new VideoFormat("h263"), //CODEC_ID_H263 + + new VideoFormat("u263"), //CODEC_ID_H263P + new VideoFormat("viv1"), + + new VideoFormat("i263"), //CODEC_ID_i263 + + new VideoFormat("dvc"), //CODEC_ID_DVVIDEO + new VideoFormat("dvcp"), + new VideoFormat("dvsd"), + new VideoFormat("dvhs"), + new VideoFormat("dvs1"), + new VideoFormat("dv25"), + + + new VideoFormat("vp31"), //CODEC_ID_VP3 + + new VideoFormat("rpza"), //CODEC_ID_RPZA + + new VideoFormat("cvid"), //CODEC_ID_CINEPAK + + new VideoFormat("smc"), //CODEC_ID_SMC + + + new VideoFormat("mp42"), // CODEC_ID_MSMPEG4V2 + new VideoFormat("div2"), + + new VideoFormat("mpg4"), // CODEC_ID_MSMPEG4V1 + + new VideoFormat("div3"), // CODEC_ID_MSMPEG4V3 + new VideoFormat("mp43"), + new VideoFormat("mpg3"), + new VideoFormat("div5"), + new VideoFormat("div6"), + new VideoFormat("div4"), + new VideoFormat("ap41"), + new VideoFormat("col1"), + new VideoFormat("col0") + + }; + Format[] frgb = new VideoFormat[] {new RGBFormat()}; + Format[] alinear=new AudioFormat[]{new AudioFormat("LINEAR")}; + //registre le renderer + PlugInManager.addPlugIn("org.llama.jmf.ByteBufferRenderer",frgb,null,PlugInManager.RENDERER); + Vector plist=PlugInManager.getPlugInList(null,null,PlugInManager.RENDERER); + //move the plugin to the top of the list + Object last=plist.lastElement(); + plist.insertElementAt(last,0); + plist.remove(plist.lastIndexOf(last)); + PlugInManager.setPlugInList(plist,PlugInManager.RENDERER); + // try{PlugInManager.commit();}catch(Exception e){e.printStackTrace();}; + System.out.println("RENDERERn"+plist.toString()); + //registre le demultiplexer + + /*PlugInManager.addPlugIn("com.omnividea.media.parser.video.Parser", + ffmpegformat, + null, + PlugInManager.DEMULTIPLEXER); */ + + Format[] In; + Format[] Out; + In=ffmpegdesc; + Out=null; + PlugInManager.addPlugIn("com.omnividea.media.parser.video.Parser", + In, + Out, + PlugInManager.DEMULTIPLEXER); + plist=PlugInManager.getPlugInList(null,null,PlugInManager.DEMULTIPLEXER); + System.out.println("DEMULTIPLEXERn"+plist.toString()); + + //registre les codecs + + + In=ffmpegformat; + Out=frgb; + PlugInManager.addPlugIn("com.omnividea.media.codec.video.NativeDecoder", + In, + Out, + PlugInManager.CODEC); + + In=ffmpegaudio; + Out=alinear; + + PlugInManager.addPlugIn("com.omnividea.media.codec.audio.NativeDecoder", + In, + Out, + PlugInManager.CODEC); + + //PlugInManager.addPlugIn("com.omnividea.media.codec.video.JavaDecoder",supportedInputFormats,defaultOutputFormats,PlugInManager.CODEC); + In=supportedInputFormats; + Out=frgb; + PlugInManager.addPlugIn("com.omnividea.media.codec.video.JavaDecoder", + In, + Out, + PlugInManager.CODEC); + plist=PlugInManager.getPlugInList(null,null,PlugInManager.CODEC); + System.out.println("CODECSn"+plist.toString()); + + //registre le package + Vector packagePrefix = PackageManager.getProtocolPrefixList(); + String myPackagePrefix = new String("com.omnividea"); + packagePrefix.add(0,myPackagePrefix); + PackageManager.setProtocolPrefixList(packagePrefix); + + System.out.println(PackageManager.getProtocolPrefixList().toString()); + + return; + } } Index: testJFjr/org/llama/jmf/JMFVideoImage.java =================================================================== --- testJFjr/org/llama/jmf/JMFVideoImage.java (revision 20168) +++ testJFjr/org/llama/jmf/JMFVideoImage.java (working copy) @@ -147,22 +147,22 @@ switch (format.getBitsPerPixel()) { case 32: pixelformat = GL12.GL_BGRA; - this.setType(Image.RGBA8888); + setFormat(Format.RGBA8); break; case 16: pixelformat = GL12.GL_BGRA; - this.setType(Image.RGBA5551); + setFormat(Format.RGB5A1); break; case 24: default: pixelformat = GL12.GL_BGR; - this.setType(Image.RGB888); + setFormat(Format.RGB8); } // TODO: not actually used yet. switch(format.getPixelStride()) { case 1: - if (this.getType() == Image.RGBA5551) { + if (getFormat() == Format.RGB5A1) { /* * setFormat will be a jME method to set the native texture image format. * (currently always GL11.GL_UNSIGNED_BYTE) @@ -205,7 +205,8 @@ this.width = size; this.height = size; - data = ByteBuffer.allocateDirect(size*size*4).order(ByteOrder.nativeOrder()); + data.clear(); + data.add(ByteBuffer.allocateDirect(size*size*4).order(ByteOrder.nativeOrder())); ready = true; inittexture = true;

Roccia85 · December 9, 2009, 2:18pm

Hi, I'm a newbie of jME video rendering and I read all your posts but I havn't understood if it works on jME2 and how can I do it…

Can someone helps me? I have to put a video in a jMEDesktop…

Have I to install something on my PC or I can add only jmf.jar and fobs4jmf.jar to my project?

Thanks