Video Texture

Hi,



i know it's possible to render a video on a texture using JMF/FOBS…

My question is, how to use it with jME2 … i read this text very carefully



http://www.jmonkeyengine.com/jmeforum/index.php?topic=10561.0



I've loaded the src ( http://www.mediafire.com/file/iumoymbvnje/src.rar )  and did the change from "JOC"


         data = new ArrayList<ByteBuffer>();
         data.add(ByteBuffer.allocateDirect(size*size*4).order(ByteOrder.nativeOrder()));



Everything runs great without errors but, nothing is displayed on the quad. I hear the sound of the media files,
but the quad is still white... I found the problem:

//tex.setImage(image);



This line is commented but it shouldn't be... Without the slashes the program aborts with this error message :
       
        at com.jme.image.Image.getEstimatedByteSize(Image.java:663)

Looking up this method shows, that the image doesn't have a format... and so there is a NullPointerException...

Now my question:

How can i solve this problem ? Maybe the topic creator could help... it seems to be that he solved this problem....


Greetings

Kr0e

Since Quaddro didn't mind spamming the board with "up up" messages I'd say spam his PM and email to the point where he says what he did :wink:



More to the point, try setting a format. Either RGB8 or RGBA8 I think.

Hi,

ok this solved the problem but creates a new one, too.



Now the error message is :



java.lang.NullPointerException

        at com.jme.scene.state.lwjgl.LWJGLTextureState.load(LWJGLTextureState.java:486)



I've tested RGB8 and RGBA8…



Any idea ?



PS:



The code which makes problems:


image.getData(0).rewind();



But i don't know which var is null... getData(0) can't be null...


->


         data = new ArrayList<ByteBuffer>();
         data.add(ByteBuffer.allocateDirect(size*size*4).order(ByteOrder.nativeOrder()));

okay,

i've solved it… I should read the tutorials more carefully the next time…

Now i've registered the ByteBufferRenderer and everything runs !

@Llama:



I'm using Ubuntu 8.10… My framerate is very small… sometimes only 15… (flv format)

I did your patch but this didn't change the speed… tomorrow i'll test it on a windows pc… maybe

it's a ubuntu problem…



Greetings from germany…



Kr0e

So could YOU post the solution this time then? For others to find in the future…

Make sure you're using the native MPEG4 plugins for JMF/FOBS for video decoding (explained in the tutorial).

Other than that, the slow down can come from the software YV12 -> RGB conversion, especially for high resolution video. Profile the code and find where the bottleneck is.

Hi,

what do you mean with "diff" ?



xD



@Momoko_Fan :



I'm using the native libs, otherwise the flv video couldn't be played… (millions of exception are trown :D)

Ok, the trick were very simple… The image didn't has a format and the ByteBuffer were null, because the methode "setSize" was never called. After registering the correct renderer (ByteBufferRenderer) by the JMStudio, everythings runs great :slight_smile: To register the renderer you have to pack the COMPILED ByteBufferRenderer in a jar file. Then you have to change the start script called "runjmstudio.sh". (You must add the new jar with the ByteBufferRenderer to the classpath)



That's it !



So, your code was completely correct llama. After making a few jme2 related changes, the code runs correct.



Greetings from Germany!



Kr0e





PS: It's not really a trick xD Llama wrote all needed things in his tutorial:

http://www.jmonkeyengine.com/jmeforum/index.php?topic=2908.0

But i didn't read it carefully the first time  ://

Please could you give us a clear diff?

So do you still have the performance issues?



I can think of 2 things… if I remember correctly my default example limits your FPS to that of the video, secondly the new version of FOBS might not be compatible with the extra performance patch anymore (though I think I have described somewhere in these threads how to patch FOBS itself).



A diff is a file that shows all changes you made. However, if you could just package up all the source files in a zip, that'd be a help too…

Kr0e said:

Hi,
what do you mean with "diff" ?

I mean this:
http://svnbook.red-bean.com/en/1.0/re09.html

Yes, i still have the performence problems… Do you remember, where you create the FPS limiter ?



I use the Fobs4JMF-0.4.1 version of FOBS… the latest version only runs of windows…



I use Java 6… maybe this is one reason for the slow motion :smiley:



By the way… i 've tested the example on a windows pc with the same result…



I'm new with the monkey engine… where could i upload the zip ?



Greetings

Ok,

at first i want to say that i upload the package when i've solved the performence problem…



I removed your "FPS-Brake"… I think it was this part :



    @Override
   protected void simpleUpdate() {
      if (!image.update(tex, false)) {
         // if we did not update the texture, wait some time, but when the
         // next frame is ready, wait no longer.
         //image.waitSome(3);
      }

   }



Now the jME shows more than 450 FPS... Thats ok! But the video still runs with 8 fps and so the "FirstPersonInput" react very "laggy"....  The console output tells me , that the program uses the native libs correctly... So maybe there is another brake ??

Greetings


PS:

I find the bottleneck!!

It's this method:


         // TODO: use this.dataformat
         GL11.glTexSubImage2D(GL11.GL_TEXTURE_2D, 0, 0, 0, videowidth, videoheight,pixelformat,
               GL11.GL_UNSIGNED_BYTE, buffer);



I think the problem is the conversion of the formats!!
I hope anybody has an idea !

PSS:

During bugfixing i noticed another problem:

At the moment the update method of the JMEVideoImage class is called in the main loop... So if this update method is slow the whole game will be slow... Now i have the idea to call this update method in a single timer thread... But then i get this exception:

Exception in thread "Timer-0" java.lang.NullPointerException
        at org.lwjgl.opengl.GL11.glDisable(GL11.java:995)

I think this is because a GL method is called outside the main thread... But how could i fix this ? I mean... If i want to disable textures or something else outside this method, the whole program will shutdown ...

Use the game task manager to post a Callable to execute your task on the OpenGL thread.

… but then you're back in the same thread. I think about the only way to properly do it is by creating a whole new context, you don't want to get involved with that right now.



And while it's true  the conversion from one format of pixels to another takes some time, it's only a few percent difference. I think the biggest issue right now is that your computer is busy getting those 450fps and not decoding the video. Try putting a small sleep in the game loop. Another quick fix could be enableling vsync.



Also, how many FPS does your video have?


The video i've tested is a flv video with 30 fps. Your ByteBufferRenderer shows 9 fps … But jME shows 450 fps… But i'm sure that the problem is this method



// TODO: use this.dataformat
         GL11.glTexSubImage2D(GL11.GL_TEXTURE_2D, 0, 0, 0, videowidth, videoheight,pixelformat,
               GL11.GL_UNSIGNED_BYTE, buffer);



When i comment this method, ByteBufferRenderer shows 29.999 Fps and jME shows 1,5K FPS... But then there is nothing displayed  :lol:

A short "System.currentMillis" before and after glTexSubImage2D shows that this method needs more then 50-70 ms... So, i don't know whether this is a "normal" time for converting pixel formats... but i think it'S not the best solution... At the moment i try pixel buffers to accelerate this method... Have anybody experience with pixel buffers ?

GReetings

What video card? Latest drivers? What kind of resolution are the videos in? Is multisampling set in the control panel or application? What format is "pixelformat" in that code?

9 fps is too low even if using glTexImage, and I don't think pixel buffers will help you much. I have made a similar application using my own binding to libavcodec and had about 70 fps.

Yes, i think you’re right :frowning: I hate ATI & Linux support… Maybe i have to install windows  :’(

I’ll test this method on a windows PC… When it runs faster then 40 FPS i’ll be happy :smiley:



Kr0e

Ok, on the windows pc the method runs great…



@ Llama: Nice work!



Greetings



Kr0e

What's your distro/ati drivers/card? Ati/AMD has released some very good open source drivers the last few months… the closed source driver is much better too.