Hello guys,
I'm working on a project where I have to overlay 3D objects on a webcam stream. Until recently I just grabbed frames from the webcam using http://www.exactfutures.com/index02.htm and then used these frames as a texture for a QUAD in JME. However since this rendering was rather slow, I looked into the Video Renderer from llama (http://www.jmonkeyengine.com/jmeforum/index.php?topic=2908.0) to render the video directly to a texture, and got it to work with my webcam. The only issue is that I can't get this to work with YUV cams, while the code from the first url I pasted here, actually does support them.
Now, I have the code of a project in .NET which does a similar thing. Only they don't render the webcam in 3D. In stead they just put the webcam images on a viewbox, which is behind the 3D scene. This way things go incredibly fast, and I would like to accomplish the same thing in Java.
Therefore my question is: Is it possible to just show a webcam stream (for instance in swing), while putting a 3D canvas with a transparent background on top of it? (Can a 3D canvas actually have a transparent background?)
I've been struggling with this for several nights now, so any help would be greatly appreciated 
Tnx alot,
Vincent
I don't think a 3D canvas with transparent background is possible… depends on the operating system.
Rendering video in 3D is actually quite fast, if you directly upload it to a texture without any mipmap creation or other useless stuff.
Tnx for your reply. Indeed you are right about the good performance of rendering video in a 3DCanvas, but that's only true if you directly render the frames to a texture, in stead of capturing the images and then using them to create a texture. And that's where the code from llama comes in, but this code only supports RGB cameras.
Anyway, I think I can try to use a codec for converting the YUV data into RGB, in the process method of the videorender.
As soon as I get it right, I will post the updated llame video render (or convertor) here.
JMF should convert YUV data to RGB if your jME Renderer object specifies it only supports RGB.
hmm, for now I'll just use this ByteBufferRenderer from llama with it's supported RGB format…
I tried to using a jmf codec (YUV2RGB), but for some reason I end up with a lot of green on my screen (also it looks like the captured image shows up twice, although it's hard to notice anything recognizable in the rendered image).
I'll probably try to add YUV support later on…