JavaCV IplImage convert to jme Image

General Discussion
1

Guess what I am looking for is an easy way to create Augmented Reality Windows app using webcam thru Javacv (or whateva).
I can easily get IplImages from the webcam but how to update a JME Image which can be used on a background in a scene
is beyond me. :slight_smile: (I’m just a simple c# programmer in disguise).
Was briefly looking at ARmonkeytoolkit but are unsure about if this is the way to go, due to it being 4 or 5 years old now.

A small well structured augmented reality windows start app would be really great to have in JME portfolio.

Harald Heide Gundersen
Norw thing

2

I don’t have a detailed example of it, but here’s one approach:

  1. Use the Java API of OpenCV (I’ve seen better performance with OpenCV 3 than 2).
  2. There are numerous examples of how to create a BufferedImage from a Mat object on the web.
  3. Convert it to an image and create a Texture out of it.
  4. Use a post filter to display the image over the whole screen. Normally post filters are drawn last so some tweaking might be necessary.

Object recycling and management is (for me) the most tricky part with this approach. It’s important to release the native resources.

Getting good cameras is also important. With Logitech c920’s it’s possible to get 30fps, but not more. Despite that, it’s a good camera to work with for AR.

Edit: Oh and tip 3: Use multithreading to grab and convert the images for best performance. At least avoid doing it on the render thread.

3

This thingie seems to work OK getting a webcam image from IplImage and onto jme Image so you can use it in material on a quadmesh as a background in a scene.
(Giving augmented reality possibilities) Please fell free to comment on the code as I am not too familiar with threading in Java.
Also as stated above: Object recycling and management is (for me) the most tricky part with this approach. It’s important to release the native resources.
Would be just beautiful if anyone had a nice suggestion as to how to handle the above.
If anyone interested in the beginning of a simple Augmented reality project using javacv and a webcam please comment and I can post how I use this thingie in my Main.

package mygame;
import com.googlecode.javacv.cpp.opencv_core.IplImage;
import com.googlecode.javacv.CanvasFrame;
import com.googlecode.javacv.cpp.opencv_highgui;
import com.jme3.scene.shape.Quad;
import com.jme3.texture.Image;
import com.jme3.texture.Texture2D;
import com.jme3.texture.plugins.AWTLoader;
import com.sun.org.apache.xerces.internal.impl.dv.util.Base64;
import java.awt.image.BufferedImage;
/**
*

  • @author HaraldHeide
    */
    public class Webcam {

    public Image webcamJmeImage;
    public boolean mNewCameraFrameAvailable;

    private BufferedImage webcamBufferedImage;
    private opencv_highgui.CvCapture capture;

    public Webcam()
    {
    Thread myImageLoop = new Thread(new ImageLoop(“ImageLoop”));
    myImageLoop.start();
    }

    public class ImageLoop implements Runnable{
    public ImageLoop(String Name){
    }

     public void run(){
     capture = opencv_highgui.cvCreateCameraCapture(0); //0=Front, 1=Back camera
     opencv_highgui.cvSetCaptureProperty(capture, opencv_highgui.CV_CAP_PROP_FRAME_HEIGHT,720);
     opencv_highgui.cvSetCaptureProperty(capture, opencv_highgui.CV_CAP_PROP_FRAME_WIDTH,1280);
     IplImage webcamIplImage = opencv_highgui.cvQueryFrame(capture);

     while(opencv_highgui.cvQueryFrame(capture) != null)
     {
         webcamBufferedImage = webcamIplImage.getBufferedImage();
         webcamJmeImage = new AWTLoader().load(webcamBufferedImage, false);
         mNewCameraFrameAvailable = true;
     }
 }

    }
    }