[Solved] Save Camera View to an Image File

Hey everyone,
what’s the easiest / cleanest way to save a camera view to an image file?

I want to place multiple cameras in a scene that film different sections and save their view to image files from time to time (e.g. every 2 seconds). Later i want to use these images e.g. to train some AI models.

I already read about the ScreenshotAppState (https://wiki.jmonkeyengine.org/docs/3.4/core/app/state/screenshots.html) but i was not able to apply this to a specific camera. I also read about the AtlasGeneratorState (https://github.com/Simsilica/SimArboreal-Editor/blob/master/src/main/java/com/simsilica/arboreal/AtlasGeneratorState.java) which is mentioned in multiple questions about screenshots in this forum but again i was not able to apply this to my project.

Can someone post the code to save the view of a specified camera to an image file? Or explain it in easy words (i am new to JMonkeyEngine)?

Thanks in advance!

Kind regards,
Nico

1 Like

Maybe not the answer you are hoping for, but I guess your problem stems from fixating on the camera directly.
A camera is (more or less) just an object with a transform and some frustum.
Everything renderable will be rendered to a framebuffer of some viewport.
In the case of the screenshot app state this is the default viewport’s framebuffer, which is the screen.

So, if you want multiple cameras at once, you probably need multiple frame buffers and some custom code or multiple viewports in general (where you can attach the app states).

Basically create ViewPorts and add your scene to the scenes list.
See LegacyApplication/SimpleApplication for the required house-keeping.
I think that’s also what the atlas state does, however obviously more specific to the use-case there.

2 Likes

Thank you very much for your response. After an extensive research yesterday i took notice of the ViewPort and FrameBuffer stuff.

I am currently working on an implementation for generating pictures using ViewPorts and Buffers, i will share it here when it woks :sunglasses:

2 Likes

As promised, here is my solution! It took me quite a wile, but now it works :sunglasses:

I was able to solve my issue after reading about TestRenderToMemory. I used this example and adapted it for my use case.

First the main class: Here all the scene setup is done. Also a second camera (i called it robCam) and view port (robView) is created. Don’t get confused by the names, but since i want to use my “offscreen” camera to be the eye of a robot the name is applicable :wink:

package panda;

import com.jme3.app.SimpleApplication;
import com.jme3.light.SpotLight;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.math.FastMath;
import com.jme3.math.Quaternion;
import com.jme3.math.Vector3f;
import com.jme3.renderer.Camera;
import com.jme3.renderer.ViewPort;
import com.jme3.scene.Geometry;
import com.jme3.scene.shape.Box;
import com.jme3.scene.shape.Quad;
import states.OffCameraState;
/**
 * This is the Main Class of your Game. You should only do initialization here.
 * Move your Logic into AppStates or Controls
 * @author normenhansen
 */
public class Main extends SimpleApplication {
    
    /** Stores the spot light source. */
    SpotLight spot = new SpotLight();
    
    /** Stores the main application. */
    static Main app;
    
    /** The width of the additional robot camera. */
    int width = 1440;
    /** The height of the additional robot camera. */
    int height = 1080;
    /** The additional robot camera. */
    Camera robCam;
    
    /** Additional ViewPort for the robot camera. */
    ViewPort robView;
    
    /** Handles the additional camera and grabs images from it. */
    OffCameraState offCam;

    public static void main(String[] args) {
        app = new Main();
        app.start();
        
    }

    @Override
    public void simpleInitApp() {
        cam = app.getCamera();
        cam.setLocation(new Vector3f(0.f, 1.f, 3.f));
        
        /* Load example geometry and material */
        Material defaultMat = new Material( assetManager, "Common/MatDefs/Misc/ShowNormals.j3md");
        Box boxShape = new Box(new Vector3f(0.f, 1.f, 0.f), 1.f, 1.f, 1.f);
        Geometry box = new Geometry("Box", boxShape);
        box.setMaterial(defaultMat);
        rootNode.attachChild(box);
        
        /* Create floor */
        Quad quadMesh = new Quad(20.f, 20.f);
        Geometry quad = new Geometry("floor", quadMesh);
        quad.rotate(-90.f * FastMath.DEG_TO_RAD, 0.f, 0.f);
        quad.setLocalTranslation(-10.f, 0.f, 10.f);
        quad.setMaterial(defaultMat);
        rootNode.attachChild(quad);
        
        /* Light */
        /** A cone-shaped spotlight with location, direction, range */
        spot.setSpotRange(100); 
        spot.setSpotOuterAngle(89 * FastMath.DEG_TO_RAD); 
        spot.setSpotInnerAngle(15 * FastMath.DEG_TO_RAD); 
        spot.setDirection(new Vector3f(0.f, -1.f, 0.f));
        spot.setPosition(new Vector3f(0.f, 9.f, 0.f)); 
        spot.setColor(ColorRGBA.White.mult(1.3f));
        rootNode.addLight(spot); 

        /* Background */
        viewPort.setBackgroundColor(ColorRGBA.Blue);
        
        /* Initialize and setup additional camera, view and buffer (this will be used for offline rendering) */
        robCam = new Camera(width, height);
        robView = renderManager.createPreView("Robot View", robCam);
        // cam
        robCam.setLocation(new Vector3f(3.f, 4.f, -3.f));
        robCam.setRotation(new Quaternion(0.27653885f, -0.3852068f, 0.122174986f, 0.87190324f));
        robCam.setFrustumPerspective(45.f, (float) (width / height), 1.f, 1000.f);
        // view
        robView.setBackgroundColor(ColorRGBA.Magenta); // note: Different background!
        robView.attachScene(rootNode);
        robView.setEnabled(true);
        robView.setClearFlags(true, true, true);

        offCam = new OffCameraState(robCam, renderManager, robView);
        stateManager.attach(offCam);
        
    }
    
    protected int loops = 0;
    
    @Override
    public void simpleUpdate(float tpf) {
        if (loops == 100) {
            offCam.saveCurrentImage("/Path/To/Image/File/image.png");
            System.out.println("Done!");
        }
        loops++;
        
    }

}

As you can see i implemented a new appstate called OffCameraState. This state handles a camera and grabs images from it. By calling the method saveCurrentImage() the current data is written to an image file. As you can see in my main class i call this method after 100 loops.

package states;

import com.jme3.app.state.AbstractAppState;
import com.jme3.post.SceneProcessor;
import com.jme3.profile.AppProfiler;
import com.jme3.renderer.Camera;
import com.jme3.renderer.RenderManager;
import com.jme3.renderer.Renderer;
import com.jme3.renderer.ViewPort;
import com.jme3.renderer.queue.RenderQueue;
import com.jme3.texture.FrameBuffer;
import com.jme3.texture.Image;
import com.jme3.util.BufferUtils;
import com.jme3.util.Screenshots;
import java.awt.geom.AffineTransform;
import java.awt.image.AffineTransformOp;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.nio.ByteBuffer;
import javax.imageio.ImageIO;

/**
 *
 * @author
 */
public class OffCameraState extends AbstractAppState implements SceneProcessor{
    
    private final Renderer renderer;
    private final ViewPort viewPort;
    
    private BufferedImage image;
    
    private ByteBuffer cpuBuf;
    private FrameBuffer offBuf;
    private final int width, height;
    
    public OffCameraState(Camera cam, RenderManager renderManager, ViewPort vp) {
        width = cam.getWidth();
        height = cam.getHeight();
        renderer = renderManager.getRenderer();
        viewPort = vp;
        
        image = new BufferedImage(width, height, BufferedImage.TYPE_INT_BGR);
        
        init();
    }
    
    /**
     * Initialization stuff.
     */
    private void init() {
        viewPort.addProcessor(this);
        
        cpuBuf = BufferUtils.createByteBuffer(width * height * 4);
        
        offBuf = new FrameBuffer(width, height, 1);
        offBuf.setDepthBuffer(Image.Format.Depth);
        offBuf.setColorBuffer(Image.Format.RGBA8);
        
        viewPort.setOutputFrameBuffer(offBuf);
    }
    
    /**
     * Get the current data from the GPU and store it in cpuBuf.
     */
    private void updateImageContents(){
        cpuBuf.clear();
        renderer.readFrameBuffer(offBuf, cpuBuf);

        Screenshots.convertScreenShot2(cpuBuf.asIntBuffer(), image);    
    }
    
    /**
     * Save the current data in cpuBuf as image.
     * 
     * @param path Specifies where the image should be saved.
     */
    public void saveCurrentImage(String path) {
        try {
            File f = new File(path);
            f.createNewFile();
            flipImage();
            ImageIO.write(image, "jpg", f);
        } catch (IOException e) {
            // TODO: Handle exception.
        }
        
    }
    
    /**
    * Credits to: https://stackoverflow.com/questions/9558981/flip-image-with-graphics2d
    */
    private void flipImage() {
        // Flip the image vertically and horizontally; equivalent to rotating the image 180 degrees
        AffineTransform tx = AffineTransform.getScaleInstance(-1, -1);
        tx.translate(-image.getWidth(null), -image.getHeight(null));
        AffineTransformOp op = new AffineTransformOp(tx, AffineTransformOp.TYPE_NEAREST_NEIGHBOR);
        image = op.filter(image, null);
    }

    @Override
    public void initialize(RenderManager arg0, ViewPort arg1) {
        // throw new UnsupportedOperationException("Not supported yet."); //To change body of generated methods, choose Tools | Templates.
    }

    @Override
    public void reshape(ViewPort arg0, int arg1, int arg2) {
        // throw new UnsupportedOperationException("Not supported yet."); //To change body of generated methods, choose Tools | Templates.
    }

    @Override
    public void preFrame(float arg0) {
        // throw new UnsupportedOperationException("Not supported yet."); //To change body of generated methods, choose Tools | Templates.
    }

    @Override
    public void postQueue(RenderQueue arg0) {
        // throw new UnsupportedOperationException("Not supported yet."); //To change body of generated methods, choose Tools | Templates.
    }

    @Override
    public void postFrame(FrameBuffer arg0) {
        /*
         * Grab the new camera image.
         */
        updateImageContents();
    }

    @Override
    public void setProfiler(AppProfiler arg0) {
        // throw new UnsupportedOperationException("Not supported yet."); //To change body of generated methods, choose Tools | Templates.
    }
}

This example runs in jMonkeyEngineSDK v3.3.0.

2 Likes