[SOLVED] Screen capture with alpha background

[joliver82]

Hi!

I’m trying to implement a simple tool to generate small png files with one 3d model and full transparent background to be used as assets for GUI in my project. I’m using ScreenshotAppState to get a screenshot of current rendered scene with alpha background but I’m not able to do so.

I’ve tried setting viewport to have alpha=0: viewPort.setBackgroundColor(new ColorRGBA(0,0,0,0)); but didn’t make the trick.

Maybe it’s not possible with ScreenshotAppState and I will need to implement it myself rendering current scene to a back buffer… but just in case here is my code:

public class Main extends SimpleApplication 
{
    static String assetStr=null;
    
    int frame=0;
    Vector3f defaultCameraLocation=new Vector3f(2, 2, 2);

    public static void main(String[] args) 
    {
        Main app = new Main();
        int width=0;
        int height=0;
        
        if(args.length<3)
        {
            System.err.println("Missing arguments");
            Main.printUsage();
            return;
        }
        else
        {
            try
            {
                width=Integer.parseInt(args[0]);
            }
            catch(Exception e)
            {
                System.err.println("Witdh must be an integer");
                Main.printUsage();
                return;
            }
            try
            {
                height=Integer.parseInt(args[1]);
            }
            catch(Exception e)
            {
                System.err.println("Height must be an integer");
                Main.printUsage();
                return;
            }
        }
        
        assetStr=args[2];
        
        AppSettings settings=new AppSettings(true);
        settings.setResolution(width, height);
        app.setSettings(settings);
        app.showSettings=false;

        app.setDisplayStatView(false);
        app.setDisplayFps(false);
        
        app.start();
    }
    
    public static void printUsage()
    {
        System.out.println("Usage:");
        System.out.println("\tscreenshot_generator <width> <height> <3d model>");
    }

    @Override
    public void simpleInitApp() 
    {
        viewPort.setClearColor(true);
        viewPort.setBackgroundColor(new ColorRGBA(0,0,0,0));
        
        flyCam.setDragToRotate(true);
        cam.setLocation(defaultCameraLocation);
        cam.lookAt(Vector3f.ZERO, Vector3f.UNIT_Y);
        
        assetManager.registerLoader(BlenderLoader.class, "blend");
        
        // find path for assets and set locator
        int lastSlash=assetStr.lastIndexOf("/");
        String assetsPath="./";
        if(lastSlash>0)
        {
            assetsPath=assetStr.substring(0, lastSlash+1);
            assetStr=assetStr.substring(lastSlash+1);
        }
        assetManager.registerLocator(assetsPath, FileLocator.class);
        
        // load model
        Spatial model=this.assetManager.loadModel(assetStr);
        rootNode.attachChild(model);
        
        // add a light
        DirectionalLight dl0=new DirectionalLight(new Vector3f(-1,-1,-1));
        rootNode.addLight(dl0);
        
        // configure screenshot
        String pngOutput=assetStr.substring(0,assetStr.lastIndexOf("."));
        ScreenshotAppState screenshot=new ScreenshotAppState(assetsPath, pngOutput);
        screenshot.setIsNumbered(false);
        screenshot.takeScreenshot();
        getStateManager().attach(screenshot);
    }
    
    @Override
    public void simpleUpdate(float tpf) 
    {
        if(frame>1)
        {
            System.exit(0);
        }
        else
        {
            ++frame;
        }
    }
}

Thanks

[Ali_RS]

Yeah, JME will drop alpha when writing image, see

jMonkeyEngine/jmonkeyengine/blob/a79be13ac83fdf421e00e6ca0c189d8f361b00d7/jme3-desktop/src/main/java/com/jme3/system/JmeDesktopSystem.java#L93

For detailed info, please see this topic

Generate transparent icon from model

I also need to generate transparent icon from model. This is my test code, but the generated png file has no transparency public class TestIconGenerator extends SimpleApplication { public static final FunctionId F_GENERATE_ICON = new FunctionId("Generate Icon"); private FrameBuffer fb; public TestIconGenerator() { } public static void main(String[] args) { TestIconGenerator main = new TestIconGenerator(); AppSettings settings = new AppSettings(true)…

I ended up rendering scene to a back buffer and using this utility class to export PNGs.

NemesisMate/util-jme3/blob/c7958446898adeffad74af3aa0eb9f0a35738085/util-jme3-base/src/main/java/com/nx/util/jme3/base/ImageUtil.java#L22

[joliver82]

Thanks for the detailed info. I’ll try it later

[Ali_RS]

By the way, this is the code I am using to generate an icon from a model. Hope you find it useful.

Most of the code is copied from pspeed AtlasGeneratorState.

public class IconGeneratorState extends BaseAppState {

    static Logger log = LoggerFactory.getLogger(IconGeneratorState.class);
    private final ArrayDeque<IconEntry> toGenerate = new ArrayDeque<>();
    private Vector3f camOffset = new Vector3f();
    private ViewPort offView;
    private Node offViewRoot;
    private FrameBuffer offBuffer;
    private boolean debugTexture = false;
    private int width;
    private int height;
    private int frameCount = -1;

    public IconGeneratorState() {
        this(256, 256);
    }

    public IconGeneratorState(int width, int height) {
        this.width = width;
        this.height = height;
    }

    public void setCameraOffset(Vector3f cameraOffset) {
        this.camOffset.set(cameraOffset);
    }

    public void generate(Spatial scene, File path) {
        generate(scene, path, Vector3f.ZERO);
    }

    public void generate(Spatial scene, File path, Vector3f cameraOffset) {
        setCameraOffset(cameraOffset);
        toGenerate.add(new IconEntry(scene, path));
        if (!offView.isEnabled()) {
            offView.setEnabled(true);
        }
    }

    @Override
    protected void initialize(Application app) {
        offViewRoot = new Node("RootNode");

        initLight();

        //setup framebuffer's cam
        Camera offCamera = app.getCamera().clone();
        offCamera.resize(width, height, true);
        float aspect = (float) width / (float) height;
        offCamera.setFrustumPerspective(45f, aspect, 0.1f, 1000f);

        offView = app.getRenderManager().createMainView("Offscreen View", offCamera);

        // create offscreen framebuffer
        offBuffer = new FrameBuffer(width, height, 1);

        //setup framebuffer's texture
        Texture2D fbTex = new Texture2D(width, height, Image.Format.RGBA8);

        //setup framebuffer to use texture
        offBuffer.setDepthBuffer(Image.Format.Depth);
        offBuffer.setColorTexture(fbTex);

        //set viewport to render to offscreen framebuffer
        offView.setOutputFrameBuffer(offBuffer);

        if (debugTexture) {
            //setup main scene
            Quad mesh = new Quad(2, 2);
            Geometry quad = new Geometry("box", mesh);

            Material mat = GuiGlobals.getInstance().createMaterial(false).getMaterial();
            mat.setTexture("ColorMap", fbTex);
            mat.getAdditionalRenderState().setBlendMode(RenderState.BlendMode.Alpha);
            quad.setMaterial(mat);
            ((SimpleApplication) getApplication()).getRootNode().attachChild(quad);
        }

        // attach the scene to the viewport to be rendered
        offView.attachScene(offViewRoot);

        offView.setClearFlags(true, true, true);
        offView.setBackgroundColor(new ColorRGBA(0f, 0f, 0f, 0f));
        offCamera.lookAtDirection(new Vector3f(0, 0, -1), Vector3f.UNIT_Y);
    }

    @Override
    protected void cleanup(Application app) {
        getApplication().getRenderManager().removePreView(offView);
    }

    @Override
    protected void onEnable() {
        offView.setEnabled(true);
    }

    @Override
    protected void onDisable() {
        offView.setEnabled(false);
    }

    @Override
    public void update(float tpf) {
        if (frameCount != -1) {
            frameCount++;
        }

        if (offView.isEnabled() && toGenerate.isEmpty()) {
            offView.setEnabled(false);
        }

        if (offView.isEnabled()) {
            if (!toGenerate.isEmpty() && offViewRoot.getQuantity() == 0) {
                // setup framebuffer's scene
                IconEntry entry = toGenerate.peek();
                // Keep parent reference for reattaching after icon generating
                entry.parent = entry.model.getParent();
                Node wrapperNode = new Node();
                wrapperNode.attachChild(entry.model);
                offViewRoot.attachChild(wrapperNode);
                wrapperNode.center();
                frameCount = 0;
            }
            offViewRoot.updateLogicalState(tpf);
            updateCamera();
        }
    }

    @Override
    public void render(RenderManager rm) {
        if (frameCount > 10) {
            Renderer renderer = rm.getRenderer();
            Image image = createFrameBufferImage(offBuffer);
            //Texture2D texture = new Texture2D(image);

            renderer.readFrameBuffer(offBuffer, image.getData(0));
            image.setUpdateNeeded();

            IconEntry entry = toGenerate.poll();
            try {
                log.info("Generating icon:" + entry.file);
                ImageUtils.writeImage(image, entry.file);
            } catch (IOException ex) {
                log.error("Error generating icon!", ex);
            }

            frameCount = -1;
            offViewRoot.detachAllChildren();
            // Reattach to it's parent, if it had one
            if (entry.parent != null) {
                entry.parent.attachChild(entry.model);
            }
        }

        offViewRoot.updateGeometricState();
    }

    private void initLight() {
        AmbientLight ambient = new AmbientLight(ColorRGBA.White);
        offViewRoot.addLight(ambient);
    }

    protected Image createFrameBufferImage(FrameBuffer fb) {
        int width = fb.getWidth();
        int height = fb.getHeight();
        int size = width * height * 4;
        ByteBuffer buffer = BufferUtils.createByteBuffer(size);
        Image.Format format = fb.getColorBuffer().getFormat();

        // I guess readFrameBuffer always writes in the same
        // format regardless of the frame buffer format
        //format = Image.Format.BGRA8;
        return new Image(format, width, height, buffer);
    }

    protected void updateCamera() {
        Camera camera = offView.getCamera();

        BoundingBox bb = (BoundingBox) offViewRoot.getWorldBound();

        Vector3f min = bb.getMin(null);
        Vector3f max = bb.getMax(null);

        float xSize = Math.max(Math.abs(min.x), Math.abs(max.x));
        float ySize = max.y - min.y;
        float zSize = Math.max(Math.abs(min.z), Math.abs(max.z));

        float size = ySize * 0.5f;
        size = Math.max(size, xSize);
        size = Math.max(size, zSize);

        // In the projection matrix, [1][1] should be:
        //      (2 * Zn) / camHeight
        // where Zn is distance to near plane.
        float m11 = camera.getViewProjectionMatrix().m11;

        // We want our position to be such that
        // 'size' is otherwise = cameraHeight when rendered.
        float z = m11 * size;

        // Add the z extents so that we adjust for the near plane
        // of the bounding box... well we will be rotating so
        // let's just be sure and take the max of x and z
        float offset = Math.max(bb.getXExtent(), bb.getZExtent());
        z += offset;
        // This creates problems because it makes way too much
        // space around the tree.  A proper solution would require
        // a bunch of math and in the end would also have to be duplicated
        // on the quad generation side or somehow stored with the atlas.
        Vector3f center = bb.getCenter();

        float sizeOffset = 0; // size - (ySize * 0.5f);

        Vector3f camLoc = new Vector3f(0, center.y, z);
        camLoc.addLocal(camOffset);
        camera.setLocation(camLoc);
        camera.lookAt(Vector3f.ZERO, Vector3f.UNIT_Y);
    }

    // Not working properly
    /*protected void savePng(File f, Image img) throws IOException {
        try (OutputStream out = new FileOutputStream(f)) {
            JmeSystem.writeImageFile(out, "png", img.getData(0), img.getWidth(), img.getHeight());
            log.debug("Icon saved at {}", f.getPath());
        }
    }*/

    private class IconEntry {
        Spatial model;
        Node parent;
        File file;

        public IconEntry(Spatial model, File file) {
            this.model = model;
            this.parent = model.getParent();
            this.file = file;
        }
    }
}

[joliver82]

Thanks @Ali_RS for the code, I’ve used a slightly modified version of it and works great!

I’m facing a weird issue, probably in my side… The capture is lighter than the realtime render:

I’ve setup lights in the exact same way for both nodes. Any idea why this could happen?

I’ll clean the code a little and post it later

EDIT: it was completely my fault, I was adding a copy of the whole root node of the scene (including lights) so I had twice the same lights Now it’s OK:

[Ali_RS]

Glad it works OK.

I am using a queue and a frame counter to add a slight delay before capturing (a dirty code to call it) because instant capturing or even one frame delay does not work correctly for me (results a blank image).

You may want to rip out the queue and frame counter things and see if instant capturing works for you.