Generate transparent icon from model


#42

That makes it more complicated though. Could be that they define white/black as “transparent”. Anyway, good to know, I didn’t know BMP supports transparency :smiley:


#43

Yeah, and any of this would require a second method else we break all existing code… so we should think about it a lot first.


#44

Just test it with bmp format and it throws this exception:

java.io.IOException: Image can not be encoded with compression type BI_RGB
	at java.desktop/com.sun.imageio.plugins.bmp.BMPImageWriter.write(BMPImageWriter.java:322)
	at com.jme3.system.JmeDesktopSystem.writeImageFile(JmeDesktopSystem.java:113)
	at com.jme3.system.JmeSystem.writeImageFile(JmeSystem.java:134)
	at com.overthemoon.main.TestIconGenerator2.savePng(TestIconGenerator2.java:142)
	at com.overthemoon.main.TestIconGenerator2.simpleRender(TestIconGenerator2.java:203)
	at com.jme3.app.SimpleApplication.update(SimpleApplication.java:271)
	at com.jme3.system.lwjgl.LwjglWindow.runLoop(LwjglWindow.java:499)
	at com.jme3.system.lwjgl.LwjglWindow.run(LwjglWindow.java:581)
	at com.jme3.system.lwjgl.LwjglWindow.create(LwjglWindow.java:423)
	at com.jme3.app.LegacyApplication.start(LegacyApplication.java:463)
	at com.jme3.app.LegacyApplication.start(LegacyApplication.java:424)
	at com.jme3.app.SimpleApplication.start(SimpleApplication.java:125)
	at com.overthemoon.main.TestIconGenerator2.main(TestIconGenerator2.java:77)

So below code is not working for bmp.

public void writeImageFile(OutputStream outStream, String format, ByteBuffer imageData, int width, int height) throws IOException {       
        ImageWriter writer = ImageIO.getImageWritersByFormatName(format).next();
        ImageWriteParam writeParam = writer.getDefaultWriteParam();
        
        BufferedImage awtImage;
        if (format.equals("jpg")) {
            JPEGImageWriteParam jpegParam = (JPEGImageWriteParam) writeParam;
            jpegParam.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
            jpegParam.setCompressionQuality(0.95f);
            awtImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_BGR);
            Screenshots.convertScreenShot2(imageData.asIntBuffer(), awtImage);
            awtImage = verticalFlip(awtImage);
        } else {
            awtImage = new BufferedImage(width, height, BufferedImage.TYPE_4BYTE_ABGR);
            Screenshots.convertScreenShot(imageData, awtImage);
        }

        ImageOutputStream imgOut = new MemoryCacheImageOutputStream(outStream);
        writer.setOutput(imgOut);
        IIOImage outputImage = new IIOImage(awtImage, null, null);
        try {
            writer.write(null, outputImage, writeParam);
        } finally {
            imgOut.close();
            writer.dispose();
        }
    }

Should we treat bmp as non transparent (BufferedImage.TYPE_INT_BGR) ?


#45

Neither it works for wbmp:

java.lang.IllegalArgumentException: Only integral single-band bilevel image is supported.
	at java.desktop/com.sun.imageio.plugins.wbmp.WBMPImageWriter.checkSampleModel(WBMPImageWriter.java:311)
	at java.desktop/com.sun.imageio.plugins.wbmp.WBMPImageWriter.write(WBMPImageWriter.java:169)
	at com.jme3.system.JmeDesktopSystem.writeImageFile(JmeDesktopSystem.java:117)
	at com.jme3.system.JmeSystem.writeImageFile(JmeSystem.java:134)
	at com.overthemoon.main.TestIconGenerator2.savePng(TestIconGenerator2.java:141)
	at com.overthemoon.main.TestIconGenerator2.simpleRender(TestIconGenerator2.java:202)
	at com.jme3.app.SimpleApplication.update(SimpleApplication.java:271)
	at com.jme3.system.lwjgl.LwjglWindow.runLoop(LwjglWindow.java:499)
	at com.jme3.system.lwjgl.LwjglWindow.run(LwjglWindow.java:581)
	at com.jme3.system.lwjgl.LwjglWindow.create(LwjglWindow.java:423)
	at com.jme3.app.LegacyApplication.start(LegacyApplication.java:463)
	at com.jme3.app.LegacyApplication.start(LegacyApplication.java:424)
	at com.jme3.app.SimpleApplication.start(SimpleApplication.java:125)
	at com.overthemoon.main.TestIconGenerator2.main(TestIconGenerator2.java:76)

#46

Ah I just notice this in javadoc of JmeSystem.writeImageFile()

It clearly says to use jpg or png, so I guess we should not care about other formats. :slightly_smiling_face:


#47

I’m criticizing the finer details here, but line 119 states:

It indicates a form of compression takes place. BMP is a raw lossless format. So maybe there’s your get-out clause for utilitarian functionality.

Edit: Then again, so is PNG. There’s a discrepancy in the javadoc there, strictly speaking.


#48

I see what you’re saying, but I think it could also be read the other way to say that it takes a raw image (uncompressed pixel stream) and converts it to a standard compressed image format. In any case, it’s a bit ambiguous and it might be a good idea to clarify.


#49

Okay, now main issue with transparent png exporting is fixed on master branch.

I have yet one small issue, see below generated png icon:

As you can see behind the leaves (marked in red) is transparent.
Any way to fix it ?

And here is my code:

public class IconGeneratorState extends BaseAppState {

    static Logger log = LoggerFactory.getLogger(IconGeneratorState.class);

    private Node root;
    private FrameBuffer offBuffer;
    private ViewPort offView;
    private Geometry wireBounds;
    private boolean debugBounds = false;
    private boolean debugTexture = false;
    private int width;
    private int height;
    private File path;

    public IconGeneratorState() {
        this(256, 256);
    }
    
    public IconGeneratorState(int width, int height) {
        this.width = width;
        this.height = height;
    }

    @Override
    protected void initialize(Application app) {
        root = new Node("RootNode:");
        Camera offCamera = new Camera(width, height);

        offView = getApplication().getRenderManager().createPreView("Offscreen View", offCamera);
        offView.setClearFlags(true, true, true);
        offView.setBackgroundColor(ColorRGBA.BlackNoAlpha);

        // create offscreen framebuffer
        offBuffer = new FrameBuffer(width, height, 1);

        //setup framebuffer's cam
        offCamera.setFrustumPerspective(45f, 1f, 1f, 1000f);
        offCamera.setLocation(new Vector3f(0f, 0f, 10f));
        offCamera.lookAt(new Vector3f(0f, 0f, 0f), Vector3f.UNIT_Y);

        //setup framebuffer's texture
        Texture2D offTex = new Texture2D(width, height, Image.Format.RGBA8);
        //offTex.setMinFilter(Texture.MinFilter.Trilinear);
        //offTex.setMagFilter(Texture.MagFilter.Bilinear);

        //setup framebuffer to use texture
        offBuffer.setDepthBuffer(Image.Format.Depth);
        offBuffer.setColorTexture(offTex);

        //set viewport to render to offscreen framebuffer
        offView.setOutputFrameBuffer(offBuffer);

        // attach the scene to the viewport to be rendered
        offView.attachScene(root);

        if (debugTexture) {
            //setup main scene
            Quad mesh = new Quad(2, 2);
            Geometry quad = new Geometry("box", mesh);

            Material mat = GuiGlobals.getInstance().createMaterial(false).getMaterial();
            mat.setTexture("ColorMap", offTex);
            //mat.getAdditionalRenderState().setBlendMode(RenderState.BlendMode.Alpha);
            quad.setMaterial(mat);
            ((SimpleApplication) getApplication()).getRootNode().attachChild(quad);
        }
        
        //initFilters();
    }
    
    private void initFilters() {
        FilterPostProcessor fpp = new FilterPostProcessor(getApplication().getAssetManager());
        fpp.addFilter(new FXAAFilter());
        fpp.addFilter(new ToneMapFilter(Vector3f.UNIT_XYZ.mult(11.0f)));//4.0f
        fpp.addFilter(new SSAOFilter(0.5f, 3, 0.2f, 0.2f));
        offView.addProcessor(fpp);
    }

    @Override
    protected void cleanup(Application app) {
        getApplication().getRenderManager().removePreView(offView);
    }

    @Override
    protected void onEnable() {
        offView.setEnabled(true);
    }

    @Override
    protected void onDisable() {
        offView.setEnabled(false);
    }

    float time;
    @Override
    public void update(float tpf) {
        root.updateLogicalState(tpf);
        root.updateGeometricState();
        updateCamera();
        time += tpf;
    }

    @Override
    public void render(RenderManager rm) {
        if (path != null && time > 0.2) {
            Renderer renderer = rm.getRenderer();
            Image image = createFrameBufferImage(offBuffer);
            //Texture2D texture = new Texture2D(image);

            renderer.readFrameBuffer(offBuffer, image.getData(0));
            //image.setUpdateNeeded();

            try {
                savePng(path, image);
            } catch (IOException ex) {
                log.error("Error generating icon!", ex);
            }

            path = null;
        }
    }

    public void generateIcon(Spatial model, File path) {
//        Vector3f bound = ((BoundingBox) model.getWorldBound()).getExtent(new Vector3f()).multLocal(2);
//        float max = Math.max(Math.max(bound.x, bound.y), bound.z);
//        float resize = baseSize / max;
//        model.scale(resize);
        // setup framebuffer's scene
        root.detachAllChildren();
        root.attachChild(model);
   
        this.path = path;
        this.time = 0;
    }

    protected void updateCamera() {

        BoundingBox bb = (BoundingBox) root.getWorldBound();

        Vector3f min = bb.getMin(null);
        Vector3f max = bb.getMax(null);

        float xSize = Math.max(Math.abs(min.x), Math.abs(max.x));
        float ySize = max.y - min.y;
        float zSize = Math.max(Math.abs(min.z), Math.abs(max.z));

        float size = ySize * 0.5f;
        size = Math.max(size, xSize);
        size = Math.max(size, zSize);

        // In the projection matrix, [1][1] should be:
        //      (2 * Zn) / camHeight
        // where Zn is distance to near plane.
        float m11 = offView.getCamera().getViewProjectionMatrix().m11;

        // We want our position to be such that
        // 'size' is otherwise = cameraHeight when rendered.
        float z = m11 * size;

        // Add the z extents so that we adjust for the near plane
        // of the bounding box... well we will be rotating so
        // let's just be sure and take the max of x and z
        //float offset = Math.max(bb.getXExtent(), bb.getZExtent());
        //z += offset;
        // This creates problems because it makes way too much
        // space around the tree.  A proper solution would require
        // a bunch of math and in the end would also have to be duplicated
        // on the quad generation side or somehow stored with the atlas.
        Vector3f center = bb.getCenter();

        float sizeOffset = 0;

        Vector3f camLoc = new Vector3f(0, center.y + sizeOffset, z);
        offView.getCamera().setLocation(camLoc);

        if (debugBounds) {
            WireBox box;
            if (wireBounds == null) {
                box = new WireBox();
                wireBounds = new Geometry("wire box", box);
                Material mat = GuiGlobals.getInstance().createMaterial(ColorRGBA.Yellow, false).getMaterial();
                wireBounds.setMaterial(mat);
                root.attachChild(wireBounds);
            } else {
                box = (WireBox) wireBounds.getMesh();
            }
            box.updatePositions(bb.getXExtent(), bb.getYExtent(), bb.getZExtent());
            box.setBound(new BoundingBox(new Vector3f(0, 0, 0), 0, 0, 0));
            wireBounds.setLocalTranslation(bb.getCenter());
            //wireBounds.setLocalRotation(leafGeom.getLocalRotation());
        }
    }

    protected void savePng(File f, Image img) throws IOException {
        try (OutputStream out = new FileOutputStream(f)) {
            JmeSystem.writeImageFile(out, "png", img.getData(0), img.getWidth(), img.getHeight());
            log.debug("Icon saved at {}", f.getPath());
        }
    }

    protected Image createFrameBufferImage(FrameBuffer fb) {
        int width = fb.getWidth();
        int height = fb.getHeight();
        int size = width * height * 4;
        ByteBuffer buffer = BufferUtils.createByteBuffer(size);
        Image.Format format = fb.getColorBuffer().getFormat();

        // I guess readFrameBuffer always writes in the same
        // format regardless of the frame buffer format
        format = Image.Format.BGRA8;
        return new Image(format, width, height, buffer);
    }
}

#50

Seems related to issues discussed in:

So, if those are separate geometry then make sure the trunk is in the opaque bucket and tree in the transparent bucket… else you may need to turn on the alpha threshold.


#51

It’s a single geometry. I enabled alpha threshold and it fixed the issue.
Thank you so much. :slightly_smiling_face:


#52

I seem to have yet another problem :frowning:

In this specific model, this is how my model looks in real-time and when exported to jpg.

but when exporting to png it looks like:

This happens only in this specific model.
I am clueless what may be the issue, anybody have any guess what may cause this ?

Edit:
I am using the same code I provided above.


#53

It’s acting like you are exporting RGB as BGR or vice-versa… ie: red and blue are flipped.


#54

hmm, then shouldn’t same effect happen when exporting to jpg too.
As you know this is how they are exported : jpg uses BufferedImage.TYPE_INT_BGR and png uses BufferedImage.TYPE_4BYTE_ABGR

BufferedImage awtImage;
        if (format.equals("jpg")) {
            JPEGImageWriteParam jpegParam = (JPEGImageWriteParam) writeParam;
            jpegParam.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
            jpegParam.setCompressionQuality(0.95f);
            awtImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_BGR);
            Screenshots.convertScreenShot2(imageData.asIntBuffer(), awtImage);
            awtImage = verticalFlip(awtImage);
        } else {
            awtImage = new BufferedImage(width, height, BufferedImage.TYPE_4BYTE_ABGR);
            Screenshots.convertScreenShot(imageData, awtImage);
        }

Edit:
I wonder if Screenshots.convertScreenShot might be faulty:


#55

Was the coloring right before your changes?


#56

Just tested it and yes it is right.

This is the png file exported with JME 3.2 (off-course there no transparency) :

I am putting the old code here:

@Override
    public void writeImageFile(OutputStream outStream, String format, ByteBuffer imageData, int width, int height) throws IOException {       
        BufferedImage awtImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_BGR);
        Screenshots.convertScreenShot2(imageData.asIntBuffer(), awtImage);

        ImageWriter writer = ImageIO.getImageWritersByFormatName(format).next();
        ImageWriteParam writeParam = writer.getDefaultWriteParam();

        if (format.equals("jpg")) {
            JPEGImageWriteParam jpegParam = (JPEGImageWriteParam) writeParam;
            jpegParam.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
            jpegParam.setCompressionQuality(0.95f);
        }

        awtImage = verticalFlip(awtImage);
        
        ImageOutputStream imgOut = new MemoryCacheImageOutputStream(outStream);
        writer.setOutput(imgOut);
        IIOImage outputImage = new IIOImage(awtImage, null, null);
        try {
            writer.write(null, outputImage, writeParam);
        } finally {
            imgOut.close();
            writer.dispose();
        }
    }

Edit:

My doubt is even Screenshots.convertScreenShot() is not working as it is supposed to or Java’s ImageIO may have issue ?


#57

TYPE_4BYTE_ABGR

Represents an image with 8-bit RGBA color components with the colors Blue, Green, and Red stored in 3 bytes and 1 byte of alpha.

I didn’t read the whole post, but is this what we want?

Shouldn’t we use

int TYPE_INT_ARGB
Represents an image with 8-bit RGBA color components packed into integer pixels.

And

`` TYPE_INT_RGB
Represents an image with 8-bit RGB color components packed into integer pixels.

for non alpha.

Or is there any reason for which BGR is preferable?


#58

In your newer code, I wonder what happens if you make it:
Screenshots.convertScreenShot(imageData.asIntBuffer(), awtImage);
…I don’t know if that nukes the alpha or not.

Or we may go back and look at the 3.0 code when it worked for PNG and see what’s different.


#59

Okay guys, I am giving up on JME’s method to write image files.
I found this ImageUtil class created by @NemesisMate and tested it on bunch of models and it works like a charm.

This is the example model exported to PNG with ImageUtil.


#60

I mean, it used to work… that I’m sure of.