[SOLVED] Create 2d Image from 3d Scene

I have an 3d Scene of an maze that I want to run on Android, but the idea is to show it as an image in the background.
The idea is to render the maze in memory, render the scene also in memory, take an “print screen” and generate an image with that, and use the image in the game play scene.
Is that possible ?

Take a look at the screenshot appstate,
you could probably create something based on this. (with jme in headless)

Or look at TestRenderToTexture.

Thanks, that was easy, I didnt notice the example :slight_smile:

It seens its not so easy as I thought …
Its getting render state change error :

State was changed after rootNode.updateGeometricState() call.

package mygame;

import com.jme3.app.SimpleApplication;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Vector3f;
import com.jme3.renderer.Camera;
import com.jme3.renderer.ViewPort;
import com.jme3.scene.Geometry;
import com.jme3.scene.shape.Box;
import com.jme3.scene.shape.Quad;
import com.jme3.texture.FrameBuffer;
import com.jme3.texture.Image;
import com.jme3.texture.Texture;
import com.jme3.texture.Texture2D;

public class Test extends SimpleApplication {

    public static void main(String[] args) {
        Main app = new Main();
        app.start();
    }

    @Override public void simpleInitApp() {

        int width = 512; int height = 512;

        Camera offCamera = new Camera(width, height);

        ViewPort offView = renderManager.createPreView("Offscreen View", offCamera);
        offView.setClearFlags(true, true, true);
        offView.setBackgroundColor(ColorRGBA.DarkGray);

        // create offscreen framebuffer
        FrameBuffer offBuffer = new FrameBuffer(width, height, 1);

        //setup framebuffer's cam
        offCamera.setFrustumPerspective(45f, 1f, 1f, 1000f);
        offCamera.setLocation(new Vector3f(0f, 0f, -5f));
        offCamera.lookAt(new Vector3f(0f, 0f, 0f), Vector3f.UNIT_Y);

        //setup framebuffer's texture
        Texture2D offTex = new Texture2D(width, height, Image.Format.RGBA8);
        offTex.setMinFilter(Texture.MinFilter.Trilinear);
        offTex.setMagFilter(Texture.MagFilter.Bilinear);

        //setup framebuffer to use texture
        offBuffer.setDepthBuffer(Image.Format.Depth);
        offBuffer.setColorTexture(offTex);

        //set viewport to render to offscreen framebuffer
        offView.setOutputFrameBuffer(offBuffer);

        // setup framebuffer's scene
        //setup3DScene();
        Box boxMesh = new Box(Vector3f.ZERO, 1,1,1);
        Material material = new Material(Main.ref_assetManager, "Common/MatDefs/Light/Lighting.j3md");
        Geometry offBox = new Geometry("box", boxMesh);
        offBox.setMaterial(material);
        offView.attachScene(offBox);

        // render the offText Texture on a quad
        Quad mesh = new Quad(9, 7);
        Geometry quad = new Geometry("quad",mesh);
        quad.setLocalTranslation((float)-9/2,(float)-7/2,0);
        Material mat = new Material(Main.ref_assetManager, "Common/MatDefs/Misc/Unshaded.j3md");
        mat.setTexture("ColorMap", offTex);
        quad.setMaterial( mat );

        rootNode.attachChild(quad);
    }

}

Do I have to do the offBox.updateLogicalState(tpf); and offBox.updateGeometricState(); on the first frame update ?

Answering to myself : Just the updateGeometriState :

@Override public void simpleUpdate(float tpf){
    if (offView.isEnabled()) {
        offBox.updateGeometricState();
    }
}

Thanks !

Ok, now its working, BUT :

When I remove the 3d objects from the scene or disable the offView, it somehow clean the texture/image…
Even if the texture is an clone of the texture from the offTex…
I did notice Texture use the CloneableSmartAsset that dont seen to be really clone the texture…
The porpose of it is to reduce the load of vertices and faces, etc, on the scene showing the maze at the background, the way it is, I am showing the image, but if I remove the 3d objects it also remove the image…
How can I clone it correctly ?

The problematic code :

        mazebackgroundImage.getMaterial().setTexture("ColorMap", mazetexture.getTexture().clone());
        mazetexture.cleanUp();

the cleanup proc :

public void cleanUp() {
    offView.setEnabled(false);
    for(int t=0;t<=renderNode.getChildren().size()-1;t++)
        renderNode.getChild(t).removeFromParent();
}

Should I open an new topic for this ?

I think you need to paint the texture to a real texture.

You can see where I create real textures that persist here:

You might have to unwrap what I’m doing a bit but it works. I use that code for generating imposters for my trees.

That looks nice, I tried to make it works but I got an error when trying to read the bytes with .getData(0):

java.lang.IllegalStateException: Unsupported buffer type: null

public static Texture cloneThisTexture(Texture tex) {
    Renderer renderer = Main.ref_renderer;
    FrameBuffer framebuffer = new FrameBuffer(tex.getImage().getWidth(), tex.getImage().getHeight(), 1);
    framebuffer.setDepthBuffer(Format.Depth);
    framebuffer.setColorTexture(new Texture2D(tex.getImage().getWidth(), tex.getImage().getHeight(), Format.RGBA8));
    renderer.readFrameBuffer(framebuffer, tex.getImage().getData(0)); // <<=== the line I got error
    Image img = createFrameBufferImage( framebuffer );
    return new Texture2D(img);
}

private static Image createFrameBufferImage( FrameBuffer fb ) {
    int width = fb.getWidth();
    int height = fb.getHeight();
    int size = width * height * 4;
    ByteBuffer buffer = BufferUtils.createByteBuffer(size);
    Image.Format format = fb.getColorBuffer().getFormat();
    format = Image.Format.RGBA8;
    return new Image(format, width, height, buffer);
}

I am not sure if I implemented it right, but I did notice something when I looked on your code, at line 135 there is:

format = Format.BGRA8;

But its invalid on jm3.0… Should it be format = Format.RGBA8; as in my translation ?
Maybe its in 3.1 and maybe its my problem ?

Also, I am getting confuse on how it works, should it be equal : ??

        Texture2D newimg = new Texture2D(mazetexture.getTexture().getImage().clone());
        newimg.equals(mazetexture.getTexture());

The funy thing is that if I remove clone its equals, but not with clone, and the clone one is empty…

Well, sure… because if you clone it then it’s a different instance so not equal anymore.

Did you try it?

Or what happens if you leave that line out and just let the color buffer’s format fall through?

At any rate, your exception indicates that the image wasn’t setup right.

Probably because you are trying to use the image from the texture (which the whole point of this is that it isn’t really an image) instead of the one you cretaed right in the next line.

You might want to step through the code in your head to understand what it’s doing instead of just randomly poking, maybe.

Did you try it?

Yes, it gives an error on compiling :

error: cannot find symbol
format = Format.BGRA8;
symbol: variable BGRA8
location: class Format

It seens this line is doing nothing thought…

About the other answers I just found out I have the FrameBuffer already, no need to convert to texture to framebuffer. I will play a bit with that :slight_smile:

We are not ever converting a texture to a frame buffer. So I have no idea what you are talking about.

So… then… no? You didn’t try what you suggested.

Edit: and anyway that line might not be needed in 3.1 because the framebuffer’s format might work.

Ok, thats a bit hard to understand.

  1. I have an “scene texture” created the same way its done on TestRenderToTexture.
  2. Since I dont need animation or anything, after the first update tick, my plan is to take this generated texture and put in an quad on screen, and remove all the scene.

I tried the FrameBuffer, but no sucess, if I just try to do this the quad shows black on screen :

        Image img = Utils.createFrameBufferImage( mazetexture.getFrameBuffer() );
        Texture2D newimg = new Texture2D( img );
        mazebackgroundImage.getMaterial().setTexture("ColorMap" , newimg ); 

Note that if I do this it works, so the texture and framebuffer is valid:

mazebackgroundImage.getMaterial().setTexture("ColorMap" , mazetexture.getTexture() );

So I am confuse … There would be much more easy if the texture/image classes has an truedeepclone() proc…

:chimpanzee_facepalm:

You have to take the frame buffer and render it to an image. So… you have to take the frame buffer and create a compatible image. Then you have to RENDER/PAINT/COPY/WHATEVER the framebuffer data from the GPU to your newly created image.

Your first attempt tried to paint the frame buffer image to the already existing framebuffer texture’s image and then created an empty image for some reason. ie: it attempted to paint an image on top of itself (which won’t work anyway because there is no image in the framebuffer texture) then it created the image.

Your second attempt appears not even to paint at all.

You have to take the frame buffer and render it to an image. So… you have to take the frame buffer and create a compatible image.

I really thought I was doing exactly this here … Maybe this image is not compatible ?
I checked both are RGBA8, so I dont know…

 Image img = Utils.createFrameBufferImage( mazetexture.getFrameBuffer() );
 Texture2D newimg = new Texture2D( img );
 mazebackgroundImage.getMaterial().setTexture("ColorMap" , newimg );

Squinting to see the code for this method but can’t see it. The old code was way broken as I’ve already described.

This is the code I toked from your repo, I just changed the format line, but I really think this makes no diference :

public static Image createFrameBufferImage( FrameBuffer fb ) {
    int width = fb.getWidth();
    int height = fb.getHeight();
    int size = width * height * 4;
    ByteBuffer buffer = BufferUtils.createByteBuffer(size);
    Image.Format format = fb.getColorBuffer().getFormat();
    format = Format.RGBA8; // ofiginal invalid one : format = Format.BGRA8;
    return new Image(format, width, height, buffer);
}

Do you think there is any problem on this ?

So, you create a blank texture and never paint to it.

Show me where you call
renderer.readFrameBuffer(framebuffer, tex.getImage().getData(0));

How else will the image end up in the image if you never render it?

This is the code that setup the texture :

    Camera offCamera = new Camera(sizew, sizeh);

    offView = Main.ref_renderManager.createPreView("Offscreen View", offCamera);
    offView.setClearFlags(true, true, true);
    offView.setBackgroundColor(ColorRGBA.Blue);

    // create offscreen framebuffer
    offBuffer = new FrameBuffer(sizew, sizeh, 1);

    //setup framebuffer's cam
    offCamera.setFrustumPerspective(45f, 1f, 1f, 10f);
    offCamera.setLocation(new Vector3f(0f, 0f, 6f));
    offCamera.lookAt(new Vector3f(0f, 0f, 0f), Vector3f.UNIT_Y);

    //setup framebuffer's texture
    offTex = new Texture2D(sizew, sizeh, Image.Format.RGBA8);
    offTex.setMinFilter(Texture.MinFilter.Trilinear);
    offTex.setMagFilter(Texture.MagFilter.Bilinear);

    //setup framebuffer to use texture
    offBuffer.setDepthBuffer(Image.Format.Depth);
    offBuffer.setColorTexture(offTex);

    //set viewport to render to offscreen framebuffer
    offView.setOutputFrameBuffer(offBuffer);

    // setup framebuffer's scene
    setup3DScene( renderNode );
    //ownerNode.attachChild(renderNode);

    // attach the scene to the viewport to be rendered
    offView.attachScene(renderNode);

    // Enable render tick update
    spatial=new Node();
    ownerNode.attachChild(spatial); spatial.addControl(this);

This is similar to the TestRenderToTexture.

Still not seeing anywhere that you are rendering to the image.