[SOLVED] Flipping the display

Troubleshooting graphics
1

Hi all,

I have a driving simulation developed in JME3, which I have been using successfully for neuroscience research.

I am currently modifying this to be compatible with an MRI scanner, and am running into an annoying problem: the screen is viewed by the participant via a mirror, which means that it is flipped in the X axis.

I have tried to flip the graphics display via the graphics card, but this apparently only has an option to rotate the display.

I have tried to embed the Lwjgl Canvas in a Swing component, and flip the graphics via its paint function, but this also fails (possible because it only uses the bounds and not the actual Graphics object??).

So I am wondering if anyone has any advice/suggestions on how to flip the display along the X axis? Either using JME itself, its dependencies, or the OS (e.g., graphics card driver - which in our case is an NVIDIA card).

Happy to provide further information!

Thanks,

Andrew

1 Like
2

not sure(would require some Renderer changes?), but i know one idea.

It is: Create Quad with flip UV x coords and render main view into this quad, while main render to just render this uv flipped quad.

maybe view setup could allow something like this too, and might be more simple. (see wiki multiple views or JME Tests → multiview test)

2 Likes
3

To me the easiest way would be to create a post-processor that samples the texture at (1.0-u, v) instead of (u,v).

Edit: could maybe start with the fog filter and strip almost everything out of it.

1 Like
4

Thanks for your quick reply, I will check out the multiple views option first.

I guess this would flip the 3D scene, but not the Nifti components I superimpose on these? I can deal with these separately if necessary.

Ideally I’d like a solution that flips the entire screen, including Nifti components / text, as this would remove the need to fiddle with the position and rendering of components - but if this is the best solution I can work with that!

1 Like
5

Thanks, that sounds like a valid option as well. I’ll try both of the above suggestions and let you know how they work out.

6

Why donot you flip your coordinates & the gravity ? & You will have flipped scene + flipped gravity to fit the scene items :grinning:through rotation, I think rotating rootNode would flip the whole image upside down

1 Like
7

I don’t know if that qualifies as flipping but before you do that ^, just rotate the camera.

2 Likes
8

I mean, it seems pretty clear to me what they mean. They want the screen to look right in a mirror, including text, whether the steering wheel is on the right or left, cars are in the left/right lane, etc.

1 Like
9

There is a quick solution to your problem, that will work regardless of the scene setup, postprocessing queues, viewports etc… , since it works on raw framebuffers, but it uses engine’s internals, so it might break in future versions of the engine, if we were to change those internals.

If you are ok with this, the solution is to mirror-blit the output framebuffer into an auxiliary framebuffer and then blit the auxiliary framebuffer back into the output framebuffer.
This is the code:


import java.lang.reflect.Field;

import com.jme3.renderer.RenderManager;
import com.jme3.renderer.opengl.GL;
import com.jme3.renderer.opengl.GLFbo;
import com.jme3.renderer.opengl.GLRenderer;
import com.jme3.texture.FrameBuffer;
import com.jme3.texture.Image.Format;

public class FrameBufferFlip {
    private static  GLFbo glfbo;
    private static FrameBuffer mirrorFb;

    private static  void init(GLRenderer renderer) {
        if(glfbo != null) return;
        try{
            for(Field f:renderer.getClass().getDeclaredFields()){
                Class t=f.getType();
                if(GLFbo.class.isAssignableFrom(t)){
                    f.setAccessible(true);
                    glfbo=(GLFbo)f.get(renderer);
                }
            }
        }catch(Exception e){
            e.printStackTrace();
        }
    }

    public static  void flip(RenderManager rm,FrameBuffer inputFb,Format colorFormat,int width,int height){
        if(!(rm.getRenderer() instanceof GLRenderer))throw new RuntimeException("Non GL renderer not supported by this method");

        GLRenderer glRenderer=(GLRenderer)rm.getRenderer();
        init(glRenderer);        // Get engine internals

        if(mirrorFb!=null&&(mirrorFb.getWidth()!=width||mirrorFb.getHeight()!=height||mirrorFb.getColorBuffer().getFormat()!=colorFormat)){
            mirrorFb.dispose();
            mirrorFb=null;
        }
        if(mirrorFb==null){
            mirrorFb=new FrameBuffer(width,height,1);
            mirrorFb.setColorBuffer(colorFormat);
        }
        
        if(inputFb != null && inputFb.isUpdateNeeded())  glRenderer.updateFrameBuffer(inputFb);
        if(mirrorFb != null && mirrorFb.isUpdateNeeded())  glRenderer.updateFrameBuffer(mirrorFb);
        
        // Flip on auxiliary fb
        glfbo.glBindFramebufferEXT(GLFbo.GL_READ_FRAMEBUFFER_EXT,inputFb==null?0:inputFb.getId());
        glfbo.glBindFramebufferEXT(GLFbo.GL_DRAW_FRAMEBUFFER_EXT,mirrorFb.getId());
        glfbo.glBlitFramebufferEXT(
            width, 0, 0, height,
            0, 0, width, height,
        GL.GL_COLOR_BUFFER_BIT, GL.GL_NEAREST);


        // Blit back to original fb
        glfbo.glBindFramebufferEXT(GLFbo.GL_READ_FRAMEBUFFER_EXT,mirrorFb.getId());
        glfbo.glBindFramebufferEXT(GLFbo.GL_DRAW_FRAMEBUFFER_EXT,inputFb==null?0:inputFb.getId());
        glfbo.glBlitFramebufferEXT(
            0, 0, width, height,
            0, 0, width, height,
        GL.GL_COLOR_BUFFER_BIT, GL.GL_NEAREST);

    }


}

The usage is

        FrameBufferFlip.flip(
            renderManager,
            YOUR_OUTPUT_FRAMEBUFFER,
            YOUR_OUTPUT_COLOR_FORMAT, //Must be the same format used in YOUR_OUTPUT_FRAMEBUFFER
            SCREEN_WIDTH,
            SCREEN_HEIGHT
        );

Call this anywhere, after you are sure the previous draw calls have been called.
In a common SimpleApplication, you can call it by overriding the update(tpf) like so

    @Override
    public void update() {
        super.update();
        FrameBufferFlip.flip(renderManager,viewPort.getOutputFrameBuffer(),Format.RGB8,viewPort.getCamera().getWidth(),viewPort.getCamera().getHeight());
    }
1 Like
10

Thanks, this seems to be exactly what I need!

I’ve implemented it exactly as above, but sadly the rendering appears to stop altogether when this is called in update().

I just upgraded the JME platform to 3.2.4 from 3.1.0, but I get the same issue. I’m not sure how to debug: I’m guessing that “blit” copies the frame buffer from the first to the second buffers passed to “bind”, and flipping X0 and X1 in this call should invert the x coordinates, but for some reason the renderer is no longer updating.

I made a small change to ensure the color formats match; they didn’t at first. This doesn’t fix the issue however:

public static  void flip(RenderManager rm,FrameBuffer inputFb,int width,int height){
    if(!(rm.getRenderer() instanceof GLRenderer))throw new RuntimeException("Non GL renderer not supported by this method");

    GLRenderer glRenderer=(GLRenderer)rm.getRenderer();
    init(glRenderer);        // Get engine internals

    if(mirrorFb!=null&&(mirrorFb.getWidth()!=width||mirrorFb.getHeight()!=height)){
        mirrorFb.dispose();
        mirrorFb=null;
    }
    if(mirrorFb==null){
        mirrorFb=new FrameBuffer(width,height,1);
        mirrorFb.setColorBuffer(inputFb.getColorBuffer().getFormat());
    }
    
    if(inputFb != null && inputFb.isUpdateNeeded())  glRenderer.updateFrameBuffer(inputFb);
    if(mirrorFb != null && mirrorFb.isUpdateNeeded())  glRenderer.updateFrameBuffer(mirrorFb);
    
    // Flip on auxiliary fb
    glfbo.glBindFramebufferEXT(GLFbo.GL_READ_FRAMEBUFFER_EXT,inputFb==null?0:inputFb.getId());
    glfbo.glBindFramebufferEXT(GLFbo.GL_DRAW_FRAMEBUFFER_EXT,mirrorFb.getId());
    glfbo.glBlitFramebufferEXT(
        width, 0, 0, height,
        0, 0, width, height,
    GL.GL_COLOR_BUFFER_BIT, GL.GL_NEAREST);


    // Blit back to original fb
    glfbo.glBindFramebufferEXT(GLFbo.GL_READ_FRAMEBUFFER_EXT,mirrorFb.getId());
    glfbo.glBindFramebufferEXT(GLFbo.GL_DRAW_FRAMEBUFFER_EXT,inputFb==null?0:inputFb.getId());
    glfbo.glBlitFramebufferEXT(
        0, 0, width, height,
        0, 0, width, height,
    GL.GL_COLOR_BUFFER_BIT, GL.GL_NEAREST);

}

The loop is still running fine, responds to input, and plays sound, and there is no Exception or warning in the console. Any ideas?

11

It seems viewPort.getOutputFrameBuffer() in your application is not the output framebuffer, there might be some code that binds another framebuffer inside the application update loop, the solution is to find the right output framebuffer and pass it to FrameBufferFlip.flip (you might try null for the default framebuffer)

Also, can you give more info about your application setup? Your change makes me think you are rendering on an offscreen framebuffer buffer. Is that the case? Is it maybe an awt canvas application?

For debug you can run the app with https://renderdoc.org/ , from there you will be able to see the gl calls and get a better understanding of the issue, you can also take a capture of the gl calls that you can share here, if you want me to take a look.

Oh, and please, double check you are calling super.update() before FrameBufferFlip.flip in your update method

1 Like
12

Passing null worked like a charm! That’s fantastic, thanks for your excellent responses :slight_smile:

It is odd, I don’t see where a new FrameBuffer is being set in the code (it is build on a fork of the opends API), but in any case using the default one does the trick.

Cheers,

Andrew

2 Likes