Integrating Skija/Skia renderer with JME

Troubleshooting graphics
1

Hello everyone,

I am building a UI library with the HumbleUI\Skija package, I’m successfully using it while building it out in my game at the moment, but I’m starting to hit performance limits using it with the bitmap rendering back end.

Currently I’m encoding the Skia buffer to PNG then uploading that as a texture to JME, this works but is too slow for complex UI to feel snappy.

Skija/Skia can use an OpenGL backend so I’m now trying to upgrade using this example:

Skia OpenGL DirectContext

apparently Skija’s DirectContext can pick up with the open GL context, thats what the documentation and examples imply?

I’ve put together the basic test case, this renders the blue cube when this line is commented out of simpleUpdate():

skia_surface = getGLSurface(app.getCamera().getWidth(), app.getCamera().getHeight());

But when that line is included, the screen is black, the blue cube does not render, however the debug stats do show the number of objects flicking from between 4/5 as you move the camera with the mouse past where the cube should be, no errors, the debug statements all print okay and the FPS stays at 60.

The test case:



import com.jme3.app.SimpleApplication;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.scene.Geometry;
import com.jme3.scene.shape.Box;
import com.jme3.system.AppSettings;
import com.jme3.system.JmeContext;
import io.github.humbleui.skija.*;

import org.lwjgl.opengl.GL11;


public class GLTest extends SimpleApplication {

    public static GLTest app;
    public Surface skia_surface = null;


    public static void main(String[] args) {

        app = new GLTest();

        AppSettings settings = new AppSettings(true);

        //settings.disp

        settings.setFullscreen(false);
        settings.setWidth(600);
        settings.setHeight(600);

        settings.setTitle("My Awesome Game");
        app.setSettings(settings);

        app.start(JmeContext.Type.Display);

    }

    @Override
    public void simpleInitApp() {

        Box b = new Box(1, 1, 1);
        Geometry geom = new Geometry("Box", b);

        Material mat = new Material(assetManager, "Common/MatDefs/Misc/Unshaded.j3md");
        mat.setColor("Color", ColorRGBA.Blue);
        geom.setMaterial(mat);

        rootNode.attachChild(geom);

    }


    public static Surface getGLSurface(int width, int height) {

        // Initialize Skija's context with JME's OpenGL context
        DirectContext skija_context = DirectContext.makeGL();
        //skija_context.
        logStatic("DirectContext ptr: " + skija_context._ptr);

        // Get the current framebuffer ID from JME
        int frame_buffer_id = GL11.glGetInteger(0x8CA6);
        //int frame_buffer_id = GL30.glGenFramebuffers();
        logStatic("frame_buffer_id: " + frame_buffer_id);

        // Create Skija's BackendRenderTarget using JME's dimensions and framebuffer
        BackendRenderTarget render_target = BackendRenderTarget.makeGL(
                width, height, /* samples */ 0, /* stencil */ 8, frame_buffer_id, FramebufferFormat.GR_GL_RGBA8
        );

        // Create a surface for drawing
        Surface surface = Surface.makeFromBackendRenderTarget(
                skija_context, render_target, SurfaceOrigin.BOTTOM_LEFT, SurfaceColorFormat.RGBA_8888, ColorSpace.getSRGB()
        );

        return surface;

    }

    @Override
    public void simpleUpdate(float tpf) {

        if (skia_surface == null) {

            skia_surface = getGLSurface(app.getCamera().getWidth(), app.getCamera().getHeight());
            log("skia_surface initialised");
            return;

        } else {

            /*
            Paint paint = new Paint();

            //paint.setBlendMode(BlendMode.DARKEN);
            paint.setMode(PaintMode.FILL);
            paint.setColor(ColorRGBA.Cyan.asIntARGB());

            Rect box_size = new Rect(100, 100, 200, 200);

            skia_surface.getCanvas().drawRect(box_size, paint);

             */

        }

    }

    public void log(String s) {
        System.out.println("Main: " + s);
    }

    public static void logStatic(String s) {
        System.out.println("Main[static]: ");
    }

}

This outputs:

Main[static]: DirectContext ptr: 1609112682016
Main[static]: frame_buffer_id: 0
Main: skia_surface initialised

I’ve included a bit of skia rendering code in simpleUpdate() thats block commented out, but that would be a valid draw call if the skia surface was set up

I’m at the limits of my knowledge with OpenGL contexts, FrameBuffer ID’s and such, i’m fumbling around not sure if I’m on the right path at the moment… Does anyone have any idea why I get the black screen?

Or is there a better way to do this integration?

Thanks

2

If the UI library provides alternate means of intercepting its calls, you might try to look at how Nifty did this sort of integration.

I don’t know if it’s helpful… but trying to coordinate direct GL context access with JME (who assumes it owns it) is probably going to be tricky.

3

I have my own UI library built with Skija: Obsidian. I did the jME integration as a standalone module: Obsidian-jME.

I’m going to second what @pspeed said and add that I’ve never worked on a project where sharing a GL context between two different renderers worked out successfully (this is not the only time I’ve done something like this - the other time was an attempt to overlay Qt over the results of a C++ OpenGL geographical renderer). The solution I ended up using for Obsidian-jME was creating a second context set up to share textures with the jME context (provided they are properly constructed, sharing textures between OpenGL contexts is supported by the spec). This means that Obsidian/Skija & jME have their own OpenGL state with no opportunity for interference, and jME and Skija are both able to render hardware accelerated frames without ever copying pixel buffers or GPU → CPU → GPU round-tripping.

If you have any interest in kicking the tires on Obsidian, I’d love to get some feedback on it - I consider it beta quality at the moment. I use it (without major issues) in my own project but haven’t put the finishing touches into a full release yet, so it’s not well known in the jME community at the moment. It has a lot of useful features - fully skinnable without changing component types or UI logic, reactive data-driven rendering, declarative rendering styles (defined in code by default, but would be quite easy to declare in json or xml), etc. All rendering is implemented in Skia/Skija, though at the moment it wouldn’t be terribly difficult to swap in a different backend should the need arise.

4 Likes