readFrameBufferWithFormat

Troubleshooting graphics
1

I’m trying to save an offscreen render to a jpg file but my current method is taking roughly 45-70 ms to save an image.

rm.getRenderer().readFrameBufferWithFormat(out, byteBuf, Image.Format.BGRA8);
pixelBuf = byteBuf.asIntBuffer();
BufferedImage img = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);

	    for (int x = 0; x < width; x++) {
	    	for (int y = 0; y < height; y++) {
	    		img.setRGB (x, y, pixelBuf.get((height - y - 1) * width + x));
	        }
	    }

…
File outFile = new File(filename);
ImageIO.write(img, “jpg”, outFile);

Is there any faster way to go from the byteBuffer in BGRA8 to RGB BufferedImage?

Thanks

2

First you should look into jmonkeyengine/ScreenshotAppState.java at master · jMonkeyEngine/jmonkeyengine · GitHub

It does what you want (except it saves to a png…), so you can use it or get inspired by it.

I don’t know the in and outs of the needed format conversion, but I’m not sure there is a fastest way to do it.
Would be interesting to know if the bottleneck of the operation is the readFrameBuffer or the iteration over the data.
If it’s the latter you could do the file conversion and write in a separate thread. It wouldn’t be faster, but it would give back the hand to the render thread immediately after the readFramebuffer, and finish in the background.

Only the readFrameBuffer needs to be done on the render thread, once you have the byteBuffer you can do whatever you want with it.

3

Thank you for your response,

So it appears that the bottleneck is actually readFrameBufferWithFormat, is there any other more efficient method of accessing this FrameBuffer?

Update: using JmeSystem.write takes roughly 55-61 ms. It seems as though the iteration over the data provides slightly better performance.

4

Have you tried using just Renderer.readFrameBuffer()? That will read the framebuffer as RGBA8. I’d guess that in most cases that’s the format it’s stored as and it will not need to be reformatted during copy (although I’d also guess that your bottleneck is the actual framebuffer transfer - not the pixel format conversions. How large is your framebuffer?). It’s also very close to your desired BufferedImage format, which could save you time during the export (which in most cases is probably negligible, but it’s a possibility at least).

5

Well then there is not much you can do. You are limited by the bandwidth of the hardware.
What are you trying to do? Why 70 ms is an issue? I guess you don’t want to do that on every frame, do you?
You could try to lower the resolution of the frame buffer…and have a smaller image…

6

Hi there, so I’ve tried loading the ByteBuffer in RGBA8 Format and unfortunately there isn’t any speed up at all :frowning:

7

It is, I’m doing processing on each frame and the processing takes roughly 100ms so with the saving in the mix I’m not even reach 10fps which is an issue. I’m using a 640x480 image which I feel is already on the smaller side. What do you mean by lower the resolution of the frame buffer?

8

Hm, that’s what I thought you’d find… on the bright side, it still could save you from doing the conversion manually.

9

Getting data from the GPU will take a fixed amount of time. The lowest “fixed amount of time” will be retrieving it without any conversion.

Writing the data to a JPG may take a variable amount of time. You can turn this into a fixed amount of time by writing the data in an uncompressed format, making sure writers are buffered, etc…

Still, once you’ve got those down to the smallest “fixed amount of time” then you are stuck.

You might take a step back and describe exactly what you are doing that requires a separate image per frame and still being interactive.

10

I really wonder why you need to write a file on a hard drive on each frame…

11

I’m using readFrameBufferWithFormat() all the time and never had a problem with it.
In mixed Swing/JME3 application I often use the following snippet to render the scene in a JPanel:

In simpleRender:

renderManager.getRenderer().readFrameBufferWithFormat(screenFrameBuffer, screenOutputBuffer, Image.Format.BGRA8);
synchronized(screenAwtImage) {
    Screenshots.convertScreenShot(screenOutputBuffer, screenAwtImage);
}
SwingUtilities.invokeLater(contentPanel::repaint)

In contentPanel’s repaint() method (inherits from JComponent):

synchronized(screenAwtImage) {
    g.drawImage(screenAwtImage, 0, 0, null);
}

It is fast enough for a 50 FPS playback. Why is it not sufficient for you?

12

I need to send the image to a C program for processing and return the results of the C program back so that I can use those results to display without any frame delay. This is done over TCP communication. I felt it was easier this way than to send byte data over.