Just a little suggestion to the JME team: why don't you include the OffScreenRenderer code into the official distribution of JME ?
It's been suggested… modifications need to be made to TextureRenderer to support OffscreenRenderer's features…
EDIT: After looking at OffscreenRenderer again, I noticed it doesn't actualy add any new features, the pixels of the screen can be read with DisplaySystem.grabScreenPixels when the pbuffer/fbo is in context (which I believe it never is since the context is switched internally in TextureRenderer.render…)
Either way, OffscreenRenderer cannot be added as of now since it doesn't have a Pbuffer fallback and its mostly identical to TextureRenderer.
I'll try to look at it during the current reworking to 2.0
Yes, I'm supposed to do that 
A merge of the two class might not be possible though, because FBO parameters are slightly different, and it might be much simpler and intuitive to keep both class separate. We'll see
Edit: Momoko_Fan, indeed TextureRenderer was my starting point, but OffscreenRenderer is not just about one new method, the FBO parameters are not the same. Don't ask me about it, I just adapted some stuff from the LWJGL wiki 
Okay I see now. This would have been easier with Pbuffer because it renders to a colorbuffer first instead of directly to a texture.
Yes, but a context switch would be needed (irrelevent right now I guess, since the biggest bottleneck is the grabScreenContent() call)
The fallback method will use this though
A little suggestion for initHeadlessDisplay() in LWJGLDisplaySystem:
/**
* <code>initHeadlessDisplay</code> creates the LWJGL window with the
* desired specifications.
*/
private void initHeadlessDisplay()
{
PixelFormat format = getFormat();
try {
headlessDisplay = new Pbuffer( 1, 1, format, null, null );
headlessDisplay.makeCurrent();
} catch ( Exception e ) {
// System.exit(1);
logger.severe("Cannot create headless window");
logger.logp(Level.SEVERE, this.getClass().toString(),
"initHeadlessDisplay()", "Exception", e);
throw new Error( "Cannot create headless window: " + e.getMessage(), e );
}
}
Looks like there's no need to create a width*height pbuffer or set the Display mode here. A 1*1 PBuffer can always be initialized even on boards that does not support TEXTURE_RECTANGLE. So we can save some graphics memory..
Also, for OffScreen rendering, it seems that the default texture renderer (LWJGLTextureRenderer) is sufficient for offscreen grabbing. I made some tests and it works very well. It is sufficient to activate() it before issuing a (LWJGL-level) glReadPixels(). I would suggest to add activate() and deactivate() in the TextureRenderer interface, so we can avoid some unneeded casts.
Cheers,
Mik
Hi everyone!
As I recently needed offscreen rendering for an application I'm developing and one of the test systems (a notebook with an ATI Mobility Radeon 9600) did not support FBO, I made an offscreen renderer that uses Pbuffer. I just modified the LWJGLPbufferTextureRenderer so it doesn't render to a texture but instead grabs the rendered content to an IntBuffer. I know that this has been suggested before, but I needed it now, so…
The LWJGLPbufferOffscreenRenderer has already been tested, and so far it works very well.
Here's the
package com.jme.renderer.lwjgl;
import java.nio.ByteBuffer;
import java.nio.ByteOrder;
import java.nio.IntBuffer;
import java.util.ArrayList;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.lwjgl.LWJGLException;
import org.lwjgl.opengl.Display;
import org.lwjgl.opengl.Drawable;
import org.lwjgl.opengl.GL11;
import org.lwjgl.opengl.GL12;
import org.lwjgl.opengl.Pbuffer;
import org.lwjgl.opengl.PixelFormat;
import com.jme.math.Vector3f;
import com.jme.renderer.Camera;
import com.jme.renderer.ColorRGBA;
import com.jme.renderer.OffscreenRenderer;
import com.jme.scene.Spatial;
import com.jme.system.DisplaySystem;
import com.jme.system.JmeException;
import com.jme.system.lwjgl.LWJGLDisplaySystem;
import com.jmex.awt.lwjgl.LWJGLCanvas;
/**
* This class is used by LWJGL to render offscreen images using a Pbuffer. The
* class is derived from LWJGLPbufferTextureRenderer. Users should <b>not</b>
* create this class directly. Instead, allow DisplaySystem to create it for
* you.
*
* @author Original: Joshua Slack, Mark Powell
* @author Changes: Michael Sattler
* @see com.jme.system.DisplaySystem#createOffscreenRenderer(int, int)
*/
public class LWJGLPbufferOffscreenRenderer implements OffscreenRenderer {
private static final Logger logger = Logger.getLogger(LWJGLPbufferOffscreenRenderer.class.getName());
private LWJGLCamera camera;
private ColorRGBA backgroundColor = new ColorRGBA(1, 1, 1, 1);
private int pBufferWidth = 16;
private int pBufferHeight = 16;
/* Pbuffer instance */
private Pbuffer pbuffer;
private Drawable sharedDrawable;
private int active, caps;
private boolean isSupported = true;
private LWJGLRenderer parentRenderer;
private LWJGLDisplaySystem display;
private boolean headless = false;
private int bpp, alpha, depth, stencil, samples;
private Camera oldCamera;
private int oldWidth;
private int oldHeight;
private IntBuffer screenshotBuffer;
/**
* Creates an LWJGLPbufferOffscreenRenderer with the given width and height.
* The settings for bit depth, alpha bits, depth buffer bits, stencil buffer
* bits and minimum samples are taken from the currently active
* DisplaySystem.
*
* @param width
* The width of the renderer
* @param height
* The height of the renderer
* @param parentRenderer
* The parent LWJGLRenderer
*/
public LWJGLPbufferOffscreenRenderer(int width, int height,
LWJGLRenderer parentRenderer) {
this(width, height, parentRenderer,
DisplaySystem.getDisplaySystem().getBitDepth(),
DisplaySystem.getDisplaySystem().getMinAlphaBits(),
DisplaySystem.getDisplaySystem().getMinDepthBits(),
DisplaySystem.getDisplaySystem().getMinStencilBits(),
DisplaySystem.getDisplaySystem().getMinSamples(),
null);
}
public LWJGLPbufferOffscreenRenderer(int width, int height,
LWJGLRenderer parentRenderer, Drawable sharedDrawable) {
this(width, height, parentRenderer,
DisplaySystem.getDisplaySystem().getBitDepth(),
DisplaySystem.getDisplaySystem().getMinAlphaBits(),
DisplaySystem.getDisplaySystem().getMinDepthBits(),
DisplaySystem.getDisplaySystem().getMinStencilBits(),
DisplaySystem.getDisplaySystem().getMinSamples(),
sharedDrawable);
}
public LWJGLPbufferOffscreenRenderer(int width, int height,
LWJGLRenderer parentRenderer, int bpp, int alpha, int depth,
int stencil, int samples, Drawable sharedDrawable) {
this.bpp = bpp;
this.alpha = alpha;
this.depth = depth;
this.stencil = stencil;
this.samples = samples;
this.sharedDrawable = sharedDrawable;
caps = Pbuffer.getCapabilities();
if (((caps & Pbuffer.PBUFFER_SUPPORTED) != 0)) {
isSupported = true;
pBufferWidth = width;
pBufferHeight = height;
// validateForCopy();
this.parentRenderer = parentRenderer;
this.display = (LWJGLDisplaySystem) DisplaySystem
.getDisplaySystem();
initPbuffer();
screenshotBuffer = ByteBuffer.allocateDirect(
pBufferWidth * pBufferHeight * 4).order(
ByteOrder.LITTLE_ENDIAN).asIntBuffer();
} else {
isSupported = false;
}
}
/**
*
* <code>isSupported</code> obtains the capability of the graphics card.
* If the graphics card does not have pbuffer support, false is returned,
* otherwise, true is returned. OffscreenRenderer will not process any scene
* elements if pbuffer is not supported.
*
* @return if this graphics card supports pbuffers or not.
*/
public boolean isSupported() {
return isSupported;
}
/**
* <code>getCamera</code> retrieves the camera this renderer is using.
*
* @return the camera this renderer is using.
*/
public Camera getCamera() {
return camera;
}
/**
* <code>setCamera</code> sets the camera this renderer should use.
*
* @param camera
* the camera this renderer should use.
*/
public void setCamera(Camera camera) {
this.camera = (LWJGLCamera) camera;
}
/**
* <code>setBackgroundColor</code> sets the OpenGL clear color to the
* color specified.
*
* @see com.jme.renderer.OffscreenRenderer#setBackgroundColor(com.jme.renderer.ColorRGBA)
* @param c
* the color to set the background color to.
*/
public void setBackgroundColor(ColorRGBA c) {
// if color is null set background to white.
if (c == null) {
backgroundColor.a = 1.0f;
backgroundColor.b = 1.0f;
backgroundColor.g = 1.0f;
backgroundColor.r = 1.0f;
} else {
backgroundColor = c;
}
if (!isSupported) {
return;
}
activate();
GL11.glClearColor(backgroundColor.r, backgroundColor.g,
backgroundColor.b, backgroundColor.a);
deactivate();
}
/**
* <code>getBackgroundColor</code> retrieves the clear color of the
* current OpenGL context.
*
* @see com.jme.renderer.Renderer#getBackgroundColor()
* @return the current clear color.
*/
public ColorRGBA getBackgroundColor() {
return backgroundColor;
}
public void render(Spatial spat) {
render(spat, true);
}
/**
* <code>render</code> renders a scene. As it recieves a base class of
* <code>Spatial</code> the renderer hands off management of the scene to
* spatial for it to determine when a <code>Geometry</code> leaf is
* reached. The result of the rendering can then be retrieved from the image
* data buffer.
*
* @param spat
* the scene to render.
* @param doClear
* <code>true</code>, if the parent renderer should clear its
* color and depth buffer before the OffscreenRenderer renders
*
*/
public void render(Spatial spat, boolean doClear) {
if (!isSupported) {
return;
}
// clear the current states since we are renderering into a new location
// and can not rely on states still being set.
try {
if (pbuffer.isBufferLost()) {
logger.warning("PBuffer contents lost - will recreate the buffer");
deactivate();
pbuffer.destroy();
initPbuffer();
}
// Override parent's last frustum test to avoid accidental incorrect
// cull
if (spat.getParent() != null) {
spat.getParent().setLastFrustumIntersection(
Camera.INTERSECTS_FRUSTUM);
}
// render and get the rendered content
activate();
switchCameraIn(doClear);
doDraw(spat);
switchCameraOut();
readToScreenshotBuffer();
deactivate();
} catch (Exception e) {
logger.logp(Level.SEVERE, this.getClass().toString(),
"render(Spatial)", "Exception", e);
}
}
// inherited docs
public void render(ArrayList<? extends Spatial> spats) {
render(spats, true);
}
public void render(ArrayList<? extends Spatial> spats, boolean doClear) {
if (!isSupported) {
return;
}
// clear the current states since we are renderering into a new location
// and can not rely on states still being set.
try {
if (pbuffer.isBufferLost()) {
logger.warning("PBuffer contents lost - will recreate the buffer");
deactivate();
pbuffer.destroy();
initPbuffer();
}
// render and get the rendered content
activate();
switchCameraIn(doClear);
for (int x = 0, max = spats.size(); x < max; x++) {
Spatial spat = spats.get(x);
// Override parent's last frustum test to avoid accidental
// incorrect cull
if (spat.getParent() != null)
spat.getParent().setLastFrustumIntersection(
Camera.INTERSECTS_FRUSTUM);
doDraw(spat);
}
switchCameraOut();
readToScreenshotBuffer();
deactivate();
} catch (Exception e) {
logger.logp(Level.SEVERE, this.getClass().toString(),
"render(ArrayList<Spatial>)", "Exception", e);
}
}
private void switchCameraIn(boolean doClear) {
// grab non-offscreen settings
oldCamera = parentRenderer.getCamera();
oldWidth = parentRenderer.getWidth();
oldHeight = parentRenderer.getHeight();
parentRenderer.setCamera(getCamera());
// swap to offscreen settings
parentRenderer.getQueue().swapBuckets();
parentRenderer.reinit(pBufferWidth, pBufferHeight);
// clear the scene
if (doClear) {
parentRenderer.clearBuffers();
}
getCamera().update();
getCamera().apply();
}
private void switchCameraOut() {
parentRenderer.setCamera(oldCamera);
parentRenderer.reinit(oldWidth, oldHeight);
// back to the non-offscreen settings
parentRenderer.getQueue().swapBuckets();
oldCamera.update();
oldCamera.apply();
}
private void doDraw(Spatial spat) {
// do offscreen scene render
spat.onDraw(parentRenderer);
parentRenderer.renderQueue();
}
private void readToScreenshotBuffer() {
screenshotBuffer.rewind(); // TODO: Is this necessary?
GL11.glReadPixels(0, 0, pBufferWidth, pBufferHeight, GL12.GL_BGRA,
GL11.GL_UNSIGNED_BYTE, screenshotBuffer);
}
private void initPbuffer() {
if (!isSupported) {
return;
}
try {
if (pbuffer != null) {
giveBackContext();
DisplaySystem.getDisplaySystem().removeContext(pbuffer);
}
pbuffer = new Pbuffer(pBufferWidth, pBufferHeight, new PixelFormat(
bpp, alpha, depth, stencil, samples), sharedDrawable);
} catch (Exception e) {
logger.logp(Level.SEVERE, this.getClass().toString(), "initPbuffer()", "Exception", e);
logger.log(Level.WARNING, "Failed to create Pbuffer.", e);
isSupported = false;
return;
}
try {
activate();
pBufferWidth = pbuffer.getWidth();
pBufferHeight = pbuffer.getHeight();
GL11.glClearColor(backgroundColor.r, backgroundColor.g,
backgroundColor.b, backgroundColor.a);
if (camera == null)
initCamera();
camera.update();
deactivate();
} catch( Exception e ) {
logger.log(Level.WARNING, "Failed to initialize created Pbuffer.",
e);
isSupported = false;
return;
}
}
public void activate() {
if (!isSupported) {
return;
}
if (active == 0) {
try {
pbuffer.makeCurrent();
display.switchContext(pbuffer);
} catch (LWJGLException e) {
logger.logp(Level.SEVERE, this.getClass().toString(), "activate()", "Exception",
e);
throw new JmeException();
}
}
active++;
}
public void deactivate() {
if (!isSupported) {
return;
}
if (active == 1) {
try {
giveBackContext();
((LWJGLRenderer)display.getRenderer()).reset();
} catch (LWJGLException e) {
logger.logp(Level.SEVERE, this.getClass().toString(),
"deactivate()", "Exception", e);
throw new JmeException();
}
}
active--;
}
private void giveBackContext() throws LWJGLException {
if (!headless && Display.isCreated()) {
Display.makeCurrent();
display.switchContext(display);
} else if (display.getCurrentCanvas() != null) {
((LWJGLCanvas)display.getCurrentCanvas()).makeCurrent();
display.switchContext(display.getCurrentCanvas());
} else if (display.getHeadlessDisplay() != null) {
display.getHeadlessDisplay().makeCurrent();
display.switchContext(display.getHeadlessDisplay());
}
}
private void initCamera() {
if (!isSupported) {
return;
}
camera = new LWJGLCamera(pBufferWidth, pBufferHeight, this);
camera.setFrustum(1.0f, 1000.0f, -0.50f, 0.50f, 0.50f, -0.50f);
Vector3f loc = new Vector3f(0.0f, 0.0f, 0.0f);
Vector3f left = new Vector3f(-1.0f, 0.0f, 0.0f);
Vector3f up = new Vector3f(0.0f, 1.0f, 0.0f);
Vector3f dir = new Vector3f(0.0f, 0f, -1.0f);
camera.setFrame(loc, left, up, dir);
}
public void cleanup() {
if (!isSupported) {
return;
}
display.removeContext(pbuffer);
pbuffer.destroy();
}
// private void validateForCopy() {
// if (pBufferWidth > DisplaySystem.getDisplaySystem().getWidth()) {
// pBufferWidth = DisplaySystem.getDisplaySystem().getWidth();
// }
//
// if (pBufferHeight > DisplaySystem.getDisplaySystem().getHeight()) {
// pBufferHeight = DisplaySystem.getDisplaySystem().getHeight();
// }
// }
public int getWidth() {
return pBufferWidth;
}
public int getHeight() {
return pBufferHeight;
}
public IntBuffer getImageData() {
return screenshotBuffer;
}
}
To create it, I also modified the method createOffscreenRenderer(int, int) introduced here, which looks like this now:
public com.jme.renderer.OffscreenRenderer createOffscreenRenderer(int width, int height) {
if ( !isCreated() ) {
return null;
}
boolean fboSupported = GLContext.getCapabilities().GL_EXT_framebuffer_object;
if (fboSupported) {
return new com.jme.renderer.lwjgl.LWJGLOffscreenRenderer( width, height,
(LWJGLRenderer) getRenderer());
}
logger.info("FBO not supported, using Pbuffer for offscreen rendering");
return new LWJGLPbufferOffscreenRenderer(width, height,
(LWJGLRenderer) getRenderer());
}
In addition, for applets to work properly with the LWJGLPbufferOffscreen renderer, I also added the follwing method to (LWJGL)DisplaySystem. If offscreen rendering is to be used in applets, this method should be used, with the applet's GL canvas passed as the third argument. Otherwise textures might not be rendered.
public OffscreenRenderer createOffscreenRenderer(int width, int height,
JMECanvas canvas) {
if ( !isCreated() ) {
return null;
}
boolean fboSupported = GLContext.getCapabilities().GL_EXT_framebuffer_object;
if (fboSupported) {
return new com.jme.renderer.lwjgl.LWJGLOffscreenRenderer( width, height,
(LWJGLRenderer) getRenderer());
}
logger.info("FBO not supported, using Pbuffer for offscreen rendering");
if (canvas instanceof AWTGLCanvas) {
return new LWJGLPbufferOffscreenRenderer(width, height,
(LWJGLRenderer) getRenderer(), (AWTGLCanvas) canvas);
}
return new LWJGLPbufferOffscreenRenderer(width, height,
(LWJGLRenderer) getRenderer());
}
EDIT: By the way, my first try was to use the LWJGLPbufferTextureRenderer to do the offscreen rendering. I simply added another constructor that creates a dummy texture for the texture renderer, added an IntBuffer to read the screen contents to in the rendering process and created a method to retrieve the IntBuffer. This also worked, but I think it's better to have extra classes to do the offscreen rendering than to have the rtt overhead all the time.
EDIT 2: Added a method to fix some problems with offscreen-rendering textures in an applet context.
If you’re looking for offscreen rendering you might want to check out jme-context’s JmePixelBuffer context, which uses Pbuffer as a base for rendering. It also doesn’t require any modifications to jme to use.
Hm, nice! Currently I'm happy with the solution I found, but maybe I can use your JmeContext in the future. At the moment I can't use multipass rendering (e.g. to render shadows) for the screenshots I take by using an offscreen renderer, but on a first look, that seems to work with your JmeContext. Also, additional windows for my app could be a nice addition.
Thanks for the hint!
A little update: I noticed that the LWJGLPbufferOffscreenRenderer does not render any textures when used in an applet context. I was able to fix this issue by passing the applet's GL canvas to the Pbuffer when it is created. This has already been edited in the post above.
I suppose this problem should also be checked in the LWLJGLPbufferTexture renderer. Probably that one also doesn't render any textures in an applet context…
Do you have any kind of test to check for this issue? I have forced PBuffer usage on some applets using imposters locally and do not see an issue.
It was just a guess, but I'll test it…