AppState for Managing One or More Render to Texture Scenes

I’ve created an AppState to help manage scenes rendered to textures. Just extend the Act class and attach it to your StateManager.

With this set of classes you can manage multiple scenes rendered to textures. Scenes can be initialized, paused and have two active states, active and active_thumb, which allows to differentiate between interactive and non-interactive states while the scene is not paused. Scenes can be transitioned between states and can even have an associated physics simulation thread which is managed by the Act class to allow easy thread safety.

With this you can easily have multiple real-time animated textures or even multiple different games running simultaneously each with their own, optional, physics simulation. To use a physics simulation just set the phy variable to an instance of PhySim in your Act's constructor or initialize(stateManager, app) method. You’ll need to extend the PhySim class to create your own physics simulation, I’ve included an example using Dyn4j.

The Act class:

import com.atr.math.GMath;
import com.jme3.app.Application;
import com.jme3.app.state.AbstractAppState;
import com.jme3.app.state.AppStateManager;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Vector3f;
import com.jme3.renderer.Camera;
import com.jme3.renderer.RenderManager;
import com.jme3.renderer.ViewPort;
import com.jme3.scene.Geometry;
import com.jme3.scene.Node;
import com.jme3.scene.Spatial;
import com.jme3.texture.Image;
import com.jme3.texture.Texture;
import com.jme3.texture.Texture2D;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicReference;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 * <p>An <code>AppState</code> that manages a scene rendered to a texture.
 * The scene can have an optional physics simulation that runs in a
 * seperate thread.</p>
 * 
 * 
 * @author Adam T. Ryder
 * <a href="http://1337atr.weebly.com">http://1337atr.weebly.com</a>
 */
public abstract class Act extends AbstractAppState {
    public final ViewPort vp;
    public final Camera cam;
    public final Node scene;
    public final Texture2D tex;
    protected final RenderManager renderManager;
    
    protected final Application app;
    
    public final Geometry viewplane;

    public final int width;
    public final int height;
    
    /**
     * <p>The <code>MODE</code> is used to indicate the execution state of the
     * scene.</p>
     * 
     * <ul>
     * <li>init - Inidicates the <code>Act</code> has not yet rendered
     * its first frame after having been instantiated.
     * {@link #initializeScene(float)} will be called, the
     * {@link MODE} will be changed to <code>MODE.active_thumb</code>
     * or what was set through {@link #setNextMode(Act.MODE mode)}.</li>
     * 
     * <li>pause - The scene is not rendered, logic is not updated and
     * the physics simulation thread has been instructed to finish.</li>
     * 
     * <li>active - The scene is rendered and logic is updated every frame.
     * The physics simulation thread is running. The scene is ready to
     * interact with the user. Methods used to update the scene in this
     * mode are {@link #preFrame(float)} and
     * {@link #actUp(float)}</li>
     * 
     * <li>active_thumb - This is a convenience {@link MODE}.
     * Everything works just like <code>MODE.active</code>, but is
     * intended to indicate that the scene is only animating and not
     * interactive. Methods used to update the scene in this mode are
     * {@link #preFrameThumb(float)} and
     * {@link #actThumb(float)}</li>
     * </ul>
     * 
     * 
     */
    public enum MODE {
        init,
        pause,
        active_thumb,
        transition,
        active
    }
    
    private AtomicReference<MODE> mode = new AtomicReference<MODE>(MODE.init);
    protected MODE lastMode = MODE.init;
    protected MODE nextMode = MODE.active_thumb;
    
    protected float transitionLength = 0.25f;
    private float transitionTme = 0;
    private int transStep = 0;
    
    protected PhySim phy;
    protected Thread phyThread;
    
    /**
     * Constructs an instance of <code>Act</code>.
     * 
     * @param actName A name to assign to the <code>Act</code>. This does
     * not need to be unique.
     * @param mode The initial {@link MODE} for the <code>Act</code>.
     * @param app Your main <code>Application</code> class.
     * @param width The width of the texture, in pixels, that the scene will
     * be rendered to.
     * @param height The height of the texture, in pixels, that the scene will
     * be rendered to.
     * @param colorFormat The <code>com.jme3.texture.Image.Format</code>
     * for the color texture.
     * @param depthFormat The <code>com.jme3.texture.Image.Format</code>
     * for the depth texture.
     * @param bilinear If true the texture will be set to use bilinear
     * filtering otherwise the texture will use nearest.
     * 
     * @see MODE
     */
    public Act(String actName, MODE mode,
        Application app, int width, int height,
        Image.Format colorFormat,
        Image.Format depthFormat, boolean bilinear) {
        this.width = width;
        this.height = height;
        this.cam = new Camera(width, height);
        this.vp = GMath.renTex(actName, this.cam, colorFormat, depthFormat);
        this.mode.set(mode);
        scene = new Node(actName);
        vp.attachScene(scene);
        
        tex = (Texture2D)vp.getOutputFrameBuffer().getColorBuffer().getTexture();
        tex.setMinFilter(bilinear ? Texture.MinFilter.BilinearNoMipMaps : Texture.MinFilter.NearestNoMipMaps);
		tex.setMagFilter(bilinear ? Texture.MagFilter.Bilinear : Texture.MagFilter.Nearest);
        
        this.app = app;
        this.renderManager = app.getRenderManager();

        float ratio = (float)height / width;
        viewplane = new Geometry(actName + ": Scene View", new CQuad(1, ratio));
        
        renderManager.getRenderer().setFrameBuffer(vp.getOutputFrameBuffer());
        renderManager.getRenderer().clearBuffers(true, true, true);
    }
    
    /**
     * Constructs an instance of <code>Act</code> with a default
     * color <code>com.jme3.texture.Image.Format</code> of <code>RGB8</code>
     * and depth <code>com.jme3.texture.Image.Format</code> of
     * <code>Depth</code> and bilinear filtering.
     * 
     * @param actName A name to assign to the <code>Act</code>. This does
     * not need to be unique.
     * @param mode The initial {@link MODE} for the <code>Act</code>.
     * @param app Your main <code>Application</code> class.
     * @param width The width of the texture, in pixels, that the scene will
     * be rendered to.
     * @param height The height of the texture, in pixels, that the scene will
     * be rendered to.
     * 
     * @see MODE
     */
    public Act(String actName, Application app,
        int width, int height) {
        this(actName, MODE.init, app, width, height, Image.Format.RGB8,
            Image.Format.Depth, true);
    }
    
    /**
    * The name the <code>Act</code> was constructed with.
    * 
    * @return The name of the <code>Act</code>.
    */
    public String getName() {
        return vp.getName();
    }
    
    /**
    * Gets the <code>Act</code>'s current {@link MODE}.
    * 
    * @return The <code>Act</code>'s current {@link MODE}.
    * 
    * @see MODE
    */
    public MODE getMode() {
        return mode.get();
    }
    
    /**
    * Sets the mode that will be set after the next transition.
    * This only applies if the current mode is <code>MODE.init</code>.
    * 
    * @param toMode The {@link MODE} to transition to after the
    * <code>Act</code> is initialized.
    * 
    * @see MODE
    */
    public void setNextMode(MODE toMode) {
        if (toMode == MODE.init || toMode == MODE.transition
                || mode.get() != MODE.init)
            return;
        
        nextMode = toMode;
    }
    
    /**
    * Set the <code>Act</code> to a new {@link MODE}. The Act will
    * transition to this new {@link MODE} over a period of time.
    * 
    * @param toMode The {@link MODE} to transition to.
    * @param length The length in seconds the transition will take. can be
    * zero or negative for an immediate change in mode.
    * 
    * @see MODE
    * @see #transitionFinished()
    */
    public void transMode(MODE toMode, float length) {
        transitionLength = length;
        
        if (toMode == MODE.transition ||
                toMode == MODE.init)
            return;
        
        nextMode = toMode;
        if (mode.get() == MODE.transition) {
            transitionTme = 0;
            transStep = 0;
        } else
            lastMode = mode.get();
        
        if (transitionLength <= 0) {
            mode.set(toMode);
            if (phy == null) {
                transitionFinished();
                return;
            }
            if (toMode == MODE.pause && phyThread != null) {
                finishPhyThread();
                while (phyThread.isAlive() && phy.updating())
                    continue;
                transitionFinished();
                return;
            }
            
            if (!phy.active.get() || phyThread == null)
                startPhyThread();
            while (phy.updating())
                continue;
            transitionFinished();
        } else
            mode.set(MODE.transition);
    }
    
    private void startPhyThread() {
        if (phyThread != null
            && phyThread.isAlive()) {
            finishPhyThread();
            while (phy.updating())
                continue;
        }
        Logger.getLogger(getClass().getName()).log(Level.INFO, "Act - " + getName() + ": Starting simulation thread.");
        phy = phy.clone();
        phyThread = new Thread(phy);
        phyThread.start();
    }
    
    private void finishPhyThread() {
        Logger.getLogger(getClass().getName()).log(Level.INFO, "Act - " + getName() + ": Finishing physics thread...");
        phy.active.set(false);
    }
    
    @Override
    public void update(float tpf) {
        switch(mode.get()) {
            case active:
                preFrame(tpf);
                while (phyThread != null && phy.updating())
                    continue;
                actUp(tpf);
                scene.updateLogicalState(tpf);
                if (phyThread != null)
                    phy.setUpdating(tpf);
                break;
            case active_thumb:
                preFrameThumb(tpf);
                while (phyThread != null && phy.updating())
                    continue;
                actThumb(tpf);
                scene.updateLogicalState(tpf);
                if (phyThread != null)
                    phy.setUpdating(tpf);
                break;
            case transition:
                if (lastMode == MODE.active_thumb) {
                    preFrameThumb(tpf);
                    while (phyThread != null && phyThread.isAlive()
                            && phy.updating())
                        continue;
                    if (nextMode == MODE.pause) {
                        transActiveToThumb(tpf);
                    } else
                        transThumbToActive(tpf);
                    break;
                }
                if (lastMode == MODE.pause || lastMode == MODE.init) {
                    preFrameThumb(tpf);
                    while (phyThread != null && phyThread.isAlive()
                            && phy.updating())
                        continue;
                    transThumbToActive(tpf);
                } else {
                    preFrame(tpf);
                    while (phyThread != null && phyThread.isAlive()
                            && phy.updating())
                        continue;
                    transActiveToThumb(tpf);
                }
                scene.updateLogicalState(tpf);
                if (phyThread != null && phy.active.get())
                    phy.setUpdating(tpf);
                break;
            case init:
                initializeScene(tpf);
                transMode(nextMode, -1);
                break;
            default:
                return;
        }

        render(tpf);
    }
    
    /**
    * <p>Called every frame while the mode is set to
    * <code>MODE.active</code>. The physics simulation is not yet
    * synchronized with the main thread. Do not access or modify the
    * physics simulation!</p>
    * 
    * @param tpf The time, in seconds, it took to render the last frame.
    */
    protected abstract void preFrame(float tpf);
    
    /**
    * <p>Called every frame while the mode is set to
    * <code>MODE.active</code>. The physics simulation is synchronized
    * with the main thread. <code>Control</code>s have not yet executed.</p>
    * 
    * @param tpf The time, in seconds, it took to render the last frame.
    * 
    * @see MODE
    */
    protected abstract void actUp(float tpf);
    
   /**
    * <p>Called every frame while the mode is set to
    * <code>MODE.active_thumb</code>. The physics simulation is not yet
    * synchronized with the main thread. Do not access or modify the
    * physics simulation!</p>
    * 
    * @param tpf The time, in seconds, it took to render the last frame.
    */
    protected abstract void preFrameThumb(float tpf);
    
    /**
    * <p>Called every frame while the mode is set to
    * <code>MODE.active_thumb</code>. The physics simulation is
    * synchronized with the main thread. <code>Control</code>s have not
    * yet executed.</p>
    * 
    * @param tpf The time, in seconds, it took to render the last frame.
    * 
    * @see MODE
    */
    protected abstract void actThumb(float tpf);
    
    private void transThumbToActive(float tpf) {
        if (transStep < 3) {
            if (transStep == 0) {
                if (phy != null && (lastMode == MODE.pause || lastMode == MODE.init)) {
                    startPhyThread();
                }
            }
            
            transStep++;
            return;
        }
        
        transitionTme += tpf;
        if (transitionLength > 0)
            thumbToActive(transitionTme / transitionLength, tpf);
        if (transitionTme >= transitionLength) {
            mode.set(nextMode);
            transitionTme = 0;
            transStep = 0;
            transitionFinished();
        }
    }

    private void transActiveToThumb(float tpf) {
        transitionTme += tpf;
        if (transitionLength > 0)
            activeToThumb(transitionTme / transitionLength, tpf);
        if (transitionTme >= transitionLength) {
            if (phy != null && nextMode == MODE.pause)
                finishPhyThread();
            mode.set(nextMode);
            transitionTme = 0;
            transStep = 0;
            transitionFinished();
        }
    }
    
    /**
    * <p>Called when transitioning from an active_thumb, pause or init
    * mode to active.</p>
    * 
    * @param perc The percentage, range 0-1+, between the beginning
    * and end of the transition length. Can be greater than 1.
    * 
    * @see MODE
    */
    protected abstract void thumbToActive(float perc, float tpf);
    
    /**
    * <p>Called when transitioning from active to active_thumb or
    * from active_thumb to pause.</p>
    * 
    * @param perc The percentage, range 0-1+, between the beginning
    * and end of the transition length. Can be greater than 1.
    * 
    * @see MODE
    */
    protected abstract void activeToThumb(float perc, float tpf);
    
    /**
    * <p>Called when a mode transition has finished. The current mode
    * will be the mode set at the end of the transition. The
    * physics simulation is synchronized.</p>
    */
    protected abstract void transitionFinished();
    
    /**
    * <p>Renders the scene and returns the <code>Texture2D</code>
    * the scene is rendered to.</p>
    * 
    * @param tpf The time, in seconds, it took to render the previos
    * frame.
    * 
    * @return The <code>Texture2D</code> the scene is rendered to.
    */
    public Texture2D render(float tpf) {
        scene.updateGeometricState();
        renderManager.renderViewPort(vp, tpf);
        
        return tex;
    }
    
    /**
    * <p>Called just before the first frame is rendered after the
    * <code>Act</code> is attached to the <code>StateManager</code>.
    * The physics simulation has not yet started. Perform your scene
    * initialization here.</p>
    * 
    * @param tpf The time, in seconds, it took to render the previous
    * frame.
    */
    protected abstract void initializeScene(float tpf);
    
    /**
     * <p>Be sure to detach your <code>Act</code>s from the
     * <code>StateManager</code> in your <code>Application</code>'s
     * <code>destroy()</code> method as this will ensure the associated
     * physics simulation thread, if there is one, will be stopped.</p>
     */
    @Override
    public void stateDetached(AppStateManager stateManager) {
        if (phyThread != null) {
            Logger.getLogger(this.getClass().getName()).log(Level.INFO, "Act - " + getName() + ": Disabling Physics...");
            phy.active.set(false);
        }
    }
}

The PhySim class:

import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicReference;
import java.util.logging.Level;
import java.util.logging.Logger;

public abstract class PhySim implements Runnable {
    protected final Act act;
    
    public final AtomicBoolean active = new AtomicBoolean(true);
    protected final AtomicBoolean updating = new AtomicBoolean(true);
    
    private float timeperframe = 0.25f;
    
    public PhySim(Act act) {
        this.act = act;
    }

    public PhySim clone() {
        return this.clone(act);
    }
    
    public abstract PhySim clone(Act act);
    
    public boolean updating() {
        return updating.get();
    }
    
    public void setUpdating(float tpf) {
        this.timeperframe = tpf;
        updating.set(true);
    }
    
    @Override
    public void run() {
        while(active.get()) {
            update(timeperframe);
            
            updating.set(false);
            while(!updating.get() && active.get())
                continue;
        }
        
        Logger.getLogger(this.getClass().getName()).log(Level.INFO, "Act - " + act.getName() + ": Exiting simulation thread.");
        updating.set(false);
    }
    
    public abstract void update(float tpf);
}

The CQuad class, which is the mesh used to display the texture:

import com.jme3.scene.Mesh;
import com.jme3.scene.VertexBuffer;

/**
 * A quad facing the positive z-axis that can be aligned to any of it's
 * corners or centered. Default alignment is centered.
 * 
 * @author Adam T. Ryder
 * <a href="http://1337atr.weebly.com">http://1337atr.weebly.com</a>
 */
public class CQuad extends Mesh {
    public enum ORIGIN {
        LeftBottom,
        LeftTop,
        RightTop,
        RightBottom,
        Center
    }
    
    private float width;
    private float height;
    private ORIGIN origin;
    
	public CQuad(float width, float height) {
		this(width, height, ORIGIN.Center);
	}
    
    public CQuad(float width, float height,
            ORIGIN origin) {
		this(width, height, origin, false);
	}
    
    public CQuad(float width, float height,
            ORIGIN origin, boolean flipYuv) {
		super();
        setWidth(width);
        setHeight(height);
        setOrigin(origin);
        updateGeometry(flipYuv);
	}
    
    public void setOrigin(ORIGIN origin) {
        this.origin = origin;
    }
    
    public void setWidth(float width) {
        this.width = width;
    }
    
    public void setHeight(float height) {
        this.height = height;
    }
    
    public void setDimensions(float width, float height) {
        this.width = width;
        this.height = height;
    }
    
    public void set(float width, float height, ORIGIN origin) {
        setOrigin(origin);
        setWidth(width);
        setHeight(height);
    }
    
    public ORIGIN getOrigin() {
        return origin;
    }
    
    public float getWidth() {
        return width;
    }
    
    public float getHeight() {
        return height;
    }
    
    public void updateGeometry() {
        updateGeometry(false);
    }
	
    /**
     * <p>Updates the mesh to reflect any changes made to dimensions or
     * alignment.</p>
     * 
     * @param flipYuv UV coordinates will be flipped along the y-axis if true.
     */
	public void updateGeometry(boolean flipYuv) {
        
        switch(origin) {
            case Center:
                setBuffer(VertexBuffer.Type.Position, 3,
                    new float[]{-width / 2, -height / 2, 0,
                                width / 2, -height / 2, 0,
                                width / 2, height / 2, 0,
                                -width / 2, height / 2, 0});
                break;
            case LeftBottom:
                setBuffer(VertexBuffer.Type.Position, 3,
                    new float[]{0, 0, 0,
                                width, 0, 0,
                                width, height, 0,
                                0, height, 0});
                break;
            case LeftTop:
                setBuffer(VertexBuffer.Type.Position, 3,
                    new float[]{0, -height, 0,
                                width, -height, 0,
                                width, 0, 0,
                                0, 0, 0});
                break;
            case RightTop:
                setBuffer(VertexBuffer.Type.Position, 3,
                    new float[]{-width, -height, 0,
                                0, -height, 0,
                                0, 0, 0,
                                -width, 0, 0});
                break;
            case RightBottom:
                setBuffer(VertexBuffer.Type.Position, 3,
                    new float[]{-width, 0, 0,
                                0, 0, 0,
                                0, height, 0,
                                -width, height, 0});
                break;
            default:
                setBuffer(VertexBuffer.Type.Position, 3,
                    new float[]{-width / 2, -height / 2, 0,
                                width / 2, -height / 2, 0,
                                width / 2, height / 2, 0,
                                -width / 2, height / 2, 0});
                break;
        }
        
        if (!flipYuv) {
            setBuffer(VertexBuffer.Type.TexCoord, 2, new float[]{0, 0,
                                                                1, 0,
                                                                1, 1,
                                                                0, 1});
        } else {
            setBuffer(VertexBuffer.Type.TexCoord, 2, new float[]{0, 1,
                                                                1, 1,
                                                                1, 0,
                                                                0, 0});
        }
        
        setBuffer(VertexBuffer.Type.Normal, 3, new float[]{0, 0, 1,
                                                            0, 0, 1,
                                                            0, 0, 1,
                                                            0, 0, 1});
        
        if (height < 0){
            setBuffer(VertexBuffer.Type.Index, 3, new short[]{0, 2, 1,
                                                              0, 3, 2});
        } else {
            setBuffer(VertexBuffer.Type.Index, 3, new short[]{0, 1, 2,
                                                              0, 2, 3});
        }
        
        updateBound();
        updateCounts();
    }
}

An example PhySim using Dyn4j:

import org.dyn4j.dynamics.World;

public class DynSim extends PhySim {
    private final World world;
    
    public DynSim(Act act, World world) {
        super(act);
        this.world = world;
    }
    
    public PhySim clone(Act act) {
        return new DynSim(act, world);
    }
    
    public void update(float tpf) {
        world.update(tpf);
    }
    
    public World getWorld() {
        return world;
    }
}

Happy holidays!

P.S. I haven’t really done a whole lot of testing on this so…

3 Likes

Happy holidays to you as well!

You have decided to use multiple threads. Couldn’t you have achieved the same with no extra threads?

On top of it I noticed:

The statement waits for the other thread. Thus, why introduce the extra thread anyway.

They physics thread runs concurrently during the render cycle then synchronizes with the main thread during the update cycle. If you have, for instance, Controls attached to your Spatials, running on the main thread, and they look at the location/rotation of simulated bodies in the physics thread you want to make sure that information is not also being accessed by the physics thread.

The physics thread runs its logic while the main thread renders the scene, concurrently, then synchronizes with the main thread so you can update your visualization objects to reflect the location/rotation of the simulated bodies.

If the main thread finishes the render cycle before the physics thread finishes a step the main thread will wait for the physics thread. If the physics thread finishes a step before the main thread is finished rendering it will wait for the main thread.

Of course if you don’t need this synchronization, though I’m not sure why you wouldn’t, you can put all your logic in preFrame(float tpf) or preFrameThumb(float tpf) both of which are called before the physics thread is synchronized.

Note that actUp(float tpf) and actThumb(float tpf) are both called while the threads are synchronized. Likewise scene.upadateLogicalState(float tpf) is also called while the threads are synchronized, this is where Control.controlUpdate(float tpf) is called.

Okay I see, for physics it might make sense to use an extra thread.

Well I suppose it probably makes sense in some situations and not so much in others. You could always use jMonkey’s built-in bullet physics, I think it’s an AppState that you attach to the StateManager.

I only played around with it once, I don’t recall much about it.