Drawing into textures once instead of every frame

Hi All,



Sorry for all the questions but hopefully this will be the last one for a while as once I’ve solved this it should be full steam ahead!



For the game I’m working on we need to render different textures onto an object, with those textures being built dynamically but changing infrequently (they will be static most of the time once created). For example a card might have a picture of a hero with that hero’s name above it and stats down the side. That name would virtually never change and the stats would rarely change but they would be different for each hero (and hence need generating separately for each one).



The obvious way forwards would be to create the textures using Java2D but I’d like to keep platform compatibility and not have to write everything twice (once for android, once for everywhere else). One reason I picked jME is because of the high quality and convenient cross-platform deployment support.



I found http://hub.jmonkeyengine.org/groups/contribution-depot-jme3/snippets/14/ but that involves pixel-level operations. It would need enhancing a lot to allow images and text to be drawn into the texture in non-painful ways. The other similar things I found were JME2 and/or used Java2D.



My other thought was to use Nifty and have Nifty create the texture and project it onto the object as described here https://wiki.jmonkeyengine.org/legacy/doku.php/jme3:advanced:nifty_gui_projection.



I tried this and it did work well but there was a problem: Framerate with 3 cards spinning in space dropped from something like 2500 fps to 900fps, this is a massive overhead and so far only with 3 cards being rendered when potentially there could be hundreds on screen.



I then investigated the way the Nifty projection works and it looks like it is being rendered into the texture every frame for every card - which obviously for mostly static data seems like massive overkill. Ideally what I’d like to do is have one Nifty instance, use that to render the textures and then save those textures for use and only go back to nifty when a value changes and the texture needs re-rendering.



I tried to set that up by detaching the Nifty view from the renderManager but I just ended up with a plain black texture so I guess I did something wrong there.



[java]

Nifty nifty = null;

Texture2D frontTexture = null;



protected void renderFront() {

nifty.fromXml(“Cards/Hero.xml”, “front”, Card.this);

}

[/java]



In card constructor:

[java]

// Will just use standard plain colour material until nifty is ready

AppContext.getJme3().enqueue(new Callable<Object>() {



@Override

public Object call() throws Exception {

if (nifty == null) {

ViewPort niftyView = new ViewPort(“NiftyView”, new Camera(320, 512));

niftyView.setClearFlags(true, true, true);

NiftyJmeDisplay niftyDisplay = new NiftyJmeDisplay(assetManager,

AppContext.getJme3().getInputManager(),

AppContext.getJme3().getAudioRenderer(),

niftyView);

nifty = niftyDisplay.getNifty();

niftyView.addProcessor(niftyDisplay);



Texture2D depthTex = new Texture2D(320, 512, Format.Depth);

FrameBuffer fb = new FrameBuffer(320, 512, 1);

fb.setDepthTexture(depthTex);



frontTexture = new Texture2D(320, 512, Format.RGBA8);

frontTexture.setMinFilter(MinFilter.Trilinear);

frontTexture.setMagFilter(MagFilter.Bilinear);



fb.setColorTexture(frontTexture);

niftyView.setOutputFrameBuffer(fb);

}



renderFront();

nifty.update();

frontMat.setTexture(“DiffuseMap”, frontTexture);



return null;

}

});

[/java]



(When I tried the usual method of projecting the view as described in the wiki everything worked perfectly so that side of things is working).



So to try and boil this down to simple questions:


  • Is there some recommended platform independent way to render into textures?

  • Is it possible to do what I'm trying to do with nifty?

    • - Render into the Texture once on demand rather than every frame

    • - Use one Nifty instance to render multiple textures (even if by rendering into one texture and then cloning it and repeating)





Sorry for the long post and thanks in advance for any suggestions people might have :)

Thanks,
Zarch

Ok, I’ve spent some more time on this today and I think I’ve cracked it. What I have is working but considering my unfamiliarity with the system I thought I’d ask if any of the more experienced jME/Nifty people here would mind taking a look. If it looks generally useful I’m happy to release this as a code fragment with whatever license is usually used for that sort of thing.







[java]

public class NiftyTexture {



private Nifty nifty = null;

private Texture2D workingTexture = null;

private ViewPort niftyView = null;

private final int width;

private final int height;

private final SimpleApplication jme3;



public NiftyTexture(int width, int height, SimpleApplication jme3) {

this.width = width;

this.height = height;

this.jme3 = jme3;



// Will just use standard plain colour material until nifty is ready

jme3.enqueue(new Callable<Object>() {



@Override

public Object call() throws Exception {

niftyView = new ViewPort(“NiftyView”, new Camera(NiftyTexture.this.width, NiftyTexture.this.height));

niftyView.setClearFlags(true, true, true);

NiftyJmeDisplay niftyDisplay = new NiftyJmeDisplay(NiftyTexture.this.jme3.getAssetManager(),

null,

NiftyTexture.this.jme3.getAudioRenderer(),

niftyView);

niftyDisplay.initialize(NiftyTexture.this.jme3.getRenderManager(), niftyView);

nifty = niftyDisplay.getNifty();

niftyView.addProcessor(niftyDisplay);



Texture2D depthTex = new Texture2D(NiftyTexture.this.width, NiftyTexture.this.height, Format.Depth);

FrameBuffer fb = new FrameBuffer(NiftyTexture.this.width, NiftyTexture.this.height, 1);

fb.setDepthTexture(depthTex);



workingTexture = new Texture2D(NiftyTexture.this.width, NiftyTexture.this.height, Format.RGBA8);

workingTexture.setMinFilter(MinFilter.Trilinear);

workingTexture.setMagFilter(MagFilter.Bilinear);

workingTexture.setWrap(Texture.WrapMode.EdgeClamp);



fb.setColorTexture(workingTexture);

niftyView.setOutputFrameBuffer(fb);



return null;

}

});

}



public Future<Texture> render(final String layout, final String screen, final ScreenController… controllers) {

// This assumes that enqueued callables are called FIFO which will ensure that the initialization

// call in the constructor is always called before render can be.

return jme3.enqueue(new Callable<Texture>() {



@Override

public Texture call() throws Exception {



RenderManager renderManager = NiftyTexture.this.jme3.getRenderManager();

nifty.fromXml(layout, screen, controllers);

renderManager.setCamera(niftyView.getCamera(), true);

renderManager.renderViewPort(niftyView, 1.0f);

renderManager.setCamera(niftyView.getCamera(), false);



return workingTexture.clone();

}

});

}



public int getHeight() {

return height;

}



public SimpleApplication getJme3() {

return jme3;

}



public int getWidth() {

return width;

}

}



[/java]

3 Likes

One mistake I already realised was doing the thread safety stuff inside the NiftyTexture class when it may well be used from the jme thread so I’ve moved that outside and/or made it optional.

Posted the tidied up class here: http://hub.jmonkeyengine.org/groups/contribution-depot-jme3/snippets/31/#message

2 Likes

I’ve posted a patch for nifty that fixes the controller problem, I’ve no idea how long it will take for that to be applied and filter through into JME though. For now I’m using reflection to access the field directly but that won’t work if a security manager gets involved.



I also ran into a problem with textures being shared/copied - essentially I didn’t seem to be able to take a copy of the texture at the end of the method running and get anything but black. If I recreate the frame buffer at the start of the render call that makes things work but that seems wasteful and I was also worried that there might be hidden race conditions. I did try creating 100 at once and checked they all created correctly though which means race conditions are at least not common.



Does anyone know how I’m supposed to take a copy of a texture/image that won’t be shared with the original? I’ve tried clone on both texture and image and even creating a new image with get/set data and I get either one image with two references or blackness.



[java]

/*

  • Copyright © 2012 Zero Separation Ltd
  • All rights reserved.

    *
  • Redistribution and use in source and binary forms, with or without
  • modification, are permitted provided that the following conditions are
  • met:

    *
    • Redistributions of source code must retain the above copyright
  • notice, this list of conditions and the following disclaimer.

    *
    • Redistributions in binary form must reproduce the above copyright
  • notice, this list of conditions and the following disclaimer in the
  • documentation and/or other materials provided with the distribution.

    *
    • Neither the name of ‘Zero Separation Ltd’ nor the names of its contributors
  • may be used to endorse or promote products derived from this software
  • without specific prior written permission.

    *
  • THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
  • "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
  • TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
  • PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
  • CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
  • EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
  • PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
  • PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
  • LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
  • NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
  • SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

    */



    package net.herodex.client.framework.utilities;



    import com.jme3.app.SimpleApplication;

    import com.jme3.niftygui.NiftyJmeDisplay;

    import com.jme3.renderer.Camera;

    import com.jme3.renderer.RenderManager;

    import com.jme3.renderer.ViewPort;

    import com.jme3.texture.FrameBuffer;

    import com.jme3.texture.Image.Format;

    import com.jme3.texture.Texture;

    import com.jme3.texture.Texture.MagFilter;

    import com.jme3.texture.Texture.MinFilter;

    import com.jme3.texture.Texture2D;

    import de.lessvoid.nifty.Nifty;

    import de.lessvoid.nifty.screen.ScreenController;

    import java.lang.reflect.Field;

    import java.util.List;

    import java.util.concurrent.Callable;

    import java.util.concurrent.Future;

    import java.util.logging.Level;

    import java.util.logging.Logger;



    /**

    *
  • @author Tim Boura - Zero Separation

    */

    public class NiftyTexture {



    private Nifty nifty = null;

    private Texture2D workingTexture = null;

    private ViewPort niftyView = null;

    private final int width;

    private final int height;

    private final SimpleApplication jme3;

    private final Texture2D depthTex;



    public NiftyTexture(int width, int height, SimpleApplication jme3) {

    this.width = width;

    this.height = height;

    this.jme3 = jme3;



    niftyView = new ViewPort("NiftyView", new Camera(NiftyTexture.this.width, NiftyTexture.this.height));

    niftyView.setClearFlags(true, true, true);

    NiftyJmeDisplay niftyDisplay = new NiftyJmeDisplay(NiftyTexture.this.jme3.getAssetManager(),

    null,

    NiftyTexture.this.jme3.getAudioRenderer(),

    niftyView);

    niftyDisplay.initialize(NiftyTexture.this.jme3.getRenderManager(), niftyView);

    nifty = niftyDisplay.getNifty();

    niftyView.addProcessor(niftyDisplay);



    depthTex = new Texture2D(NiftyTexture.this.width, NiftyTexture.this.height, Format.Depth);

    }



    public Future<Texture> renderFuture(final String layout, final String screen, final ScreenController… controllers) {

    return jme3.enqueue(new Callable<Texture>() {



    @Override

    public Texture call() throws Exception {

    return render(layout, screen, controllers);

    }

    });

    }



    public Texture render(final String layout, final String screen, final ScreenController… controllers) {



    FrameBuffer fb = new FrameBuffer(NiftyTexture.this.width, NiftyTexture.this.height, 1);

    fb.setDepthTexture(depthTex);



    workingTexture = new Texture2D(NiftyTexture.this.width, NiftyTexture.this.height, Format.RGBA8);

    workingTexture.setMinFilter(MinFilter.Trilinear);

    workingTexture.setMagFilter(MagFilter.Bilinear);

    workingTexture.setWrap(Texture.WrapMode.EdgeClamp);



    fb.setColorTexture(workingTexture);

    niftyView.setOutputFrameBuffer(fb);



    RenderManager renderManager = NiftyTexture.this.jme3.getRenderManager();

    nifty.fromXml(layout, screen, controllers);

    renderManager.setCamera(niftyView.getCamera(), true);

    renderManager.renderViewPort(niftyView, 1.0f);

    renderManager.setCamera(niftyView.getCamera(), false);

    try {

    // Reflection hack to remove controller until there is a proper way

    Field field = nifty.getClass().getDeclaredField("registeredScreenControllers");

    field.setAccessible(true);

    List<ScreenController> regList = (List<ScreenController>) field.get(nifty);

    for (ScreenController c: controllers)

    regList.remove©;



    } catch (IllegalArgumentException ex) {

    Logger.getLogger(NiftyTexture.class.getName()).log(Level.SEVERE, null, ex);

    } catch (IllegalAccessException ex) {

    Logger.getLogger(NiftyTexture.class.getName()).log(Level.SEVERE, null, ex);

    } catch (NoSuchFieldException ex) {

    Logger.getLogger(NiftyTexture.class.getName()).log(Level.SEVERE, null, ex);

    } catch (SecurityException ex) {

    Logger.getLogger(NiftyTexture.class.getName()).log(Level.SEVERE, null, ex);

    }



    return workingTexture;

    }



    public int getHeight() {

    return height;

    }



    public SimpleApplication getJme3() {

    return jme3;

    }



    public int getWidth() {

    return width;

    }

    }

    [/java]