HUD support

Hi,

as I have already threatened, this is another topic I am looking into currently.

What I have got so far is the following:

Starting from SimpleGame I created an instance of a HUD class derived from Node, which will contain all my HUD stuff. This node is not added to the rootNode.



Here is my simpleUpdate in my app class:


    protected void simpleUpdate() {
        float tpf = timer.getTimePerFrame();
        world.step(tpf);
        float speed = botNode.currentSpeed.length();
        hud.setSpeed(speed);
        hud.updateGeometricState(tpf, true);
        hud.updateRenderState();
    }


World is my physics simulation. As you can see, in my simple test I have chosen to somehow display the speed of the avatar in the HUD.

Here is simpleRender from the app class:


   protected void simpleRender() {
      display.getRenderer().setOrthoCenter();
      display.getRenderer().draw(hud);
      display.getRenderer().unsetOrtho();
   }



And here, finally, is my HUD class:


public class HUD extends Node {
   
   private static final float ORIGINX = -290f;
   private static final float WIDTHX = 100f;
   
   private Spatial box;
   
   public HUD(String name) {
      super(name);
      box = new Box("hudbox", new Vector3f(0f, 0f, 0f), WIDTHX, 10f, 1f);
      
        TextureState ts = DisplaySystem.getDisplaySystem().getRenderer().getTextureState();
        ts.setTexture(TextureManager.loadTexture(getClass().getClassLoader()
                .getResource("flaresmall.jpg"), Texture.MM_LINEAR,
                Texture.FM_LINEAR, true));
        ts.setEnabled(true);
       
        box.setRenderState(ts);

        AlphaState as = DisplaySystem.getDisplaySystem().getRenderer().getAlphaState();
        as.setEnabled(true);
        as.setBlendEnabled(true);
        as.setSrcFunction(AlphaState.SB_SRC_ALPHA);
        as.setDstFunction(AlphaState.DB_ONE);
        as.setTestEnabled(true);
        as.setTestFunction(AlphaState.TF_GREATER);
        box.setRenderState(as);

        MaterialState ms = DisplaySystem.getDisplaySystem().getRenderer().getMaterialState();
        ColorRGBA color = new ColorRGBA(1f, 0f, 0f, 1f);
        ms.setAmbient(color);
        ms.setDiffuse(color);
        ms.setAlpha(0.5f);
        ms.setEnabled(true);
        box.setRenderState(ms);
       
        PointLight light = new PointLight();
        light.setDiffuse(new ColorRGBA(1.0f, 1.0f, 1.0f, 1.0f));
        light.setAmbient(new ColorRGBA(0.5f, 0.5f, 0.5f, 1.0f));
        light.setLocation(new Vector3f(0, 0, 100));
        light.setEnabled(true);

        LightState ls = DisplaySystem.getDisplaySystem().getRenderer().getLightState();
        ls.setEnabled(true);
        ls.attach(light);
        setRenderState(ls);

      attachChild(box);
      box.setLocalTranslation(new Vector3f(ORIGINX, -260f, 0f));
      updateRenderState();
   }
   
   public void setSpeed(float newSpeed) {
      box.setLocalTranslation(new Vector3f(ORIGINX - ((1 - newSpeed/30f) * WIDTHX / 2) , -260f, 0f));
      box.setLocalScale(newSpeed / 30f);
      updateWorldBound();
   }

} /* public class HUD extends Node */


As you can see, what I do is draw a texture on a box (could be a quad, I know) and then I scale and translate this box to reflect the speed.

Well the good thing is that I figured out how to draw in ortho mode. The bad thing is that I have something different in mind than what I have achieved by now.
If you look at typical huds, they often consist of a number of fixed bitmaps, which surround the different value displays. So if I understand things correctly, I would have to create a mask texture to mask out the area of the HUD display from the prespective rendering in the background (at least as long as the display is nonrectangular) and place it onto a quad, then I'd provide the bitmap to draw the static parts (which must match the mask in its outer form) and place it on another quad which gets drawn after the mask and then I would need to draw another bitmap containing the progress bar or whatever value representation chosen, which I would have to create dynamically like in painting into a Graphics2d from an ImageIcon and then changing that into a texture (which I suppose is not a very fast way to do it) before I can place it on a third quad. Or am I again missing something? Can I simply draw a "box" or "line" or "bitmap" somewhere in ortho mode, representing the value, without having to provide a trimesh onto which it is projected (and a mask and a light...).

Some of you must have had the same questions in the past, as HUD information is present in almost any kind of game I can think of. How did you solve these problems?

Maybe some of the code used in ImageWidget would help.

I’m not knowledgable enough to know how easily it mixes with the standard 3d display. I’ve never tried drawing an image widget in the same viewport as my 3d view.



If nothing else, I would guess you could write an abstract HUD display object using the LWJGLImageWidget as a starting point.



-Mike

I’m actually going to be sitting down and tackling this issue in my own project over the next week or two (as work permits)… If I come up with any helpful comments I’ll post them here.

@mkienenb:

I had a look at the ImageWidget and the Widget stuff in general the other day. I tried TestWidgetButtonLightSwitch, as this test program combines a Widget and a 3D rendered scene, but I noticed that the widget was always drawn behind the scene rather than in front of it and couldn’t work out how to change it, so I did not look further into Widgets. Maybe if someone could explain a bit more about the widget lib - I found very little comment there and I don’t understand enough OpenGL yet to understand what it is doing from looking at the source only.

"batman.ac" wrote:
@mkienenb:
I had a look at the ImageWidget and the Widget stuff in general the other day. I tried TestWidgetButtonLightSwitch, as this test program combines a Widget and a 3D rendered scene, but I noticed that the widget was always drawn behind the scene rather than in front of it and couldn't work out how to change it, so I did not look further into Widgets. Maybe if someone could explain a bit more about the widget lib - I found very little comment there.

I wish I could help, but I am equally ignorant. My guess in this situation is that the widget code is being drawn before the 3d code, and you'd need to switch the ordering somehow.
"batman.ac" wrote:
I don't understand enough OpenGL yet to understand what it is doing from looking at the source only.
Well, I also barely understand OpenGL. I have no training, and no reference material. In fact, I "wrote" LWJGLImageWidget by google searches and guesswork. I guess what I'm trying to say is, "Don't let ignorance stop you from doing the improbable." :)

Spent the whole evening now trying to get WidgetImage to work inside my app and it never rendered. Debugging said, that it had no world bound and therefore was ignored, but I couldn’t figure out, where the problem is.



Instead I now come up with a different idea. I’ll try to extend com.jme. image.Image by some infrastructure to have a paint method, which allows for custom painting into a Graphics2D context using all appropriate methods. The Graphics2D is provided by an internally stored BufferedImage and then always after painting it updates the internal ByteBuffer of the Image. Which hopefully will be reflected in the texture then. Maybe I’ll have time to try this out before the weekend. This will still mean, that I have to provide a quad to map the texture onto, but maybe I can solve some of the other problems.

You might also look at the com.jme.scene.BillboardNode class and see if that’s helpful. From what I’ve heard, it sounds like a 2d drawing area that’s always orthogonal to the viewer.

I’ve played around a bit and came up with two solutions now.

I’ve managed to write a PaintableImage class, which allows one to paint into an Image/Texture using Graphics2D calls. Since the changed image data must always be uploaded to the graphics adapter first (eg. by calling setTextureId(0) for the texture in the update call) this obviously is slow. But since you can use all the power of Graphics2D it’s also a nice method to paint more complex Textures once and then have them ready for usage.



The other method uses a simple texture created from a png file containing alpha information, whose complete left half is painted and whose complete right half is transparent. Depending on the gauge value I like to display, I simply shift the texture coordinates accordingly. It works nicely and looks quite promising.



If there is any interest, I can post the code here or send it per eMail.

I’d love to see screenshots to see the results.

Instead of sending screenshots I give you the source. First of all here’s the PaintableImage class.


/*
 * Created on 26.05.2004
 */
import java.awt.Graphics2D;
import java.awt.image.BufferedImage;
import java.nio.ByteBuffer;
import java.util.logging.Level;

import com.jme.image.Image;
import com.jme.util.LoggingSystem;

/**
 * This class provides a method to create an image for usage in a texture which
 * can be repainted during execution by calling graphics2d methods.
 *
 * @author batman
 */
public abstract class PaintableImage extends Image {
   private BufferedImage backImg;
   private ByteBuffer scratch;
   /**
    * Creates a new PaintableImage.
    *
    * @param width
    *            The fixed width of this image.
    * @param height
    *            The fixed height of this image.
    * @param hasAlpha
    *            Whether this image shall contain alpha information.
    */
   public PaintableImage(int width, int height, boolean hasAlpha) {
      super();
      try {
         backImg = new BufferedImage(width, height, hasAlpha
               ? BufferedImage.TYPE_4BYTE_ABGR
               : BufferedImage.TYPE_3BYTE_BGR);
         setType(hasAlpha
               ? com.jme.image.Image.RGBA8888
               : com.jme.image.Image.RGB888);
         setWidth(backImg.getWidth());
         setHeight(backImg.getHeight());
         scratch = ByteBuffer.allocateDirect(4 * backImg.getWidth()
               * backImg.getHeight());
      } catch (IllegalArgumentException e) {
         LoggingSystem.getLogger().log(Level.WARNING,
               "Problem creating buffered Image: " + e.getMessage());
      }
   }

   /**
    * This method must be called, when the state of the image has changed in a
    * way that it needs to be repainted.
    */
   public void refreshImage() {
      Graphics2D g = backImg.createGraphics();
      paint(g);
      g.dispose();

      /* get the image data */
      byte data[] = (byte[]) backImg.getRaster().getDataElements(0, 0,
            backImg.getWidth(), backImg.getHeight(), null);
      scratch.clear();
      scratch.put(data, 0, data.length);
      scratch.rewind();

      setData(scratch);
   }

   /**
    * Implement this method so that it contains your drawing calls. The image
    * state is preserved so if you want to clear the image to completely redraw
    * it, you must provide the function yourself.
    *
    * @param graphicsContext
    *            Tha Graphics2D context in which the painting can be performed.
    */
   public abstract void paint(Graphics2D graphicsContext);
}



As you can see, this is an abstract class, so here's a simple implementation for it:


/*
 * Created on 26.05.2004
 */
import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.geom.AffineTransform;

/**
 * This is just an example for a PaintableImage that accepts a value between
 * 0 and 100 and paints itself as a box reflecting this value.
 * @author batman
 */
public class GaugeImage extends PaintableImage {
   private int value = 0;

   public GaugeImage() {
      super(128, 128, true);
        /* call refreshImage once so it is initialized */
        refreshImage();
   }
   
   public void setValue(int newValue) {
      if (newValue != value && newValue <= 100 && value >= 0) {
         value = newValue;
         refreshImage();
      }
   }
   
   /**
    * @see PaintableImage#paint(java.awt.Graphics2D)
    */
   public void paint(Graphics2D g) {
        AffineTransform af = new AffineTransform(1, 0, 0, -1, 0, getHeight());
        g.setBackground(new Color(0f, 0f, 0f, 0f));
      g.clearRect(0, 0, getWidth(), getHeight());
      g.setColor(new Color(1f, 0f, 1f, 1f));
      g.fillRect(0, 0, value * getWidth() / 100 , getHeight() - 15);
        g.setTransform(af);
        g.setColor(new Color(0f, 1f, 0f, 1f));
        g.drawString(Integer.toString(value), 10, 10);
   }

}



And here finally is a simple test class.


/*
 * Created on 26.05.2004
 */
import com.jme.app.SimpleGame;
import com.jme.bounding.BoundingBox;
import com.jme.image.Texture;
import com.jme.math.Quaternion;
import com.jme.math.Vector3f;
import com.jme.renderer.ColorRGBA;
import com.jme.scene.Node;
import com.jme.scene.shape.Cylinder;
import com.jme.scene.shape.Quad;
import com.jme.scene.state.AlphaState;
import com.jme.scene.state.TextureState;

/**
 * Simple Test frame for PaintableImage
 * @author batman
 */
public class TestPaintableImage extends SimpleGame {

    private Quaternion rotQuat = new Quaternion();
    private float angle = 0;
    private Vector3f axis = new Vector3f(1, 1, 0);
    private Cylinder t;
    private GaugeImage img;
    private Texture texture;
    private TextureState ts;
    private Node hud;
    private int lastValue = 0;

    /**
     * Entry point for the test,
     * @param args
     */
    public static void main(String[] args) {
        TestPaintableImage app = new TestPaintableImage();
        app.setDialogBehaviour(FIRSTRUN_OR_NOCONFIGFILE_SHOW_PROPS_DIALOG);
        app.start();
    }

    /**
     * Performs recalculations
     */
    protected void simpleUpdate() {
        /* recalculate rotation for the cylinder */
        if (timer.getTimePerFrame() < 1) {
            angle = angle + (timer.getTimePerFrame() * 1);
            if (angle > 360)
                angle = 0;
        }

        rotQuat.fromAngleAxis(angle, axis);
        t.setLocalRotation(rotQuat);
       
        /* now derive some useless value for the gauge */
        int value = (int)(angle * 8f);
        /* since this is a slooooow operation, do it only if needed */
        if (value != lastValue) {
            lastValue = value;
           img.setValue(value);
            /* force re-apply of the texture, otherwise the changed image won't be uploaded */
           texture.setTextureId(0);
        }
    }

    /**
     * builds the trimesh.
     * @see com.jme.app.SimpleGame#initGame()
     */
    protected void simpleInitGame() {
        display.setTitle("Cylinder Test");
       
        /* create a rotating cylinder so we have something in the background */
        t = new Cylinder("Cylinder", 6, 18, 5, 10);
        t.setModelBound(new BoundingBox());
        t.updateModelBound();
        rootNode.attachChild(t);
       
        /* now create the gauge */
        img = new GaugeImage();
        img.setValue(0);
       
        /* create a texture and apply the image to this */
        texture = new Texture();
        texture.setApply(Texture.AM_MODULATE);
        texture.setBlendColor(new ColorRGBA(1, 1, 1, 1));
        texture.setCorrection(Texture.CM_PERSPECTIVE);
        texture.setFilter(Texture.MM_LINEAR);
        texture.setImage(img);
        texture.setMipmapState(Texture.FM_LINEAR);
       
        /* now create a texture state for this texture */
        ts = display.getRenderer().getTextureState();
       
        ts.setTexture(texture);
        ts.setEnabled(true);

        /* we want some nice alpha blending of the ortho node */
        AlphaState as = display.getRenderer().getAlphaState();
        as.setBlendEnabled(true);
        as.setSrcFunction(AlphaState.SB_SRC_ALPHA);
        as.setDstFunction(AlphaState.DB_ONE_MINUS_SRC_ALPHA);
        as.setTestEnabled(true);
        as.setTestFunction(AlphaState.TF_GREATER);
        as.setEnabled(true);
       
        hud = new Node("hud");
       
        /* create a hud which shall receive the texture */
        Quad q = new Quad("gauge", 128f, 128f);
       
        hud.attachChild(q);

        hud.setRenderState(ts);
        hud.setRenderState(as);
        hud.updateRenderState();
       
        lightState.setTwoSidedLighting(true);
    }
   
    /**
     * @see com.jme.app.SimpleGame#simpleRender()
     */
    protected void simpleRender() {
        /* at the end of the render process, draw the hud */
        display.getRenderer().setOrthoCenter();
        display.getRenderer().draw(hud);
        display.getRenderer().unsetOrtho();
    }
   

}

"mojomonk" wrote:
I'd love to see screenshots to see the results.

Basically, being quite talkative, I've set up a blog on our clan server to document the progress while writing my game. There I have a screenie showing the hud gauge using the left side opaqe/right side transparent texture on the lower left.

http://www.cli-clan.de/tiki/tiki-view_blog.php?blogId=4

Ignore the fact, that most of the page is in german and green XD

I like! Nice screen shot. How are you holding map data in memory? (format)

I currently use my own format. The map file describes all textures in use, then all floor tiles, then all wall tiles (and later some in-map objects. The particle system will be some sort of power-up field). I parse them, create the quads (own format so I can spare the translation), place them under top nodes (sorted from back to front) and then I assign the texture states. I do collision detection only against the walls and terrain following only against the floor, which saves quite a few calculations.



BTW, completely off-topic question: does anybody use a non-US keyboard?!? I do have the feeling, that I have problems identifying all keys for a german keyboard. We do not have OPENBRACKET, we have "ö"…

Sorry, US keyboard here, but would it help if you could match on the ascii or unicode value of the key pressed? Might be something to look into.

Surely. The point is that german keyboards have more keys than us keyboards and these additional keys (like Alt Gr) do not generate ascii codes but are meta keys which normally allow for different characters when pressed together with another key.

I like games which let the gamer freely decide which keys to use since most gamers I know have a unique favorite keyboard layout. Many keys I use are meta keys (eg. I use right control for jumping and right shift for crouching in 1st person shooters).



I will write a simple test prog which returns me all key codes that jME can identify. If I come up with something interesting and commonly usable, I’ll post it here.


/* now derive some useless value for the gauge */
int value = (int)(angle * 8f);
/* since this is a slooooow operation, do it only if needed */
if (value != lastValue) {
lastValue = value;
img.setValue(value);
/* force re-apply of the texture, otherwise the changed image won't be uploaded */
texture.setTextureId(0);
}
}



should be:


        /* now derive some useless value for the gauge */
        int value = (int)(angle * 8f);
        /* since this is a slooooow operation, do it only if needed */
        if (value != lastValue) {
            lastValue = value;
           img.setValue(value);
            /* force re-apply of the texture, otherwise the changed image won't be uploaded */
          // SHOULD BEl
           textureState.delete(0);
        }
    }



just dont want you to fall in the same pit hole I did. Check your app to make sure it isn't doing that.

You’re absolutely right, thanks!



Interestingly I was asking how to remove textures from the graphics adapter memory in another thread, but I forgot about this app.