A GUI toolkit in the hand is worth two in the bush

Taking a look at the existing UI stuff, I can only assume no one is actually using it to do anything meaningful. The UIEditBox, for example, doesn’t have a way to get or set the text being edited. That would seem to be somewhat necessary in normal use (not to mention it entering a space or an apostrophe when typed rather than spelling those words out). Further there seems to be no way to react to interface events, indeed, no notion of them at all. Various other things one would expect to find in a user interface toolkit are conspicuously lacking.



There has been much talk of a new UI system over the past four months, but it seems to remain elusively out of our grasp (apologies in advance to guurk). Given that the next major phase in my application (a commercial game) is entirely user interface related and having turned up no better options after a few hours of intensive Googling, I’m faced with the following choices:


  1. Write my own user interface library from scratch using the input and graphics primitives in JME


  2. Port some existing GL-targeted interface toolkit to JME


  3. Wait patiently for manna to fall from the sky



    I have 8 hours of productive work every day to devote to accomplishing this goal (though my employers would probably disapprove of me spending all day doing #3), so I’m fully prepared to undertake #1 or #2 starting tomorrrow morning. I just wanted to pipe in here, say hello (since I haven’t been on the forums before) and confirm that no one is indeed lurking in the bushes with some sort of usable user interface system that they just haven’t told anyone about.



    Per #2, I took a look at CEGUI and while it’s a fine toolkit, porting it outright would be rather difficult. I also dug up some info on GLOW and NGL/NUI, both interesting but also C++. Better, most likely to develop a new framework with similar principles in mind.



    While I appreciate the notion of a broadly usable toolkit that would magically work with every engine and GL-binding under the sun (apologies in advance to Schabby), what I need is a robust, useful toolkit that works with JME, so that’s what I’ll be working on.



    I have already integrated JME with our networking toolkit (http://www.threerings.net/code/narya/) which required the addition of an event queue to the main application loop (network events are processed in the spare time in a fixed framerate rendering loop), I am inclined to go with an event-based architecture for the user interface system as well, given that most people are familiar with that approach.



    Anyhow, I’ll make the code available in a publicly accessible version control repository, so anyone can track the progress of development and make use of it as things are being developed (boy I would love to see more source code from the rest of you guys, there are a lot of one off samples, but nothing that demonstrates sensible application organization). The (trivial) integration of Narya and JME is already publicly available in Narya’s Subversion repository, along with a few other bits.



    Onward ho!

Hey, thanks for ending the lurking and posting! :wink: I have to agree, GUI is a big area where jME is lacking. Largely that is because we have focused attention elsewhere, anticipating GUI additions in a future release. Believe me… I’ve longed to rip out certain sections of GUI code (the old old stuff) for a while now.



So, needless to say, I’m impressed and happy you are looking into GUI! :slight_smile: I write Swing apps all day for my day job and would love an event driven model, assuming it was robust enough to work in the context of the gaming world.



Please let us know if we can help in any way. I wish you the best.



PS: I saw Puzzle Pirates at GDC… Were you involved in writing that? Very cool Java game! Congrats on getting picked up.

Oops. I’m not used to phpBB forums that let one post without logging in.

Hi samskivert, i remeber i have done a little gui system just like u wanted, its rather missy but it works fine; just a little screen to show you what i mean:

http://www.geocities.com/doufish/1.png



if you wish i can send you the source, but tomorow just to tied things up for you. I am sure that it will help some how, and may be cut you development time dramatically.



PM me your contact information if you are intersted in it.



Outrunner.

samskivert,



First, welcome. Second, congratulations on Puzzle Pirates. The fact that you guys are going to be using jME really excites me. Third, the GUI…



You are absolutely right, we’ve floundered a bit in this area. First there was the widget library, which was great but stagnated. Then the Guurk’s GUI library which is great, but stagnating as nothing is being updated. As with everything, real world got in the way.



I know people would be thrilled if you got a complete library together. Also, there is no ownership of code here, so if you think Guurk’s library is a good start, by all means, take it from there. It might save you some time.



Last, (this is for everyone) I have just completed the major project for this semester’s class. Why did I let my wife talk me into going back to school? Work will be slowing down in about 2 weeks as we end a cycle on a contract we are trying to win. Therefore, I will soon have some free time! You happen to notice I wasn’t around much? Probably not, damn Renanse and his grand standing! :stuck_out_tongue: Also, I’m trying to work with my boss into allowing me to write a jME project over the summer, if so, I’ll be able to devote pretty much all day to jME.



So, in summary, welcome… uh… matey, a working GUI would be great, and I’ll be back.

Grandstanding? :stuck_out_tongue: Someone’s gotta do it. Dirty job, and all that…

 if you think Guurk's library is a good start, by all means, take it from there. It might save you some time.



Is Guurk's library the stuff in the com.jme.ui package or is it elsewhere?

yer, guruks is the UI. Theres also a widget library which is feature-rich, but it doesn’t go well with jme’s design…



DP

The stuff guurk has “recently” shown off on the forums is not in cvs, so if you wanted to look at that at all you’d need to talk to him.

Ah, I completely missed the com.jme.widget package. Somehow even my years of developing on the Amiga, where widgets were called widgets, failed to come to my aid.



That looks significantly more substantial than what’s in com.jme.ui. I’ll definitely take a look at that stuff as well.

I looked over the widget stuff and I see what DP means when he says it doesn’t go well with JME’s design. I’m assuming Guurks stuff expands upon what’s already in com.jme.ui, which handles input at far too low a level for comfort (from what I can see, every interface object simply has update() called on it and it has to figure out everything and anything it needs to know from the InputHandler).



Given these fine excuses to reinvent the wheel, I’ve gone ahead and gotten started on a new toolkit, and I’ve got most of the basic input event system written which processes raw input from the MouseInput and KeyInput into events and will track things like the interface component under the mouse (so that it can dispatch mouse events to it) as well as which interface component has focus (so that it can dispatch key events to it).



I need to give some consideration on how to play nicely with the existing InputHandler system. For one, I want to “disable” any key-related InputHandler activity while a component has focus. For another, I’d like to maintain the state of the modifiers (whether alt, control, shift, etc. are pressed) at all times so that it will be possible to report that someone shift-clicked a button or held the shift key down, then clicked a text component, giving it focus and started typing with the shift key down. I think the best way to accomodate this is to allow the InputHandler to do its non-buffered input processing immediately following a poll(), then allow my event dispatcher to process the keyboard events in a buffered manner with calls to next() and whatnot.



For the time being I’m just glossing over that whole business and making progress on other fronts. In any case, anyone who’s interested can see how things are coming together by checking the code out of subversion:



svn checkout svn://samskivert.com/jme-bui/trunk jme-bui



There are Windows GUI Subversion clients (TortoiseSVN being the one I set the artists up with) or you can get one that works on the command line. There’s not much to look at yet except what I mentioned above about event dispatch. I’m out of town from tomorrow afternoon through the weekend, but I expect to make rapid progress next week, so I’ll keep updating this thread with anything of interest.

samskivert, it looks like your manna has just fallen from the sky! :wink:



http://www.jmonkeyengine.com/jmeforum/viewtopic.php?t=1684



3. Wait patiently for manna to fall from the sky


DP

It’s amazing timing as I was in the process of putting together a website for what I have been working on and was linking to this forum from there and saw the post from Guurk. It’s a good day for user interfaces. I’ll certainly be taking a look at the new toolkit but given that I am now well underway on one that does exactly what I need (one of the fringe benefits of reinventing the wheel), I doubt I’ll be able to resist continuing to work on it.



So for any interested parties, I put together a little web page and a zip archive for those not Subversion enabled to see how things are coming along on BUI (the Bannana User Iterface, it seemed apropos for the Java Monkey Engine).



http://samskivert.com/code/jme-bui/



The state of my efforts is as follows:

  • I have a working event dispatch system which is used by the toolkit to create an environment very similar to the AWT in terms of mouse and keyboard events and the way components construct their behavior by responding to events
  • I have an event and "action" listener system where entities can register with components to hear about events (both basic events like mouse and keyboard and component specific events, currently only ActionEvent)
  • The toolkit has a notion of focus which indicates where keyboard events are delivered, though keyboard focus traversal is not yet implemented
  • I have a working label component, button component, text input component and multiline text display component (though more work needs to be done on that to better support scrolling). As you can see my focus has been more on text display as the roadblock that started this whole project was getting chat working in my game.
    [/list:u]I have also managed to integrate the basic JME polled input handler system with the event-driven input handler system used by BUI. It works by telling the InputDispatcher (which does event-driven input) about the InputHandler and the dispatcher only calls update() on the handler when no interface component has focus. This seems to generally do what one expects.

    The font rendering currently happens through Text instances which presents a number of challenges. Firstly, I would have support for colored text in BTextArea but the only way I can figure out to "color" a font is to maintain a modified version of the font texture image in each desired color (clearly not scalable). Perhaps I'm being naive as I don't have much experience in the wild wild world of 3D. Perhaps there's some way to recolor a texture on the fly.

    Font support in general needs serious work. I was reluctant to go down the route of texturing a Quad for each character as that seems very resource intensive, but I suspect JME will need better font rendering support in the Renderer if this is to be at all high-performance. The present "interface" between JME and fonts whereby a font must be presented as a texture in a special format seems limiting. I cringe at the thought of how I'm going to support Chinese with its ten thousand plus characters. I'm punting on that for now, but it is inevitably going to come up when we get to localizing our game.

    There's a (native) library called FTGL that would provide the basis for more sophisticated font support, but introducing another native library dependency is not likely to make me any friends. Alternatively, something could probably be worked out by using the AWT font rendering support and a character cache of some sort. Whether JME wants to introduce a dependency on the AWT is another question.

    One other question from my 3D noobie self. I tried to create component backgrounds that can be stretched in a way that I find particularly useful which is to cut the source image up into nine sections and only scale the center sections:

    +


    +
    +
    +
    | unscaled |  <- scaled ->  | unscaled |
    +
    +
    +
    +
    |    ^     |       ^        |    ^     |
    |  scaled  |  <- scaled ->  |  scaled  |
    |    v     |       v        |    v     |
    +
    +
    +
    +
    | unscaled |  <- scaled ->  | unscaled |
    +
    +
    +
    +


    However, my naive implementation of this which is to create nine quads and provide them with texture coordinates that map them to each section of the image with the corner sections set to the exact size of that section of the image and the central sections scaled to whatever size is needed, still results in fuzzyness. Perhaps it's the texture filtering, but I was unable to turn that off without JME freaking out. Is there a well-known way to get a pixel accurate rendering of a bitmap on the screen? Take a look at TiledBackground.java in the BUI distribution if you want to see what I'm currently doing.

    Anyhow, I hope BUI turns out to be useful to others using JME and I hope its value outweighs the cost of introducing yet another UI library to divide up peoples' attention. I would certainly appreciate any feedback anyone cared to give on the way things are done now as this is the first time I've done anything in 3D and I'm thus susceptible to making any and all rookie mistakes.

Awesome. Screenshots?

Not much to see except the text input and display but here it is:







Anyone else seeing the strange half-character offset when rendering text with Text objects? Maybe I’m doing something improperly or maybe Text objects are just “supposed” to be positioned half a character off in the negative x direction.

Ooo looks pretty cool so far. I can’t wait till the scrollbar works! :smiley:

What about the buttons? How do they look? Oh, and is it possible that you can use the coordinates of the mouse to do live highlighting? (Stretching that box in the background?)

is it possible that you can use the coordinates of the mouse to do live highlighting?

It's definitely possible, but selections and cut and paste are not even close to being implemented at the moment.

First let me say, your design sounds very promising, and the results show you are having luck. I’ll try to answer the questions the best I can, I don’t know the answers to all of them myself.



Font rendering (especially via Text) is very basic, and very substandard. This is something that needs to be replaced (with a way to generate font maps dynamically) and I can understand your frustration in this regard. You’ll notice that both of the other GUI work have their own Text replacements. I have ideas for this, but each time I go to implement it something comes up. Any ideas on this front would be welcome with open arms.



Coloring of the font should be possible with the setTextColor(ColorRGBA) method of Text class. Does this not work correctly? Wouldn’t suprise me if it didn’t.



Texture Fuzzyness - Couple tips are: make sure you are not using mipmaps if you don’t need them (they will blur), try turning off texture compression, try a larger texture image.



BUI looks great (I think your design sounds solid), and I look forward to seeing it more. When I get back to working on jME in a week or two, my first goal is to clean up the packaging of the API. Such that there will be the core jar, with external jars for specific feature sets. This will make it easier to switch in and out things such as GUI tools.



While I understand you have your own vision for your GUI, you and Guurk might want to at least get in touch to see if there is any common ground.



Keep it up.



P.S. Puzzle Pirates is addicting, I find myself Bilging quite often. If you see Oneye that’s me.

Coloring of the font should be possible with the setTextColor(ColorRGBA) method of Text class. Does this not work correctly? Wouldn't suprise me if it didn't.

That works fine. I just foolishly didn't notice it and was trying to use setSolidColor().

Texture Fuzzyness - Couple tips are: make sure you are not using mipmaps if you don't need them (they will blur), try turning off texture compression, try a larger texture image.

When I try to turn off mipmapping I get a funny error. Here's how I set up the texture:

        Texture texture = TextureManager.loadTexture(
            source, Texture.MM_NONE, Texture.FM_LINEAR);
        texture.setWrap(Texture.WM_WRAP_S_WRAP_T);
        _twidth = texture.getImage().getWidth();
        _theight = texture.getImage().getHeight();
        _tstate = DisplaySystem.getDisplaySystem().getRenderer().
            createTextureState();
        _tstate.setEnabled(true);
        _tstate.setTexture(texture);


and here's the error:

org.lwjgl.opengl.OpenGLException: Invalid value (1281)
        at org.lwjgl.opengl.Util.checkGLError(Util.java:56)
        at org.lwjgl.opengl.Display.update(Display.java:544)
        at com.jme.renderer.lwjgl.LWJGLRenderer.displayBackBuffer(LWJGLRenderer.java:516)
        at com.jme.app.BaseGame.start(BaseGame.java:74)
        at com.jme.bui.tests.LayoutTest.main(LayoutTest.java:119)


I'm using this texture on nine quads, each with texture coordinates set up like so:

    /** Contains texture coordinates for each of the nine sections into
     * which we divide the background image. */
    protected static Vector2f[][] _tcoords = new Vector2f[9][4];

    /**
     * These map points on a 4x4 grid to texture coordinates. Consider the
     * following grid:
     *
     * <pre>
     * 12 13 14 15
     *  8  9 10 11
     *  4  5  6  7
     *  0  1  2  3
     * </pre>
     *
     * Each of the nine sections is defined by four of the grid
     * coordinates. For example, the upper left section is 12, 8, 9, 13 and
     * we proceed in row major order from there.
     */
    protected static final int[] TCOORDS = {
        4, 0, 1, 5,
        5, 1, 2, 6,
        6, 2, 3, 7,
        8, 4, 5, 9,
        9, 5, 6, 10,
        10, 6, 7, 11,
        12, 8, 9, 13,
        13, 9, 10, 14,
        14, 10, 11, 15
    };

    static {
        Vector2f[] coords = new Vector2f[4*4];
        int idx = 0;
        for (int yy = 0; yy < 4; yy++) {
            for (int xx = 0; xx < 4; xx++) {
                coords[idx++] = new Vector2f(xx/3f, yy/3f);
            }
        }
        for (int ii = 0; ii < TCOORDS.length/4; ii++) {
            _tcoords[ii][0] = coords[TCOORDS[4*ii]];
            _tcoords[ii][1] = coords[TCOORDS[4*ii+1]];
            _tcoords[ii][2] = coords[TCOORDS[4*ii+2]];
            _tcoords[ii][3] = coords[TCOORDS[4*ii+3]];
        }
    }


The idea being each quad will display one section of the texture and I scale the size of the "internal" quads based on how big the whole component needs to be. It works, aside from the weird error when I try to turn off mipmapping and of course the blurriness that you can see in the screen shot above. The original texture looks like this:



Anyhow, if there's anything obviously wrong, I'd be happy to hear it. Otherwise I'll keep fiddling around.

Looks good, I could use your input and experience maybe merging our work together.



The half character thing seems like it’s probably the standard means of positioning objects within the 2D environment where the center of the geometry of the object is the object’s position within the worldspace. Thus all my computations have to account for -height/2 and -width/2 to figure out position according to how people normally reference screen position.