3d gui?

What benefits are derived and how do people see the actualization of a 3D GUI?  I've been thinking about what gains can be achieved through a 3D GUI rather than a 2D GUI and I've had somewhat of a hard time determining what real benefits can be seen so I thought I'd ask the community.

To be clear, we're not talking about GUI in a 3D environment like JMEDesktop supports, but rather GUI components with depth.

Interesting question.

Reminds me of the 3D desktop discussion. To put everything on the table lets consider some things:

1 - A 3D scene is good to represent real-world objects, that's a fact

2 - Should a GUI be based on real-world objects analogies or not?

3 - If so, we have to consider the handling as well

Today I don't think it's really necessary, because:

  • A 3D GUI can make the player confuse about which objects come from the scene and which ones are GUI elements (usability questions about #1)
  • A 2D GUI can serve well. It's a good analogy of a glass pane with touch buttons and screen (#2). The handling is straigh forward (#3)

    There'se also the whole discussion about the ANTI-MAC Interface, that says users nowadays don't need so many real-world analogies anymore, but that's controversial.

    This is my first opinion on this subject.

Let's say you have a "backpack" in your game you can put items in? Why shouldn't the items in the "backpack" screen be in 3D, and allow you to use some nice 3D effects when you handle it? But that's just a very functional example. Take it a small step further… why shouldn't normal object in your GUI have any 3D animation effects when you hover over them or press them? Some of these could be very functional as well, for example scaling an object up

Or to bring it back to it's most basic form: why should you have a 2D GUI emulating a 3D look for a button if you can have true 3D button? Why have some fake 2D highlight effect when you hover over an object, if I can just put BloomPass on my 3D object or use it as a particle emitter?

IMHO I don't see any reason at all why you should have a GUI capable only of 2D in a 3D enviroment, other than that it's very hard to create a reusable and styleable 3D GUI API. I'd love to see someone try though, and I think once you manage it, you get some real extra benifits too, such as a very strong cohesion between jME scenegraph API and the UI.

I guess you can split this into roughly 3 areas:

  • a GUI focused input and eventing system for scenegraph objects (not just things like click events, but character input, focus handeling, etc)
  • a 3D aware layout system ("GuiNodes" that will arrange the positions of their childeren)
  • an API for styling/themes.

    You'd probably still end up with a basic layout pattern that is 2D oriented in most cases, though I think if you're willing to put some time into it the 3d elements will really enhance it both functionaly and for looking good. I also think for some cases a real 3D GUI could lead for some very innovative solutions for things that are problematic in current 2D GUI's, eg for quick access to things, or "information overload".

    As for the user confusing the game and the 3D GUI, I think we should have a little faith in either our users or our own desgin skills.

    If you look at games, they usually have 3D elements botched onto their 2D GUIs anyway. That doesn't mean they have a pleasant and non-limiting API to do that though…

DP, are willing to take this further and try implement it? After Llama's args I can see the benefits.

I’m actually using a very simple 3D GUI in some project. It uses PickDevice to do that “GUI focused input and eventing system for scenegraph objects”. Though it is “just things like click events”  :slight_smile:

Here a screenshot - the ‘gui’ is the bunch of spinning boxes on the right:


(edit: changed into url)

llama said it all. And when it comes to implement themes support, i dont think its too hard to separate the gui handling and the visual representation. For example you have the GUI concept of a button. It can have keyboard focus, can be in pressed state and so on. The visualization can be completly seperated. So one theme uses a sphere, that gets smaller, if pressed, and changes color or effects, when focused or hovered. The only problematic controls would be all kinds of text input controls.

Maybe such 3D GUI Elements, well Buttons mostly, could also be used in Point and Click games like (J)Monkey Island.

At some point, when I was using Java3D for graphics, I was trying to develop a character creation and animation program… sort of bones animation. The thing is that point and click directly on each bone sometimes is confusing, and thus I decided to add some 3D components to change some of the parameters… I used a box as a component to specify the position relative to a point (translation) of a joint. A cone to depict the orientation of the joint in such a way that it pointed in the same direction as the joint, and a sphere for general scene rotation.

Of course this was a custom GUI design for the application, and therefore never intended as a GUI library, but it is just an example of the benefits you can get from exploiting a 3D environment into a GUI design.

I am all for the 3D GUI.

Insightful feedback.  Now, relating outside of games (I know this is a game forum, so this may be difficult) can anyone think of some real-world examples where TRUE 3D is beneficial (as opposed to giving the appearance of 3D such as scaling, animation effects, etc) and useful to the application?  I'm contemplating the idea of adding functionality to jSeamless for 3D (granted on most implementations it would be 2D trickery) and writing an OpenGL implementation that would exploit that functionality that could be used in games but also in regular desktop applications.

Is anyone interested in a 3D mail client?

This way I could have an animated Dacshundt bitting the mailman while waiting for pop3, or even the dog stealing the mail bag from the postman at the end of the animation. :slight_smile:

The uses of 3D GUI are severely limited by the fact that screens are still 2D, 3D guis are mostly aesthetic than function at this point, IMO.

The uses of 3D GUI are severely limited by the fact that screens are still 2D, 3D guis are mostly aesthetic than function at this point, IMO.

Then why the hell are we making 3d games?! :P

3 GUI could help you visualize what you are doing. For example, compare 4 virtual desktops on linux vs 4 sides of the cube in beryl.

Well, that really depends on what kind of spatial relations you will have for your mails, accounts, etc. The traditional tree-like structure is fairly well handled in 2D. Most people have not even thought about what third dimension could be used, and that is why we tend to store information hierarchically on trees (because we can draw them in 2D).

What you are asking for is uncharted territory. Is a little bit like asking who could use and how a 3D desktop environment. There are many ways, but no definite answer. Beryl/Compiz are a good attempt, but I have seen some other very impressive technologies in research stage.

The capability should not hinder the development of ideas… even though we mostly think everything related to information in 2D, the technology should be there for when we start thinking differently.

Have you heard of the "life-stream" model of organizing information? Maybe it's possible to implement a nice stream of messages (a bit like vista's new win-tab way of switching tasks).

Lifestream was a concept invented by Eric Freeman and David Gelernter.

You mean something like this?


IMO, such a GUI is mainly for fun and aesthetics, definitly NOT usability.

Think about something like Fitt’s law: Particletree » Visualizing Fitts’s Law

I think it’s difficult to take into account when all you have to interact in your 3D world is a 2D screen and a 2D mouse.

And what about the Principle of least astonishment?

Camera movement also means target movement. If you use “point and click”, the previous principe is violated (IMO). Moving your avatar towards the target would still be OK (as the relative position between the avatar and the target remains the same), but slooooow.

(3D games are another matter: fun and aesthetics are more important than for a UI)

This is definitely moderately uncharted territory.  Such is why I posed the question.  I'm curious of what ideas people have for a 3D GUI that is not a game.

All non-game GUIs today (at least that I know of) have you in a single place and you manipulate the world around you.  This is the opposite of most 3D games.  What about a 3D GUI that is more like MMORPG?  You have an avatar and your desktop is your virtual world where you can start and run applications.  You can organize those applications in this 3D world and customize to your needs?

That might be something there. Then you can have all your running applications as Nodes on your "desk-scape" represented by some sort of 3D object depending on the application, and you can switch to the application by "walking up to it and stepping through the door". Exiting the application is similar and takes you back to your "desk-scape"

darkfrog said:

What about a 3D GUI that is more like MMORPG?

I think the problem with all known (at least by me) 3D desktop concepts is that they are too similar to MMO type games.  Walking from one place to another takes time for example.  This is as it should be in an MMO, but in a desktop application you care about getting to where you want as fast as possible.  I think it's possible to create an intuitive and efficient desktop experience in 3D that can even be faster once you're used to it than a 2D desktop, but it would require a LOT of work to accomplish.

This is great feedback. :slight_smile:

If you ask me, it is all about the usability. Take the example of navigating the filesystem to a file in a directory. Assuming you know the path to the directory:

In 1D (command line) you would have to type the path linearly (with autocompletion, perhaps), until you reach the desired file.

In 2D (finder/explorer/konqueror window) you will have to still go search the pieces of the path in order, but at each time you have the other dimension help you choose the right piece of path in a contextual way.

In 3D I would like to see something like each file being in a point in space, clustered together with hierarchical relations with visual clues (i.e. files in the same directory are in the same place roughly) and I would like to be able to direct myself in the approximate direction of the file with a variable speed (to accommodate for different users). If the visual clues are good, I would be able to approach the general direction of the file, without having to actually traverse each particle of the path.