Virtual Reality library architecture discussion (continued from OpenVR Available. Convert?)

Don’t forget not everyone is going to be building a VR application from scratch or targeting VR exclusively. We want a library that is easy to convert existing applications into VR & make it easy to switch between VR and non-VR modes. The current library does this very well already. 4089, for example, will run in non-VR mode if no VR equipment is detected, and seamlessly run in VR mode when VR equipment is detected. Virtually no code, including GUI construction, needs to be changed because it is handled by the guiNode system which gets instanced in the VRApplication. I’ve been using this library commercially in 2 games on Steam, so perhaps I’m the only one seeing the added value with the current structure.

Ultimately, I don’t want to expose the developer to more complexity if we can hide it inside the library itself. VRApplication gives us more flexibility in what we can automate & integrate. I believe it is safe to assume games (including VR ones) will have a user interface, so I’d rather have it be an “opt-out” GUI management system (perhaps via preconfigureVR call) than an “opt-in”.

2 Likes

What if there’s an option in the game to turn VR on and off?

We need to target the most common use cases first, and make them very simple & easy to use. Right now, games are overwhelming being developed not for VR. We need to make it easy to convert those into VR games & make sure library integration is not the biggest hurdle. Also, people don’t want to bet all their development time on VR taking off, so we don’t want to require people to develop exclusively for VR or exclusively for non-VR.

Getting caught up in edge cases, like implementing in-game switching between VR & non-VR modes distracts us from the core purposes of the library: to make VR development easy & accessible. I don’t see why there couldn’t be a simple VRApplication.setVRMode(boolean enabled) function to accomplish this (that would handle removing/adding AppStates, moving GUI elements to the GUI bucket etc.), but again, it is an edge case we don’t need to focus on now.

We don’t have OpenVR distortion working yet, but are close. Then, we need to get tracked input working. Let’s focus on those pieces, which are far more needed for progress than structural changes (e.g. “don’t fix what isn’t broken”).

Well, design-wise, the issue I have with VRApplication is the same one I have with SimpleApplication… it’s doing so much crap for you that you can’t easily override.

In the future, there will be no guiNode field precisely for this reason. (The thing I hate the most about SimpleApplication is all of the protected fields.) You can call app.getGuiNode() but it will be looking up the GuiNodeAppState and calling the getGuiNode() on that. Thus if you were to have a VrGuiNodeAppState that extended that to provide something else then the regular BaseApplication would be automatically picking that up.

So you could have a VrAppState that an app could attach to configure whatever additional stuff it wanted but the app itself would still just be extending BaseApplication like any other regular application.

Personally, I’d rather have a library doing all that crap for me, so I could focus on my game development. I want the VR library to be easy to use first, and easily overwritten second. If someone really wants to get into the nitty gritty of a BaseApplication, they can always skip using the pre-packaged & ready to go VRApplication. However, I suspect most people will want to get right into VR & game development, and not managing GUI AppStates.

Ultimately, this is why I branched my own jME3 & virtual reality library. If you want to break it into smaller pieces, I’ll take the time to make it whole again & easy to use in my libraries.

Yeah, alright… but just to be sure we are talking about the same thing:

Your way:

public class MyGame extends VRApplication {
   public static void main( String... args ) {
       MyGame game = new MyGame();
       game.start();
   }

   public MyGame() {
      super(new SomeGameSpecificAppState(), 
         new SomeOtherGameSpecificAppState());
   }
}

My way:

public class MyGame extends BaseApplication {
   public static void main( String... args ) {
       MyGame game = new MyGame();
       game.start();
   }

   public MyGame() {
      super(new VrAppState(),
         new SomeGameSpecificAppState(), 
         new SomeOtherGameSpecificAppState());
   }
}

The difference is that with the second way, I can easily turn a regular application into a VR app without forcing specialization.

You could structure it that way. There are many ways to solve this problem. In my solution, you have the power to handle VrAppState() addition & initialization automatically, based on hardware detection. Converting regular applications into VR applications would be changing the extends to VRApplication instead of BaseApplication, and virtually nothing else should need modification (like it currently is). Down the road, there might be other complexities that are more easily solved by having control over the application, instead of just an AppState. An update wouldn’t require developers to make changes to their BaseApplication in that case.

However, the primary problem I am seeing now: we are debating structure when structure isn’t our current problem, OpenVR distortion is. This thread has taken an unexpected detour, delaying overall progress of the library. We got off track & I’d love to get back on it.

Sure… but you posted additional arguments so if I let them go without comment then they stand on their own. For some reason the “Here are 900 points about why you are wrong and let’s stop talking about it now” thing always frustrates me. :smile:

The main VrAppState itself could/should be handling this hardware detection and attaching whatever other additional things it needs (or not).

Except if my app already extends a different application base class from some other library that did this.

This is the primary reason why composition should be preferred over inheritance (specialization). In a single inheritance model, “subclassing to gain behavior” automatically rules out a whole bunch of options.

Then JME is broken. That’s what we’re trying to fix. When BaseApplication is (mostly) a dumb-passthrough for app states then there should be no real reason to extend it (even applications wouldn’t have to extend it). You should be able to do everything without extending.

As long as you are forward-thinking about your design and try to encapsulate as much as possible in app states (even if you still have VRApp to set this for the user) then everything is golden.

I have two main reasons for butting my nose in here:

  1. because it’s a great use-case to make sure we are doing the right flexibility changes to the engine to support your various use-cases.
  2. because a bit of discussion might save tons of refactoring later when we go to incorporate this “officially”. Your prime concern is producing games. Our prime concern is producing a long-term useable and supportable engine.
2 Likes

I hear you. What frustrates me, as someone who is trying to do some project management, is trying to tackle one problem at a time. I want to get OpenVR distortion working & restructuring slips in and takes over the discussion. As I said 5 hours ago, “after distortion is working, we can then evaluate changes to the VRapplication and other structures on its own merits.”

2 hours later, I restated “We don’t have OpenVR distortion working yet, but are close. Then, we need to get tracked input working. Let’s focus on those pieces, which are far more needed for progress than structural changes (e.g. “don’t fix what isn’t broken”).”

I’m trying to fix OpenVR distortion, that is our current task for the OpenVR library. Throwing restructuring discussion in the middle is just throwing a wrench in the gears at this point. We don’t know what the library will look like when it works, or exactly what it requires for elegant jME3 integration. Let’s discuss that when the library works & we have time to spare to make things pretty.

I’m putting off further replies about refactoring this library, because this library doesn’t work yet.

Fixing the core functionality of the library should take precedence to restructuring. Right now, the library’s architecture works very good with jME3 & I was able to easily integrate it into 2 commercial products on Steam. We have been complimented on how easy it was to use. However, we need to convert to OpenVR; mixing restructuring & conversion at the same time is messy. OpenVR should work first, with the existing architecture, which we know already works. In my opinion, “fixing it later” rarely happens because it is so often not worth the effort.

To add my 2 cents to the discussion: “fixing it later” is exactly what I’m doing with OpenRTS.

OpenRTS is awesome, and solves a lot of problems that the wiki just leaves as an exercise for the reader. And those problems can be quite hard if you don’t have experience at solving them. In fact, my game does not solve them very well.

The problem is that everyting in OpenRTS is tangled together and a bit hard to dissect into AppStates, but that is what I’m doing, because I want to solve my problems in the “OpenRTS-way”. I think that in the long run I’m saving time by making GoldMonkey & friends.

Develoment is driven by the application’ needs, so I think that even if VRAppState isn’t done from the beginning, it will be done eventually. :smile:

Yes, well… understand that the needs of a game are different than the needs of a framework. Momentum in a framework generally means that when something is added that it’s really hard to remove later. See: SimpleApplication, See: nearly every protected field in JME, etc.

To add my 2 cents too, for the Oculus Rift I refactored your lib so that I don’t have the OVRApplication anymore.
All is handled by app states. It works fine.
The huge advantage for me is that I can start the app in classic mode or VR mode just with and if statement and adding an appstate or not.

1 Like

Really? Did you have any problems in doing so? The reason OVRApplication was created in the first place was that for some reason, the HMD had to be initialized in the static context, before the application thread was started. Otherwise it would crash. I suspected some JNI/JNA threading issue (of which I am certainly no expert).
Maybe the issue has disappeared with later firmware. That’s good news in any case!

Well I guess it has been fixed, because I never had this issue.
But actually, I call the “preconfigureOVRApp” in the appstate constructor. This constructor is called in the MainApplication (that extends SimpleApplication) constructor. And this constructor is called from a static context (in the static void main), so maybe I worked around the issue without knowing it.

Also I didn’t spend a lot of time on the refactor, it was like brute force copy/paste from the app to the AppState, I guess more clever things could be done about it.
But it works fine.

1 Like

I think two things are overlooked sometimes:

  1. app state has ‘construction time’ as one of it’s lifecycle steps (just like any object but easy to overlook when you are thinking of real methods).
  2. app allows passing app states on its constructor.
1 Like

I’m coding VRApplication to handle starting in classic mode or VR mode, based on hardware detection, automatically without any code on the developer’s side. My goal is for virtually everything to be the same between classic & VR development. My VRApplication will handle everything possible in making what you write into a VR & non-VR app.

With that said, I appreciate anyone who takes the time to transform what I’m doing into something easier to incorporate into the main jME3 distribution, whether that is making it just an AppState or something else with a smaller footprint. I understand there are different needs & different architectures may be more fitting for those needs. Whatever I write, I’ll do my best to answer for if problems arise when converting to someone else’s repository.

If you want I’ll clean up what I did, and I’ll make a pull request on your project. You’ll be free to take it or not.
Though it’s been a long time since I pulled your project a lot of things may have changed :stuck_out_tongue:

Not much have changed since last week, my fork is running in an AppState and with regular jMonkeyEngine. In case anyone has those requirements. I have yet to test the GUI.
One problem I have is that once I’ve run the VR application, I can’t run without VR mode or any other non-VR class in the same project. Even other projects are affected. When starting, the application is running (logs), but the screen is all black.
I suspected it having to do with either swapbuffers or Pause on lost focus. Reverting them in settings works for other projects, but not the project having the VR lib.

Make sure “use custom distortion” is disabled from my VR library… I’ve noticed that creates “black” problems when not actually using VR. There also is a “disable swap buffers completely” VR option, make sure that isn’t enabled… also just make sure whatever final scene viewport is being sent to the display (OutputBuffer should be null), and not a texture (that’d be normally sent a compositor).