OpenVR Available, Convert?

I’d like to add: let’s try to fix one thing at a time. I don’t believe we should be refactoring the whole library’s structure at the same time as getting OpenVR distortion working. We might introduce unexpected bugs that would distract us from making OpenVR progress if we try to do two things at once. After distortion is working, we can then evaluate changes to the VRapplication and other structures on its own merits. We know the current setup worked nicely with the Rift, so lets just swap our what is necessary to get OpenVR working first.

No dice with the chaperone file. I’ll give them some more time to answer.

What makes me think it’s not the null driver is that the display that is shown when launching Steam VR seems to have distortion (the out of bounds areas are not square).

My approach to this is simply KISS. I want the simplest possible solution without any unnecessary overhead. Once a test application is running one can ponder about general changes. But without the simplest possible solution it’s much more difficult to assess what is the least common denominator.
The OpenVR API (so far) is much more direct than the Oculus Rift drivers so I see no reason to add more complexity to it just for the sake of sticking to old conventions. We also have the matter of controllers and base stations to consider, which is not at all covered in the existing project.

But it’s still to early to say what will be required in the end. I want to see some hardware working. I tried with my DK1 but it wasn’t detected.

Edit: There is no refactoring done at all up to this point. Just an AppState handling everything.

I’ve commited the WIP. Tell me if you have any luck with hooking up your OVR to this.

2 Likes

I moved 14 posts to a new topic: Virtual Reality library architecture discussion

Thank you for the commit. I’ll have to boot into Windows to play with it.

I’m a big fan of KISS too, but don’t forget to apply KISS in the eyes of our end users too. Making a piece of the library “simple” might be offloading complexity to the VR developer, which isn’t good. If we can get rid of filters and streamline the library due to OpenVR being more direct – great, but I don’t want it to be at the cost of ease of use for developers. VRapplication & guiNode management was a simplification of VR development, irrespective of what SDK we use, in my opinion.

I did put in some VR input code into VRapplication, as seen here:

… using the VRInput object, meant to abstract out whatever input is available:

https://github.com/phr00t/jmonkeyengine-virtual-reality/blob/master/src/jmevr/input/VRInput.java

What SDK version do you have? It sounded like v0.4.4 SDK was the most compatible with SteamVR, while the latest is v0.6.0.1. I’ll have to plug in my DK2 to test.

The problem with “we can always go back and fix it later”, is that it rarely happens. As you expand the library, the temporary functions get more entangled and difficult to weed out.
Therefor I think that debating general issues about the library is important and beneficial for it. It helps us make better decisions while developing it further. Especially with this spare-time and online development, it’s the best forum we have.

I think the issue we’re facing here is that everything is kept in the same thread. This discussion should be split into several threads so that information that may be crucial for the issue currently at hand is not obstructed by more general discussions. That way the devs can also choose what discussion to involve themselves in and at their leisure.

I’ll see if I can restructure the last posts of this thread into a new one.

Edit: Discussion now continued here: Virtual Reality library architecture discussion (continued from OpenVR Available. Convert?)

Thank you for moving that section elsewhere.

I’m looking over the commit now.

This is precisely why I don’t want to mix restructure changes with OpenVR conversion:

SEVERE: Uncaught exception thrown in Thread[jME3 Main,5,main]
java.lang.NullPointerException
	at jmevr.util.VRGuiNode.attachChild(VRGuiNode.java:137)
	at jmevr.TestOpenVR.initTestScene(TestOpenVR.java:76)
	at jmevr.TestOpenVR.simpleInitApp(TestOpenVR.java:53)
	at com.jme3.app.SimpleApplication.initialize(SimpleApplication.java:229)
	at com.jme3.system.lwjgl.LwjglAbstractDisplay.initInThread(LwjglAbstractDisplay.java:131)
	at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:212)
	at java.lang.Thread.run(Thread.java:745)

VRGuiNode is trying to grab the VR hardware from VRapplication, but you unexpectedly side-stepped VRapplication to extend SimpleApplication in the new TestOpenVR. Now I have to muddle through stuff like this to get back to distortion correction.

Or you just skip adding the gui node. It doesn’t do anything. in TestOpenVR.

Well, the crosshair is a small test of the GUI stuff, which was in TestVRApplication. It should still be part of the test & not be skipped. Getting distortion working shouldn’t have to affect that piece (unless, of course, you are trying to make restructural changes at the same time). I have to head out for a bit, but I hope to get back to this later tonight. I’m trying to find compromise between VRApplication & the changes you made, so we can get back on the original track. I notice you are using fixed 512x512 frame buffer sizes, and not the OpenVR texture determination size available in “prepareCameraResolution”, part of the original VRAppState. I can get that fixed too. Together, we should be able to make a jump in progress here & I just want to make sure among all this debate, I really appreciate your help!

Well, it is WIP. I just wanted to check it in so that you had something to look at besides this thread. The 512 are coming straight from the jme test of offscreen rendering. I think it’s a big advantage that we can set a custom size and not be restricted to the actual camera (like we were originally in the oculus filters)

Edit: I actually don’t mind working on these bits on my own until it works. That may make things easier. I generally just want some advice for now.

Priority one should be to get distortion working, we can worry about architecture later :wink:

Seriously. I think we can look at general abstraction once OpenVR is working. The current VRAppState expects too much to be useful as a generic appstate.

The longer we can keep OpenVR separate, the easier it will be to make it work.

Sorry for spamming the thread. But to get this thing on track:
I’m pretty sure that the java code is behaving OK, what data is coming is displayed correctly. There are printouts of the TexCoords coming from the JNA part and they’re all uniformly distributed. I think this is where the issue is. Do you have any control/oversight over what’s passed there? If it’s coming unmodified from the c+±core, I think we need to look outside of our part.
If you can try with a real HMD that would be a great start, I think.

[quote=“phr00t, post:95, topic:32413”]
Together, we should be able to make a jump in progress here & I just want to make sure among all this debate, I really appreciate your help![/quote]

We both want to see this work. You’ve done a tremendous amount of ground work on this plugin. Getting to where we are with the rendering in such a short time is much due to that. I basically just had to hook things up.

I’m still seeing just black on my end. Not sure why. Did you make any changes to shaders or something else that didn’t get checked in? It is getting late & I need to head to bed, so I can’t really look too much deeper into it now. I first was trying to fit things back into VRApplication just to get it to run without crashing, but I see you are initializing the hardware & compositor in the AppState, so the two need some more reconciliation than I was hoping for. It is possible I broke something when trying to merge things. It is a headache trying to work on just distortion, when you wrote initialization all over again in different classes…

I’ll look more into it tomorrow… feel free to tinker with what code you have. When either one of us gets something working, we’ll have to merge our methods (which might be a little messy now).

The reason you’re having problems is because I made a partial commit due to not wanting to break backwards-compatibility.
I just made a commit to mitigate this by removing the VRGuiNode dependency.

If you have a desperate urge to merge this WIP test case with the existing architecture before distortion is working, that is your headache. I’d much rather see what is required to get OpenVR to work at all with jMonkeyEngine before cramming additional features in there. But I’ve made that clear before.

Now that there is a working test case committed (and let me know if it isn’t working, ie TestOpenVR doesn’t display the image), let me know if you want to discuss how to get distortion working. Otherwise I’m taking a break from this. It’s already taking too much energy that I rather spend on constructive things.

I wasn’t expecting a partial commit, VRGuiNode being broken or a rewrite of initialization in a whole new class structure. We are taking a step forward in distortion progress, and 2 steps back elsewhere. I’d love to just work on distortion, but now I have to be concerned with restoring lost functionality & reconciling the library’s structure. It isn’t about cramming additional features in, it is maintaining existing features that should be modular and separate from the distortion piece.

As the library grows, we can’t create separate test cases for each new feature we want to implement. It is a headache to tie it back into the main project. We should have worked on distortion as a modularized part of the existing library.

I’m going to start some of that work now, so we can get back to an integrated distortion solution.

I moved your test case back into the existing library & revived the VRGuiNode stuff. I tried to keep everything you’ve done as whole, only moving out hardware & compositor initialization back into VRApplication. I moved the “observer” code out of the OpenVRAppState initialization, just incase it isn’t set yet (to avoid a null crash). I updated both test cases. Unfortunately, I don’t know why I’m still only seeing a black screen. I tried making the distortion meshes be rendered as solid white, and I can’t see them. I’m not sure what in the move is causing the problem… I’m using your OpenVRAppState & am not using the StereoCameraControl, as you did in your test case. You can see the commit here:

Hmm. I synced your changes and TestOpenVR worked for me.
I’m not using your jme build though so I had to comment out the start method of VRApplication. But I doubt that’s it. On the other hand, you could try removing the setSwapBuffer calls. I don’t think it’s necessary for OpenVR.

Edit: I also looked through the commit. I didn’t spot anything.

That was it! It was the setSwapBuffer call. Removed it, and I’m getting the two views. Thanks!

OK, big update:

New things working: VRGuiNode, FOV, camera projection, frustrum culling and camera resolution based on OpenVR requirements.

I moved some things out of OpenVRAppState & into VRApplication. I renamed the AppState to OpenVRCamControl (since that is all it is doing now). I plan on hiding the AppState from the developer & managing it from VRApplication for simplicity. Developers will only need to make their Application a VRApplication and set things like VRApplication.setObserver(player). I also want to avoid any problems with changes away from SimpleApplication to BaseApplication, as VRApplication can accommodate that. You can see how simple this can be by the improvements to TestOpenVR.java

No need to create a guiNode or AppState yourself.

@rickard & I had an offline discussion, and he may fork this library because he may want to approach the same problem from a different angle with a different structure. At the very least, we will continue to work together on the tricky bits.

About distortion: it is hard to tell if we are seeing none because we have no Rift hooked up (and are using a “null driver”), or if it because we are doing something wrong in the code. I plan to hook my Rift up as soon as possible to see if any other behavior happens.

I still need to get post processing filters cloned to both views, something that worked previously.

I have an oddity with the GUI when it is placed in a specific spot (which I think is around [0,0,0]). It appears as if the crosshair is somehow being rendered from within the distortion scene (that should only have the two distortion meshes). I checked the main viewport, and it only has the scene attached with the two distortion geometries… so how is the crosshair drawing over them both? Screenshot: