OpenVR Available, Convert?

david_bernard_31 · May 3, 2015, 5:02pm

Sorry, VR is low in my TODO list. But @jherico seems to continue to work on it.

phr00t · May 3, 2015, 6:56pm

I ran OpenVR through JNAerator, targeting BridJ (which supposedly is faster & has better C++ support), creating a Java API:

https://github.com/phr00t/jmonkeyengine-virtual-reality/tree/master/src/openvr_api

I went a bit farther than jherico – I made sure to embed all of the OpenVR libraries, so this should already start working for all operating systems:

https://github.com/phr00t/jmonkeyengine-virtual-reality/tree/master/src/lib

I added a new VRHMD object for OpenVR to the jMonkeyEngine Virtual Reality library:

github.com

phr00t/jMonkeyVR/blob/master/src/jmevr/input/OpenVR.java

/*
 * To change this license header, choose License Headers in Project Properties.
 * To change this template file, choose Tools | Templates
 * and open the template in the editor.
 */
package jmevr.input;

import com.jme3.math.Matrix4f;
import com.jme3.math.Quaternion;
import com.jme3.math.Vector2f;
import com.jme3.math.Vector3f;
import com.jme3.renderer.Camera;
import com.sun.jna.Memory;
import com.sun.jna.Pointer;
import java.nio.FloatBuffer;
import java.nio.IntBuffer;
import java.nio.LongBuffer;
import java.util.Locale;
import java.util.concurrent.TimeUnit;
import jmevr.app.VRApplication;

This file has been truncated. show original

I already tested the VR_Initialize() function, which currently returns 110. 110 means the OpenVR runtime isn’t installed (which it isn’t on my machine, yet). Good news is – that is an expected result! Looks like we will get to integrating this sooner than expected. Next step will be to install the OpenVR runtime & start figuring out how to integrate rendering. I forsee the traditional Oculus Rift VRHMD object become obsolete and deprecated.

phr00t · May 5, 2015, 3:04pm

After installing SteamVR (start the Steam client, go to Library → Tools, right click SteamVR & go to Install) & running the library, I now get this error:

HmdError_Init_HmdNotFound (108) - Either no HMD was attached to the system or the HMD could not be initialized.

… which is also expected, since I don’t have one plugged in! My Rift ironically isn’t hooked up to my primary development machine, so I’m hoping there is a way we can test/implement much of the functionality without an actual HMD connected (like we were able to do with the Rift).

phr00t · May 5, 2015, 4:04pm

Small update to display any errors in English during initialization:

https://github.com/phr00t/jmonkeyengine-virtual-reality/commit/2db6467c269d3151f021ddc8e49b595ca57aa0d4

@rickard, do you want to look this library over & perhaps start contributing rendering integration? It looks like this is what we need to call to get pose information:

Dealing with Pointers in Java with BridJ is a little funky, but it seems to be working. It might get a bit more complicated with more complex object types, though.

rickard · May 6, 2015, 5:51am

I’ll try to, this weekend.
Most likely I’ll use your classes and try to build a minimum viable test based on the official example to confirm that my expectations on how it works are correct. Then we can see how to get it into the current architecture.
From skimming through the example, it seems it’s much more direct than the oculus implementation, more akin to how cardboard works (supplying the matrix transforms for each eye directly).

phr00t · May 6, 2015, 1:41pm

OK, sounds good. I’d much prefer you handle this, since you have more experience in this area. Hopefully most of the code can go inside the OpenVR.java implemented interface. However, I’m sure there are places among the library where changes will be needed too (like where you see “VRhardware instanceof OculusRift” will probably need corresponding “VRhardware instanceof OpenVR” sections). Ultimately, we are probably going to remove the Oculus Rift stuff (since it is already obsolete, and going to be even more obsolete when the big v0.6 SDK changes come).

phr00t · May 7, 2015, 2:10pm

Status update, which will be important for you @rickard:

I cleaned up the BridJ interface so it is easier to read & understand. The “Pointer” interface is much easier to use if you don’t specify what kind of pointer it is (e.g. just leave “Pointer” instead of “Pointer{IVRSystem}”).

I also noticed there might be some instances where BridJ thought an argument was a Pointer, but isn’t. vRGetStringForHmdError, for example, wants just the long value – not a pointer to a long. I updated the Openvr_apiLibrary.java as seen in this commit (near the bottom):

https://github.com/phr00t/jmonkeyengine-virtual-reality/commit/ba0877400ccd695805a49b11d7285be63cc58263

I have a strong feeling functions like GetFloatTrackedDeviceProperty’s hmdDeviceIndex argument will work the same, which if so, still needs to be fixed.

EDIT: another quick edit, since hmdDeviceIndex seems to be an int, not a long. Still need to determine if its looking for a pointer to an int, or just the int value. It appears it wants just the int, but BridJ auto-generated a setup for a pointer to an int… have to test.

rickard · May 9, 2015, 3:42pm

@phr00t
Progress is pretty slow as I’m getting to grips with the example application and the library.
I’m currently working on mapping the UpdateHMDMatrixPose() method from the example. Just so we don’t do any duplicate work. I’ll continue from there.

rickard · May 10, 2015, 1:19pm

I’ve checked in my progress from this weekend.
Basically there are methods for getting various pose transforms from the HMD, but not put into any context yet. It’s 100% not tested.
Btw, it’s easier to intialize the Pointers, but there’s a whole lot of casting once you want to use them for something.

phr00t · May 12, 2015, 4:12pm

Great, thank you for the help. I haven’t had a chance to really look it over yet – been working on a Spermination update The SteamVR team has some instructions on testing with a Null Driver here:

http://steamcommunity.com/app/250820/discussions/0/613956964603712981/#c613956964604147408

I haven’t gotten those instructions to work yet, but I admit I need to spend a little more time trying. We need to figure out how much needs to be filled out before we get something worth testing against a Rift (printing out pose information would be a great step). I’ll be able to look more at this in the next few days.

rickard · May 12, 2015, 5:23pm

I think the basic stuff is there for reading out poses. That should be enough for testing against with a StereoCameraControl. It seems the preferred way of rendering is to use the supplied viewports and projection matrices. This can be added once the basics are confirmed. I can look into that over the coming weeks, but this weekend is packed.

phr00t · May 12, 2015, 6:39pm

OK, checked in some more progress:

https://github.com/phr00t/jmonkeyengine-virtual-reality/commit/edb2898c7cf32dd295543cff36ffe71a72c0634d

Flagged Oculus Rift SDK-specific stuff deprecated. There might be spots where Rift structures are used, like OvrSizei, that need to be replaced. I changed one to Vector2f.

Is it possible to dump HMDInfo? It seems like the only variable used is pupillary distance. Perhaps a few more variables will be needed for the shader filter, though? I’d like to remove code & structure excess where possible.

The TestVRApplication project now runs up to the updatePose() function. Unfortunately, it crashes because I’m still getting “No HMD present (108)” when trying to initialize. I have no HMD hooked up, so this error is mostly expected, although I’m trying to use the “Null Driver” method I posted about above. Asking for help on that here:

Perhaps hooking it up to a Rift DK1 or DK2 would get us some results.

phr00t · May 12, 2015, 6:55pm

Another quick update – getting the projection matrix & texture render size:

https://github.com/phr00t/jmonkeyengine-virtual-reality/commit/45b1c18a2420db8ee911ba0396ddebc2307aaff7

rickard · May 12, 2015, 7:06pm

I left a TODO when initializing the composer. I couldn’t find how to get hold of the version number there. Without it being initialized, it’s not going to work.

phr00t · May 12, 2015, 7:28pm

This should do it:

https://github.com/phr00t/jmonkeyengine-virtual-reality/commit/858ab4c291b82e40220f90d4a866092fd71e9b61

phr00t · May 12, 2015, 7:41pm

Another quick update – I don’t think you need to allocate a long for the compositor. The VRGetGenericInterface probably initializes the memory needed for whatever interface is being returned. I made the change here:

https://github.com/phr00t/jmonkeyengine-virtual-reality/commit/dce02efa5ff65b6147b05918083381cbcb6376f7

phr00t · May 14, 2015, 3:19pm

From Valve:

“The compositor still needs some work before it will be fully operational on both linux and osx, but you can use the sdk interfaces to do your own distortion correction, etc. in the meantime.”

Still trying to get the debug environment working to get past the “Hmd not found (108)” error without an HMD plugged in:

Made a little progress, but it looks like some arguments might be broken on the vrserver tool. Awaiting Valve’s response… anyone with a Rift & Windows is free to try the library sooner, before I get to it though!

Momoko_Fan · May 14, 2015, 3:47pm

By the way, there’s OSVR SDK coming out in July 2015:
http://www.osvr.com/

This one is actually open source (Apache 2.0 license), unlike OpenVR which is a proprietary SDK and not open at all.
This means we can actually integrate OSVR with jMonkeyEngine without concerns over license.

They still haven’t provided the source code yet though, will be waiting patiently…

phr00t · May 14, 2015, 3:58pm

Thank you for the heads-up. We’ll see how the project develops.

My gut is to stick with OpenVR, though. Valve is obviously a huge player, already incorporating OpenVR into the largest digital distribution platform as SteamVR. The Vive is currently the most promising piece of hardware, which OpenVR (and SteamVR) is the primary driver for. People already have Oculus Rifts, which is being used against OpenVR already.

OSVR looks promising, but OpenVR is larger & has a head start at accomplishing unification of VR hardware. The open-source angle is nice, but doesn’t trump other factors (in my humble opinion).

rickard · May 16, 2015, 11:22am

What worries me about the OSVR is that it doesn’t seem to have any (positional) head-tracking, which simply is a must for any decent VR experience (it’s extendable, I know).
I also “registered my interest” many months ago, but I have yet to receive any newsletter from them.

That said, provided there is some way of getting positional tracking I’ll get one. I’ll probably sit around waiting for someone to make a java port, though.