Oculus Rift Support

toolforger · June 2, 2014, 5:42am

@jherico said: Incorporation and use of Oculus license covered binaries in an engine doesn't impact the engine itself, or Unreal and other engines wouldn't be able to support the Oculus at all. If Oculus decides that JOVR isn't meeting the terms of the license, they will come after me, not jMonkeyEngine, and I will do my best to accommodate their requirements, even if it means jamming the entire libOVR C/C++ source code into the jar file.

You have a point there.
Though OVR might have given the big guys a separate license to do this.

I doubt they intended anything fishy though.
Now that they’re owned by Facebook, policy might have changed. It’s not very likely since Facebook is generally friendly towards open source and is also interested in developer adoption.
Still, it’s better to be safe than sorry. Facebook might sell OVR if it turns out that OVR doesn’t meet their expectations, and in the end it might be owned by Oracle or somebody as lawyer-happy as Oracle, and the new owner might decide to sue neither you nor the JME project but anybody who’s making money off OVR+JME.

I do not think that the probability that anything will happen is not very high.
However, if something does happen, anybody who build his family income on OVR+JME will be bankrupt.
It’s the classic low-probability, high-damage risk. You have a couple of these in every project, but you minimize those that you can - and this risk can be minimized:

My recommendation would be to contact them and ask them about the situation.
Preferrably after asking FSF/OSI/Creative CC for advice how best to proceed.

toolforger · June 2, 2014, 5:45am

@pspeed said: Can point to where in the apache license it says this? Or are you also changing the subject now to the Occulus SDK license?

Sorry, I was in “Oculus Rift” mode and didn’t notice that people had started talking about the details of the Apache license.
AFAIK mixing Apache and MIT licensed code is okay. I’ve never heard to the contrary, and my reading of the licenses does not indicate any possible problem.

jherico · June 2, 2014, 5:31pm

@rickard said: Played around a bit yesterday, but i can't get around the IllegalMemoryAccess error. Have to wait and see if jherico can give any info on it.

If you can show me the code, or come up with a minimum reproducible version, I’ll see if I can diagnose the issue.

When I have time I’m hoping to embellish the Java code to do a little validation of the input values to ensure that any errors come from the Java side with a reasonable description of what went wrong, as opposed to the black-box error you get out of a crash in the DLL.

rickard · June 5, 2014, 1:01pm

@jherico said: If you can show me the code, or come up with a minimum reproducible version, I'll see if I can diagnose the issue.
When I have time I’m hoping to embellish the Java code to do a little validation of the input values to ensure that any errors come from the Java side with a reasonable description of what went wrong, as opposed to the black-box error you get out of a crash in the DLL.

Hi.

I’m a bit hampered atm since a lightning took out my screen, modem and graphics card a couple of days ago. I’ll try to get to the code on the other machine.

jherico · June 7, 2014, 8:35am

@rickard said: I'm a bit hampered atm since a lightning took out my screen, modem and graphics card a couple of days ago. I'll try to get to the code on the other machine.

Well, let me know if I can help. Just point me at a repository.

I’ve released a new version of the jar: 0.3.2.3. It changes the name of the core classes to be a little more Java like, but it also adds a bunch of validation to the passed functions to try to avoid some of the crash situations.

There will be a 0.3.2.4 soon which won’t change nearly as much as .3, but will address some bugs in the Oculus SDK.

rickard · June 8, 2014, 3:20pm

Here’s what the initialization code looks like:

OculusRift.java:
[java]
ovrLib = OvrLibrary.INSTANCE;
ovrLib.ovr_Initialize();

    loadedHmd = ovrHmd.createDebug(OvrLibrary.ovrHmdType.ovrHmd_DK1);
    
    initHMDSuccess = loadedHmd != null;
    info.createFakeValues();
    if( initHMDSuccess ){
        updateHMDInfo();
        System.out.println(info);
        loadedHmd.startSensor(OvrLibrary.ovrSensorCaps.ovrSensorCap_Orientation | OvrLibrary.ovrSensorCaps.ovrSensorCap_YawCorrection |
                              OvrLibrary.ovrSensorCaps.ovrSensorCap_Position,
                              OvrLibrary.ovrSensorCaps.ovrSensorCap_Orientation);

        OculusRiftUtil.configureRendering(loadedHmd, loadedHmd.getDesc());[/java]

OculusRiftUtil.configureRendering
[java]
EyeRenderDesc eyeRenderDescs[] = (EyeRenderDesc[]) new EyeRenderDesc().toArray(2);
FovPort fovPorts[] = (FovPort[])new FovPort().toArray(2);
for (int eye = 0; eye < 2; ++eye) {
{
// JNA weirdness 1
FovPort defaultEyeFov = hmdDesc.DefaultEyeFov[eye];
fovPorts[eye] = defaultEyeFov;
FovPort.ByValue fovPort = new FovPort.ByValue();
fovPort.DownTan = defaultEyeFov.DownTan;
fovPort.UpTan = defaultEyeFov.UpTan;
fovPort.LeftTan = defaultEyeFov.LeftTan;
fovPort.RightTan = defaultEyeFov.RightTan;
// projections[eye] = RiftUtils.toMatrix4f(
// OvrLibrary.INSTANCE.ovrMatrix4f_Projection(
// fovPort, 0.1f, 10000f, (byte) 1));

// TextureHeader eth = eyeTextures[eye].Header;
// eth.TextureSize = hmd.getFovTextureSize(eye, fovPort, 1.0f);
// eth.RenderViewport.Size = eth.TextureSize;
// eth.RenderViewport.Pos = new OvrVector2i(0, 0);
// eth.API = OvrLibrary.ovrRenderAPIType.ovrRenderAPI_OpenGL;

// frameBuffers[eye] = new FrameBuffer(eth.TextureSize.w, eth.TextureSize.h);
// eyeTextures[eye].TextureId = frameBuffers[eye].getTexture().id;
}
}

RenderAPIConfig rc = new RenderAPIConfig();
rc.Header.API = OvrLibrary.ovrRenderAPIType.ovrRenderAPI_OpenGL;
rc.Header.RTSize = hmdDesc.Resolution;
rc.Header.Multisample = 1;
for (int i = 0; i < rc.PlatformData.length; ++i) {
  rc.PlatformData[i] = 0;
}
rc.write();
int distortionCaps = 0
    | OvrLibrary.ovrDistortionCaps.ovrDistortionCap_Chromatic
    | OvrLibrary.ovrDistortionCaps.ovrDistortionCap_TimeWarp
    | OvrLibrary.ovrDistortionCaps.ovrDistortionCap_Vignette
    | OvrLibrary.ovrDistortionCaps.ovrDistortionCap_NoSwapBuffers
    ;

byte configureResult = hmd.configureRendering(
rc, distortionCaps, fovPorts, eyeRenderDescs);[/java]

I think this is as per your example application. In this case the exception is thrown when calling configureRendering.
At first I did not try to call configureRendering then it crashed when calling:
pose = hmd.beginEyeRender(eye);
or
hmd.endEyeRender(eye, pose, eyeTexture);
or
hmd.endFrame();

Any ideas?

yombo · June 8, 2014, 5:05pm

I had the same problem, and solved it by calling configureRendering() in the rendering thread, as the begin…() and end…() functions.

Hope it helps.

rickard · June 8, 2014, 8:32pm

@yombo said: I had the same problem, and solved it by calling configureRendering() in the rendering thread, as the begin...() and end...() functions.
Hope it helps.

I think it does.
I guess the filter calls are on the render thread, but the initialization is not.
I’ll check it out, thanks.

jherico · June 12, 2014, 2:25am

It’s possible that it’s threading, or order of operations. The configureRendering method should only be called after an OpenGL context has been created and is currently active, as internally it makes a lot of OpenGL calls.

phr00t · June 16, 2014, 5:20pm

I updated jMonkeyEngine’s Oculus library (we need a name for this library, how about MonkeyRift?) to support JOVR 0.3.2.4 (the latest). @rickard, can you test to make sure everything works as expected?

Also, any progress with the SDK-side distortion filter, using jherico’s suggestions?

normen · June 16, 2014, 5:26pm

Anyone up to making this a plugin? The audio/video recorder one even supports extracting natives, maybe you can have a look at that.

rickard · June 16, 2014, 8:11pm

@phr00t said: Also, any progress with the SDK-side distortion filter, using jherico's suggestions?

Sorry, no progress. I simply don’t have any time until in a couple of weeks (and the network card in the computer had broken down too). Then on the other hand, I should have ample.

@normen: I actually set it up as a plugin at one point, so I should be able to do it again. I agree that’s probably the most suitable usage for it. Just have to make it stable, first.

rickard · June 19, 2014, 2:01pm

Found some time to work on the Oculus this afternoon, but ran into an old problem.

It was indeed like yombo suggested. The OculusRift was initialized outside of the application, like it was in the old architecture:
[java]public static void main(String[] args) {
OculusRift.initialize();
App app = new App();
app.start();
}[/java]

Changing it to inside the application instead fixed the illegalmemoryaccess exception.

HOWEVER

Doing so, the OVR doesn’t return any information post the hmdInfo. No pose data is available.
This reminds me of why i placed the initialization outside of the application. With my old JNI code, it had to be there for the OVR to be initialized, at all. So this is sort of one step further, at least…

I’ll continue with this, just wanted to give an update.

Edit: Managed to get the sensors working with some hacking, getting the latest JOVR and apparently also the guava library is needed now.

rickard · June 19, 2014, 5:44pm

@phr00t (and anyone else interested)

Major breakthrough! There is SDK distortion.

Now i have to go and pick up some food before i starve. I’ll see if i can find some more time to work on this tonight.

yombo · June 19, 2014, 5:53pm

@rickard said: @phr00t (and anyone else interested)
Major breakthrough! There is SDK distortion.

Now i have to go and pick up some food before i starve. I’ll see if i can find some more time to work on this tonight.

I’m glad it works!

phr00t · June 19, 2014, 5:57pm

@rickard said: @phr00t (and anyone else interested)
Major breakthrough! There is SDK distortion.

Now i have to go and pick up some food before i starve. I’ll see if i can find some more time to work on this tonight.

NICE! I did check in JOVR 0.3.2.4 into the SVN, which had very minor changes. You should be able to just merge that with your own. Now that we have SDK distortion, we need to remove the previous distortion routines – or have you done that already?

rickard · June 19, 2014, 9:40pm

@phr00t said: NICE! I did check in JOVR 0.3.2.4 into the SVN, which had very minor changes. You should be able to just merge that with your own. Now that we have SDK distortion, we need to remove the previous distortion routines -- or have you done that already?

There’s still some work left. It’s not working entirely as it should yet, but it’s rendering. I’m also going to see if we can use the pose directly, and dump the CamControl, should lower the latency a tad, perhaps.

rickard · June 20, 2014, 10:17am

It’s working decently now. I didn’t remove the StereoCamControl since it’s still needed if we want to do a combination control of rift + mouse for example.
New appstate instead of StereoCamAppState (i’ll deprecate it), OVRAppState and new filters instead of BarrelDistortionFilter, OculusFilter.
There’s still more work to do on this, but the basics are working. Unconfirmed if it’s actually any 3d effect, right now, but I don’t see why there shouldn’t be.
Really unconvenient, but it seems you must initialize the hmd and start the sensors outside of the application.

Changelist:

Initial commit for SDK distortion and rendering
Rendering works in 1280x800 for now.
See TestSkyBox and TestStereoCam for examples.
Use OVRAppState instead of StereoCammAppState
Must call OculusRift.initSensors from a static context outside of the application thread.

phr00t · June 21, 2014, 1:31am

OK, I put some work into this. It looks like you made OVRAppState from an old version of StereoCamAppState, since many of my improvements to it, like GUI distance setting, wasn’t included. So, I merged your changes with the updated StereoCamAppState, so that state now has all the goodies. TestStereoCams uses StereoCamAppState again (we can rename/merge it to OVRAppState when things get settled).

I also made loadedHmd get set to a debug HMD if no Rift is present (as is in my situation). I also added guava-libraries to the project, since apparently that is now needed for some “Preconditions” methods in Rift initialization. I also consolidated initSensors and initialize into one method.

Unfortunately, I get a native crash in this line, in OculusRiftUtil.java:

[java]
configureResult = hmd.configureRendering(rc, distortionCaps, fovPorts);
[/java]

Here is the complete stack trace:

[java]
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [libc.so.6+0xa20bd] envz_strip+0x989d
C [jna2797404871268425972.tmp+0xc3d1f] OVR::CAPI::GL::DistortionRenderer::Initialize(ovrRenderAPIConfig_ const*, unsigned int)+0x2b
C [jna2797404871268425972.tmp+0xba7ed] OVR::CAPI::HMDState::ConfigureRendering(ovrEyeRenderDesc_, ovrFovPort_ const, ovrRenderAPIConfig_ const*,
unsigned int)+0x2d3
C [jna2797404871268425972.tmp+0xbc29e] ovrHmd_ConfigureRendering+0x47
C [jna8293897992565732083.tmp+0x127a0] ffi_call_unix64+0x4c

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j com.sun.jna.Native.invokeInt(JI[Ljava/lang/Object;)I+0
j com.sun.jna.Function.invoke([Ljava/lang/Object;Ljava/lang/Class;Z)Ljava/lang/Object;+108
j com.sun.jna.Function.invoke(Ljava/lang/Class;[Ljava/lang/Object;Ljava/util/Map;)Ljava/lang/Object;+262
j com.sun.jna.Library$Handler.invoke(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;)Ljava/lang/Object;+316
j com.sun.proxy.$Proxy0.ovrHmd_ConfigureRendering(Lcom/oculusvr/capi/Hmd;Lcom/oculusvr/capi/RenderAPIConfig;I[Lcom/oculusvr/capi/FovPort;[Lcom/oculu
svr/capi/EyeRenderDesc;)B+37
j com.oculusvr.capi.Hmd.configureRendering(Lcom/oculusvr/capi/RenderAPIConfig;I[Lcom/oculusvr/capi/FovPort;)[Lcom/oculusvr/capi/EyeRenderDesc;+79
j oculusvr.util.OculusRiftUtil.configureRendering(Lcom/oculusvr/capi/Hmd;Lcom/oculusvr/capi/HmdDesc;)[Lcom/oculusvr/capi/EyeRenderDesc;+180
j oculusvr.input.OculusRift.initialize()V+99
j oculusvr.TestStereoCams.main([Ljava/lang/String;)V+0
v ~StubRoutines::call_stub
[/java]

I’m wondering if this is just because I’m trying to use a debug HMD instance. I checked in my changes to the SVN – @rickard, could you see if you get this same crash, or does it “just work”?

EDIT: If this “just works” for you, I wonder if SDK-side distortion is possible without a HMD? If not, may still have to use the BarrelDistortion filter (or just no filter at all – just side-by-side).

yombo · June 21, 2014, 1:47am

@phr00t said:

SDK distortion and rendering works without a Rift, it is also my case