OSVR HDK 1.3 running on JME3 and jMonkeyVR: Setup guide

User Code & Projects virtual-reality
1

Written: 2016-03-21
Note: I’m only allowed two links, so all links are in raw text format. Sorry, I can’t put in any screen shots either (the JME forum software won’t let me because this is my first post).
Reddit cross posting (better formatting)

2016-03-24: -->Please refer to the above Reddit posting, for the most recent version. The version under here is essentially a stale copy.

Introduction

I recently managed to get an OSVR HDK 1.3 HMD working with jMonkeyEngine and jMonkeyVR, and I thought some of you might be interested in some of the things I learned along the way, as well as providing a reference for people attempting the same thing in the future.

Like with a lot of VR things, and particularly with OSVR it seems, there’s a dearth of documentation out there, especially when it comes to debugging technical issues. So hopefully this tutorial will help fill in that gap some, and save you some frustration if you’re also attempting to get OSVR working with jMonkey.

Before we get started

Just so you know where I’m coming from, before this project I had some experience with OSVR, having used it with Unity 5, Unreal 4, and WebVR, but I had almost no experience with Steam, SteamVR/OpenVR, jMonkey, or jMonkeyVR in and of themselves and certainly no experience in interfacing them with OSVR.

My system is an HP Envy DV7 laptop, Windows 10, gpu Nvidia 650M, driver 364.51, Java 1.8.0_73, OSVR-Core-Snapshot-v0.6-1074-g11272ab-build243-vs12-64bit, jMonkeyEngine SDK 3.1-alpha, jMonkeyVr github checkout on 2016-03-15.

This thread](jmonkeyvr not working with osvr openvr emulator · Issue #2 · phr00t/jMonkeyVR · GitHub) from the jMonkeyVR github](GitHub - phr00t/jMonkeyVR: Easy, free & universal VR development solution: jMonkeyEngine + OpenVR/OSVR = develop on any operating system, for any operating system, for any VR device (Rift, Vive, FOVE, StarVR, OSVR etc.). Supports SteamVR & rendering via instancing, which can greatly improve performance!) is the inspiration and source for this write-up. You may find this thread to be of some use if you’re looking for more detail.

I assume you already have OSVR](https://osvr.github.io/) installed, know how to start the osvr_server.exe, and can get an OSVR application like Unity Palace demo](GitHub - OSVR/OSVR-Unity-Palace-Demo: A simple example OSVR Unity 5.6 project.) to run in full VR-mode. I also assume you have the jMonkey Engine](http://jmonkeyengine.org/), Steam](http://store.steampowered.com/about/), and Steam VR](How to try SteamVR :: SteamVR General Discussions) installed.

I do not assume you have jMonkeyVR ](GitHub - phr00t/jMonkeyVR: Easy, free & universal VR development solution: jMonkeyEngine + OpenVR/OSVR = develop on any operating system, for any operating system, for any VR device (Rift, Vive, FOVE, StarVR, OSVR etc.). Supports SteamVR & rendering via instancing, which can greatly improve performance!) installed.

This is just what I did to get it working on my system. I am not saying this is the only way, or even the best way. If you have a better way, please post a comment. In my experience it was a far from trivial experience, with a lot of random futzing involved.

Setup

  1. Make sure you have the OSVR Tracker View App](http://access.osvr.com/binary/osvr-tracker-view) installed.

While technically not a requirement, this is your go-to tool for making sure your OSVR HMD is working properly. You will need to invoke it a lot if you’re attempting to debug any OSVR problem.

  1. Install the OSVR SteamVR driver](GitHub - OSVR/SteamVR-OSVR: An OSVR plugin for SteamVR, providing SteamVR support for OSVR HMDs.). The github repo](GitHub - OSVR/SteamVR-OSVR: An OSVR plugin for SteamVR, providing SteamVR support for OSVR HMDs.) has some good instructions about how to get Steam VR installed, but I didn’t build from the source, but downloaded the binary](Version SteamVR-OSVR-Win/v0.1-86-g00093fb-core-v0.6-1159-ga309a49 - osvr) instead.

Basically, all this does is install an OSVR driver into your SteamVR directory

!picture osvr_driver_screen_shot
img src="/uploads/default/original/2X/f/f9a8919c4e988bf079c72ff5a4a763c15d2acb96.png" width=“690” height=“347”>

  1. Start osvr_server.exe, if not already started:

    vt5491@hp_laptop_envy /c/Program Files (x86)/OSVR/OSVR-Core-Snapshot-v0.6-1074-g11272ab-build243-vs12-64bit/bin
    $ ./osvr_server.exe

    [OSVR Server] Using default config file - pass a filename on the command line to use a different one.
    [OSVR Server] Using config file ‘osvr_server_config.json’
    [OSVR Server] Constructing server as configured…
    [OSVR] Adding search path “C:\Program Files (x86)\OSVR\OSVR-Core-Snapshot-v0.6-1074-g11272ab-build243-vs12-64bit\bin\osvr-plugins-0”
    [OSVR] Adding search path “C:\Program Files (x86)\OSVR\OSVR-Core-Snapshot-v0.6-1074-g11272ab-build243-vs12-64bit\bin/osvr-plugins-0\RelWithDebInfo”
    [OSVR] Adding search path “C:\Program Files (x86)\OSVR\OSVR-Core-Snapshot-v0.6-1074-g11272ab-build243-vs12-64bit\bin/osvr-plugins-0”
    [OSVR Server] Loading auto-loadable plugins…
    [OSVR Oculus Rift] Initializing Oculus API…

Note: how it says “Oculus Rift”. I don’t know why this is. As a general rule, I find the messages spit out by osvr_server to be pretty useless, so don’t put too much stock into them.

  1. Test the OSVR to OpenVr mapping by running the OpenVR demo app at:

    c:Program Files (x86)/Steam/steamapps/common/SteamVR/demo/bin/win32/hellovr_sdl.exe
    I was able to get this app to work “out of the box”. It will work with just about any config file you start OSVR server with, and whether you’re running in duplicate or extended monitor mode. In other words, this app is much more forgiving than a jMonkeyVR app. So if you can’t get this app to work, you won’t be able to get your jMonkey app to work.

It should display a simple test pattern scene in your HMD only i.e. direct mode.

Getting a jMonkeyVR app to run on your OSVR HDK

Overview

There were four basic “gotchas” I experienced. These are the things that are not mentioned anywhere:

  1. I had to explicitly configure my java runtime to use my nvidia graphics card.
  2. You need to properly set up your project to use phroot’s jMonkeyVr build.
  3. You need to run osvr_server.exe with the default config and in extended mode
  4. Default keyboard events like “Esc= exit” don’t work when in VR mode. You have to code them explicitly
  5. Power-cycling your OSVR HMD is a very handy procedure.

For your test app, you have several choice here: one, you can download a previously built OpenVR app such as Spermination](http://store.steampowered.com/app/363460/), use a free demo](jMonkeyVR-Demo.zip - Google Drive), or you can build your own app using jMonkeyEngine.

I originally tried to get my own app to work, but then ended up trying to get spermination to work as a baseline (presumably the steam app has no library or configuration issues).

But in this write-up we’ll just start with your own app. How you get your own app to run pretty much applies to getting a Steam openVR app to run, so both bases are covered.

Procedure

  1. When starting your SDK, explicitly specify “use Graphics processor”:
    ! high_performance_right_click_screen_shot
    img src="/uploads/default/original/2X/1/1866a001566d4ac2ee3f87147caade857ffe10eb.png" width=“561” height=“500”>

  2. Set up your project and use the proper jar files.
    Obviously, if you’ve never used the SDK to build an app before, it’s advisable to get a non-VR app running first. I’ll assume you are able to get a non-VR jMonkey app to run.

Basically what this means is you want to use the libs from jMonkeyVr and install them into your project. Use Shift-click to select multiple jars.

a) Create a JME3 Basic game, and delete all the original libraries except for your assets and JDK:

!delete_libs_step_1_screen_shot
img src="/uploads/default/original/2X/a/a9eb7379382b2b243c8e680ee52edfe88755570e.png" width=“690” height=“373”>

b) download phroots JME build](GitHub - phr00t/jmonkeyengine: jMonkeyEngine is a 3D game engine for adventurous Java developers. It’s open source, cross platform and cutting edge. And it is all beautifully documented. This is Phr00t's version of that library; better performance, bleeding-edge commits, support for Virtual Reality & multithreading.) from github. When I didn’t user this build, and just used the default jars, I was getting compile errors on the shaders:

SEVERE: Uncaught exception thrown in Thread[jME3 Main,5,main]
com.jme3.renderer.RendererException: compile error in: ShaderSource[name=jmevr/shaders/Unshaded.frag, defines, type=Fragment, language=GLSL150]
0(4) : error C7101: Macro FRAGMENT_SHADER redefined

c) add the jar files in phroots JME3 build from ‘lib’ and ‘dist/lib’ into your projects Libraries (right-click and select Add Jar/Folder).

Note: it’s important that you add ‘lib’ first and then ‘dist/lib’, otherwise I was getting:

Exception in thread "main" java.lang.SecurityException: sealing violation: package org.lwjgl.opengl is sealed

d) add JMonkeyVr.jar and jna-4.2.1.jar from the jMonkeyVr root directory (download this git repo too)

Here’s a screen shot of the end of my libraries:
!libs_complete_screen_shot
img src="/uploads/default/original/2X/c/c5a827a23d6c333296c24faafb4e1a61dbf15171.png" width=“690” height=“194”>

e) make sure your (still non-vr) app runs.

  1. Make sure java uses your GPU.

This step might not be necessary on desktops (?)

This may only be a problem on laptops, or computers that have an internal GPU on the motherboard and a separate graphics card. Typically when you run an app that needs accelerated graphic, the computer is supposed to smart enough to figure this out switch from the default of the internal graphics card to your nVidia card. But for some reason, on my laptop, the computer wasn’t able to figure this out when running Java. So I had to explicitly set it.

a) There are two ways to bring up your GPU’s control panel:
i) right-click on a file you want to change and select
! gpu_screen_shot_1
img src="/uploads/default/original/2X/7/702cd80688b76d948e05656e3fe21cbd272c1bc9.png" width=“670” height=“500”>

b) invoke it from your menu bar in the tray icons near your clock in the bottom right of your desktop.

c) Either way, you should see a program like this:
!nvidia_control_panel
img src="/uploads/default/original/2X/9/92bfe0445073114e8150e9429adad45ace042803.png" width=“592” height=“500”>

d) While I think you only need to add the java jre, I just added everything I could find: the (jre, jdk) x (java, javaw, javaws) + the indirection files.

java= java for window apps
javaw= java from the console
javasw= java web start

see the above screen print for the list. For instance my jre java.exe is located here:
/c/Program Files/Java/jre1.8.0_73/bin/java.exe

The indirection programs are symlinks that point to the actual binaries:

vt@hp_laptop_envy  /c/ProgramData/Oracle/Java/javapath
$ ls -l
total 0
lrwxrwxrwx 1 Vince 197610 46 Mar 15 15:07 java.exe -> /c/Program Files/Java/jre1.8.0_73/bin/java.exe*
lrwxrwxrwx 1 Vince 197610 47 Mar 15 15:07 javaw.exe -> /c/Program Files/Java/jre1.8.0_73/bin/javaw.exe*
lrwxrwxrwx 1 Vince 197610 48 Mar 15 15:07 javaws.exe -> /c/Program Files/Java/jre1.8.0_73/bin/javaws.exe*

e) I received the following errors before I forced java to use the GPU:

*** Error! ***

First thing to do is verify your Java installation. Install the latest from http://java.com/getjava
and make sure to uninstall old versions.

If still having trouble, check the game's forum for help & report this: I'd love to get this addressed!

Thread: main, Version: v1.26

java.lang.Error: Invalid memory access
        at com.sun.jna.Native.invokeInt(Native Method)
        at com.sun.jna.Function.invoke(Function.java:390)
        at com.sun.jna.Function.invoke(Function.java:323)
        at com.sun.jna.Function.invoke(Function.java:275)
        at com.sun.jna.CallbackReference$NativeFunctionHandler.invoke(CallbackReference.java:646)
        at com.sun.proxy.$Proxy59.apply(Unknown Source)
        at jmevr.util.OpenVRViewManager.sendTextures(OpenVRViewManager.java:159)
        at jmevr.app.VRApplication.update(VRApplication.java:588)
        at com.jme3.system.lwjgl.LwjglWindow.runLoop(LwjglWindow.java:372)
        at com.jme3.system.lwjgl.LwjglWindow.run(LwjglWindow.java:456)
        at com.jme3.system.lwjgl.LwjglWindow.create(LwjglWindow.java:299)
        at com.jme3.app.Application.start(Application.java:446)
        at jmevr.app.VRApplication.start(VRApplication.java:310)
        at Init.Main.main(Main.java:137)

  1. Put you computer in extended mode:
    ! extended_mode_screen_shot
    img src="/uploads/default/original/2X/8/8d71b410e633163cbf0ca92986cee10023dadf09.png" width=“577” height=“500”>

  2. Run osvr_server.exe with no configs:

    vt@hp_laptop_envy /c/Program Files (x86)/OSVR/OSVR-Core-Snapshot-v0.6-1074-g11272ab-build243-vs12-64bit/bin
    $ ./osvr_server.exe
    [OSVR Server] Using default config file - pass a filename on the command line to use a different one.
    [OSVR Server] Using config file ‘osvr_server_config.json’
    [OSVR Server] Constructing server as configured…
    [OSVR] Adding search path “C:\Program Files (x86)\OSVR\OSVR-Core-Snapshot-v0.6-1074-g11272ab-build243-vs12-64bit\bin\osvr-plugins-0”
    [OSVR] Adding search path “C:\Program Files (x86)\OSVR\OSVR-Core-Snapshot-v0.6-1074-g11272ab-build243-vs12-64bit\bin/osvr-plugins-0\RelWithDebInfo”
    [OSVR] Adding search path “C:\Program Files (x86)\OSVR\OSVR-Core-Snapshot-v0.6-1074-g11272ab-build243-vs12-64bit\bin/osvr-plugins-0”
    [OSVR Server] Loading auto-loadable plugins…
    [OSVR Oculus Rift] Initializing Oculus API…
    [OVR 1] 03/22/16 05:23:04: [CAPI] LibOVR module is located at C:\WINDOWS\SYSTEM32\LibOVRRT64_0_8.dll

    [OVR 1] 03/22/16 05:23:04: [IPC] Call Stats for SetInitialState : x1: C2S=32.5002, S2C=22.6646, Runtime=54.7372 (microseconds) C2Ssize=80, S2Csize=81 (bytes)

    [OVR 1] 03/22/16 05:23:04: [IPC] Call Stats for IsLatencyTesterAvailable : x1: C2S=6.84214, S2C=4.27634, Runtime=0.855268 (microseconds) C2Ssize=33, S2Csize=18 (bytes)

    [OSVR Oculus Rift] Oculus Rift initialized.
    [OSVR Server] Loading plugins…

  3. Modify your source.

Note: the code I added is denoted by “//vt add” and “//vt end” brackets. Use VRApplication instead of SimpleAppliction:

package mygame;

import com.jme3.app.SimpleApplication;
import com.jme3.material.Material;
import com.jme3.math.ColorRGBA;
import com.jme3.math.Vector3f;
import com.jme3.renderer.RenderManager;
import com.jme3.scene.Geometry;
import com.jme3.scene.shape.Box;
//vt add
import jmevr.app.VRApplication;
import com.jme3.input.KeyInput;
import com.jme3.input.controls.ActionListener;
import com.jme3.input.controls.KeyTrigger;
import com.jme3.scene.Node;
import com.jme3.scene.Spatial;
//vt end

/**
 * test
 * @author normenhansen
 */
//public class Main extends SimpleApplication {
public class Main extends VRApplication {

    public static void main(String[] args) {
        Main app = new Main();
        app.start();
    }

	//vt add
    protected Geometry player;
    Spatial observer;
	//vt end

    @Override
    public void simpleInitApp() {
        Box b = new Box(1, 1, 1);
        Geometry geom = new Geometry("Box", b);

        Material mat = new Material(assetManager, "Common/MatDefs/Misc/Unshaded.j3md");
        mat.setColor("Color", ColorRGBA.Blue);
		geom.setMaterial(mat);
		player = geom;
		
		rootNode.attachChild(geom);
		//vt add
		this.initInputs();
		observer = new Node("observer");

        observer.setLocalTranslation(new Vector3f(8.0f, 0.0f, 0.0f));
        
        VRApplication.setObserver(observer);
        rootNode.attachChild(observer);
		//vt end
    }

    //vt add
    private void initInputs() {
		System.out.println("InitInput: entered");
    	inputManager.addMapping("forward", new KeyTrigger(KeyInput.KEY_W));
    	inputManager.addMapping("backward", new KeyTrigger(KeyInput.KEY_S));

        inputManager.addMapping("exit", new KeyTrigger(KeyInput.KEY_ESCAPE));

		ActionListener acl = new ActionListener() {
			public void onAction(String name, boolean keyPressed, float tpf) {
                if(name.equals("forward")){
					System.out.println("initInput: forward key pressed");
					observer.move(VRApplication.getFinalObserverRotation().getRotationColumn(2).mult(1f));
	    		}
				else if(name.equals("backward")){
					System.out.println("initInput: backward key pressed");
					observer.move(VRApplication.getFinalObserverRotation().getRotationColumn(2).mult(-1f));
//                        moveForward = false;
	    		}
				else if (name.equals("exit")) {
					System.out.println("initInput: exit key pressed");
					System.out.println("initInput: now exiting");
					System.exit(0);
				}
			}
		};
			
        inputManager.addListener(acl, "forward");
        inputManager.addListener(acl, "backward");
        inputManager.addListener(acl, "exit");
    }    
    //vt end

    @Override
    public void simpleUpdate(float tpf) {
        //TODO: add update code
   	 player.rotate(0, 2*tpf, 0.4f * tpf);
    }

    @Override
    public void simpleRender(RenderManager rm) {
        //TODO: add render code
    }
}
  1. Bring up Tracker and make sure your HMD is tracking:

! bad_tracker
img src="/uploads/default/original/2X/0/02d687f443f8ac3f95f848c9aedac51bf0f5f72d.png" width=“645” height=“500”>
!good_tracker
img src="/uploads/default/original/2X/8/8828f3803baedbcb758fa8c6da34c7431cbb7d25.png" width=“645” height=“500”>

Note: if you see the “bad tracker” image, try power-cycling your HMD. Wait for about 5 seconds to get the “device detaching” sound. Then plug back in and wait for the “device attaced” sounds. After this, you should see the “good tracker” image. You may need to cycle the tracker app to refresh.

You don’t have to unplug it at the outlet power source. Just use unplug the power that is going into the belt clip. Also note, I had to do this a lot — dozens of times, because launching a game can from JME cause you to go into “bad tracker” mode.

If you don’t see good tracker, then your app will not run in VR mode

  1. Launch your app
    Press f6, and now you should see a little black window, and you should see a head tracked, split screen image on the HMD i.e. your classic successful VR mode.
6 Likes
2

Thanks for your awesome tutorial. Appreciate Your Work :grinning: