Google Cardboard support

If you look in CardboardState there is a
private Spatial observer;

The transformation of this is applied to the view in the update method.

In other words, if you set the observer field when initializing the application (or whenever), that spatial will affect the camera.

Hello,

Im so happy others with more 3D math and GVR experience is looking into getting this working :smile:

I’m happy to add you as collaborators on my github repo if you like (I will be working more on the issues around mid January, I hope)

Thanks for that! I managed to set an observer but I seemed to be getting an odd rotation and the camera wasn’t moving either.

I had a look in cardboard state and changed a couple of things - firstly I added a line to set the location of the observer to the world translation which put the camera in the right place.

Secondly I changed the order in which the quaternions are multiplied as the weird rotation was still there. I changed these around and this seemed to correct it. I’ve tested it both with an observer now and without and it seems to be ok. :slight_smile:

I’ve included the modified update method below if anyone else needs it:

    // left eye
    tempVars.quat1.set(context.getOrientation());

    tempVars.quat1.toAngles(tempAngles);
    tempAngles[0] = -tempAngles[0];
    tempAngles[2] = -tempAngles[2];
    tempVars.quat1.fromAngles(tempAngles);
    
    tempVars.tempMat4.set(context.getLeftEye().getEyeView());
    tempVars.tempMat4.toTranslationVector(tempVars.vect1);
    if(observer != null){
        tempVars.quat1.set(observer.getLocalRotation().multLocal(tempVars.quat1));
        tempVars.vect1.set(observer.getWorldTranslation());
    }
    camLeft.setFrame(tempVars.vect1, tempVars.quat1);
    
    // right eye
    tempVars.tempMat4.set(context.getRightEye().getEyeView());
    tempVars.tempMat4.toTranslationVector(tempVars.vect1);
    if(observer != null){
        tempVars.vect1.set(observer.getWorldTranslation());
    }
    camRight.setFrame(tempVars.vect1, tempVars.quat1);


    tempVars.release();

I think all my test cases were based on cam only, so it might be that the feature has never been tested properly. I’ve pushed this fix to the repo. Thanks!

I used Robinns Code to move the viewing point (observer). It seemed to work, but I had a lot of really weird rotation of the observer.

After a while, I changed

tempVars.quat1.set(observer.getLocalRotation().multLocal(tempVars.quat1));

to

tempVars.quat1.set(observer.getLocalRotation().mult(tempVars.quat1));

Now it seems to work right for me. Thank you very much, again!

Yup, that makes sense. Thanks for reporting back.
However, wouldn’t just

tempVars.quat1.multLocal(observer.getLocalRotation());

yield the same result?

I don’t know about this case but in general the order of operations of multiplying two quats is very important.

a.mult(b) is not the same as b.mult(a).

You’re right, of course. I see no reason as to why that wouldn’t be true in this case too.

I wanted to see what happens when using

tempVars.quat1.multLocal(observer.getLocalRotation());

and it seems to mirror the rotation.

But If you comment the

    tempVars.quat1.toAngles(tempAngles);
    tempAngles[0] = -tempAngles[0];
    tempAngles[2] = -tempAngles[2];
    tempVars.quat1.fromAngles(tempAngles);

block out, AND use the upper line, it seems to be correct again. (At least in my application, which is only 2D (to name it: tetris). So I’m not sure, if this covers all test cases.)

I think he was just trying to save a Quaternion creation. Your way does more work and may not produce the right answer anyway. (at some point a quaternion’s rotation might spill over into the angle you aren’t inverting in a way that randomly causes issues.)

It’s not really mirroring… it’s just applying the rotations in the wrong order which in some cases may appear to be mirroring.

This effect is easy to see. Let’s say you have a rotation (A) about the x axis of 90 degrees and a rotation (B) about the y axis of 90 degrees. Apply these to your head. ie: A is nodding yes, B is shaking your head no.

First B then A. You will be looking up at the ceiling with your chin over your shoulder.

If A then B you will be looking at the wall over your shoulder with your ear on your shoulder.

Remember, rotations are relative to the current reference frame.

…and I wouldn’t call either of those a mirror image of the other.

Actually, he removes the code that does that extra work by that comment. There’s a discrepancy originating from the matrix that is received from the cardboard that was inverted in that way. So this requires some testing but might both save the Quaternion creation and those extra operations.

Edit: I’m not sure what I’m saying, but there’s always been some issues with the rotation (evident higher up in the thread). Perhaps the order in which I applied the rotations was wrong to begin with.

Ah… I see.

I’d have to know more context to be able to help further. I don’t have enough background on what led up to this and don’t have the time to dig for it, unfortunately.

If someone wants to take a step back and describe what you start with and what you want to get then I might have advice.

You’re right, I was trying to save creating an additional quaternion - I think originally I put the result in tempVars.quat2 but it looked like I could get away without needing to. I will say now though that I am still new to Java and coding in general, so there is a chance that I am doing something stupid.

I have had a couple more peculiar issues with the cardboard plugin related to rotation as well though, but I still want to test them fully - everything seemed fine until I realised that after some time it seemed like the phone had lost the orientation for forward.

I’m hoping to do some experiments in the next couple of days, then I will get back to you.

I unfortunately don’t know much about graphic programming and rotations - so don’t count much on that. I’d love to learn more about this one day, but at the moment, I don’t have the time, as this seems to be a very complex subject.
Thank you pspeed for your explanation about rotations, this makes the problem quite clear.

I just wanted to report a new issue I found and on which I’m working on currenty. I used a bluetooth gamepad with my application before I tried to integrate it with the cardboard plugin and this worked quite well. After integrating, it still is able to ‘find’ the gamepad (joystickEventsEnabled is set to true), but no events seem to get to the event queue.
I hope to solve this soon :slight_smile:

Yeah, I don’t know enough about what the starting conditions or the ending desire is.

To me it seems like you’d start with some head orientation and then from that you’d want to create left and right eye orientations that slightly converge somewhere down that line. (They are also offset left/right of course based on a desired occular distance.)

All of that is super easy to calculate if that’s really the desired starting and ending state.

I’ll take a look at the orientation issue again “when I have time”. I have more info to go on now, regarding the end results. At least it seems to be working as expected for the time being, even if it’s not optimal.

The new issue, hmm. I don’t recall the cardboard integration doing anything specific with event queues. Will be interesting to see what you come up with.

In case it matters… since it’s in my head and I’m avoiding day job work for a minute…

Off the top of my head:

Vector3f headLoc = ...
Quaternion headRot = ...
float convergenceDistance = ...  // down the center line
float eyeOffset = ...  // from the center line
float angle = FastMath.atan2(convergenceDistance, eyeOffset);
Vector3f leftLoc = headLoc.add(headRot.mult(Vector3f.UNIT_X.mult(eyeOffset));
Vector3f rightLoc = headLoc.subtract(headRot.mult(Vector3f.UNIT_X.mult(eyeOffset));
Quaternion leftEyeRot = new Quaternion().fromAngles(0, -angle, 0);
Quaternion rightEyeRot = new Quaternion().fromAngles(0, angle, 0);
Quaternion leftEyeRotWorld = headRot.mult(leftEyeRot);
Quaternion righttEyeRotWorld = headRot.mult(righttEyeRot);

Possibly adjust for the fact that I often get left-hand/right-hand backwards so the eye rotations and/or positions might be swapped. (Also note: technically the local rightEyeRot is also just leftEyeRot.inverse() before converting to world.)

Anyway… now this won’t be distracting my hind-brain. Probably you don’t have that starting information maybe.

After I spent yesterday to much time to solve the gamepad issue (perhaps this wasn’t helpful…), I solved it today quite quickly. The line

mView.setFocusable(true);

was missing in createView in CardboardContext. I put it just l after the lines

    androidInput.setView(mView);
    androidInput.loadSettings(settings);

So, the event queue itself wasn’t the problem, but rather that the fragment view wasn’t really focused. Now the application reacts again to my gamepad input.

Aha. Have you forked the project? Could you create a pull request for the change, you think? No worries if you can’t, it can be a bit tricky.

I just cloned it before, but now I forked it and made a pull request.

1 Like