Google Cardboard support

Looks OK to me.
I have a vague memory of the setting “VR mode” possibly causing an issue like this. Ie, if VR mode is disabled the lib won’t render the second eye. That should be located in the CardboardActivity somewhere, or possible CardboardContext

Hi to everybody, I’ve been lurking the forum for a while, but it is my first post here :relaxed:

I remember having the same problem.
Unluckily I tinkered with the original code in variuos places long time ago, so I can’t be 100% sure about the result, but It seemed to me that changing the right viewport clear flags in the setupFiltersAndViews function in the CardboardState.java fixed the problem.
As I said I’m not very sure about it of course, but you can give a try.
This is my function:

private void setupFiltersAndViews() {
        camRight = camLeft.clone();
        
        viewPortRight = app.getRenderManager().createMainView("Right viewport", camRight);
        viewPortRight.setClearFlags(false, false, true);
        viewPortRight.attachScene(this.app.getRootNode());

        /*I commented the following lines, differently from the original file, but I don't think it should influence the result*/
        //camLeft.setViewPort(0.0f, 0.5f, 0.0f, 1.0f);
        //camRight.setViewPort(0.5f, 1, 0.0f, 1f);

        
    }
1 Like

If it is confirmed to work I’ll push a fix, unless you’d rather make a pull request.

I tried turning on the VR Mode yesterday, but it had no effect. I called (CardboardView)frameLayout.getChildAt(0)).getVRMode() in the fragment, which told me, that the VR Mode seems already to be activated.

But with McToo’s fix, it’s working now. Thank you both so much! I can’t imagine, how long it would have taken me to find it out myself.

Hello,

I’ve been tinkering with the cardboard plugin as well and have managed to get it working. Thanks to everyone above for this!

I was wondering if there is any way to get the move the cardboard camera with this plugin? I had a look at cardboard state and noticed that in initialisation it places the camera at zero - overriding any motion set on the camera in the main application.

Essentially I want to get the location of the main application plugin and apply it to the cardboard plugin - has this been done or should I try it myself?

If you look in CardboardState there is a
private Spatial observer;

The transformation of this is applied to the view in the update method.

In other words, if you set the observer field when initializing the application (or whenever), that spatial will affect the camera.

Hello,

Im so happy others with more 3D math and GVR experience is looking into getting this working :smile:

I’m happy to add you as collaborators on my github repo if you like (I will be working more on the issues around mid January, I hope)

Thanks for that! I managed to set an observer but I seemed to be getting an odd rotation and the camera wasn’t moving either.

I had a look in cardboard state and changed a couple of things - firstly I added a line to set the location of the observer to the world translation which put the camera in the right place.

Secondly I changed the order in which the quaternions are multiplied as the weird rotation was still there. I changed these around and this seemed to correct it. I’ve tested it both with an observer now and without and it seems to be ok. :slight_smile:

I’ve included the modified update method below if anyone else needs it:

    // left eye
    tempVars.quat1.set(context.getOrientation());

    tempVars.quat1.toAngles(tempAngles);
    tempAngles[0] = -tempAngles[0];
    tempAngles[2] = -tempAngles[2];
    tempVars.quat1.fromAngles(tempAngles);
    
    tempVars.tempMat4.set(context.getLeftEye().getEyeView());
    tempVars.tempMat4.toTranslationVector(tempVars.vect1);
    if(observer != null){
        tempVars.quat1.set(observer.getLocalRotation().multLocal(tempVars.quat1));
        tempVars.vect1.set(observer.getWorldTranslation());
    }
    camLeft.setFrame(tempVars.vect1, tempVars.quat1);
    
    // right eye
    tempVars.tempMat4.set(context.getRightEye().getEyeView());
    tempVars.tempMat4.toTranslationVector(tempVars.vect1);
    if(observer != null){
        tempVars.vect1.set(observer.getWorldTranslation());
    }
    camRight.setFrame(tempVars.vect1, tempVars.quat1);


    tempVars.release();

I think all my test cases were based on cam only, so it might be that the feature has never been tested properly. I’ve pushed this fix to the repo. Thanks!

I used Robinns Code to move the viewing point (observer). It seemed to work, but I had a lot of really weird rotation of the observer.

After a while, I changed

tempVars.quat1.set(observer.getLocalRotation().multLocal(tempVars.quat1));

to

tempVars.quat1.set(observer.getLocalRotation().mult(tempVars.quat1));

Now it seems to work right for me. Thank you very much, again!

Yup, that makes sense. Thanks for reporting back.
However, wouldn’t just

tempVars.quat1.multLocal(observer.getLocalRotation());

yield the same result?

I don’t know about this case but in general the order of operations of multiplying two quats is very important.

a.mult(b) is not the same as b.mult(a).

You’re right, of course. I see no reason as to why that wouldn’t be true in this case too.

I wanted to see what happens when using

tempVars.quat1.multLocal(observer.getLocalRotation());

and it seems to mirror the rotation.

But If you comment the

    tempVars.quat1.toAngles(tempAngles);
    tempAngles[0] = -tempAngles[0];
    tempAngles[2] = -tempAngles[2];
    tempVars.quat1.fromAngles(tempAngles);

block out, AND use the upper line, it seems to be correct again. (At least in my application, which is only 2D (to name it: tetris). So I’m not sure, if this covers all test cases.)

I think he was just trying to save a Quaternion creation. Your way does more work and may not produce the right answer anyway. (at some point a quaternion’s rotation might spill over into the angle you aren’t inverting in a way that randomly causes issues.)

It’s not really mirroring… it’s just applying the rotations in the wrong order which in some cases may appear to be mirroring.

This effect is easy to see. Let’s say you have a rotation (A) about the x axis of 90 degrees and a rotation (B) about the y axis of 90 degrees. Apply these to your head. ie: A is nodding yes, B is shaking your head no.

First B then A. You will be looking up at the ceiling with your chin over your shoulder.

If A then B you will be looking at the wall over your shoulder with your ear on your shoulder.

Remember, rotations are relative to the current reference frame.

…and I wouldn’t call either of those a mirror image of the other.

Actually, he removes the code that does that extra work by that comment. There’s a discrepancy originating from the matrix that is received from the cardboard that was inverted in that way. So this requires some testing but might both save the Quaternion creation and those extra operations.

Edit: I’m not sure what I’m saying, but there’s always been some issues with the rotation (evident higher up in the thread). Perhaps the order in which I applied the rotations was wrong to begin with.

Ah… I see.

I’d have to know more context to be able to help further. I don’t have enough background on what led up to this and don’t have the time to dig for it, unfortunately.

If someone wants to take a step back and describe what you start with and what you want to get then I might have advice.

You’re right, I was trying to save creating an additional quaternion - I think originally I put the result in tempVars.quat2 but it looked like I could get away without needing to. I will say now though that I am still new to Java and coding in general, so there is a chance that I am doing something stupid.

I have had a couple more peculiar issues with the cardboard plugin related to rotation as well though, but I still want to test them fully - everything seemed fine until I realised that after some time it seemed like the phone had lost the orientation for forward.

I’m hoping to do some experiments in the next couple of days, then I will get back to you.

I unfortunately don’t know much about graphic programming and rotations - so don’t count much on that. I’d love to learn more about this one day, but at the moment, I don’t have the time, as this seems to be a very complex subject.
Thank you pspeed for your explanation about rotations, this makes the problem quite clear.

I just wanted to report a new issue I found and on which I’m working on currenty. I used a bluetooth gamepad with my application before I tried to integrate it with the cardboard plugin and this worked quite well. After integrating, it still is able to ‘find’ the gamepad (joystickEventsEnabled is set to true), but no events seem to get to the event queue.
I hope to solve this soon :slight_smile:

Yeah, I don’t know enough about what the starting conditions or the ending desire is.

To me it seems like you’d start with some head orientation and then from that you’d want to create left and right eye orientations that slightly converge somewhere down that line. (They are also offset left/right of course based on a desired occular distance.)

All of that is super easy to calculate if that’s really the desired starting and ending state.