jMonkey and Augmented Reality

Hi all,



For my final year University dissertation I am going to be focusing on Augmented Reality in computer games and was hoping to use jARToolKit (http://sourceforge.net/projects/jartoolkit/) but currently it only supports GL4Java, JOGL and Java3D. Currently I have zero experience with all of these 3 API’s and was wandering how easy it would be to integrate any of them with jMonkey?


Well, JOGL and GL4Java are GL bindings more than graphics or game engines (which would be what jME is)… Java3D is more similar to jME in this respect. I would recommend you to check LWJGL and see if it makes more sense to use it directly instead of jME…

If I was going to use something directly I would have to use either Java3D, JOGL or GL4Java but I would preferably be able to use all the features of jME as well which is why I wanted to integrate JOGL or GL4Java into jME. I noticed in the main description for jME it says that there are plans to support JOGL, has anything else been mentioned about that? I've only just discovered jME so for all I know it may be going into 2.0

If I were you, I'd ping jARToolKit to add lwjgl support.

I’ve (atleast I think) sent a message to the developers of jARToolKit asking them about it and I’ve also found another AR kit - http://nyatla.jp/nyartoolkit/wiki/index.php?FrontPage.en which seems to of been recently updated but I can’t see an email address on the site and the forum is empty so not sure if it’s even updated atall.

Writing a jME renderer for any OpenGL implementation is easy… You just replace all GL11/GL12/etc in the LWJGL renderer with the corresponding OpenGL wrapper class for the implementation.

Sounds like you've had some experience with custom renderers. Any tips for me as to where to begin? My first and main question is that would this be the sort of thing where I can start my project in jME as is to get the game up and running and then when I want to move on to the augmented reality portion I could then swap out all the calls like you mentioned?

I haven't actually done this because I never had the need, if I had done this I would have released the renderer(s) to jME by now. Anyway, what I was saying was mostly theory; considering that the API being used is still OpenGL, you only need to adapt the method of calling to whatever wrapper you're using

I was able to put together an AndroidRenderer in about a day or so, so yeah, it's not too bad.  One of the researchers at NASA Ames put together a JOGL port that sounds fairly thorough.  They were open to contributing it back, but I imagine that will be after lots of navigation through government red tape.

Hey, I’m trying to do the same for my university project…



Look i got to render the camera buffer as the scene background, but I’m having some problems setting the camera position and projection. When i have the pattern in the middle of the screen, it almost looks like it would lie on the pattern, but when I move the pattern the 3D Object moves too, but it goes slow, it means when the pattern reaches the left, right, top or bottom side of the camera the 3D object doesn’t reach it yet.



Look an example…















Who considers that can help me, i will thank you really much…

Then you can give me you e-mail and i will send you the source code.

Hey, m8 I am also doing on my final years project in AR lol, and decided to use Artoolkit Plus and port the necessary functions to java using JNI.



In the beggining I had a similar problem and it was due to the camera FOV, you need to get the data from the projection matrix and applay it to the camera.



I am doing something like this:



                float[] matrixProjection = testeMulti.getProjection();
                cam.setLocation(new Vector3f(2, 6, 0)); //this part is the only one I wasnt able to figure out where to automaticly set...

                float fovy = (float) Math.toDegrees(Math.atan(1 / matrixProjection[5]) * 2);
                float aspect = matrixProjection[5] / matrixProjection[0];
                cam.setFrustumPerspective(fovy, aspect, (float) 1, 1000);

I'm using the code you give me, but it doesn't solve anything…



double[] matrix = arEngine.getCameraTransformMatrix();
cam.setLocation(new Vector3f(2, 6, 0));

float fovy = (float) Math.toDegrees(Math.atan(1 / matrix[5]) * 2);
float aspect = (float) (matrix[5] / matrix[0]);
cam.setFrustumPerspective(fovy, aspect, (float) 1, 1000);



And when I get the transformation matrix for the marker from the ARtoolkit, I set it to the Node and i have to invert Y and Z angles; and set the X location to -X. If i don't do it, the object behave so different to the marker rotation and translation.


               boolean found = pattern.getTransMatrix(matrix);


      if (found) {
         setCullMode(CULL_DYNAMIC);

         Matrix4f mat4f = new Matrix4f(
         (float) matrix[0], (float) matrix[4], (float) matrix[8], -(float) matrix[12],
         (float) matrix[1], (float) matrix[5], (float) matrix[9], (float) matrix[13],
         (float) matrix[2], (float) matrix[6], (float) matrix[10], (float) matrix[14],
         (float) matrix[3], (float) matrix[7], (float) matrix[11], (float) matrix[15]);


         Vector3f location = mat4f.toTranslationVector();
         Quaternion rotation = mat4f.toRotationQuat();


         rotation.z *= -1.0f;
         rotation.y *= -1.0f;

         setLocalTranslation(location);
         setLocalRotation(rotation);

      } else if (!keepTransform)
         setCullMode(CULL_ALWAYS);



Here you can see the problem I have...
http://asoesis.sinfoci.net/swf/TestAR/TestAR.htm

I was looking another related topic with the same problem.
http://www.jmonkeyengine.com/jmeforum/index.php?topic=8427.msg66610#msg66610

humm strange but how were u setting the camera before ?

@jonal

It looks like you have the wrong field of view. What fov to use depends on the camera (and the amount of zoom). Unless you happen to know what it is you have to find out using trial and error. I would make it possible to change the fov using the keyboard. Then set a marker at the edge of screen and adjust the fov until it fits.



Btw, I've never used artoolkit, so there might be something in there that would help.

yes tom it  does, we can get a projection matrix from it.

Ok, I was tired trying to set the artoolkit' camera projection in the jME camera, so I did what Tom said me I used the keyboard to change x, y, z, fov and I get it to work almost perfectly…  :smiley:



But,  :frowning: I found that when I move the pattern in depth the Y value get the same error than before… But when I move the pattern at the same distance to the camera It works perfect.



Ahhmm I forgot to say that when I set up the camera with the keyboard I didn't use the artoolkit's getCameraMatriz() anymore.

Sorry I gave you some bad advice there. You should configure artoolkit. It has to have the right config to calculate the projection matrix. Have a look here: http://www.hitl.washington.edu/artoolkit/documentation/usercalibration.htm



Then you should try to use artoolkits getCameraMatrix()