I was thinking about building a framework for doing 3D gesture based input. I have done a proof of concept in 2D using a regular mouse, but I am having trouble finding a 3D device that give the type of output needed.
With all of the easily/cheaply available 6 axis mice, this seemed like it would be easy, but my experience with the couple of them that I have tried so far is that they register with the OS as a regular mouse. So, the hardware is 3 axis of gyro and 3 axis of accelerometer, but only 2 axis output like any regular mouse would give.
Has anyone had any experience with an air mouse or similar device that will give access to the raw output or something resembling 3D motion output?
Iāve seen it done in java once. Iāll edit in a link later, when Iām not on work internet. IIRC, it involved dropping down to the USB device level, and essentially writing a custom driver. I never had the hardware, so have not actually tested.
Itās a plugin for the ArtOfIllusion 3d modeling suite. Part of it is a framework for interacting with aoiās scene graph, part is a driver for a 3dConnexion Space Navigator.
Some discussion is interspersed around the main project forum dut the original author is not very active in the community at the moment.
@Mark_Pyrizhok@sailsman63 Thanks for the suggestions. I was really aiming for something using an air mouse because of how inexpensive and ubiquitous they are. I might have to just go the route of trying to roll a custom driver. Although, Iām not sure if thatās possible because I donāt know how much of the processing is done on the device.
There is also the possibility of doing it with custom hardware, but I expect it would be near impossible to get users to build their own. I had built a prototype about a year ago, but donāt have a working example at the moment. For anyone who is interested, the parts I used were as follows. I ordered all of the parts on aliexpress for about $12 total.
Arduino pro mini atmega328 (make sure you get one with a pre-installed bootloader)
Actually, the joystick library could be used for many different types of custom hardware. You can send whatever values you want for the axes/button values and interpret them to mean whatever you want inside your app.
EDIT: I forgot to mention, if you want to use the joystick library, it has to be direct USB connection and not bluetooth. In that case you need to use arduino leonardo or āpro microā instead of āpro miniā
@jayfella I hadnāt thought of using the wheel as part of the gesture, but thatās definitely an interesting idea. After trying this a little bit, it seems that drawing a gesture with the mouse while rolling the scroll wheel at the same time requires some serious skill. This is probably okay, as long as the gestures are simple. It may also get easier with a little practice. I think Iāll do the rest of the implementation this way and see how it feels after using for a while.
I think that @jayfella meant that he expected the third axis motion to automatically map out as scroll wheel in the default āmouseā interpretation of the data.
Ah, I guess I misunderstood that. Unfortunately they donāt seem to give any mouse wheel events at all. Or at least not for couple of ones that I have tried. They only register button presses, DX, and DY. It is possible that there as some models that do something like that.
@Martha_Simons Not really sure what you are getting at ā¦
For anyone still following this thread. I have gotten a google daydream controller which and am going to try to write a small native library in Windows to connect to it. Then write a JNI wrapper to get output from it into the JVM.
I was able to easily get to the correct BLE characteristic on Linux using the bluetoothctl utility, so hopefully it wonāt be too hard to do it programatically on window.
Iāll post again later when I have made some progress.
I wrote a small C# program to connect to the daydream controller. That part was actually pretty easy, but figuring out how to interpret the data was a bit harder as it is a mix of signed (13 bit twoās complement) and 8 bit unsigned integers . Since you canāt do JNI directly to C#, I had to write a C++/CLI bridge. Then write JNI wrappers for the bridge which is a PITA. There is still quite a bit of clean up to do, but I have the basics working.
I made notes, so if you hear of anyone else wanting to do this, send them my way.
