6-axis mouse / air mouse


#1

Hello All,

I was thinking about building a framework for doing 3D gesture based input. I have done a proof of concept in 2D using a regular mouse, but I am having trouble finding a 3D device that give the type of output needed.

With all of the easily/cheaply available 6 axis mice, this seemed like it would be easy, but my experience with the couple of them that I have tried so far is that they register with the OS as a regular mouse. So, the hardware is 3 axis of gyro and 3 axis of accelerometer, but only 2 axis output like any regular mouse would give.

Has anyone had any experience with an air mouse or similar device that will give access to the raw output or something resembling 3D motion output?


#2

I’ve seen it done in java once. I’ll edit in a link later, when I’m not on work internet. IIRC, it involved dropping down to the USB device level, and essentially writing a custom driver. I never had the hardware, so have not actually tested.


#3

I’ve played a bit with LeapMotion. You can use java for it.
~70$
https://www.leapmotion.com/


#4

The project I had in mind was this: https://sourceforge.net/projects/aoisp/files/HIDPlugin/0.85/

It’s a plugin for the ArtOfIllusion 3d modeling suite. Part of it is a framework for interacting with aoi’s scene graph, part is a driver for a 3dConnexion Space Navigator.

Some discussion is interspersed around the main project forum dut the original author is not very active in the community at the moment.


#5

@Mark_Pyrizhok @sailsman63 Thanks for the suggestions. I was really aiming for something using an air mouse because of how inexpensive and ubiquitous they are. I might have to just go the route of trying to roll a custom driver. Although, I’m not sure if that’s possible because I don’t know how much of the processing is done on the device.

There is also the possibility of doing it with custom hardware, but I expect it would be near impossible to get users to build their own. I had built a prototype about a year ago, but don’t have a working example at the moment. For anyone who is interested, the parts I used were as follows. I ordered all of the parts on aliexpress for about $12 total.

  1. Arduino pro mini atmega328 (make sure you get one with a pre-installed bootloader)
  2. MPU-6050
  3. SPP-C Bluetooth serial pass-through
  4. Micro USB 5V 1A lithium battery charger board
  5. A LiPo battery from a RC copter kit.

There are many examples of handling standard serial input for Java, but if you don’t want to mess with that. You can use this excellent library to have it register with the OS as a joystick https://github.com/MHeironimus/ArduinoJoystickLibrary.

Actually, the joystick library could be used for many different types of custom hardware. You can send whatever values you want for the axes/button values and interpret them to mean whatever you want inside your app.

EDIT: I forgot to mention, if you want to use the joystick library, it has to be direct USB connection and not bluetooth. In that case you need to use arduino leonardo or “pro micro” instead of “pro mini”


#6

Sorry, I misunderstood completely what you meant by a 6-axis mouse.

A quick Google name me think that most of the processing is done on the host system, not the device.

If you can’t find data sheets, you might need to set up a sniffer like Wireshark, and do some experimenting.


#7

Yeah, it looks like that might be the only way. I think I’ll just put the idea on hold for now. Thanks again for the help.


#8

Isn’t the scroll wheel your third axis?


#9

@jayfella I hadn’t thought of using the wheel as part of the gesture, but that’s definitely an interesting idea. After trying this a little bit, it seems that drawing a gesture with the mouse while rolling the scroll wheel at the same time requires some serious skill. This is probably okay, as long as the gestures are simple. It may also get easier with a little practice. I think I’ll do the rest of the implementation this way and see how it feels after using for a while.


#10

I think that @jayfella meant that he expected the third axis motion to automatically map out as scroll wheel in the default ‘mouse’ interpretation of the data.


#11

Ah, I guess I misunderstood that. Unfortunately they don’t seem to give any mouse wheel events at all. Or at least not for couple of ones that I have tried. They only register button presses, DX, and DY. It is possible that there as some models that do something like that.


#12

@jayfella :thinking:


#13

@Martha_Simons Not really sure what you are getting at …

For anyone still following this thread. I have gotten a google daydream controller which and am going to try to write a small native library in Windows to connect to it. Then write a JNI wrapper to get output from it into the JVM.

I was able to easily get to the correct BLE characteristic on Linux using the bluetoothctl utility, so hopefully it won’t be too hard to do it programatically on window.

I’ll post again later when I have made some progress.


#14

@sailsman63 @Mark_Pyrizhok I just wanted to thank you guys again and let you know where I ended up.

I wrote a small C# program to connect to the daydream controller. That part was actually pretty easy, but figuring out how to interpret the data was a bit harder as it is a mix of signed (13 bit two’s complement) and 8 bit unsigned integers . Since you can’t do JNI directly to C#, I had to write a C++/CLI bridge. Then write JNI wrappers for the bridge which is a PITA. There is still quite a bit of clean up to do, but I have the basics working.

I made notes, so if you hear of anyone else wanting to do this, send them my way.