Kinect Game Template – A christimas gift for the monkeys

Contribution Depot
21

@glaucomardano: Is there a graceful way to kill all Java child threads when using your Kinect template? They are still left behind on exit blocking access to Kinect. I could write a bit of code to enumerate all child threads and interrupt them one by one (hoping for them to die out). There must be a simpler way to do this? In the meantime, I have to kill them by hand.

22

Well, in OpenNIKinectInput.java I’ve wrote a piece of code in update() method for killing the thread, but idk this doesn’t work. I did something like executor.purge() or executor.shutdown() or something like that. It should shutdown all its submitted threads when the input is destroyed…

23

So you don’t have to have XBox to make use of Kinect? o.O

24
@dariuszg-jagielski said:
So you don't have to have XBox to make use of Kinect? o.O


No ;).
25

Oh, didn’t know. But you know what I’m waiting for? Software that would basically turn any webcam with solid (>2Mpix) resolution into Kinect.

26

Can’t understand what you mean.

27

Well, Kinect is basically camera with additional software (drivers), so I’m waiting for someone to write such software that would work with any ordinary webcam, so I wouldn’t be forced to buy expensive Kinect

28
@dariuszg-jagielski said:
Well, Kinect is basically camera with additional software (drivers), so I'm waiting for someone to write such software that would work with any ordinary webcam, so I wouldn't be forced to buy expensive Kinect

Ordinary webcams have an IR filter and cannot see the IR web the Kinect projects into the room (and no IR projector for that matter).
29
@glaucomardano said:
Well, in OpenNIKinectInput.java I've wrote a piece of code in update() method for killing the thread, but idk this doesn't work. I did something like executor.purge() or executor.shutdown() or something like that. It should shutdown all its submitted threads when the input is destroyed...


I think this is a common problem with Java. When you control the threads from the main update method then when the application quits there is no-one to kill the background threads. They still live on in the background, in this case holding onto the Kinect resources. Some people create a 'daemon' thread that is responsible for killing all other threads. When the daemon thread is the only one left behind the JDK will exit. See:

http://oreilly.com/catalog/expjava/excerpt/index.html

It has a good section on the life of a thread discussing daemon threads.
30
@dariuszg-jagielski said:
Well, ... so I wouldn't be forced to buy expensive Kinect


If Kinect is too expensive, get ASUS Xtion PRO (or LIVE). These devices are much cheaper than Kinect and OpenNI (which developed an interface useful in JME) people recommend it on their web site. However second hand Kinect for XBOX 360 (which also work with OpenNI) sell for less than a $100.
31
@ironfrown said:
I think this is a common problem with Java.

No, this is a common problem with code that doesn't handle its threads properly.
32

Here is the first result of using Glauco’s template with OpenNI, NITE and Tomoto S. Washio’s bridge to Kinect 4W SDK:



http://www.youtube.com/watch?v=gUBX5Q1xTDU



The gestures need to be very clear, Kinect tends to lag a bit, sometimes it misses gestures. I think it behaves better when continually tracking hand movement rather than trying to interpret gestures. Perhaps tracking skeleton would be easier than a hand? May be there are too many layers of interfacing? Need to check this against a direct use of Kinect 4W SDK from C#.

By the way - the mad waving of my hands is done to reverse the push actions - I still cannot figure out how to execute pull. According to the NITE manual, you should be able to receive "push" event when the hand is moving away from Kinect - this is not happening.

Jacob
1 Like
33

WOW! Nice job :). About the gesture recognitions, I’ve implemented some custom full body gestures recognitions in the app that I did for my thesis, this was based in FAAST, but I did them myself, by reading this chapter of the Killer Game Programming in Java. The gestures works pretty well, but I need to update the project so you can use them, I’ll do it as soon. Cya.

34

That would be great. Also check out if PUSH is not supposed to return a negative velocity when moving the hand away from the sensor (I will also check my code as to what are priorities in handling these events).

35

The push gesture is handled by NITE.

36

@glaucomardano , Can I use your kinect project template as library?

I load model from blender to your kinect project template, but there are error , these error :



WARNING: Failed to find loader: com.jme3.scene.plugins.blender.BlenderModelLoader

java.lang.ClassNotFoundException: com.jme3.scene.plugins.blender.BlenderModelLoader

at java.net.URLClassLoader$1.run(URLClassLoader.java:366)

at java.net.URLClassLoader$1.run(URLClassLoader.java:355)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:354)

at java.lang.ClassLoader.loadClass(ClassLoader.java:423)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)

at java.lang.ClassLoader.loadClass(ClassLoader.java:356)

at java.lang.Class.forName0(Native Method)

at java.lang.Class.forName(Class.java:186)

at com.jme3.asset.DesktopAssetManager.registerLoader(DesktopAssetManager.java:135)

at com.jme3.asset.AssetConfig.loadText(AssetConfig.java:74)

at com.jme3.asset.DesktopAssetManager.(DesktopAssetManager.java:88)

at com.jme3.system.JmeDesktopSystem.newAssetManager(JmeDesktopSystem.java:56)

at com.jme3.system.JmeSystem.newAssetManager(JmeSystem.java:87)

at com.jme3.app.Application.initAssetManager(Application.java:164)

at com.jme3.app.Application.initialize(Application.java:499)

at com.jme3.app.SimpleApplication.initialize(SimpleApplication.java:204)

at com.jme3.system.lwjgl.LwjglAbstractDisplay.initInThread(LwjglAbstractDisplay.java:131)

at com.jme3.system.lwjgl.LwjglAbstractDisplay.run(LwjglAbstractDisplay.java:207)

at java.lang.Thread.run(Thread.java:722)

37

Well, it’s missing the blender libraly. Btw the kinect template source code is out of date, and I won’t maintain it as I’m going to start doing android games as soon. Btw I’ll have to upload the project source code so somebody can check or keep it.

38

@glaucomardano



What classes have you changed, can you give the class code?

I want to learn how you made the library code…,

there is no .java file in your library in kinect template

39

I’ve done some changes in jmonkey engine to support the kinect input, and added its jar to the kinect template. It was some months ago, as you want to see how I did it, I’ll be uploading the projects in dropbox ;). I’ll post here when I do it.

40

I want to make my model move following my hand move using kinect, but I’m so confused i am newbe here.

I’ve merged the code from Killer Game Programming in Java (gestures) that I have modified into runnable (implement runnable in gestureDetec class) n run it as new thread in my simple application, I want my model move following my hand update by point control in that class, but there are error while application running

#

A fatal error has been detected by the Java Runtime Environment:

#

EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000007fee848fd22, pid=5336, tid=1228

#

JRE version: 7.0-b147

Java VM: Java HotSpot™ 64-Bit Server VM (21.0-b17 mixed mode windows-amd64 compressed oops)

Problematic frame:

C [XnVNITE64_1_5_2.dll+0x2fd22]

#

Failed to write core dump. Minidumps are not enabled by default on client versions of Windows

#

An error report file with more information is saved as:

C:UsersAmeenulDocumentsjmeProjSlashWordhs_err_pid5336.log

#

If you would like to submit a bug report, please visit:

http://bugreport.sun.com/bugreport/crash.jsp

The crash happened outside the Java Virtual Machine in native code.

See problematic frame for where to report the bug.

#









this is my class



[java]

package mygame;



import com.jme3.app.SimpleApplication;

import com.jme3.light.DirectionalLight;

import com.jme3.math.Vector3f;

import com.jme3.scene.Spatial;

import com.jme3.system.AppSettings;

import com.primesense.NITE.HandEventArgs;

import com.primesense.NITE.HandPointContext;

import com.primesense.NITE.IdEventArgs;

import com.primesense.NITE.PointControl;

import kinect.GestureDetect2;

import kinect.PositionInfo;

import org.OpenNI.GeneralException;

import org.OpenNI.IObservable;

import org.OpenNI.IObserver;



/** Sample 1 - how to get started with the most simple JME 3 application.

  • Display a blue 3D cube and view from all sides by
  • moving the mouse and pressing the WASD keys. */

    public class mySimple extends SimpleApplication {

    Spatial Kar;

    public static void main(String[] args){

    mySimple app = new mySimple();

    AppSettings settings = new AppSettings(true);

    settings.setResolution(1024,768);

    settings.setFullscreen(true);

    app.setSettings(settings);

    app.setShowSettings(false);

    app.start(); // start the game



    }



    @Override

    public void simpleInitApp() {

    Kar = assetManager.loadModel(“Models/Font/B/B.j3o”);

    Kar.setName(“B”);

    Kar.setLocalTranslation(-1.0f, -1.5f, -0.6f);

    DirectionalLight sun = new DirectionalLight();

    sun.setDirection(new Vector3f(-0.1f, -0.7f, -1.0f));

    Kar.addLight(sun);

    rootNode.attachChild(Kar);

    movKar mK= new movKar();

    Thread th=new Thread(mK);

    th.start();



    }



    class movKar extends GestureDetect2

    {



    @Override

    protected PointControl initPointControl() {

    PointControl pointCtrl = null;

    try {

    pointCtrl = new PointControl();



    // create new hand point

    pointCtrl.getPointCreateEvent().addObserver( new IObserver<HandEventArgs>() {

    public void update(IObservable<HandEventArgs> observable, HandEventArgs args)

    {

    pi = new PositionInfo( args.getHand() );

    System.out.println(pi);

    }

    });





    // hand point has moved

    pointCtrl.getPointUpdateEvent().addObserver( new IObserver<HandEventArgs>() {

    public void update(IObservable<HandEventArgs> observable, HandEventArgs args)

    {

    HandPointContext handContext = args.getHand();

    if (pi == null)

    pi = new PositionInfo(handContext);

    else

    pi.update(handContext);

    //System.out.println(pi); // commented out to reduce output

    Kar.setLocalTranslation(pi.getPosition().getX()*0.05f,pi.getPosition().getY()0.05f,0.0f);



    }

    });



    // destroy hand point

    pointCtrl.getPointDestroyEvent().addObserver( new IObserver<IdEventArgs>() {

    public void update(IObservable<IdEventArgs> observable, IdEventArgs args)

    {

    int id = args.getId();

    System.out.printf(“Point %d destroyed:n”, id);

    if (pi.getID() == id)

    pi = null;

    }

    });



    }

    catch (GeneralException e) {

    e.printStackTrace();

    }

    return pointCtrl;

    }



    }





    }





    [/java]



    [java]

    package kinect;





    // GestureDetect.java

    // Andrew Davison, November 2011, ad@fivedots.psu.ac.th



    /     Use OpenNI’s Gesture Generator and Hands Generator with

    some of NITE’s detectors (controls, listeners)

    getting data every frame. They are connected to a source

    of their required data (usually the Session Manager).


  1. Wave Detector
  • recognizes hand movement as a wave. A wave is usually

    4 direction changes within a specified time limit


  1. Push Detector
  • recognizes hand movement as a push, towards the sensor and away


  1. Swipe Detector
  • recognizes hand movement as a swipe, either up,

    down, left or right, followed by the hand resting


  1. Circle Detector
  • recognizes hand movement as a circular motion;
  • needs a full circle in either direction to start;
  • clockwise is positive, anti-clockwise is negative.

    ** NOTE: the listener in initCircleDetector() is commented out

    because CircleDetector.getCircleEvent() is declared private

    in the API


  1. Steady Detector
  • recognizes when a hand isn’t moving for some time



    All these are subclasses of PointControl.

    /





    import java.awt.    ;

    import java.awt.event.;

    import java.util.    ;



    import org.OpenNI.;

    import com.primesense.NITE.    ;







    public class GestureDetect2 implements Runnable

    {

    // OpenNI and NITE vars

    private Context context;

    private SessionManager sessionMan;



    private boolean isRunning = true;

    protected PositionInfo pi = null; // for storing current hand point info





    public GestureDetect2()

    {



    configOpenNI();

    configNITE();



    System.out.println();

    System.out.println(“Make a click gesture to start the session”);

    // try {

    // while (isRunning) {

    // context.waitAnyUpdateAll();

    // sessionMan.update(context);

    // }

    // context.release();

    // }

    // catch (GeneralException e) {

    // // e.printStackTrace();

    // }





    } // end of GestureDetect()







    private void configOpenNI()

    // set up the Gesture and Hands Generators in OpenNI

    {

    try {

    context = new Context();



    // add the NITE Licence

    License licence = new License(“PrimeSense”, “0KOIk2JeIBYClPWVnMoRKn5cdY4=”);

    context.addLicense(licence);



    HandsGenerator handsGen = HandsGenerator.create(context); // OpenNI

    handsGen.SetSmoothing(0.1f);

    // 0-1: 0 means no smoothing, 1 means ‘infinite’

    // setHandEvents(handsGen);



    GestureGenerator gestureGen = GestureGenerator.create(context); // OpenNI

    // setGestureEvents(gestureGen);



    context.startGeneratingAll();

    System.out.println(“Started context generating…”);

    }

    catch (GeneralException e) {

    // e.printStackTrace();

    System.exit(1);

    }

    } // end of configOpenNI()







    private void configNITE()

    // set up the NITE detectors

    {

    try {

    sessionMan = new SessionManager(context, “Click”, “RaiseHand”); // NITE

    // main focus gesture(s), quick refocus gesture(s)

    //sessionMan = new SessionManager(context, “Wave”, “RaiseHand”);

    setSessionEvents(sessionMan);



    // create point, wave, push, swipe, circle, steady NITE detectors;

    // connect them to the session manager

    PointControl pointCtrl = initPointControl();

    sessionMan.addListener(pointCtrl);



    WaveDetector wd = initWaveDetector();

    sessionMan.addListener(wd);



    PushDetector pd = initPushDetector();

    sessionMan.addListener(pd);



    SwipeDetector sd = initSwipeDetector();

    sessionMan.addListener(sd);



    CircleDetector cd = initCircleDetector();

    sessionMan.addListener(cd);



    SteadyDetector sdd = initSteadyDetector();

    sessionMan.addListener(sdd);

    }

    catch (GeneralException e) {

    e.printStackTrace();

    System.exit(1);

    }

    } // end of configNITE()







    //
set event processing callbacks


// private void setHandEvents(HandsGenerator handsGen)
// // create HandsGenerator callbacks
// {
//// try {
//// // when hand is created
//// handsGen.getHandCreateEvent().addObserver(
//// new IObserver<ActiveHandEventArgs>() {
//// public void update(IObservable<ActiveHandEventArgs> observable,
//// ActiveHandEventArgs args)
//// {
//// int id = args.getId();
//// Point3D pt = args.getPosition();
//// float time = args.getTime();
//// System.out.printf("Hand %d located at (%.0f, %.0f, %.0f), at %.0f secsn",
//// id, pt.getX(), pt.getY(), pt.getZ(), time);
//// // System.out.println("objek observer create ke : "+this.toString());
//// }
//// });
////
//// // when hand is destroyed
//// handsGen.getHandDestroyEvent().addObserver(
//// new IObserver<InactiveHandEventArgs>() {
//// public void update(IObservable<InactiveHandEventArgs> observable,
//// InactiveHandEventArgs args)
//// {
//// int id = args.getId();
//// float time = args.getTime();
//// System.out.printf("Hand %d destroyed at %.0f secs n", id, time);
//// System.out.println("objek observer destroy ke : "+this.toString());
//// }
//// });
//// }
//// catch (StatusException e) {
//// e.printStackTrace();
//// }
// } // end of setHandEvents()
//
//
//
// private void setGestureEvents(GestureGenerator gestureGen)
// // create GestureGenerator callback
// {
//// try {
//// // when gesture is recognized
//// gestureGen.getGestureRecognizedEvent().addObserver(
//// new IObserver<GestureRecognizedEventArgs>() {
//// public void update(IObservable<GestureRecognizedEventArgs> observable,
//// GestureRecognizedEventArgs args)
//// {
//// String gestureName = args.getGesture();
//// Point3D idPt = args.getIdPosition();
//// // hand position when gesture was identified
//// Point3D endPt = args.getEndPosition();
//// // hand position at the end of the gesture
//// System.out.printf("Gesture "%s" recognized at (%.0f, %.0f, %.0f); ended at (%.0f, %.0f, %.0f)n",
//// gestureName, idPt.getX(), idPt.getY(), idPt.getZ(),
//// endPt.getX(), endPt.getY(), endPt.getZ() );
//// System.out.println("objek gesturOBS ke : "+this.toString());
//// }
//// });
//// }
//// catch (StatusException e) {
//// e.printStackTrace();
//// }
// } // end of setGestureEvents()
//


private void setSessionEvents(SessionManager sessionMan)
// create session callbacks
{
try {
// when a focus gesture has started to be recognized
// sessionMan.getSessionFocusProgressEvent().addObserver(
// new IObserver<StringPointValueEventArgs>() {
// public void update(IObservable<StringPointValueEventArgs> observable,
// StringPointValueEventArgs args)
// {
// Point3D focusPt = args.getPoint();
// float progress = args.getValue();
// String focusName = args.getName();
// System.out.printf("Session focused at (%.0f, %.0f, %.0f) on %s [progress %.2f]n",
// focusPt.getX(), focusPt.getY(), focusPt.getZ(), focusName, progress);
// }
// });
//
// // session started
// sessionMan.getSessionStartEvent().addObserver( new IObserver<PointEventArgs>() {
// public void update(IObservable<PointEventArgs> observable, PointEventArgs args)
// {
// Point3D focusPt = args.getPoint();
// System.out.printf("Session started at (%.0f, %.0f, %.0f)n",
// focusPt.getX(), focusPt.getY(), focusPt.getZ());
// }
// });


// session end
sessionMan.getSessionEndEvent().addObserver( new IObserver<NullEventArgs>() {
public void update(IObservable<NullEventArgs> observable, NullEventArgs args)
{
System.out.println("Session ended");
isRunning = false; // causes loop in constructor to end, so program exits
}
});
}
catch (StatusException e) {
e.printStackTrace();
}
} // end of setSessionEvents()



//
set up detectors and callbacks
// for point, wave, push, swipe, circle, steady



protected PointControl initPointControl()
{
PointControl pointCtrl = null;
try {
pointCtrl = new PointControl();

// create new hand point
pointCtrl.getPointCreateEvent().addObserver( new IObserver<HandEventArgs>() {
public void update(IObservable<HandEventArgs> observable, HandEventArgs args)
{
pi = new PositionInfo( args.getHand() );
System.out.println(pi);
}
});


// hand point has moved
pointCtrl.getPointUpdateEvent().addObserver( new IObserver<HandEventArgs>() {
public void update(IObservable<HandEventArgs> observable, HandEventArgs args)
{
HandPointContext handContext = args.getHand();
if (pi == null)
pi = new PositionInfo(handContext);
else
pi.update(handContext);
//System.out.println(pi); // commented out to reduce output
}
});

// destroy hand point
pointCtrl.getPointDestroyEvent().addObserver( new IObserver<IdEventArgs>() {
public void update(IObservable<IdEventArgs> observable, IdEventArgs args)
{
int id = args.getId();
System.out.printf("Point %d destroyed:n", id);
if (pi.getID() == id)
pi = null;
}
});

}
catch (GeneralException e) {
e.printStackTrace();
}
return pointCtrl;
} // end of initPointControl()




private WaveDetector initWaveDetector()
{
WaveDetector waveDetector = null;
try {
waveDetector = new WaveDetector();

// some wave settings; change with set
int flipCount = waveDetector.getFlipCount();
int flipLen = waveDetector.getMinLength();
System.out.println("Wave settings -- no. of flips: " + flipCount +
"; min length: " + flipLen + "mm");

// callback
// waveDetector.getWaveEvent().addObserver( new IObserver<NullEventArgs>() {
// public void update(IObservable<NullEventArgs> observable, NullEventArgs args)
// {
// System.out.println("Wave detected");
// System.out.println(" " + pi); // show current hand point
// }
// });
}
catch (GeneralException e) {
e.printStackTrace();
}
return waveDetector;
} // end of initWaveDetector()




private PushDetector initPushDetector()
{
PushDetector pushDetector = null;
try {
pushDetector = new PushDetector();

// some push settings; change with set
float minVel = pushDetector.getPushImmediateMinimumVelocity();
// minimum velocity in the time span to define as push, in m/s

float duration = pushDetector.getPushImmediateDuration();
// time used to detect push, in ms

float angleZ = pushDetector.getPushMaximumAngleBetweenImmediateAndZ();
// max angle between immediate direction and Z-axis, in degrees

System.out.printf("Push settings -- min velocity: %.1f m/s; min duration: %.1f ms; max angle to z-axis: %.1f degs n",
minVel, duration, angleZ);

// callback
// pushDetector.getPushEvent().addObserver( new IObserver<VelocityAngleEventArgs>() {
// public void update(IObservable<VelocityAngleEventArgs> observable,
// VelocityAngleEventArgs args)
// {
// System.out.printf("Push: velocity %.1f m/s, angle %.1f degs n",
// args.getVelocity(), args.getAngle());
// System.out.println(" " + pi); // show current hand point
// }
// });
}
catch (GeneralException e) {
e.printStackTrace();
}
return pushDetector;
} // end of initPushDetector()




private SwipeDetector initSwipeDetector()
{
SwipeDetector swipeDetector = null;
try {
swipeDetector = new SwipeDetector();

// some swipe settings; change with set
System.out.println("Swipe setting -- min motion time: " +
swipeDetector.getMotionTime() + " ms");

// general swipe callback
// swipeDetector.getGeneralSwipeEvent().addObserver(
// new IObserver<DirectionVelocityAngleEventArgs>() {
// public void update(IObservable<DirectionVelocityAngleEventArgs> observable,
// DirectionVelocityAngleEventArgs args)
// {
// System.out.printf("Swipe %s: velocity %.1f m/s, angle %.1f degs n",
// args.getDirection(), args.getVelocity(), args.getAngle());
// System.out.println(" " + pi); // show current hand point
// }
// });
//
// // callback for left swipes only;
// swipeDetector.getSwipeLeftEvent().addObserver(
// new IObserver<VelocityAngleEventArgs>() {
// public void update(IObservable<VelocityAngleEventArgs> observable,
// VelocityAngleEventArgs args)
// {
// System.out.printf("*Left* Swipe: velocity %.1f m/s, angle %.1f degs n",
// args.getVelocity(), args.getAngle());
// }
// });
}
catch (GeneralException e) {
e.printStackTrace();
}
return swipeDetector;
} // end of initSwipeDetector()




private CircleDetector initCircleDetector()
{
CircleDetector circleDetector = null;
try {
circleDetector = new CircleDetector();

// print some circle settings
System.out.println("Circle setting -- min-max radius: " +
circleDetector.getMinRadius() + " - " +
circleDetector.getMaxRadius() +" mm");


// callback: PROBLEM: getCircleEvent() is defined as private!
/*
circleDetector.getCircleEvent().addObserver( new IObserver<CircleEventArgs>() {
public void update(IObservable<CircleEventArgs> observable, CircleEventArgs args)
{
Circle circle = args.getCircle();
Point3D center = circle.getCenter();
System.out.printf("Circle: center (%.0f, %.0f, %.0f), radius %.0f, times %dn",
center.getX(), center.getY(), center.getZ(),
circle.getRadius(), args.getTimes());
}
});
*/
}
catch (GeneralException e) {
e.printStackTrace();
}
return circleDetector;
} // end of initCircleDetector()




private SteadyDetector initSteadyDetector()
{
SteadyDetector steadyDetector = null;
try {
steadyDetector = new SteadyDetector();
System.out.println("Steady settings -- min duration: " +
steadyDetector.getDetectionDuration() + " ms");
System.out.printf(" max movement: %.3f mmn",
steadyDetector.getMaxDeviationForSteady());

// callback
// steadyDetector.getSteadyEvent().addObserver( new IObserver<IdValueEventArgs>() {
// public void update(IObservable<IdValueEventArgs> observable,
// IdValueEventArgs args)
// {
// System.out.printf("Hand %d is steady: movement %.3fn",
// args.getId(), args.getValue());
// System.out.println(" " + pi); // show current hand point
// }
// });
}
catch (GeneralException e) {
e.printStackTrace();
}
return steadyDetector;
} // end of initSteadyDetector()



//

// public static void main(String args[])
//{ new GestureDetect(); }

public void run() {
try {
while (isRunning) {
context.waitAnyUpdateAll();
sessionMan.update(context);
}
context.release();
}
catch (GeneralException e) {
// e.printStackTrace();
}
}

} // end of GestureDetect class

[/java]