Download FingerTracker / Read the documentation / FingerTracker source on Github

FingerTracker 10 fingers

FingerTracker is a Processing library that does real-time finger-tracking from depth images. It is based on work done by Murphy Stein at NYU. It uses fast marching squares to find the contour of the hand and then estimates finger endpoints by looking for inflections in the curvature of the contour. It can work with depth maps from either OpenNI or libfreenect (it expects the depth maps to be scaled to the 500-2047 range provided by the Kinect drivers).

Using FingerTracker with the Kinect

To use FingerTracker with the Kinect, you need a Processing library that lets you access the depth images from the Kinect. The two main options are Dan Shiffman’s libfreenect wrapper or SimpleOpenNI. Both libraries offer access to the depth and color images from the Kinect. In addition, SimpleOpenNI also offers access to the other advanced features available from the OpenNI Kinect middleware such as suser detection, skeleton tracking, gesture detection, etc.

Finger-tracking vs. multi-touch

FingerTracker’s technique for detecting fingers is not based on anything specific about human anatomy or any sophisticated techniques for detecting digits. Instead, it is based on the shape of a particular region of a depth image. It looks for contours at a particular depth and then selects the areas where the contours taper-down into finger-like points.

In practice, this means that FingerTracker is, in someways, similar to a multi-touch library. It simply tells you a series of interesting points where a person or object has touched a particular plane in space. It does not tell you the identity of any particular finger or the continuity of which point corresponds to which finger over time (though there are some tricks for figuring these things out).

Installation and Usage

To install Isolines, download the library and unzip it into your Processing libraries folder. Restart Processing and FingerTracker should show up under the “File > Examples” menu under “Contributed Libraries”.

Here is a complete usage example with explanation inline as comments:

// import the fingertracker library
// and the SimpleOpenNI library for Kinect access
import fingertracker.*;
import SimpleOpenNI.*;

// declare FignerTracker and SimpleOpenNI objects
FingerTracker fingers;
SimpleOpenNI kinect;
// set a default threshold distance:
// 625 corresponds to about 2-3 feet from the Kinect
int threshold = 625;

void setup() {
  size(640, 480);
  // initialize your SimpleOpenNI object
  // and set it up to access the depth image
  kinect = new SimpleOpenNI(this);
  // mirror the depth image so that it is more natural

  // initialize the FingerTracker object
  // with the height and width of the Kinect
  // depth image
  fingers = new FingerTracker(this, 640, 480);
  // the "melt factor" smooths out the contour
  // making the finger tracking more robust
  // especially at short distances
  // farther away you may want a lower number

void draw() {
  // get new depth data from the kinect
  // get a depth image and display it
  PImage depthImage = kinect.depthImage();
  image(depthImage, 0, 0);

  // update the depth threshold beyond which
  // we'll look for fingers
  // access the "depth map" from the Kinect
  // this is an array of ints with the full-resolution
  // depth data (i.e. 500-2047 instead of 0-255)
  // pass that data to our FingerTracker
  int[] depthMap = kinect.depthMap();

  // iterate over all the contours found
  // and display each of them with a green line
  for (int k = 0; k < fingers.getNumContours(); k++) {
  // iterate over all the fingers found
  // and draw them as a red circle
  for (int i = 0; i < fingers.getNumFingers(); i++) {
    PVector position = fingers.getFinger(i);
    ellipse(position.x - 5, position.y -5, 10, 10);
  // show the threshold on the screen
  text(threshold, 10, 20);

// keyPressed event:
// pressing the '-' key lowers the threshold by 10
// pressing the '+/=' key increases it by 10 
void keyPressed(){
  if(key == '-'){
    threshold -= 10;
  if(key == '='){
    threshold += 10;