Animating in-game avatars using real time motion capture data is highly appealing and is becoming more widespread due to the accessibility of researchers and developers to consumer priced depth sensors [Microsoft Kinect 2010]. Depth sensors allow for a cheap and robust motion capture solution which can be naturally adapted to games. However, in spite of the many advantages of using real-time motion capture data for animating avatars in games, there are two major challenges. The first is due to the limitations of the current tracking techniques in producing smooth, noise free, accurate animation in real-time. The second, and more acute problem, stems from the fact that in most games, the movements of the animated avatar are expected to be more expressive than the player's actual movements. In such cases, one would like to visually enhance the player's motion to display exaggerated or even super-natural motions.

We present a robust framework for tracking skeleton joints in real-time by using a single time-of-flight depth sensor. The framework is able to remove the background noise inherent in time-of-flight cameras, detect multiple people, and track up to 30 joints of free motion for each person. The approach has several advantages over traditional motion capture, as it is a cheap alternative to magnetic and optical systems, and requires no markers whatsoever. Unlike markerless systems based on RGB cameras [Duetscher et al. 2000; Kehl and Van Gool 2006], our framework yields dependable results at an interactive rate using a single camera.