Gesture recognition methods based on intensity or color images often suffer from low efficiency and lack of robustness. In this paper, we employ a new laser-based camera that produces reliable low-resolution depth images at video rates. By decomposing and recognizing hand poses as finger states (finger poses and finger inter-relations), we achieve robust hand pose recognition in real-time (30 frames/second).

Gesture interfaces have long been pursued in the context of portable computing and immersive environments. However, such interfaces have been difficult to realize, in part due to a lack of frameworks for their design and implementation. This paper presents a framework for automatically producing a gesture interface based on a simple interface description. Rather than defining hand positions in a low-level high-dimensional joint angle space, we describe and recognize gestures in a "lexical" space, in which each hand pose is decomposed into elements in a finger-pose alphabet. The alphabet and underlying rules are defined as a gesture notation system called GeLex. By implementing a generic hand pose recognition algorithm, and a mechanism to adapt it to a specific application based on a general interface description, developing a gesture interface becomes straightforward.

This paper describes SmartCanvas, an intelligent desk system that allows a user to perform freehand drawing on a desk or similar surface with gestures. Our system requires one camera and no touch sensors. The key underlying technique is a vision-based method that distinguishes drawing gestures and transitional gestures in real time, avoiding the need for "artificial" gestures to mark the beginning and end of a drawing stroke. The method achieves an average classification accuracy of 92.17%. Pie-shaped menus and a "rotate-to-and-select" approach eliminate the need for a fixed menu display, resulting in an "invisible" interface.

A number of investigators have had success using domain specific prior knowledge to produce improved superresolution images of faces ("hallucinating faces"). These efforts address the scenario where a face image is obtained from a low-resolution camera. A related but less studied problem occurs when the missing information is the result of occlusion rather than low camera resolution, as in the case when a person is wearing sunglasses. Recently Hwang and Lee [14] introduced the first algorithm for solving this reconstruction "inpainting" problem. In the current work we report results of a psychological study that provides independent evidence regarding the validity of the face reconstruction task, and we demonstrate an improved reconstruction approach using a positive, local linear representation. The positive, local mixture operates on real-world images without manual intervention in many cases, and provides demonstrably lower reconstruction error than is obtainable with a global representation.

In some applications, a local or "parts based" representation is preferable to global basis functions, such as those used in Fourier and principal component analysis. In applications that require human understanding and editing of the data, it is also desirable that the basis functions be in some sense as "simple" as possible. This means, for example, that the basis functions should not have Gabor-like ripples if such ripples are not a prominent feature of the data to be represented. The paper introduces a direct local basis construction. Specifically, we show that local bases result from maximizing an appropriate redefinition of pairwise orthogonality while maintaining the ability to represent the data. The resulting basis functions are competitive with (and for some applications superior to) those obtained from existing algorithms, and the construction does not require that the basis coefficients be statistically non-Gaussian or independent, as would be the case with an independent component analysis approach.