We propose a temporal spotting method of a hand gesture from a low-resolution far-infrared image sequence captured by a far-infrared sensor array. The sensor array captures the spatial distribution of far-infrared intensity as a thermal image by detecting far-infrared waves emitted from heat sources. It is difficult to spot a hand gesture from a sequence of thermal images captured by the sensor due to its low-resolution, heavy noise, and varying duration of the gesture. Therefore, we introduce a voting-based approach to spot the gesture with template matching-based gesture recognition. We confirm the effectiveness of the proposed temporal spotting method in several settings.

This paper proposes a Deep Learning-based action recognition method from an extremely low-resolution thermal image sequence. The method recognizes daily actions by humans (e.g. walking, sitting down, standing up, etc.) and abnormal actions (e.g. falling down) without privacy concerns. While privacy concerns can be ignored, it is difficult to compute feature points and to obtain a clear edge of the human body from an extremely low-resolution thermal image. To address these problems, this paper proposes a Deep Learning-based action recognition method that combines convolution layers and an LSTM layer for learning spatio-temporal representation, whose inputs are the thermal images and their frame differences cropped by the gravity center of human regions. The effectiveness of the proposed method was confirmed through experiments.

In this paper, we propose a simple and effective depth upsampling technique using self-guided residual interpolation. The original residual interpolation requires guidance information such as high-resolution RGB color image. However, self-guided residual interpolation requires only a single depth map. In the proposed algorithm, a tentative estimation of a high-resolution depth map is first generated from an input low-resolution depth map. Then, re-interpolation is applied to the residual domain, which is defined by differences between the input depth map and the tentative estimate. A precise high-resolution depth map is obtainable by interpolating in the residual domain. Experimental results demonstrate that our algorithm can outperform state-of-the-art depth map upsampling algorithms.

We propose a novel image feature for textureless object detection. The feature is based on quantized gradient orientations those have been shown to be robust to cluttered backgrounds and illumination changes. We make this feature robust to the appearance changes of a targeted object itself induced by its transformations and small deformations. In our proposed method, we add small random values to the similarity transformation parameters and synthesize many model images. Then quantized orientations are extracted on these images and the orientations are cumulated at each pixel. The frequencies of selected features are utilized as weights when calculating scores. Our proposed feature is evaluated on publicly available dataset and achieve top-class performance both in speed and detection accuracy compared to state-of-the-art techniques.

Linear discrimination analysis (LDA) is one of the most popular feature extraction and classifier design techniques. It maximizes the Fisher-ratio between between-class scatter matrix and within-class scatter matrix under a linear transformation, and the transformation is composed of the generalized eigenvectors of them. However, Fisher criterion itself can not decide the optimum norm of transformation vectors for classification. In this paper, we show that actually the norm of the transformation vectors has strong influence on classification performance, and we propose a novel method to estimate the optimum norm of LDA under the ranking loss, re-weighting LDA. On artificial data and real databases, the experiments demonstrate the proposed method can effectively improve the performance of LDA classifiers. And the algorithm can also be applied to other LDA variants such as non parametric discriminant analysis (NDA) to improve theirs performance further.

Online Boosting is an effective incremental learning method which can update weak classifiers efficiently according to the object being trackedt. It is a promising technique for online object tracking to adapt tothe appearance variations of objects during tracking process. However, proposed online-boosting based tracking methods update and select weak classifiers from fixed the offline learned weak classifiers, which might not be an optimal selection for object appearance variations. In this paper, we propose a new feature adjusting strategy for online boosting called Soft Decision Feature. We combine it with online real AdaBoost to achieve better tracking performance in scenes with human pose and posture variations. Experiment result demonstrates that it can successfully deal with the human posture variation scenes that conventional online boosting tracking methods fails to deal with.

Tracking objects in low frame rate (LFR) video or with abrupt motion poses two main difficulties which most conventional tracking methods can hardly handle: 1) poor motion continuity and increased search space; 2) fast appearance variation of target and more background clutter due to increased search space. In this paper, we address the problem from a view which integrates conventional tracking and detection, and present a temporal probabilistic combination of discriminative observers of different lifespans. Each observer is learned from different ranges of samples, with different subsets of features, to achieve varying levels of discriminative power at varying cost. An efficient fusion and temporal inference is then done by a cascade particle filter which consists of multiple stages of importance sampling. Experiments show significantly improved accuracy of the proposed approach in comparison with existing tracking methods, under the condition of LFR data and abrupt motion of both target and camera.

Tracking object in low frame rate video or with abrupt motion poses two main difficulties which conventional tracking methods can barely handle: 1) poor motion continuity and increased search space; 2) fast appearance variation of target and more background clutter due to increased search space. In this paper, we address the problem from a view which integrates conventional tracking and detection, and present a temporal probabilistic combination of discriminative observers of different lifespans. Each observer is learned from different ranges of samples, with different subsets of features, to achieve varying level of discriminative power at varying cost. An efficient fusion and temporal inference is then done by a cascade particle filter which consists of multiple stages of importance sampling. Experiments show significantly improved accuracy of the proposed approach in comparison with existing tracking methods, under the condition of low frame rate data and abrupt motion of both target and camera.

Scale invariant feature transform (SIFT) proposed by Lowe has been widely and successfully applied to object detection and recognition. However, the representation ability of SIFT features in face recognition has rarely been investigated systematically. In this paper, we proposed to use the person-specific SIFT features and a simple non-statistical matching strategy combined with local and global similarity on key-points clusters to solve face recognition problems. Large scale experiments on FERET and CAS-PEAL face databases using only one training sample per person have been carried out to compare it with other non person-specific features such as Gabor wavelet feature and local binary pattern feature. The experimental results demonstrate the robustness of SIFT features to expression, accessory and pose variations.

Robust visual tracking is always a challenging but yet intriguing problem owing to the appearance variability of target objects. In this paper we propose a novel method to handle large changes in appearance based on online real-value boosting, which is utilized to incrementally learn a strong classifier to distinguish between objects and their background. By incorporating online real boosting into a particle filter framework, our tracking algorithm shows a strong adaptability for different target objects which undergo severe appearance changes during the tracking process.

In recent years, boosting has been successfully applied to many practical problems in pattern recognition and computer vision fields such as object detection and tracking. As boosting is an offline training process with beforehand collected data, once learned, it cannot make use of any newly arriving ones. However, an offline boosted detector is to be exploited online and inevitably there must be some special cases that are not covered by those beforehand collected training data. As a result, the inadaptable detector often performs badly in diverse and changeful environments which are ordinary for many real-life applications. To alleviate this problem, this paper proposes an incremental learning algorithm to effectively adjust a boosted strong classifier with domain-partitioning weak hypotheses to online samples, which adopts a novel approach to efficient estimation of training losses received from offline samples. By this means, the offline learned general-purpose detectors can be adapted to special online situations at a low extra cost, and still retains good generalization ability for common environments. The experiments show convincing results of our incremental learning approach on challenging face detection problems with partial occlusions and extreme illuminations.

Correlation is one of the most widely used similarity measures in machine learning like Euclidean and Mahalanobis distances. However, compared with proposed numerous discriminant learning algorithms in distance metric space, only a very little work has been conducted on this topic using correlation similarity measure. In this paper, we propose a novel discriminant learning algorithm in correlation measure space, Correlation Discriminant Analysis (CDA). In this framework, based on the definitions of within-class correlation and between-class correlation, the optimum transformation can be sought for to maximize the difference between them, which is in accordance with good classification performance empirically. Under different cases of the transformation, different implementations of the algorithm are given. Extensive empirical evaluations of CDA demonstrate its advantage over alternative methods. 1.

We developed a fast and robust head pose and gaze estimation system. This system can detect facial points and estimate 3D pose angles and gaze direction under various conditions including facial expression changes and partial occlusion. We need only one face image as input and do not need special devices such as blinking LEDs or stereo cameras. Moreover, no calibration is needed. The system shows a 95% head pose estimation accuracy and 81% gaze estimation accuracy (when the error margin is 15 degrees). The processing time is about 15 ms/frame (Pentium4 3.2 GHz). Acceptable range of facial pose is within a yaw (left-right) of ±60 degrees and within a pitch (up-down) of ±30 degrees.

For applications based on facial image processing, pose variation is a difficult problem. In this paper, we propose a gender and age estimation system that is robust against pose variations. The acceptable facial pose range is a yaw (left-right) from -30 degrees to +30 degrees and a pitch (up-down) from -20 degrees to +20 degrees. According to our experiments on several large databases collected under real environments, the gender estimation accuracy is 84.8% and the age estimation accuracy is 80.9% (subjects are divided into 5 classes). The average processing time is about 70 ms/frame for gender estimation and 95 ms/frame for age estimation (Pentium4 3.2 GHz). The system can be used to automatically analyze shopping customers and pedestrians using surveillance cameras.

We propose a framework for face recognition performed in 3D space. A 3D facial model consisting of a sparse depth map is constructed from stereo images using isoluminance lines for the stereo matching. By searching for arcs whose radiuses are of certain ranges, we can locate the candidate irises very efficiently. After the pose of the face is detected, the 3D model is transformed into a canonical pose. Recognition is performed by calculating the mean differences in depth between the corresponding data points in the 3D test model and all the models in the database. The corresponding data points refer to a pair of closest data points in two 3D models. We show that even without using any 2D features, we can do face recognition depending on the depth information only. The 3D pose detecting algorithm makes it possible for our face recognition algorithm to be pose invariant.

The authors have been developing a novel method for programming a robot, called the assembly-plan-from-observation (APO) method. The APO method aims to build a system that has threefold capabilities. It observes a human performing an assembly task, it understands the task based on this observation, and it generates a robot program to achieve the same task. This paper concentrates on the APO's main loop, which is task recognition. Using object recognition results, the task recognition module determines what kind of assembly task is performed. A previous system, recognizes assembly tasks which only handle polyhedral objects. The system reported here handles curved objects and other mechanical contacts as well. The authors define task models for these cases and show that task models are useful in recognizing assembly tasks, and that it is possible to generate robot motion commands for repeating the same assembly task.