In this paper, we propose the Interest Meter (IM), a system making the computer conscious of user's reactions to measure user's interest and thus use it to conduct video summarization. The IM takes account of users' spontaneous reactions when they view videos. To estimate user's viewing interest, quantitative interest measures are devised based on the perspectives of attention and emotion. For estimating attention states, variations of user's eye movement, blink, and head motion are considered. For estimating emotion states, facial expression is recognized as positive or neural emotion. By combining characteristics of attention and emotion by a fuzzy fusion scheme, we transform users' viewing behaviors into quantitative interest scores, determine interesting parts of videos, and finally concatenate them as video summaries. Experimental results show that the proposed concept “editing by viewing” works well and may provide a promising direction to consider the human factor in video summarization.

In this paper, we propose the Interest Meter (IM), a system making computer conscious of user's reactions, to measure user's interest in real time. The Interest Meter takes account of users' spontaneous reactions when users interact with computers. In this work, we analyze variations of user's eye movement, blink, head motion, and facial expression. Furthermore, we propose an algorithm to combine those signals into interest score and determine important parts of video shots when people watch raw home videos. Experimental result shows that this new type of editing mechanism can effectively generate home video summaries.

Instead of using global-appearance information for visual tracking, as adopted by many methods, we propose a tracking-by-parts (TBP) approach that uses partial appearance information for the task. The proposed method considers the collaborations between parts and derives a probability propagation framework by encoding the spatial coherence in a Bayesian formulation. To resolve this formulation, a TBP particle-filtering method is introduced. Unlike existing methods that only use the spatial-coherence relationship for particle-weight estimation, our method further applies this relationship for state prediction based on system dynamics. Thus, the part-based information can be utilized efficiently, and the tracking performance can be improved. Experimental results show that our approach outperforms the factored-likelihood and particle reweight methods, which only use spatial coherence for weight estimation.

In this paper, we propose a new approach, appearance-guided particle filtering (AGPF), for high degree-of-freedom visual tracking from an image sequence. This method adopts some known attractors in the state space and integrates both appearance and motion-transition information for visual tracking. A probability propagation model based on these two types of information is derived from a Bayesian formulation, and a particle filtering framework is developed to realize it. Experimental results demonstrate that the proposed method is effective for high degree-of-freedom visual tracking problems, such as articulated hand tracking and lip-contour tracking.

Although extremely tedious, collecting multiangle facial prototypes of new users is essential for face-related applications such as detection, tracking, recognition, and pose estimation. To reduce the workload, a user-friendly interface that is designed to collect multiangle facial prototypes in a fast and tractable manner was proposed. A user-friendly interface called dynamic cursor is designed to guide users in the operation of orientation-enforced video acquisition. A new problem, which is called the orientation-enforced face-detection problem, is defined and solved for multiangle facial-prototype collection. The method proposed in this correspondence can help in building a fast and user-friendly face-registration module for face-related applications.

We propose a model-based tracking method, called appearance-guided particle filtering (AGPF), which integrates both sequential motion transition information and appearance information. A probability propagation model is derived from a Bayesian formulation for this framework, and a sequential Monte Carlo method is introduced for its realization. We apply the proposed method to articulated hand tracking, and show that it performs better than methods that only use either sequential motion transition information or only use appearance information.

With the advances in imaging technologies for robot or machine vision, new imaging devices are being developed for robot navigation or image-based rendering. However, to satisfy some design criterion, such as image resolution or viewing ranges, these devices are not necessarily being designed to follow the perspective rule and, thus, the imaging rays may not pass through a common point. Such generalized imaging devices may not be perspective and, therefore, their poses cannot be estimated with traditional techniques. In this paper, we propose a systematic method for pose estimation of such a generalized imaging device. We formulate it as a nonperspective n point (NPnP) problem. The case with exact solutions, n=3, is investigated comprehensively. Approximate solutions can be found for n>3 in a least-squared-error manner by combining an initial-pose-estimation procedure and an orthogonally iterative procedure. This proposed method can be applied not only to nonperspective imaging devices but also perspective ones. Results from experiments show that our approach can solve the NPnP problem accurately.

Summary form only given. A new representation method, the multistage color-depth panoramic map (or panomap), is proposed for compressing 3D graphic objects. The idea of the proposed method is to transform a 3D graphic object, including both the shape and color information, into a single image. Existing image compression techniques can then be applied for compressing the panomap structure, which can achieve a highly efficient representation due to its regularity. In our experiments, compressing the color part of a CMP with a lossy method (JPEG) and the depth part with a lossless one (PNG) achieves good reconstruction quality with low bit rates.

In this paper we present a systematic method for pose estimation of such a generalized imaging device. We reformulate it as a non-perspective n point (NPnP) problem. The case with exact solutions, for n=3, is investigated comprehensively. Approximate solutions can also be found for n>3 with our approach in a least-squared-error manner. The proposed method can be used not only to perspective imaging devices, but also non-perspective ones.