In this paper, we propose a novel vehicle tracking system under a surveillance camera. The proposed system tracks vehicles by using constrained multiple-kernel facilitated with Kalman filtering, and then continuously updates the position and the orientation by adopting a systematically built 3-D vehicle model in an evolutionary computing framework. The proposed system can thus successfully track vehicles under occlusion as facilitated by the obtained 3-D geometry of vehicles. Experimental results have shown the favorable performance of the proposed system, which can successfully track vehicles while maintaining the knowledge of 3-D vehicle geometry.

This paper proposes a sender-driven peer selection scheme, including estimation of packet loss propagation, evaluation of peers' contributions, and peer selection based on child-peers' contributions, for mesh-based peer-to-peer (P2P) video streaming systems. The proposed packet loss propagation model takes into account the link packet drop rate, peer dynamics, and forward error correction protection to capture the heterogeneous packet loss behavior of individual substreams transmitted over a mesh network. The evaluation of candidate peers' contributions is modeled through Markov random fields to significantly reduce complexity. Simulation results demonstrate that our peer selection scheme significantly mitigates packet loss in a mesh-based P2P network, compared to other state-of-the-art schemes.

This paper proposes a distributed packet protection mechanism that can minimize the packet loss probability for mesh based P2P video streaming systems. The proposed scheme combines a peer selection method with forward error correction (FEC) codes. The parent peers select the child peers, which can achieve the minimal packet loss probability compared to other candidate child peers, to transmit the FEC redundant substream. Moreover, the proposed scheme utilizes a packet loss model to estimate the packet loss probability in a mesh based P2P network. The packet loss propagation among peers is modeled through Markov random field (MRF). Simulation results demonstrate that our scheme can effectively mitigate packet loss in a mesh-based P2P network.

In baseball games, different release points of pitchers form several kinds of pitching styles. Different pitching styles possess individual advantages. This paper presents a novel pitching style recognition approach for automatic generation of game information and video annotation. First, an effective object segmentation algorithm is designed to compute the body contour and extract the pitcher's body. Then, star skeleton is used as the representative descriptor of the pitcher posture for pitching style recognition. The proposed approach has been tested on broadcast baseball video and the promising experimental results validate the robustness and practicability.

This paper proposes a model to estimate the packet loss probability in a mesh-based P2P network. Because of the irregular mesh structure, packet loss estimation for a mesh-based P2P network is more complicated than that in a tree-based network. The proposed model takes into account the channel packet drop rate, peer dynamics, and FEC protection to capture the heterogeneous packet loss behavior of individual video substreams transmitted over the irregular transmission paths of a mesh network. The simulation results show that the proposed packet loss model can accurately estimate the packet loss in a mesh-based P2P network. Based on the proposed model, we also propose a peer selection mechanism which can effectively mitigate packet loss propagation by selecting at a parent-peer the candidate child-peers that can achieve the minimal packet loss probability compared to others, to transmit the FEC redundant substream.

We extend the object class invariant (OCI) model to age classification, for pose variant and occluded faces. With the OCI model, we first localize faces from images captured in arbitrary views, and then determine the most distinctive features. Relationships between feature points and the invariant vector are described in terms of geometry and appearance information, in the form of a probabilistic model. In contrast to previous works on age classification/estimation, we emphasize that this method is especially useful for faces captured in real-world situations.

This paper proposes a two-step prototype-face-based scheme of hallucinating the high-resolution detail of a low-resolution input face image. The proposed scheme is mainly composed of two steps: the global estimation step and the local facial-parts refinement step. In the global estimation step, the initial high-resolution face image is hallucinated via a linear combination of the global prototype faces with a coefficient vector. Instead of estimating coefficient vector in the high-dimensional raw image domain, we propose a maximum a posteriori (MAP) estimator to estimate the optimum set of coefficients in the low-dimensional coefficient domain. In the local refinement step, the facial parts (i.e., eyes, nose and mouth) are further refined using a basis selection method based on overcomplete nonnegative matrix factorization (ONMF). Experimental results demonstrate that the proposed method can achieve significant subjective and objective improvement over state-of-the-art face hallucination methods, especially when an input face does not belong to a person in the training data set.

We introduce local feature points to achieve face clustering for consumer photos. After combining eigenfaces with context information like clothes, we further investigate the usage of local feature points to match face images. The relationships between face images are constructed by feature matching and then described as a graph. Outliers in the results of preliminary clustering are detected and are re-clustered according to matching characteristics. We report complete performance comparison for different datasets and show that the proposed method has superior performance than conventional approaches.

This paper points out that different local feature points provide different impacts to near-duplicate detection and related applications. Aiming to automatic representative photo selection, we develop three feature classification methods, i.e., point-based, region-based, and pLSA-based classification, to differentiate local feature points described by SIFT descriptors. We investigate the performance of these classification methods, and discuss how they influence near-duplicate detection and extended applications. Experiments show that, with effective feature classification, more accurate representative selection results can be achieved.

For consumer photos, this work clusters faces with large variations in lighting, pose, and expression. After matching face images by local feature points, we transform matching situations into a novel representation called visual sentences. Then, visual language models are constructed to describe the dependency of image patches on faces. With the probabilistic framework, we develop a clustering algorithm to group the same individual's face images into the same cluster. An interesting observation about evaluating face clustering performance is proposed, and we demonstrate the superiority of the proposed visual language model approach.

We address the problem of human action understanding of the upper human body from video sequences. Time-sequential images expressing human actions are transformed to sequences of feature vectors containing the configuration of the human body. A human is modeled as a collection of body parts, linked in a kinematic structure. The relation of the joints is used to estimate the human pose. A proposed layered HMM framework decomposes the human action recognition problem into two layers. The first layer models the actions of two arms individually from low-level features. The second layer models the interrelationship of two arms as an action. Experiments with a set of six types of human actions demonstrate the effectiveness of our proposed scheme, and the comparisons with other HMM systems show the robustness.

This paper presents a novel method for reducing a baseball video segment from one batter to next batter into a more compact pitch-by-pitch video by pitching ball trajectory detection. The pitch-by-pitch video shows the complete pitching and batting process and largely reduces the source video data, making pitching analysis of broadcast baseball sequences an easier task. The proposed method has been tested for several long sequences, and promising results are reported.

This paper proposes a music accompaniment system capable of catching the tempo of music. The original signal is first reduced to a detection function revealing the pulses of music. To induce the tempo and locate beats, a Fourier analysis-based algorithm with high practicality and generality is designed that it is robust to beat strength and not restricted to specific music genres. The regularity and periodicity of music beats are extracted and even the missing beats are recovered. The performance is validated using a comprehensive testing data set, and results in both formal objective experiments and subjective listening evaluations show convincible performance. Moreover, an interactive interface is designed that users are allowed to select an instrument to accompany the music based on the obtained tempo.

Forward error correction codes have been shown to be a feasible solution either in application layer or in link layer to fulfill the need of quality of service for multimedia streaming over the fluctuant channels. In this paper, we propose FEC-distortion optimization algorithms to efficiently utilize the bandwidth for better video quality. The optimization criterions are based on the unequal error protection by taking account of the error drifting problems from both temporal motion compensation and inter-layer prediction of H.264/MPEG-4 AVC scalable video coding. Also, it can adapt to the content-dependent quality contribution of each video frame in a video layer. Lightweight error-concealment is also incorporated with the proposed algorithms for better H.264 SVC streaming. For some applications where either computation might be the bottleneck or the upper bound of non-decodable probability of each video layer is specified, alternative bandwidth allocation algorithm is provided with the trade-off of slight quality degradation.