A robust method for multi-level video representation based on the mean shift analysis (MSA) of low-level visual features is proposed in this paper. By tuning the bandwidth of MSA, video representation from the coarse level to the fine level can be achieved. This representation form provides a flexible scheme for content-based video analysis such as summarization, classification, and retrieval. Compared with the conventional k-means or fuzzy c-means algorithms, our method can adjust the resolution of representation in a more straightforward way, and is more robust since it does not need to initialize the cluster centers

We propose a statistical approach to estimate object motion with MPEG motion vectors. A model with two normal distribution terms is applied to represent the simplified object motion. One term models the noise embedded in the motion vector field produced at the encoding stage and the other term models the randomness of the true object motion. Experiments with vehicle motion estimation from MPEG traffic video are used to evaluate the proposed algorithm. The influence of time window, frame size and reference frame distance are investigated. The vehicle speeds can be estimated with a high accuracy of up to 85-92%.

Content-based video analysis calls for efficient video representation. In this paper, a novel multi-level representation of video is proposed based on the principle components derived from low-level visual features. It can characterize the video content from the coarse level to the fine level according to its intrinsic structure. This representation form provides a flexible scheme for video content analysis such as summarization, classification, and retrieval. A newly proposed subspace method, kernel based PCA, is explored to achieve this conveniently. The application in keyframe extraction is investigated to demonstrate the benefits of this representation.

In this paper, we proposed a robust moving video object segmentation algorithm using features in the MPEG compressed domain. We first cluster the motion vectors and produce a motion mask. Then, a difference mask at 8 x 8 block size is extracted from the DC image by background subtraction method. Finally, the motion mask and the difference mask are combined conditionally to generate the final object mask based on a set of rules that are application specified and is obtained with learning or heuristic based methods. The experimental results show that this object segmentation scheme is more robust than those using DC image or motion vectors only.

In this demonstration, we present a unified framework for semantic shot classification in sports videos. Unlike previous approaches, which focus on clustering by aggregating shots with similar low-level features, the proposed scheme makes use of domain knowledge of specific sport to perform a top-down video shot classification, including identification of video shots classes for each sport, and supervised learning and classification of given sports video with low-level and middle-level features extracted from the sports video. It's observed that for each sport we can predefine a small number of semantic shot classes, 5--10, which cover 90 to 95 % of sports broadcasting video. With supervised learning method, we can map the low-level features to middle-level semantic video shot attributes such as dominant object motion (a player), camera motion patterns, and court shape, etc. On the basis of the appropriate fusion of those middle-level shot attributes, we classify video shots into the predefined video shot classes, each of which has a clear semantic meaning. The proposed method has been tested over 3 types of sports videos: tennis, basketball, and soccer. Good classification results ranging from 80~95% have been achieved. The proposed framework provides a generic solution for sports video semantic shot classification, which can be adapted to a new sport type easily. With correctly classified sports video shots further structural and temporal analysis will be greatly facilitated.