Semantic high-level event recognition of videos is one of most interesting issues for multimedia searching and indexing. Since high-level events are usually domain-specific, a generic framework which can adapt itself to new domains without or with a few modifications is needed. To this end, this paper presents a generic framework for video event classification using temporal context of interval-based multimodal features. In the framework, a co-occurrence symbol transformation method is proposed to explore full temporal relations among multiple modalities in probabilistic HMM event classification. The results of our experiments on baseball video event classification demonstrate the superiority of the proposed approach.

Mining melody structure patterns from music query data is one of the most interesting issues of multimedia data mining. In this paper, we introduce a new kind of melody structure pattern, called emerging melody pattern (EMP), for knowledge discovery from music query data streams. EMPs are defined as music data items with melody item-sets whose support increase significantly from one sliding window to another window from streaming melody data sequences. An efficient data mining algorithm, called MEMPA (Mining Emerging Melody Pattern Algorithm), is proposed to discover all EMPs from music query data. In the framework of MEMPA, a prefix tree-based structure, called EMP-tree (Emerging Melody Pattern tree), is developed for storing EMPs effectively from current stream sliding windows. Experimental results show that the proposed algorithm is an efficient method for discovering all EMPs from streaming melody sequences.

Mining frequent melody structures from music data is one of the most important issues in multimedia data mining. In this paper, we proposed an efficient online algorithm, called BVMDS (Bit-Vector based Mining of Data Streams), to mine all frequent temporal patterns over sliding windows of music melody sequence streams. An effective bit-sequence representation is used in BVMDS to reduce the time and memory needed to slide the windows. An effective list structure is used to overcome the performance bottleneck of previous work, FTP-stream. Experiments show that the BVMDS algorithm outperforms FTP-stream algorithm, and just scans the streaming data once.

We address the problem of human action understanding of the upper human body from video sequences. Time-sequential images expressing human actions are transformed to sequences of feature vectors containing the configuration of the human body. A human is modeled as a collection of body parts, linked in a kinematic structure. The relation of the joints is used to estimate the human pose. A proposed layered HMM framework decomposes the human action recognition problem into two layers. The first layer models the actions of two arms individually from low-level features. The second layer models the interrelationship of two arms as an action. Experiments with a set of six types of human actions demonstrate the effectiveness of our proposed scheme, and the comparisons with other HMM systems show the robustness.

Despite a lot of research efforts in baseball video processing, little work has been done in analyzing the detailed process and ball movement of the batting content. This paper proposes a novel system to automatically summarize the progress of each batting in baseball videos. Utilizing the strictly-defined specifications of the baseball field, the system recognizes the spatial patterns in each frame and identifies what region of the baseball field is currently focused. Finally, an annotation string which abstracts the batting content is generated. With the annotation strings, the system is able to make descriptions and provide exploration for baseball videos, so that users can be given a further insight into the game quickly. The experiments on broadcast baseball videos of MLB and JPB show promising results.

Despite a lot of research efforts in sports video processing, little work was done in volleyball video analysis due to the high density of players on the court and the complicated overlapping of player-player or ball-player, which lead to great complexity of object tracking for advanced analysis. For ball detection and trajectory extraction in volleyball videos, this paper presents a physics-based scheme which utilizes the motion characteristics to extract ball trajectory from lots of moving objects. The experiments of ball tracking show encouraging results. Moreover, based on game-specific properties, the ball trajectory can be exploited to recognize set types for tactics inference and to detect basic actions in the volleyball game for close-up presentation.

This paper presents a novel method for reducing a baseball video segment from one batter to next batter into a more compact pitch-by-pitch video by pitching ball trajectory detection. The pitch-by-pitch video shows the complete pitching and batting process and largely reduces the source video data, making pitching analysis of broadcast baseball sequences an easier task. The proposed method has been tested for several long sequences, and promising results are reported.