This paper conducts a formal study of the shot boundary detection problem. First, a general formal framework of shot boundary detection techniques is proposed. Three critical techniques, i.e., the representation of visual content, the construction of continuity signal and the classification of continuity values, are identified and formulated in the perspective of pattern recognition. Meanwhile, the major challenges to the framework are identified. Second, a comprehensive review of the existing approaches is conducted. The representative approaches are categorized and compared according to their roles in the formal framework. Based on the comparison of the existing approaches, optimal criteria for each module of the framework are discussed, which will provide practical guide for developing novel methods. Third, with all the above issues considered, we present a unified shot boundary detection system based on graph partition model. Extensive experiments are carried out on the platform of TRECVID. The experiments not only verify the optimal criteria discussed above, but also show that the proposed approach is among the best in the evaluation of TRECVID 2005. Finally, we conclude the paper and present some further discussions on what shot boundary detection can learn from other related fields

Image annotation has been an active research topic in recent years due to its potentially large impact on both image understanding and Web image search. In this paper, we target at solving the automatic image annotation problem in a novel search and mining framework. Given an uncaptioned image, first in the search stage, we perform content-based image retrieval (CBIR) facilitated by high-dimensional indexing to find a set of visually similar images from a large-scale image database. The database consists of images crawled from the World Wide Web with rich annotations, e.g. titles and surrounding text. Then in the mining stage, a search result clustering technique is utilized to find most representative keywords from the annotations of the retrieved image subset. These keywords, after salience ranking, are finally used to annotate the uncaptioned image. Based on search technologies, this framework does not impose an explicit training stage, but efficiently leverages large-scale and well-annotated images, and is potentially capable of dealing with unlimited vocabulary. Based on 2.4 million real Web images, comprehensive evaluation of image annotation on Corel and U. Washington image databases show the effectiveness and efficiency of the proposed approach.

Two open problems remain unsolved in the content based video retrieval area. Firstly, how to find useful information and express it by different features. Secondly, how to fuse the heterogeneous information together to boost the retrieval performance beyond any single component. The paper presents two kinds of timing information and their use in concept detection in news video, and a novel non-linear information fusion method to combine timing cues with other information from different sources. Experiments on the TRECVID 2004 dataset show that timing cues can boost performance when combined with other information.

In this paper, we propose a unified shot boundary detection framework by extending the previous work of graph partition model with temporal constraints. To detect both the abrupt transitions (CUTs) and gradual transitions (GTs, excluding fade out/in) in a unified way, we incorporate temporal multi-resolution analysis into the model. Furthermore, instead of ad-hoc thresholding scheme, we construct a novel kind of feature to characterize shot transitions and employ support vector machine (SVM) with active leaning strategy to classify boundaries and non-boundaries. Extensive experiments have been carried out on the platform of TRECVID benchmark. The experimental results show that the proposed framework outperforms some others and achieves satisfactory results.

The Yeung-Mintzer (Y-M) image authentication scheme has been well studied. Several vulnerabilities and modified schemes to fix them have been reported. We propose a novel oracle attack on the Y-M scheme and its variations. Our attack is very different from the previously proposed attacks. A single authenticated image plus access to a verifier (oracle) is enough in our attack. The verifier returns if a testing image is authentic or not. Locations of tampered pixels are not needed. To launch the attack, a single pixel is modified and the resulting image is sent to the verifier. Observation of outputs of the verifier is used to deduce the secret mapping functions and the embedded logo within an uncertainty of two possibilities. The deduced mapping functions are then used to modify the content of an authenticated image without detection or to authenticate an arbitrary image of the same size. Note that the logo is not used in the forgery so sophisticated protection of the logo cannot thwart the attack. Our attack is very efficient. Only 255 trials are needed to attack an 8-bit grayscale image and 765 trials for a 24-bit color image. The proposed attack can also be applied to attack pixel-wise variations of the Y-M scheme proposed to fix the previously reported vulnerabilities.

To improve the naturalness of reconstructed speech, nonlinear speech models are paid more and more attention in recent years. A nonlinear speech model for speech synthesis based on support vector machine (SVM) is presented firstly. After speech signal is embedded into phase space, nonlinear map in the model is obtained with support vector regression. It is shown in the experiments that for some pieces of speech, not only can speech be perfectly reconstructed by the system, but also jitter and shimmer in the original signal is preserved. However, the output of the system is quite different from the original one for other pieces. The reason is that the sub-bands with different frequency in the original signal can not be perfectly described by a SVM-based autoregressive model trained with one set of training parameters. Consequently, a multi-band model is then proposed. After the original speech is decomposed into several bands through wavelet packet decomposition, a nonlinear dynamical model based on SVM is constructed for each sub-band signal. It is shown in the experiments that the stability of such system is improved.

This paper presents a new rule induction algorithm based on the extension matrix theory, and uses it to learn prosodic phrasing rules automatically for Chinese text-to-speech. Firstly, the basic idea of our algorithm is introduced. Secondly, we collected 937 sentences from news programs and built a corpus for modeling Chinese prosody, a group of feature variables are also proposed. Lastly, the data is divided into two parts: training set and test set, and the experimental results show that our method achieves higher comprehensibility, better accuracy and fewer rules than other algorithms. And the generated rules are quite similar to hand-crafted ones.

The World Wide Web has created major a innovation in the way knowledge is transferred. Since 1998, we have offered a Web-based multimedia technology course for on-campus and off-campus graduates in our country. In this paper, we describe the teaching and learning system we have developed, together with the model for teaching and learning, and we report the results of using it over the past three years.

A novel method of relevance feedback is presented based on support vector machine learning in the content-based image retrieval system. A SVM classifier can be learned from training data of relevance images and irrelevance images marked by users. Using the classifier, the system can retrieve more images relevant to the query in the database efficiently. Experiments were carried out on a large-size database of 9918 images. It shows that the interactive learning and retrieval process can find correct images increasingly. It also shows the generalization ability of SVM under the condition of limited training samples.

This paper describes several experiments designed to train prosodic phrasing models for Chinese TTS systems and to investigate the underlying rules that control Chinese prosody. First, we collected 559 sentences from news programs and built a large corpus for modeling Chinese prosody. Second, we selected 20 features and used classification and regression trees (CART) and transformational rule-based learning (TRBL) techniques to generate phrasing rules automatically. Lastly, we propose a computer aided error-driven method of designing rule templates, and integrate it into the TRBL algorithm. The experimental results show that we achieve a high success rate of 94.5%, and we also get a set of well comprehensible rule templates which may give us insights into the relationship between Chinese syntax and p rosody.

More and more universities and colleges are providing online courses not only for on-campus students but also for off-campus students. Tutors have to consider the differences between on- and off-campus students in order to improve effective instruction. Comparisons are made in this paper between on- and off-campus performances in online learning from four areas: learning time, path of browsing courseware, intercommunication and adaptability towards online learning. The last two areas are emphasized. Multiple approaches were adopted to collect data, which include questionnaires, posted documents, online logs, interviews and observations. This study shows that the rush time of online learning, paths of browsing courseware and favourite intercommunication means of on- and off-campus students are similar. But there are also some differences between these two groups such as competence of self-learning, enthusiasm of interpersonal exchange, dependence on tutors, feeling of learning stress, etc.