Categorizing web-based videos is an important yet challenging task. The difficulties arise from large data diversity within a category, lack of labeled data, and degradation of video quality. This paper presents a large scale video taxonomic classification scheme (with more than 1000 categories) tackling these issues. Taxonomic structure of categories is deployed in classifier training. To compensate for the lack of labeled video data, a novel method is proposed to adapt the web-text documents trained classifiers to video domain so that the availability of a large corpus of labeled text documents can be leveraged. Video content based features are integrated with text-based features to gain power in the case of degradation of one type of features. Evaluation on videos from hundreds of categories shows that the proposed algorithms generate significant performance improvement over text classifiers or classifiers trained using only video content based features.

Automatic categorization of videos in a Web-scale unconstrained collection such as YouTube is a challenging task. A key issue is how to build an effective training set in the presence of missing, sparse or noisy labels. We propose to achieve this by first manually creating a small labeled set and then extending it using additional sources such as related videos, searched videos, and text-based webpages. The data from such disparate sources has different properties and labeling quality, and thus fusing them in a coherent fashion is another practical challenge. We propose a fusion framework in which each data source is first combined with the manually-labeled set independently. Then, using the hierarchical taxonomy of the categories, a Conditional Random Field (CRF) based fusion strategy is designed. Based on the final fused classifier, category labels are predicted for the new videos. Extensive experiments on about 80K videos from 29 most frequent categories in YouTube show the effectiveness of the proposed method for categorizing large-scale wild Web videos.

Modeling and recognizing landmarks at world-scale is a useful yet challenging task. There exists no readily available list of worldwide landmarks. Obtaining reliable visual models for each landmark can also pose problems, and efficiency is another challenge for such a large scale system. This paper leverages the vast amount of multimedia data on the Web, the availability of an Internet image search engine, and advances in object recognition and clustering techniques, to address these issues. First, a comprehensive list of landmarks is mined from two sources: (1) ~20 million GPS-tagged photos and (2) online tour guide Web pages. Candidate images for each landmark are then obtained from photo sharing Websites or by querying an image search engine. Second, landmark visual models are built by pruning candidate images using efficient image matching and unsupervised clustering techniques. Finally, the landmarks and their visual models are validated by checking authorship of their member images. The resulting landmark recognition engine incorporates 5312 landmarks from 1259 cities in 144 countries. The experiments demonstrate that the engine can deliver satisfactory recognition performance with high efficiency.

We present a technical demonstration of a world-scale touristic landmark recognition engine. To build such an engine, we leverage ~21.4 million images, from photo sharing websites and Google Image Search, and around two thousand web articles to mine the landmark names and learn the visual models. The landmark recognition engine incorporates 5312 landmarks from 1259 cities in 144 countries. This demonstration gives three exhibits: (1) a live landmark recognition engine that can visually recognize landmarks in a given image; (2) an interactive navigation tool showing landmarks on Google Earth; and (3) sample visual clusters (landmark model images) and a list of 1000 randomly selected landmarks from our recognition engine with their iconic images.

We present a higher-level visual representation, visual synset, for object categorization. The visual synset improves the traditional bag of words representation with better discrimination and invariance power. First, the approach strengthens the inter-class discrimination power by constructing an intermediate visual descriptor, delta visual phrase, from frequently co-occurring visual word-set with similar spatial context. Second, the approach achieves better intra-class invariance power, by clustering delta visual phrases into visual synset, based their probabilistic dasiasemanticspsila, i.e. class probability distribution. Hence, the resulting visual synset can partially bridge the visual differences of images of same class. The tests on Caltech-101 and Pascal-VOC 05 dataset demonstrated that the proposed image representation can achieve good accuracies.

A typical automatic face recognition system is composed of three parts: face detection, face alignment and face recognition. Conventionally, these three parts are processed in a bottom-up manner: face detection is performed first, then the results are passed to face alignment, and finally to face recognition. The bottom-up approach is one extreme of vision approaches. The other extreme approach is top-down. In this paper, we proposed a Markovian stochastic mixture approach for combining bottom-up and top-down face recognition: face recognition is performed from the results of face alignment in a bottom-up way, and face alignment is performed based on the results of face recognition in a top-down way. By modeling the mixture face recognition as a stochastic process, the recognized person is decided probabilistically according to the probability distribution coming from the stochastic face recognition, and the recognition problem becomes that ldquowho the most probable person is when the stochastic process of face recognition goes on for an infinite long durationrdquo. This problem is solved with the theory of Markov chains by properly modeling the stochastic process of face recognition as a Markov chain. As conventional face alignment is not suitable for this mixture approach, discriminative face alignment is proposed. And we also prove that the Markovian mixture face recognition results only depend on discriminative face alignment, not on conventional face alignment. Our approach can surprisingly outperform the face recognition performance with manual face localization, which is demonstrated by extensive experiments.

The phenomenal growth of video on the Web and the increasing sparseness of meta information associated with it forces us to look for signals from the video content for search/information retrieval and browsing based corpus exploration. A large chunk of users' searching/browsing patterns are centered around people present in the video. Doing it at scale in videos remains hard due to a) the absence of labeled data for such a large set of people and b) the large variation of pose/illumination/expression/age/occlusion/quality etc in the target corpus. We propose a system that can learn and recognize faces by combining signals from large scale weakly labeled text, image, and video corpora. First, consistency learning is proposed to create face models for popular persons. We use the text-image co-occurrence on the web as a weak signal of relevance and learn the set of consistent face models from this very large and noisy training set. Second, efficient and accurate face detection and face tracking is applied. Last, the key faces in each face track is select by clustering to get compact and robust representation. The face tracks are further clustered to get more representative key faces and remove duplicate key faces. For each cluster of face tracks, a combination of majority voting and probabilistic voting is done with the automatically learned models. The effectiveness of our framework is demonstrated by results on image and video corpora, in which we can achieve 92.68% in 37 million images and 80% top-5-precision in 1500 hours videos.

In this paper, we present a model based on a multi-resolution, multi-source and multi-modal (M3) bootstrapping framework that exploits knowledge of sub-domains for concept detection in news video. Because the characteristics and distributions of data in different sub-domains are different, we model and analyze the video in each sub-domain separately using a transductive framework. Along with this framework, we propose a "pseudo-Vapnik combined error bound" to tackle the problem of imbalanced distribution of training data in certain segments of sub-domains. For effective fusion of multi-modal features, we utilize multi-resolution inference and constraints to permit evidences from different modal features to support each other. Finally, we employ a bootstrapping technique to leverage unlabeled data to boost the overall system performance. We test our framework by detecting semantic concepts in the TRECVID 2004 dataset. Experimental results demonstrate that our approach is effective.

A novel fast recursive minimum mean square error successive interference cancellation (MMSE-SIC) algorithm with optimal detection order for vertical Bell Laboratories layered space-time (V-BLAST) systems is proposed. In this algorithm, the MMSE Alter matrices and the optimal detection order are successively computed from the previously obtained filter matrices according to simple recursive pseudoinverse formulas, so that the algorithmic complexity is reduced significantly, especially for the practical number of transmit/receive antennas.

In this paper, we present the soft version of the Generalized Probabilistic Data Association (GPDA) algorithm, which has two implementation modes: successive GPDA (S-GPDA) and parallel GPDA (P-GPDA), and apply it to V-BLAST architecture employing joint iterative detection and decoding. In addition, we investigate the soft interference cancellation with minimum-mean-square-error criterion (SIC-MMSE) algorithm, and prove that it is equivalent to P-GPDA with one inner-loop (detector) iteration. Simulation results are provided to verify this conclusion. From the simulation results, we can also see the following findings when sufficient innerloop/outer-loop (detector/decoder) iterations are employed: 1). the performance of P-GPDA is a little better than that of S-GPDA. 2). P-GPDA with one inner-loop iteration, which provides a good approximation to the maximum a posteriori (MAP) detection performance, has better performance than P-GPDA with more inner-loop iterations.

In this paper, we propose a novel low-complexity recursive minimum mean square error successive interference cancellation (MMSE-SIC) algorithm with optimal detection order for vertical Bell Laboratories layered space-time (V-BLAST) systems. Different from the conventional V-BLAST detection algorithm, the proposed algorithm is capable of computing both the nulling vectors and the optimal detection order from the previous computational results via simple recursive methods, so that the algorithmic complexity is reduced significantly. Complexity and performance analyses show the proposed algorithm not only guarantees the optimal detection performance but is computationally more efficient than the existing algorithms.

Maximum-likelihood detection (MLD) algorithm is the optimal detection scheme for vertical Bell Laboratories layered space-time (V-BLAST) system. However, it is generally infeasible to implement MLD algorithm in practical systems due to its high computational complexity. In this paper, we utilize two parallel interference cancellation techniques including pre-cancellation algorithm and post-cancellation algorithm to simplify MLD algorithm to a computationally efficient suboptimal detection algorithm, herein called generalized parallel interference cancellation (GGPIC) algorithm. Numerical analysis shows GPIC algorithm can approximately achieve the MLD performance with considerably low and stable computational complexity. Since most steps of GPIC algorithm are parallel, it can be accelerated significantly via parallel processing methods. With near-optimal performance, stable and low complexity and fast processing speed, it is typically suitable for real-time high-speed systems.

The conventional detection algorithm for vertical Bell Laboratories layered space-time (V-BLAST) systems suffers from high computational complexity. In this paper, we develop a new fast recursive V-BLAST detection algorithm, called Greville-to-Inverse-Greville (GIG) algorithm. The core of this algorithm is a computationally efficient recursive pseudoinverse algorithm that can use the weight matrix of the first iteration to recursively compute all the other weight matrices. It can be considered as the inverse of the well-known Greville algorithm, and thus we call it Inverse Greville (IG) algorithm. By applying Greville algorithm to compute the weight matrix of the first iteration and IG algorithm to determine the others, we get the proposed GIG algorithm. Compared with the other existing fast algorithms, GIG algorithm has lower computational complexity and faster calculating speed for any number of transmit/receive antennas, even without any loss of performance.

Person identification is very important in the domain of multimedia news as it is often the focus of events in news stories and interest of searchers. However, this detection is impeded by the imprecise audio/visual analysis tools. In this paper, we describe a multimodal and multi-faceted approach to person-x detection in news video. We make use of multimodal features extracted from text, visual and audio inherent in news video. We also incorporate multiple external sources of news from Web and parallel news archives to extract location and temporal profile of the persons. We call this second source of information the multi-faceted context. The multimodal, multi-faceted information is then fused using a rank boosting approach. Experiments on TRECVID 2003 and 2004 search queries demonstrate that our approach is effective

In this paper, we propose double-orthogonal coded space-time-frequency spreading code-division multiple-access (DOC-STFS-CDMA) scheme for the downlink orthogonal frequency-division multiplexing (OFDM) systems employing multiple transmit and receive antennas. First, we introduce the so-called STFS-CDMA and the corresponding minimum mean-square error (MMSE) detection algorithms for three scenarios with different user information. Then, we propose a new class of spreading codes, called double-orthogonal code (DOC), to match the MMSE detection algorithms for STFS-CDMA system. With the special double orthogonality, DOC can effectively exploit space, time, and frequency diversity to enhance the performance of MMSE detectors, and provide the well-balanced signal-to-interference-and-noise ratio to different users. Finally, the performance gain of the proposed scheme over conventional schemes is shown by computer simulation.

With the increasing number of people who can afford to make videos to record their lives, home videos play a more and more important role in multimedia. Video abstraction is an efficient way to help review such a huge amount of home videos. A home video abstraction technique combining audio and video features is presented. The audio contents are firstly classified as silence, pure speech, non-pure speech, music and background sound using support vector machines (SVMs). Then, non-pure speech is further classified into song and other non-pure speech using SVM, and background sound is classified into laughter, applause, scream and others using hidden Markov models (HMMs). For video contents, motion level and blur degree are acquired. Finally, video segments containing special features, such as speech, laughter, song, applause, scream, and specified motion level and blur degree, are extracted as the main parts of the abstract. The remaining parts of the abstract are generated using key frame information. Experimental results show that the proposed algorithm can extract the desired parts of a home video to generate satisfactory video abstracts.

One new joint frame synchronization and carrier frequency offset estimation method for orthogonal frequency-division multiplexing (OFDM) system is proposed. The same training-symbol-block (TSB) is needed for both frame synchronization and carrier frequency offset estimation. The carrier frequency offset estimation including acquisition and tracking, and acquisition can be further divided into preacquisition and fine acquisition. As soon as frequency offset acquisition finished, timing synchronization is also performed at the same time. The acquisition range is as large as one half of overall signal bandwidth. The theoretical variance error lower bound for our frequency offset tracking algorithm is also derived in this paper.