Television news has become the predominant way of understanding the world around us, but individual news broadcasters can frame or mislead an audience's understanding of political and social issues. We are developing a computer system that can automatically identify highly biased television news and encourage audiences to seek news stories from contrasting viewpoints. But can computers identify the ideological perspective from which a news video was produced? We propose a method based on an empathic pattern of visual concepts: news broadcasters holding contrasting ideological beliefs appear to emphasize different subsets of visual concepts. We formalize the emphatic patterns and propose a statistical model. We evaluate the proposed model on a large broadcast news video archive with promising experimental results.

A number of researchers have been building high-level semantic concept detectors such as outdoors, face, building, to help with semantic video retrieval. Our goal is to examine how many concepts would be needed, and how they should be selected and used. Simulating performance of video retrieval under different assumptions of concept detection accuracy, we find that good retrieval can be achieved even when detection accuracy is low, if sufficiently many concepts are combined. We also derive suggestions regarding the types of concepts that would be most helpful for a large concept lexicon. Since our user study finds that people cannot predict which concepts will help their query, we also suggest ways to find the best concepts to use. Ultimately, this paper concludes that "concept-based" video retrieval with fewer than 5000 concepts, detected with a minimal accuracy of 10% mean average precision is likely to provide high accuracy results in broadcast news retrieval.

In this paper we investigate how to automatically determine if two document collections are written from different perspectives. By perspectives we mean a point of view, for example, from the perspective of Democrats or Republicans. We propose a test of different perspectives based on distribution divergence between the statistical models of two collections. Experimental results show that the test can successfully distinguish document collections of different perspectives from other types of collections.

In contrast to traditional video retrieval that represents visual content with low-level features (e.g. color and texture), emerging concept-based video retrieval allows users to search video archives by specifying a limited number of high-level concepts (e.g. outdoors and car). Recent studies have demonstrated the feasibility of concept-based retrieval, but a fundamental question remains: what kinds of concepts should we index? We analyze a large video archive annotated with more than a thousand high-level concepts, and develop guidelines for choosing concepts of high utility to video retrieval

Wearable physiological sensors can provide a faithful record of a patient's physiological states without constant attention of caregivers. A computer program that can infer human activities from physiological recordings will be an valuable tool for physicians. In this paper we investigate to what extent current machine learning algorithms can correctly identify human activities from physiological sensors. We further identify two challenges that developers need to address. The first problem is that the labels of training data are inevitably noisy due to difficulties of annotating thousands hours of data. The second problem lies in the continuous nature of human activities, which violates the independence assumption made by many learning algorithms. We approach the first problem of noisy labeling in the multiple-label framework, and develop a conditional Markov models to take temporal context into consideration. We evaluate the proposed methods on 12,000 hours of the physiological recordings. The results show that support vector machines are effective to identify human activities from physiological signals, and efforts of disambiguating noisy labels are worthwhile

We present an efficient system for video search that maximizes the use of human bandwidth, while at the same time exploiting the machine's ability to learn in real-time from user selected relevant video clips. The system exploits the human capability for rapidly scanning imagery augmenting it with an active learning loop, which attempts to always present the most relevant material based on the current information. Two versions of the human interface were evaluated, one with variable page sizes and manual paging, the other with a fixed page size and automatic paging. Both require absolute attention and focus of the user for optimal performance. In either case, as users search and find relevant results, the system can invisibly re-rank its previous best guesses using a number of knowledge sources, such as image similarity, text similarity, and temporal proximity. Experimental evidence shows a significant improvement using the combined extremes of human and machine power over either approach alone.

Abstract Named entities are major constituents of a document but are usually unknown words. This work proposes a systematic way of dealing with formulation, transformation, translation, and transliteration of multilingual‐named entities. The rules and similarity matrices for translation and transliteration are learned automatically from parallel‐named‐entity corpora. The results are applied in cross‐language access to collections of images with captions. Experimental results demonstrate that the similarity‐based transliteration of named entities is effective, and runs in which transliteration is considered outperform the runs in which it is neglected.

No abstract available.

Multimedia corpora increasingly consist of data from multiple sources, with different characteristics that can be exploited by specialized applications. This paper focuses on video classification over multiple-source collections, and addresses the question whether classifiers should train from individual sources or from a full data set across all sources. If training separately, how can rank lists from different sources be merged effectively? We formulate the problem of merging ranked lists as learning a function mapping from local scores to global scores, and propose a learning method based on logistic regression. In our experiments we find that source characteristics are very important for video classification. Moreover, our method of learning mapping functions performs significantly better than merging methods without explicitly learning the mapping junctions.

Broadcast news programs are well-structured video, and timing can be a strong predictor for specific types of news reports. However, learning a classifier using timing features may not be an easy task when training data are noisy. We approach the problem from the generative model perspective, and approximate the class density in a non-parametric fashion. The results show that timing is a simple but extremely effective feature, and our method can achieve significantly better performance than a discriminative classifier.

Web image retrieval is a challenging task that requires efforts from image processing, link structure analysis, and Web text retrieval. Since content-based image retrieval is still considered very difficult, most current large-scale Web image search engines exploit text and link structure to "understand" the content of the Web images. However, local text information, such as caption, filenames and adjacent text, is not always reliable and informative. Therefore, global information should be taken into account when a Web image retrieval system makes relevance judgment. We propose a re-ranking method to improve Web image retrieval by reordering the images retrieved from an image search engine. The re-ranking process is based on a relevance model, which is a probabilistic model that evaluates the relevance of the HTML document linking to the image, and assigns a probability of relevance. The experiment results showed that the re-ranked image retrieval achieved better performance than original Web image retrieval, suggesting the effectiveness of the re-ranking method. The relevance model is learned from the Internet without preparing any training data and independent of the underlying algorithm of the image search engines. The re-ranking process should be applicable to any image search engines with little effort.

Video classification is the first step toward multimedia content understanding. When video is classified into conceptual categories, it is usually desirable to combine evidence from multiple modalities. However, combination strategies in previous studies were usually ad hoc. We investigate a meta-classification combination strategy using Support Vector Machine, and compare it with probability-based strategies. Text features from closed-captions and visual features from images are combined to classify broadcast news video. The experimental results show that combining multimodal classifiers can significantly improve recall and precision, and our meta-classification strategy gives better precision than the approach of taking the product of the posterior probabilities.