Current state-of-the-art news video retrieval systems mainly focus on automated speech recognition (ASR) text to perform retrieval. This paradigm greatly affects retrieval performance as ASR text alone is not sufficient to provide an accurate representation of the entire news video. In this paper, we describe our automated retrieval framework which fuses the multimodal features and event structures present in news video to support precise news video retrieval. The contributions of this paper are: (a) we uncover and employ temporal event clusters to provide additional information during story level retrieval; and (b) we integrate other modality features with text features and incorporate event clusters for pseudo relevance feedback (PRF) in shot level re-ranking. Experiments performed on video search task using the TRECVID 2005/06 dataset show that the proposed approach is effective.

Existing video research incorporates the use of relevance feedback based on user-dependent interpretations to improve the retrieval results. In this paper, we segregate the process of relevance feedback into 2 distinct facets: (a) recall-directed feedback; and (b) precision-directed feedback. The recall-directed facet employs general features such as text and high level features (HLFs) to maximize efficiency and recall during feedback, making it very suitable for large corpuses. The precision-directed facet on the other hand uses many other multimodal features in an active learning environment for improved accuracy. Combined with a performance-based adaptive sampling strategy, this process continuously re-ranks a subset of instances as the user annotates. Experiments done using TRECVID 2006 dataset show that our approach is efficient and effective.

Earlier research in news video has been focusing mainly on improving retrieval accuracies given the limited amount of extractable video semantics. In this paper, we propose an enhancement to news video searching by leveraging extractable video semantics coupled with relevant external information resources to support event-based analysis; leading to discovery of topic hierarchy for browsing key events and supporting question answering (QA). We introduce topic browsing based on news structures obtained through hierarchical clustering and threading, with emphasis on interesting events determined by measuring the amount of "web activities" on these events on Blog sites. For QA, we employ extensive query analysis to obtain various query features in addition to the topic hierarchical structures to answer both context-oriented and visual-oriented questions. Our main contributions includes: (a) combining multimodal event information extracted from news video, web news articles and news blogs to support event analysis, (b) introducing topic evolution browsing based on users' interest and (c) extending QA on top of topic hierarchy to handle various types of specialized video queries. Experiments performed on 70 hours of multilingual news from TRECVID 2005 dataset shows that the proposed approach is effective and appealing to users.

Person identification is very important in the domain of multimedia news as it is often the focus of events in news stories and interest of searchers. However, this detection is impeded by the imprecise audio/visual analysis tools. In this paper, we describe a multimodal and multi-faceted approach to person-x detection in news video. We make use of multimodal features extracted from text, visual and audio inherent in news video. We also incorporate multiple external sources of news from Web and parallel news archives to extract location and temporal profile of the persons. We call this second source of information the multi-faceted context. The multimodal, multi-faceted information is then fused using a rank boosting approach. Experiments on TRECVID 2003 and 2004 search queries demonstrate that our approach is effective