Government agencies, corporations, and police departments are plagued by information overload. The inability of fully analyze fragments of data scattered across the organizations reduces productivity, and more and more of these fragments are being gathered every day thanks to tools like the Internet and digital audio/video recorders. However, since much of this information is stored in computer systems and networks, it is possible to develop tools that will automatically analyze, assemble and associate the fragments so that precious human resources can focus on the analysis and interpretation of just those fragments that may contain valuable insight. Media analytics for actionable intelligence (MAAI) is a combination of automatic, semi-automatic and manual tools which provide three basic levels of analysis: automatic services, manual feedback, and higher level mining (e.g. timeline, social network, hypothesis generation) which allow investigators/analysts to act more efficiently and accurately.

In this paper we describe a novel approach for jointly modeling the text and the visual components of multimedia documents for the purpose of information retrieval(IR). We propose a novel framework where individual components are developed to model different relationships between documents and queries and then combined into a joint retrieval framework. In the state-of-the-art systems, a late combination between two independent systems, one analyzing just the text part of such documents, and the other analyzing the visual part without leveraging any knowledge acquired in the text processing, is the norm. Such systems rarely exceed the performance of any single modality (i.e. text or video) in information retrieval tasks. Our experiments indicate that allowing a rich interaction between the modalities results in significant improvement in performance over any single modality. We demonstrate these results using the TRECVID03 corpus, which comprises 120 hours of broadcast news videos. Our results demonstrate over 14 % improvement in IR performance over the best reported text-only baseline and ranks amongst the best results reported on this corpus.

This paper describes a bootstrapping methodology for semi– automatic semantic annotation of a “mini–corpus” that is conventionally annotated manually to train an initial parser used in natural language understanding (NLU) systems. We propose to cast the problem of semantic annotation as a classification problem: each word is assigned a unique set of semantic tag(s) and/or label(s) from the universal tag/label set. This approach enables “local” semantic annotation resulting in partially annotated sentences. The proposed method reduces the annotation time and cost that forms a major bottleneck in the development of NLU systems. We present a set of experiments conducted on the medical domain “mini– corpus” that contains 10K hand–annotated sentences. Three annotation methods are compared: parser (baseline), similarity and classification–based annotations. The support vector machine (SVM) based classification scheme is shown to outperform both similarity and parsed–based annotation.

No abstract available.