This paper addresses the focus of this special issue by analyzing the potential contribution of linguistic content and other nonimage aspects to the processing of audiovisual data. It summarizes the various ways in which linguistic content analysis contributes to enhancing the semantic annotation of multimedia content, and, as a consequence, to improving the effectiveness of conceptual media access tools. A number of techniques are presented, including the time-alignment of textual resources, audio and speech processing, content reduction and reasoning tools, and the exploitation of surface features

No abstract available.

Small XML elements are often estimated relevant by the retrieval model but they are not desirable retrieval units. This paper presents a generic model that exploits the information obtained from small elements. We identify relationships between small and relevant elements and use this linking information to reinforce the relevance of other elements before removing the small ones. Our experiments using the INEX testbed show the effectiveness of our approach.

The structural features of XML components are an extra source of information that should be used in a content-oriented retrieval task on this type of documents. In this paper we explore one of the structural features from the INEX collection [1] that could be used in content-oriented search. We analyse the gain this knowledge could add to the performance of an information retrieval system and present a first approach on how this structural information could be extracted from a relevance feedback process to be used as priors in a language modelling framework.

This paper presents a (preliminary) analysis of the evaluation results obtained on the TRECVID 2003 search task. We study in particular the effects of combining multiple representations on retrieval: multiple representations of video content (speech and visual) and of the user information need (multiple visual examples). We conclude from our multi-modal retrieval experiments the following working hypothesis: even though the automatic speech recognition run is usually better than the visual run, matching against both modalities ensures robustness against choosing the wrong content representation. For the same reason, using multiple visual examples to represent the user information needs is preferable over using a single designated example only.

The language modeling approach to retrieval is based on the philosophy that the language in a relevant document follows the same distribution as that in the query. This same philosophy can also be applied to content-based image and video retrieval, where the only difference lies in the definition of 'language'. Previous results on the TRECVID 2003 corpus have demonstrated that the visual content can be captured successfully by a continuous Gaussian mixture model. This paper investigates whether modeling the visual content by a discrete multinomial model (as used in full-text retrieval) is also viable. We compare the retrieval effectiveness obtained on the TRECVID 2003 corpus when using continuous vs. discrete keyframe models.

We describe the application of a probabilistic multimedia model to video retrieval. From video shots, we compute Gaussian-mixture models that capture correlations in time and space, such as the appearance and disappearance of objects. These models improve the precision of "query by example/s" results in the TRECVID 2003 collection when compared to models that only make use of static visual information. Furthermore, integrated with information from automatic speech recognition (ASR) transcripts, they outperform ASR only results.

The main conclusion from the metrics-based evaluation of video retrieval systems at TREC's video track is that non-interactive image retrieval from general collections using visual information only is not yet feasible. We show how a detailed analysis of retrieval results -- looking beyond mean average precision (MAP) scores on topical relevance -- gives significant insight in the main problems with the visual part of the retrieval model under study. Such an analytical approach proves an important addition to standard evaluation measures.

An important class of searches on the world-wide-web has the goal to find an entry page (homepage) of an organisation. Entry page search is quite different from Ad Hoc search. Indeed a plain Ad Hoc system performs disappointingly. We explored three non-content features of web pages: page length, number of incoming links and URL form. Especially the URL form proved to be a good predictor. Using URL form priors we found over 70% of all entry pages at rank 1, and up to 89% in the top 10. Non-content features can easily be embedded in a language model framework as a prior probability.

We present a framework in which probabilistic models for textual and visual information retrieval can be integrated seamlessly. The framework facilitates searching for imagery using textual descriptions and visual examples simultaneously. The underlying Language Models for text and Gaussian Mixture Models for images have proven successful in various retrieval tasks.