We address the problem of identifying images of persons in large collections, such as the Web, without an existing face image database. We describe a method and a system that automatically constructs an initial face image database for a person using textual evidence obtained from the Web, and then uses this database for identifying images of that person. The initial retrieval results are obtained via text/HTML analysis and face detection. An internal clustering process groups visually similar faces among these initial results and builds a facial database. This database is then used by a face recognizer. The outputs of the textual and visual evidence modules are combined using Dempster-Shafer (1976) evidence combination formula. We present the results of an experimental evaluation where the system was able to improve upon the detection-only method when text/HTML analysis performed poorly.

Content based indexing of multimedia has always been a challenging task. The enormity and the diversity of the multimedia content on the web adds another dimension to this challenge. In this paper, we examine ways of combining visual and textual information for content based indexing of multimedia on the web. In particular, we examine different methods of combining evidences due to face detection, Text/HTML analysis and face recognition for identifying person images. We provide experimental evaluation of the following strategies: i) Face detection on the image followed by Text/HTML analysis of the containing page; ii) face detection followed by face recognition; iii) face detection followed by a linear combination of evidences due to text/HTML analysis and face recognition; and iv) face detection followed by a Dempster-Shafer combination of evidences due to text/HTML analysis and face recognition. These strategies were implemented in an automatic web search agent named Diogenes1 and compared against some well known web image search engines. The latter includes commercial systems such as Alta Vista, Lycos and Ditto, and a research prototype, WebSEEk. We report the results of our experimental retrievals where Diogenes outperformed these search engines for celebrity image queries in terms of average precision.

In this work, we examine evidence combination mechanisms for classifying multimedia information. In particular, we examine linear and Dempster-Shafer methods of evidence combination in the context of identifying personal images on the World Wide Web. An automatic web search engine named Diogenes1 searches the web for personal images and combines different pieces of evidence for identification. The sources of evidence consist of input from face detection/recognition and text/HTML analysis modules. A degree of uncertainty is involved with both of these sources. Diogenes automatically determines the uncertainty locally for each retrieval and uses this information to set a relative significance for each evidence. To our knowledge, Diogenes is the first image search engine using Dempster-Shafer evidence combination based on automatic object recognition and dynamic local uncertainty assessment. In our experiments Diogenes comfortably outperformed some well known commercial and research prototype image search engines for celebrity image queries.

Storage and retrieval of multimedia has become a requirement for many contemporary information systems. These systems need to provide browsing, querying, navigation, and, sometimes, composition capabilities involving various forms of media. In this survey, we review techniques and systems for image and video retrieval. We first look at visual features for image retrieval such as color, texture, shape, and spatial relationships. The indexing techniques are discussed for these features. Nonvisual features include captions, annotations, relational attributes, and structural descriptions. Temporal aspects of video retrieval and video segmentation are discussed next. We review several systems for image and video retrieval including research, commercial, and World Wide Web-based systems. We conclude with an overview of current challenges and future trends for image and video retrieval.

Image retrievaf based on semantic contents involves extraction, modelling and indexing of content information.While extraction of abstract contents is a hard problem, it is only part of the bigger picture.In this paper we use knowledge about the semantic contents of images to improve retrieval effectiveness.In particular we use Word Net, an electronic Iexicaf system for query and database expansion.Our content model facilitates novel uses of WordNet.We also propose a new normalization formula, an object significance scheme and evaluate their effectiveness with real user experiments.We describe the experiment setup and provide quantitative evaluation of each technique.

We make use of a refined E-R model to represent the contents of pictures. We propose remedies to handle mismatches which may arise due to differences in perception of picture contents. An iconic user interface for visual query construction is presented. A naive user can specify his/her intention without learning a query language. A function which computes the similarity between a picture and a user's description is provided. Pictures which are sufficiently close to the user description, as measured by the similarity function, are retrieved. We present the results of a user-friendliness experiment to evaluate the user interface as well as retrieval effectiveness. Encouraging retrieval results and valuable lessons are obtained.>

Since the development of QBE, over fifty visual query languages have been proposed to facilitate easy database access. Although these languages have introduced some very useful paradigms, a number of these have some severe limitations, such as: (a) not extending beyond the relational model (b) not considering negation and safety, formally (c) using ad hoc constructs, with no analysis of expressivity or complexity done, etc. Note that visual database access is an important issue being revisted, with the emergence of different flavors of object-oriented databases. We believe that there is a need for developing a unified visual query language.