Applications with street navigation have been recently introduced on mobile phone devices. A major part of existing systems use integrated GPS as input for indicating the location. However, these systems often fail or make abrupt shifts in urban environment due to occlusion of satellites. Furthermore, they only give the position of a person and not the object of his attention, which is just as important for localization based services. In this paper we introduce a system using mobile phones built-in cameras for navigation and localization using visual information in accordance with the way we as humans navigate. The introduced method uses local features for extraction of natural feature points from images which are compared to a database for localization. The system is tested and evaluated in a real urban environment and the result shows very high success rate.

Interest points have been used as local features with success in many computer vision applications such as image/video retrieval and object recognition. However, a major issue when using this approach is a large number of interest points detected from each image and created a dense feature space. This influences the processing speed in any runtime application. Selecting the most important features to reduce the size of the feature space will solve this problem. Thereby this raises a question of what makes a feature more important than the others? In this paper, we present a new technique to choose a subset of features. Our approach differs from others in a fact that selected feature is based on the context of the given image. Our experimental results show a significant reduction rate of features while preserving the retrieval performance.

In literature, existing researches on building recognition mainly concentrate on scales, rotations, and viewpoints variance. In urban environment, large temporal variations of weather and lighting conditions should also be considered as major challenges for robust recognition. For instances, there are differences between images captured during daytime and nighttime, especially significant changes in building appearances between seasons because of the differences in light setting. To date, these large temporal variation issues have not been fully investigated. In this paper, we therefore focus on constructing a system that deals with the temporal difference factors in recognizing urban buildings. In order to build such a system, two main criteria are raised, namely the efficiency of the recognition algorithm and the speed for interactive search purpose. For recognition purpose, we exploit the MOPS features (Multi-scale Oriented Patches) in [2], which extract features of patches around interest points. To speed up the searching process, we employ the vocabulary tree based search technique in [12]. Our final system shows high performance in recognizing buildings under significant temporal variations with a fast processing reaction.

At one end of the spectrum, research in interactive content-based retrieval concentrates on machine learning methods for effective use of relevance feedback. On the other end, the information visualization community focuses on effective methods for conveying information to the user. What is lacking is research considering the information visualization and interactive retrieval as truly integrated parts of one content-based search system. In such an integrated system, there are many degrees of freedom like the similarity function, the number of images to display, the image size, different visualization modes, and possible feedback modes. To base the optimal values for all of those on user studies is unfeasible. We therefore develop search scenarios in which tasks and user actions are simulated. From there, the proposed scheme is optimized based on objective constraints and evaluation criteria. In such a manner, the degrees of freedom are reduced and the remaining degrees can be evaluated in user studies. In this article, we present a system that integrates advanced similarity based visualization with active learning. We have performed extensive experimentation on interactive category search with different image collections. The results using the proposed simulation scheme show that indeed the use of advanced visualization and active learning pays off in all of these datasets.

In this paper we present the methods underlying the MediaMill semantic video search engine. The basis for the engine is a semantic indexing process which is currently based on a lexicon of 491 concept detectors. To support the user in navigating the collection, the system defines a visual similarity space, a semantic similarity space, a semantic thread space, and browsers to explore them. We compare the different browsers and their utility within the TRECVID benchmark. In 2005, we obtained a top-3 result for 19 out of 24 search topics. In 2006 for 14 out of 24.

In this paper, we argue to learn dissimilarity for interactive search in content based image retrieval. In literature, dissimilarity is often learned via the feature space by feature selection, feature weighting or by adjusting the parameters of a function of the features. Other than existing techniques, we use feedback to adjust the dissimilarity space independent of feature space. This has the great advantage that it manipulates dissimilarity directly. To create a dissimilarity space, we use the method proposed by Pekalska and Duin, selecting a set of images called prototypes and computing distances to those prototypes for all images in the collection. After the user gives feedback, we apply active learning with a one-class support vector machine to decide the movement of images such that relevant images stay close together while irrelevant ones are pushed away (the work of Guo ). The dissimilarity space is then adjusted accordingly. Results on a Corel dataset of 10000 images and a TrecVid collection of 43907 keyframes show that our proposed approach is not only intuitive, it also significantly improves the retrieval performance.

In this technical demonstration we showcase the MediaMill system. A search engine that facilitates access to news video archives at a semantic level. The core of the system is an unprecedented lexicon of 100 automatically detected semantic concepts. Based on this lexicon we demonstrate how users can obtain highly relevant retrieval results using query-by-concept. In addition, we show how the lexicon of concepts can be exploited for novel applications using advanced semantic visualizations. Several aspects of the MediaMill system are evaluated as part of our TRECVID 2005 efforts.

In any CBIR system, visualization is important, either to show the final result to the user or to form the basis for interaction. Advanced systems use 2D similarity based visualization which shows not only the information of one image itself but also the relations between images. A problem in interactive 2D visualization is the overlap between the images displayed. This obviously reduces the search capability. Simply spreading the images on the screen space will not preserve the relations between them. In this paper, we propose a visualization scheme which reduces the overlap as well as preserves the general distribution of the images displayed. Results show that an effective balance between display of structures and limited overlap can be achieved

We consider the interaction with salient details in the image i.e. points, lines, and regions. Interactive salient detail definition goes further than summarizing the image into a set of salient details since the saliency of details depends on the context, the application and the user. We propose an interaction framework for salient details from the perspective of the user, which dynamically updates the user- and context-dependent definition of saliency based on relevance feedback. A number of instantiations of the framework are presented.

In this presentation, we present a system for interactive search in video archives. In our view, interactive search is a four-step process composed of indexing, filtering, browsing, and ranking. We have experimentally verified, using 22 groups of two participants each, how users apply these steps in the interactive search and how well they perform.