Comics combine visual art, text, and narrative pacing, making them highly engaging but difficult to search using metadata or conventional content-based retrieval. Existing comic search platforms often ignore what comic fans often seek, such as a visual style, character type, or fast narrative flow. LLM or AI-based search performs well on text and images but is often opaque, leaving users unsure why results are relevant. We introduce ESCOMIC, an adaptive and explainable search system for multimodal comic content. In a query-by-example setting, ESCOMIC combines low-level visual and textual features with high-level comic facets (e.g., genre, character gender, story pace) in a facet-based re-ranking framework. Implicit feedback from simple interactions (e.g., hovering) dynamically adapts facet weights to evolving intent. ESCOMIC provides global explanations that show how facets shape ranking and local explanations that compare top-k results to the query. Due to the lack of comic retrieval benchmarks, we conduct a controlled user study with eye-tracking and appropriate baselines. Results show higher retrieval effectiveness, satisfaction, and trust with adaptive, content-aware explanations. Our findings highlight the value of transparent multimodal search and motivate future work using LLM and RAG based narrative explanations.

The rise of Gen-AI and LLMs often makes it difficult for users to trust retrieved results.The risk of IR systems using LLMs and being susceptible to misinformation can be tackled under the lens of explainability in AI.The topic of explainability in AI, machine learning (XAI) and information retrieval (IR) has been explored through various methods, yet few incorporate user feedback to adapt explanations.In this work, we present an XAI-driven extension to the classic relevance feedback model in IR, incorporating user feedback in the process of explaining the model behavior to the user.Our proposed model, RelEx, introduces XAI-specific elements, including key phrase vectors, text summaries, and contextual phrases combined with a neural ranker.RelEx interactively gathers user feedback, adapting search results based on the modified query and contextual vectors.We further introduce a novel additive similarity scheme that combines document similarity with key-phrase overlap.Retrieval performance is empirically evaluated on multiple benchmark datasets.In the absence of ground truth explanations, we assess explainability and assessability via user studies, where RelEx exhibit promising results.

We present X-Vision, an explainable AI (XAI) driven image retrieval system based on a re-ranking approach to support non-expert users. We generate textual explanations such as, ''This image is similar to query image in color by Y%, shape by Z%'' along with visual explanations that compare image features. Besides the XAI goal of making AI systems transparent, we address the semantic gap between user's perception and model ranking, which arises in content based image retrieval (CBIR). We attempt to explain the notion of similarity in images in a query-by-example scenario, starting with relatively simple features such as color, texture, objects, background-foreground segments, moving to semantic representations learned from hidden layers of deep networks. The base retrieval model compares the query vector with other image feature vectors to create rankings. This result list is transferred to a semantic feature space that allows rule-based re-rankings. The core contribution of this work is a re-ranking algorithm for generating explanations. Our re-ranking improves retrieval performance (MAP) when compared with a base ranker, a random baseline, and recent CBIR baseline rankers on PASCAL VOC data. We evaluate XAI focused aspects of user trust in an eye-tracker based user study, we find that explanations supported users in the search process and understanding the notion of similarity.

Explainable AI (XAI) is currently a vibrant research topic. However, the absence of ground truth explanations makes it difficult to evaluate XAI systems such as Explainable Search. We present an Explainable Search system with a focus on evaluating the XAI aspect of Trustworthiness along with the retrieval performance. We present SIMFIC 2.0 (Similarity in Fiction), an enhanced version of a recent explainable search system. The system retrieves books similar to a selected book in a query-by-example setting. The motivation is to explain the notion of similarity in fiction books. We extract hand-crafted interpretable features for fiction books and provide global explanations by fitting a linear regression and local explanations based on similarity measures. The Trustworthiness facet is evaluated using user studies, while the ranking performance is compared by analysis of user clicks. Eye tracking is used to investigate user attention to the explanation elements when interacting with the interface. Initial experiments show statistically significant results on the Trustworthiness of the system, paving way for interesting research directions that are being investigated.

Being able to differentiate between search activities a user is currently engaged in is crucial for adaptive information retrieval systems in order to provide appropriate support to the user in time. In this paper, we conduct an investigation into modeling two kinds of search activities: exploratory activities and fact-finding activities during web search. Specifically, we consider the case where a user is conducting consecutive fact-finding searches on multiple topics. This information behavior is also known as multitasking search. The goal of our research is to build models of users' web information-seeking behavior in order to differentiate between search activities caused by exploratory search tasks and consecutive fact-finding search tasks while users are still searching. In order to build search process models, we have designed and conducted a user study where the participants interact with a common web search engine. Based on the gathered log data, we built search models based on (Hidden) Markov Models and analyze the results. Our results yield to classification rates between 73.6% and 92.1% using different Markov Models and different data configurations. Furthermore, even with interaction sequences of a limited length, the models reach a classification rate of 85.6% within the first four interactions and about 89% with 30 and more interactions. Using a reduced data set the models still reach an accuracy of 87.7%. That is, within a single session the trained models can be used to detect the search activity to provide appropriate user support.

Exploration in large multimedia collections is challenging because the user often navigates into misleading directions or information areas. The vision of our project is to develop an assistive technology that is able to support the individual user and enhance the efficiency of an ongoing exploratory search. Such a technical search aid should be able to find out about the user's current interests and goals. Respective parameters can be found in the central and in the peripheral nervous system as well as in overt behavior. Therefore, we aim at using eye movements, pupillometry and EEG to assess respective information. Here, we describe the set-up and the first results of a preliminary user study investigating the effects of searching an image collection on eye movements and pupil dilations. First data show that numbers of fixation, fixation durations as well as pupil dilations differ systematically when looking at a subsequently selected target as compared with not selected items. These results support our vision that further research additionally investigating EEG can in fact result in better predicting the searchers goals and next choices.

Solid decision making should be ideally based on clear arguments that can be justified by trustworthy information sources. However, argument spaces can quickly get quite complex and it is very often hard to trace the line of arguments found in literature or social media such as blogs and forums. In this paper, we propose a framework for a decision supporting interactive information retrieval system using methods for argument exploration based on textual documents. This concept is supported by a prototype that focuses on the actual analysis of the retrieved arguments in order to obtain a justified decision. For that we use a simplified argumentation graph with nodes as arguments and simple attacking and supporting relations. A web-based plausibility value is propagated (using ranked-based argumentation semantics) through the network for estimating the quality of the arguments. This is based on a web search for documents that support these arguments. The final decision can further be supported by chosing certain preferred interpretations of an abstract dialectical framework, leading to an integrated view of searching, creating, analysing and deciding.

No abstract available.

We present first results of a logfile analysis on web search engines for children. The aim of this research is to analyse fundamental facts about how children's web search behaviour differs from that of adults. We show differences to previous results, which are often based on small lab experiments. Our large-scale analysis suggests that children search queries are more information-oriented and shorter on average. Children indeed make a lot of spelling errors and often repeat searches and revisit web pages.

The goal of the proposed tool multi Searcher is to answer this research question: can we expect people to be able to get information from text in languages they can not read or understand? The proposed tool multi Searcher provides users with interactive contextual information that describes the translation in the user's own language so that the user has a certain degree of confidence about the translation. Therefore, the user is considered as an integral part of the retrieval process. The tool provides possibilities to interactively select relevant terms from contextual information in order to improve the translation and thus improve the cross lingual information retrieval (CLIR) process.

Abstract In this paper we present a language‐independent approach for conflation that does not depend on predefined rules or prior knowledge of the target language. The proposed unsupervised method is based on an enhancement of the pure n‐gram model that can group related words based on various string‐similarity measures, while restricting the search to specific locations of the target word by taking into account the order of n‐grams. We show that the method is effective to achieve high score similarities for all word‐form variations and reduces the ambiguity, i.e., obtains a higher precision and recall, compared to pure n‐gram‐based approaches for English, Portuguese, and Arabic. The proposed method is especially suited for conflation approaches in Arabic, since Arabic is a highly inflectional language. Therefore, we present in addition an adaptive user interface for Arabic text retrieval called “araSearch”. araSearch serves as a metasearch interface to existing search engines. The system is able to extend a query using the proposed conflation approach such that additional results for relevant subwords can be found automatically.

Although clustering under constraints is a current research topic, a hierarchical setting, in which a hierarchy of clusters is the goal, is usually not considered. This paper tries to fill this gap by analyzing a scenario, where constraints are derived from a hierarchy that is partially known in advance. This scenario can be found, e.g., when structuring a collection of documents according to a user specific hierarchy. Major issues of current approaches to constraint based clustering are discussed, especially towards the hierarchical setting. We introduce the concept of hierarchical constraints and continue by presenting and evaluating two approaches using them. The approaches cover the two major fields of constraint based clustering, i.e. instance and metric based constraint integration. Our objects of interest are text documents. Therefore, the presented algorithms are especially fitted to work for these where necessary. Despite showing the properties and ideas of the algorithms in general, we evaluated the case of constraints that are unevenly scattered over the instance space, which is very common for real-world problems but not satisfyingly covered in other work so far.

Hierarchical classification refers to an extension of the standard classification problem, in which labels must be chosen from a class hierarchy. In this paper, we look at hierarchical classification from an information retrieval point of view. More specifically, we consider a scenario in which a user searches a document in a topic hierarchy. This scenario gives rise to the problem of predicting an optimal entry point, that is, a topic node in which the user starts searching. The usefulness of a corresponding prediction strongly depends on the search behavior of the user, which becomes relevant if the document is not immediately found in the predicted node. Typically, users tend to browse the hierarchy in a top-down manner, i.e., they look at a few more specific subcategories but usually refuse exploring completely different branches of the search tree. From a classification point of view, this means that a prediction should be evaluated, not solely on the basis of its correctness, but rather by judging its usefulness against the background of the user behavior. The idea of this paper is to formalize hierarchical classification within a decision-theoretic framework which allows for modeling this usefulness in terms of a user-specific utility function. The prediction problem thus becomes a problem of expected utility maximization. Apart from its theoretical appeal, we provide first empirical results showing that the approach performs well in practice.

A hierarchical structure can provide efficient access to information contained in a collection of documents. However, such a structure is not always available, e.g. for a set of documents a user has collected over time in a single folder or the results of a Web search. We therefore investigate in this paper how we can obtain a hierarchical structure automatically, taking into account some background knowledge about the way a specific user would structure the collection. More specifically, we adapt a hierarchical agglomerative clustering algorithm to take into account user specific constraints on the clustering process. Such an algorithm could be applied, e.g., for user specific clustering of Web search results, where the user's constraints on the clustering process are given by a hierarchical folder or bookmark structure. Besides the discussion of the algorithm itself, we motivate application scenarios and present an evaluation of the proposed algorithm on benchmark data

In this article we describe results of our research on the disambiguation of user queries using ontologies for categorization. We present an approach to cluster search results by using classes or “Sense Folders” (prototype categories) derived from the concepts of an assigned ontology, in our case WordNet. Using the semantic relations provided from such a resource, we can assign categories to prior, not annotated documents. The disambiguation of query terms in documents with respect to a user-specific ontology is an important issue in order to improve the retrieval performance for the user. Furthermore, we show that a clustering process can enhance the semantic classification of documents, and we discuss how this clustering process can be further enhanced using only the most descriptive classes of the ontology. © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 693–709, 2006.