Comics combine visual art, text, and narrative pacing, making them highly engaging but difficult to search using metadata or conventional content-based retrieval. Existing comic search platforms often ignore what comic fans often seek, such as a visual style, character type, or fast narrative flow. LLM or AI-based search performs well on text and images but is often opaque, leaving users unsure why results are relevant. We introduce ESCOMIC, an adaptive and explainable search system for multimodal comic content. In a query-by-example setting, ESCOMIC combines low-level visual and textual features with high-level comic facets (e.g., genre, character gender, story pace) in a facet-based re-ranking framework. Implicit feedback from simple interactions (e.g., hovering) dynamically adapts facet weights to evolving intent. ESCOMIC provides global explanations that show how facets shape ranking and local explanations that compare top-k results to the query. Due to the lack of comic retrieval benchmarks, we conduct a controlled user study with eye-tracking and appropriate baselines. Results show higher retrieval effectiveness, satisfaction, and trust with adaptive, content-aware explanations. Our findings highlight the value of transparent multimodal search and motivate future work using LLM and RAG based narrative explanations.

The rise of Gen-AI and LLMs often makes it difficult for users to trust retrieved results.The risk of IR systems using LLMs and being susceptible to misinformation can be tackled under the lens of explainability in AI.The topic of explainability in AI, machine learning (XAI) and information retrieval (IR) has been explored through various methods, yet few incorporate user feedback to adapt explanations.In this work, we present an XAI-driven extension to the classic relevance feedback model in IR, incorporating user feedback in the process of explaining the model behavior to the user.Our proposed model, RelEx, introduces XAI-specific elements, including key phrase vectors, text summaries, and contextual phrases combined with a neural ranker.RelEx interactively gathers user feedback, adapting search results based on the modified query and contextual vectors.We further introduce a novel additive similarity scheme that combines document similarity with key-phrase overlap.Retrieval performance is empirically evaluated on multiple benchmark datasets.In the absence of ground truth explanations, we assess explainability and assessability via user studies, where RelEx exhibit promising results.

We present X-Vision, an explainable AI (XAI) driven image retrieval system based on a re-ranking approach to support non-expert users. We generate textual explanations such as, ''This image is similar to query image in color by Y%, shape by Z%'' along with visual explanations that compare image features. Besides the XAI goal of making AI systems transparent, we address the semantic gap between user's perception and model ranking, which arises in content based image retrieval (CBIR). We attempt to explain the notion of similarity in images in a query-by-example scenario, starting with relatively simple features such as color, texture, objects, background-foreground segments, moving to semantic representations learned from hidden layers of deep networks. The base retrieval model compares the query vector with other image feature vectors to create rankings. This result list is transferred to a semantic feature space that allows rule-based re-rankings. The core contribution of this work is a re-ranking algorithm for generating explanations. Our re-ranking improves retrieval performance (MAP) when compared with a base ranker, a random baseline, and recent CBIR baseline rankers on PASCAL VOC data. We evaluate XAI focused aspects of user trust in an eye-tracker based user study, we find that explanations supported users in the search process and understanding the notion of similarity.

Explainable AI (XAI) is currently a vibrant research topic. However, the absence of ground truth explanations makes it difficult to evaluate XAI systems such as Explainable Search. We present an Explainable Search system with a focus on evaluating the XAI aspect of Trustworthiness along with the retrieval performance. We present SIMFIC 2.0 (Similarity in Fiction), an enhanced version of a recent explainable search system. The system retrieves books similar to a selected book in a query-by-example setting. The motivation is to explain the notion of similarity in fiction books. We extract hand-crafted interpretable features for fiction books and provide global explanations by fitting a linear regression and local explanations based on similarity measures. The Trustworthiness facet is evaluated using user studies, while the ranking performance is compared by analysis of user clicks. Eye tracking is used to investigate user attention to the explanation elements when interacting with the interface. Initial experiments show statistically significant results on the Trustworthiness of the system, paving way for interesting research directions that are being investigated.