SCAI (https://scai.info) celebrates its 10th anniversary this year and we would like to invite our core research community to join us. Since our first workshop started back at ICTIR 2017 in Amsterdam, we came a long way and would like to use this opportunity to reflect on it together. With the advent of large language models, conversational AI has emerged as a primary paradigm for search-intensive tasks. However, despite the vast success of conversational AI, there are major shortcomings in existing solutions that offer promising opportunities for the next breakthroughs which we would like to promote further. The focus of this edition will be on the personalization of conversational search systems, with a featured session for the former TREC shared task "Interactive Knowledge Assistance Track" (iKAT) reintroduced this year at SCAI. In combination with a panel discussion, invited presentations and keynote talks from major industry representatives, a lively poster session, and a separate break-out session featuring hands-on evaluation of the top-notch conversational AI systems, we plan for a full-day dense and highly engaging workshop.

This study presents a comprehensive reproducibility analysis and extension of the Setwise prompting method for zero-shot ranking with Large Language Models (LLMs), as proposed by Zhuang et al. We evaluate the method's effectiveness and efficiency compared to traditional Pointwise, Pairwise, and Listwise approaches in document ranking tasks. Our reproduction confirms the findings of Zhuang et al., highlighting the trade-offs between computational efficiency and ranking effectiveness in Setwise methods. Building on these insights, we introduce Setwise Insertion, a novel approach that leverages the initial document ranking as prior knowledge, reducing unnecessary comparisons and uncertainty by prioritizing candidates more likely to improve the ranking results. Experimental results across multiple LLM architectures - Flan-T5, Vicuna, and Llama - show that Setwise Insertion yields a 31% reduction in query time, a 23% reduction in model inferences, and a slight improvement in reranking effectiveness compared to the original Setwise method. These findings highlight the practical advantage of incorporating prior ranking knowledge into Setwise prompting for efficient and accurate zero-shot document reranking.

This reproducibility study analyzes and extends the paper ''Ax- iomatic Causal Interventions for Reverse Engineering Relevance Computation in Neural Retrieval Models,'' which investigates how neural retrieval models encode task-relevant properties such as term frequency. We reproduce key experiments from the original paper, confirming that information on query terms is captured in the model encoding. We extend this work by applying activa- tion patching to Spanish and Chinese datasets and by exploring whether document-length information is encoded in the model as well. Our results confirm that the designed activation patching method can isolate the behavior to specific components and to- kens in neural retrieval models. Moreover, our findings indicate that the location of term frequency generalizes across languages and that in later layers, the information for sequence-level tasks is represented in the CLS token. The results highlight the need for further research into interpretability in information retrieval and reproducibility in machine learning research. Our code is available at https://github.com/OliverSavolainen/axiomatic-ir-reproduce.