Users often turn to online forums when searching for known books, movies, or games that they cannot identify through conventional search engines. These “tip-of-the tongue” requests present a unique challenge, appearing highly variable in formulation, context, and specificity. So far, these could mostly only be solved by other humans answering in forums. Generative AI is believed to help solve these specific questions. In this work, we manually annotated 150 requests each for books, games, and movies in the casual leisure domain to study the differences between solved and unsolved requests and identify factors that influence their difficulty. We compare human responses in forum threads with the performance of a Large Language Model (LLM) under similar conditions. Specifically, we investigate how the formulation of requests affects human and LLM success; how item properties impact LLM retrieval; how interaction and feedback within a thread shape human and LLM performance; and whether increasing the information provided to an LLM improves its chances of solving the request. Our findings offer new insights into what makes these known-item search problems easier or harder to solve. This study contributes to a better understanding of complex search behavior and the role of LLMs in helping with difficult casual-leisure information needs.

The rapidly increasing popularity of LLM-powered chatbots has led to them being used for a increasing number of different tasks by the general public.One of these tasks is searching for information instead of using a search engine.Previous work has shown that complex search tasks can be problematic for traditional search engines to solve, but little is known about the capability of LLMs on the same task.We compared four LLMs on their capability to answer a specific type of complex search task: known-item requests from casual leisure domains.We constructed a test collection by gathering known-item requests for books, games and movies from online forums along with verified answers by the original requester.We prompted four LLMs multiple times with the same prompt and analyzed the results with respect to accuracy and the degree to which answers were fabricated by the LLM.Our results show that LLMs are not particularly effective in fulfilling these complex casual leisure needs, but there are are big differences between LLMs and across domains.

Across all scientific fields, there is an increased focus on the impact of scientific research: what academic and societal benefits does it provide? This question has spurred the development of a variety of different approaches to impact assessment, each appropriate in different circumstances. In this paper, we study the academic impact of the CHIIR community through a comprehensive analysis of the work published in the 2016-2023 CHIIR conference series. We collect citation counts, citing documents, and altmetrics scores for all CHIIR publications to determine their academic impact across a variety of different attributes of the CHIIR publications. In addition, we analyze a subset of citation contexts in the papers that have cited CHIIR publications to analyze how they are being used and what that means for their potential impact. Finally, we attempt to predict which properties of CHIIR publications are most predictive of future impact.

In this paper, we present the results of an initial study of the research, sharing, and re-use practices at the CHIIR conference through a systematic analysis of all CHIIR papers published from 2016 to 2022. We find that CHIIR is a conference predominantly focused on empirical, multi-methods research that over the years has undergone a focusing in terms of the type of research methods that are being used. A modest number of papers re-use existing data and design resources, but infrastructure component re-use is much more rare. Only a fraction of CHIIR papers actually share their own resources, which suggests that there is much to gain in terms of reproducibility of research presented at CHIIR and could potentially be used to support changes in reviewing practices.

We studied the collaboration patterns of CHIIR authors, and found that most papers are collaborative. A core of 33% of the CHIIR researchers are directly connected and frequently co-author, and several disconnected clusters also make frequent CHIIR contributions. We also studied citation impact of the CHIIR papers and show that in relation to research design type, theoretical and empirical papers tend to receive more citations than resource papers. With regards to sharing and re-use, papers that share at least one resource tend to have significantly higher citation impact—in particular when sharing data resources and design resources. Re-using resources does not significantly increase citation impact in itself.

Work in Progress Share on Third Workshop on Building towards Information Interaction and Retrieval Resources Re-use (BIIRRR 2022) Authors: Toine Bogers Aalborg University, Denmark Aalborg University, DenmarkView Profile , Maria Gäde Humboldt-Universität zu Berlin, Germany Humboldt-Universität zu Berlin, GermanyView Profile , Mark Michael Hall The Open University, United Kingdom The Open University, United KingdomView Profile , Marijn Koolen Huygens Institute for the History of the Netherlands, Royal Netherlands Academy of Arts and Sciences, Netherlands Huygens Institute for the History of the Netherlands, Royal Netherlands Academy of Arts and Sciences, NetherlandsView Profile , Vivien Petras Humboldt-Universität zu Berlin, Germany Humboldt-Universität zu Berlin, GermanyView Profile , Paul Thomas Microsoft, Australia Microsoft, AustraliaView Profile Authors Info & Claims CHIIR '22: ACM SIGIR Conference on Human Information Interaction and RetrievalMarch 2022 Pages 374–376https://doi.org/10.1145/3498366.3505838Published:14 March 2022Publication History 0citation24DownloadsMetricsTotal Citations0Total Downloads24Last 12 Months24Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

Natural language descriptions of geographical locations are used frequently in daily life and there is a motivation to create systems that generate such descriptions automatically, for purposes such as documentation of where events have taken place, where a person is located, where photos were taken and where plants and animals are located. Typically location descriptions combine references to named geographical features with vague spatial relational terms, such as near, north of and at that relate locations to the features. Here we describe a system for generating location descriptions, that combines spatial templates, that model the applicability of different spatial relations relative to a reference location, with toponyms in the vicinity of the described location that are selected according to aspects of salience. The toponyms are retrieved from a gazetteer service based on OpenStreetMap for which we create a hierarchical feature classification scheme to facilitate selection of toponyms according to distinctiveness of their feature types and other aspects of salience. The advantages of the approach are demonstrated in a user study, relative to an existing state of the art system and to other baseline approaches that include manually created captions and the automated methods of two widely used photo captioning systems.

This perspective paper on resource re-use intends to draw the attention of the interactive information retrieval (IIR) community to the challenges of research documentation and archiving for future use. Resources are understood as encompassing research designs, research data and research infrastructures. It proposes eight principles for improving the re-use of resources in the IIR community and presents concrete steps on how to achieve them. A five-level system for data archiving and documentation envisions increasingly open and stable documentation and access infrastructures.

Increasing re-use in Interactive Information Retrieval (IIR) has been an ongoing aim in IIR for a significant amount of time, however progress has been limited and patchy. While re-use of some study aspects can be difficult due to the varied nature of IIR studies, the use of pre- and post-task self-reported measures is widespread and relatively standardised. Nevertheless, re-use of elements in this area is also limited, in part because systems used to implement them are not able to exchange question, instruments, or complete study setups. To address this, this paper presents a standardised, but extendable, format for IIR survey instrument exchange.

Share on Workshop on Barriers to Interactive IR Resources Re-use (BIIRRR 2019) Authors: Toine Bogers Aalborg University Copenhagen, Copenhagen, Denmark Aalborg University Copenhagen, Copenhagen, DenmarkView Profile , Samuel Dodson University of British Columbia, Vancouver, Canada University of British Columbia, Vancouver, CanadaView Profile , Luanne Freund University of British Columbia, Vancouver, Canada University of British Columbia, Vancouver, CanadaView Profile , Maria Gäde Humboldt-Universität zu Berlin, Berlin, Germany Humboldt-Universität zu Berlin, Berlin, GermanyView Profile , Mark Hall Martin-Luther-Universität Halle-Wittenberg, Halle, Germany Martin-Luther-Universität Halle-Wittenberg, Halle, GermanyView Profile , Marijn Koolen Royal Netherlands Academy of Arts and Sciences, Amsterdam, Netherlands Royal Netherlands Academy of Arts and Sciences, Amsterdam, NetherlandsView Profile , Vivien Petras Humboldt-Universität zu Berlin, Berlin, Germany Humboldt-Universität zu Berlin, Berlin, GermanyView Profile , Nils Pharo Oslo Metropolitan University, Oslo, Norway Oslo Metropolitan University, Oslo, NorwayView Profile , Mette Skov Aalborg University, Aalborg, Denmark Aalborg University, Aalborg, DenmarkView Profile Authors Info & Claims CHIIR '19: Proceedings of the 2019 Conference on Human Information Interaction and RetrievalMarch 2019 Pages 389–392https://doi.org/10.1145/3295750.3298965Published:08 March 2019Publication History 1citation75DownloadsMetricsTotal Citations1Total Downloads75Last 12 Months8Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

The goal of this workshop is to serve as a starting point for a community-driven effort to design and implement a platform for the collection, organization, maintenance, and sharing of resources for IIR experimentation. As in all scientific endeavors, progress in IIR research is contingent on the ability to build on previous ideas, approaches, and resources. However, we believe there to be a number of barriers to reproducibility and re-use of resources in IIR research: the fragmentary nature of how the community»s resources are organized, the lack of awareness of their existence, documentation and organization of the resources, the nature of the typical research publication cycle, and the effort required to make such resources available. We believe that an online platform dedicated to the collection and organization of IIR resources could be a promising way of overcoming these barriers. The workshop therefore aims to serve both as a brainstorming opportunity about the shape this iRepository should take, as well as a way of building support in the community for its implementation.

When searching for books, people frequently have to deal with content that is in a language different from their own. However, research on multilingual systems has generally focused on the user interface's language rather than the content language. In this paper, we describe and compare early results from the multilingual aspects in the Interactive Social Book Search (iSBS) task at CLEF 2014 and 2015. A preliminary analysis of usage patterns for native English and non-native English speakers indicates an influence of language skills on search behaviour during goal-oriented and casual leisure tasks. Based on previous experiences and results, strengths and challenges of IIR studies are discussed.

Information Retrieval (IR) research has traditionally focused on serving the best results for a single query - so-called ad hoc retrieval. However, users typically search iteratively, refining and reformulating their queries during a session. A key challenge in the study of this interaction is the creation of suitable evaluation resources to assess the effectiveness of IR systems over sessions. This paper describes the TREC Session Track, which ran from 2010 through to 2014, which focussed on forming test collections that included various forms of implicit feedback. We describe the test collections; a brief analysis of the differences between datasets over the years; and the evaluation results that demonstrate that the use of user session data significantly improved effectiveness.

Current Information Retrieval systems for digital cultural heritage support only the actual search aspect of the information seeking process. This demonstration presents the second PATHS system which provides the exploration, analysis, and sense-making features to support the full information seeking process.

Many aspects of spatial language concerned with relationships between spatial entities are essentially vague. Current GIS technology provides very little support for dealing with this vagueness, partially because there is a lack of quantitative data and models for vague spatial relations. This paper presents an experiment that looks at quantifying spatial prepositions. In the context of image captions, the cardinal directions are analysed in an existing set of image captions, with respect to the spatial distribution of the locations of the target object (figure) and the reference object (ground). Future work will focus on using these results to improve current GIS solutions in a wide variety of scenarios.