Understanding how working memory (WM) capacity influences cognitive load (CL) during interactive information retrieval (IIR) tasks is critical for designing effective user interfaces. This knowledge supports minimizing cognitive overload, optimizing performance, and tailoring systems to individual cognitive profiles. This study examines CL dynamics and emotional responses during two types of search tasks: fact-checking (FC) and decision-making (DM), among individuals with varying WM capacity, and explores the relationship between CL and emotional states. CL was estimated in near-real-time using pupil diameter signals and analyzed across task phases (begin, mid, end). Participants were divided into high and low WM groups based on N-back task performance. Results show that CL was higher during DM tasks, especially for low WM individuals, while no group differences emerged during FC tasks. CL was highest at the beginning of the tasks and decreased over time, suggesting cognitive adaptation. Analysis of emotions revealed that high WM individuals exhibited positive correlations between CL and valence, joy, and engagement, whereas low WM individuals showed stronger associations with confusion and surprise, particularly during DM tasks. These findings highlight the importance of WM capacity in shaping cognitive and emotional experiences, offering insights for personalized and adaptive system design.

Attention is a crucial construct in Interactive Information Retrieval (IIR) activities and has become an area of growing interest among CHIIR researchers. Despite this importance, explicit definitions are rare. Many studies refer to “attention” without specifying which process or aspect is under examination, and often operationalize it using practical indicators (e.g., eye fixations, dwell time, cursor traces, interaction logs) without a clear conceptual framework guiding measurement choices, raising concerns about the comparability of findings. This paper reviews how attention has been defined, operationalized, and measured in CHIIR publications from the conference’s inception in 2016 through 2025. We searched the ACM Digital Library for "attention" within CHIIR proceedings, finding 296 results. After filtering and using semantic similarity, we narrowed it to 45 relevant papers, from which 19 were selected for in-depth review based on their clear relevance to attention. Drawing on theoretical frameworks and empirical findings from psychology and cognitive science, we analyze how CHIIR researchers define and apply the concept of attention in various contexts. Our review uncovers a variety of interpretations, aspects, and measurement approaches to attention, which reflect broader challenges in bridging cognitive theory and information interaction research. We discuss key issues underlying these differing uses of the term “attention,” and outline possible directions for advancing conceptual clarity and methodological robustness of attention research within CHIIR. We hope this perspective paper raises researchers’ awareness of the need to clearly define attention, thereby promoting greater rigor and reproducibility in future IIR studies.

Information relevance judgment is a fundamental cognitive decision performed by users during information seeking and reading tasks.Inferring relevance decisions from neurophysiological signals, particularly through eye-tracking and EEG recordings, opens possibilities for real-time relevance detection.To support this line of research, we present g-Rel-READER, a unique dataset comprising eye-tracking and EEG recordings collected during reading and relevance judgment of short news stories in a question-answering task.The dataset provides screen coordinates for each word, enabling temporal and spatial alignment between eye-tracking data, EEG signals, and the specific words being read.Question-relevant words are explicitly marked, facilitating analysis of relevance judgment processes.

The International Workshop on NeuroPhysiological Approaches for Interactive Information Retrieval (NeuroPhysIIR'25) aims to bringing together researchers from information science, humancomputer interaction, cognitive neuroscience, and related fields, to foster cross-disciplinary collaboration and accelerate progress in neurophysiologically-informed IIR research.As the third edition following successful workshops at SIGIR'15 [5] and CHIIR'17 [6], we anticipate that the interactive nature of this workshop will not only raise awareness but also lower the entry barriers for engaging with this exciting research area within the wider IIR community.Workshop website: https://neurophysiir.github.io/chiir2025/.

Relevance assessment is fundamental to Interactive Information Retrieval (IIR), and understanding the underlying cognitive processes is essential for optimizing user experience.To investigate how relevance and confirmation bias relate to cognitive load, we conducted a within-subject, lab-based, eye-tracking experiment (N=32).We applied the Low/High Index of Pupillary Activity (LHIPA) to measure cognitive load.LHIPA was calculated based on changes in pupil diameter and was unaffected by lighting conditions.Our results suggest that during relevance assessment: (1) for topically irrelevant documents, cognitive load is higher when users perceive them as relevant; for topically relevant documents, cognitive load is higher when users perceive them as irrelevant; (2) users with higher levels of topic familiarity and interest expend more cognitive load; and(3) users with higher predisposition to confirmation bias invest less cognitive load.These findings demonstrate that during relevance assessment, users' perceived relevance, topical knowledge, interest, and confirmation bias tendency influence cognitive load levels.This research has implications for improving user interface design to facilitate more accurate relevance judgments and more effective, unbiased information retrieval.

We contribute a comprehensive dataset to study user attention and purchasing behavior on Search Engine Result Pages (SERPs). Previous work has relied on mouse movements as a low-cost large-scale behavioral proxy but also has relied on self-reported ground-truth labels, collected at post-task, which can be inaccurate and prone to biases. To address this limitation, we use an eye tracker to construct an objective ground-truth of continuous visual attention. Our dataset comprises 2,776 transactional queries on Google SERPs, collected from 47 participants, and includes: (1)~HTML source files, with CSS and images; (2)~rendered SERP screenshots; (3)~eye movement data; (4)~mouse movement data; (5)~bounding boxes of direct display and organic advertisements; and (6)~scripts for further preprocessing the data. In this paper we provide an overview of the dataset and baseline experiments (classification tasks) that can inspire researchers about the different possibilities for future work.

An oft-repeated ideal of personal information management (PIM) is to have “the right information, at the right time, in the right place…” for the current need. But the technologies and innovations that bring us ever closer to this ideal carry costs as well as benefits. In this ninth in a series of PIM workshops, we give closer, critical consideration to the “right time, right place” ideal of PIM. Can we manage the potential downsides involved in achieving this ideal, while preserving its obvious benefits? Or should we revise our ideal of PIM?

Misinformation is an important topic in the Information Retrieval (IR) context and has implications for both system-centered and user-centered IR. While it has been established that the performance in discerning misinformation is affected by a person’s cognitive load, the variation in cognitive load in judging the veracity of news is less understood. To understand the variation in cognitive load imposed by reading news headlines related to COVID-19 claims, within the context of a fact-checking system, we conducted a within-subject, lab-based, quasi-experiment (N=40) with eye-tracking. Our results suggest that examining true claims imposed a higher cognitive load on participants when news headlines provided incorrect evidence for a claim and were inconsistent with the person’s prior beliefs. In contrast, checking false claims imposed a higher cognitive load when the news headlines provided correct evidence for a claim and were consistent with the participants’ prior beliefs. However, changing beliefs after examining a claim did not have a significant relationship with cognitive load while reading the news headlines. The results illustrate that reading news headlines related to true and false claims in the fact-checking context impose different levels of cognitive load. Our findings suggest that user engagement with tools for discerning misinformation needs to account for the possible variation in the mental effort involved in different information contexts.

Based on self-efficacy theory, our study investigated the relationship between perceived eHealth literacy and information search outcome. Information search outcome was measured by knowledge gained and the change in confidence level. We developed two hypotheses suggesting that high perceived eHealth literacy participants will gain more knowledge and become more confident in their post-search answers. A quasi-experimental study was conducted. 17 participants each from high and low perceived eHealth literacy groups used Google search engine to search for three topics, each with three factual questions. The results showed that both perceived eHealth literacy groups were able to find the correct answers, but only high perceived eHealth literacy participants were more confident in their search outcome. The finding corroborates the positive relationship between efficacy and outcome expectations in self-efficacy theory.

We conducted a lab-based eye-tracking study to investigate how interactivity of an AI-powered fact-checking system affects user interactions, such as dwell time, attention, and mental resources involved in using the system. A within-subject experiment was conducted, where participants used an interactive and a non-interactive version of a mock AI fact-checking system, and rated their perceived correctness of COVID-19 related claims. We collected web-page interactions, eye-tracking data, and mental workload using NASA-TLX. We found that the presence of the affordance of interactively manipulating the AI system's prediction parameters affected users’ dwell times, and eye-fixations on AOIs, but not mental workload. In the interactive system, participants spent the most time evaluating claims’ correctness, followed by reading news. This promising result shows a positive role of interactivity in a mixed-initiative AI-powered system.

Collecting participant search logs is an integral part of interactive IR research. Today's existing approaches are either piecemeal solutions, and/or require cumbersome setups. We present YASBIL, a two-component logging solution comprising a browser extension and a WordPress plugin. The browser extension logs the browsing activity in the participants' machines. The WordPress plugin collects the logged data into the researcher's data server. The logging works on any webpage, without the need to own or have knowledge about the HTML structure of the webpage. YASBIL also offers ethical data transparency and security towards participants, by enabling them to view and obtain copies of the logged data, as well as securely upload the data to the researcher's server over an HTTPS connection. We posit that ease of installation and use will make YASBIL especially suitable for remote user-studies, and longitudinal studies in IR.

We propose an image-classification method to predict the perceived-relevance of text documents from eye-movements. An eye-tracking study was conducted where participants read short news articles, and rated them as relevant or irrelevant for answering a trigger question. We encode participants' eye-movement scanpaths as images, and then train a convolutional neural network classifier using these scanpath images. The trained classifier is used to predict participants' perceived-relevance of news articles from the corresponding scanpath images. This method is content-independent, as the classifier does not require knowledge of the screen-content, or the user's information-task. Even with little data, the image classifier can predict perceived-relevance with up to 80% accuracy. When compared to similar eye-tracking studies from the literature, this scanpath image classification method outperforms previously reported metrics by appreciable margins. We also attempt to interpret how the image classifier differentiates between scanpaths on relevant and irrelevant documents.

We investigate the relationship between search behavior, eye -tracking measures, and learning. We conducted a user study where 30 participants performed searches on the web. We measured their verbal knowledge before and after each task in a content-independent manner, by assessing the semantic similarity of their entries to expert vocabulary. We hypothesize that differences in verbal knowledge-change of participants are reflected in their search behaviors and eye-gaze measures related to acquiring information and reading. Our results show that participants with higher change in verbal knowledge differ by reading significantly less, and entering more sophisticated queries, compared to those with lower change in knowledge. However, we do not find significant differences in other search interactions like page visits, and number of queries.

The field of neuroscience has fruitfully contributed to a wide variety of other fields, for example, economics, marketing and information systems, where the broad adoption and influence of neurophysiological (NP) research tools led to the creation of several new subfields, including neuroeconomics.

Search effort is an important aspect of Interactive Information Retrieval (IIR). Prior findings show that higher cognitive ability searchers tend to perform more actions than lower ability searchers. In an eye-tracking lab study we investigated the effects of working memory (WM) on search effort. The findings show that higher WM searchers perform more actions and that most significant differences are in time spent on reading results pages. We also show that behavior of high and low WM searchers changes differently in the course of a search task performance.

We investigate query characteristics and click behavior on SERPs of children in grades 6 and 8 (ages 11 and 13, respectively). We employ Retrospective Think-Aloud (RTA) protocol to elicit children's thought processes while clicking on results and to identify the sources of information that shaped these processes. We analyze the effect of grade level and task type on query characteristics and click behavior. Early findings show statistical significance across the three tasks in relation to query characteristics and between younger and older children in relation to query entry duration, count of web pages visited and count of result pages clicked on in SERPs. Our study confirms findings reported in previous research, including large-scale search logs studies, and reveals new findings in some areas.

The workshop will be the second in a series, building upon a successful workshop held at the SIGIR 2015 conference. The main aim is to focus on a narrow but highly important set of topics that have been identified in the last workshop and are of importance to IR and IIR researchers. The core theme of the workshop will be challenges is using and applying neurophysiological experimental methodologies and their applicability and adaptation in the context of IR and IIR research studies. The goal will be to clarify some of the major neurophysiological methodological concepts and constructs relevant to IR and IIR, expand awareness of critical parameters associated with neurophysiological devices and equipment, and provide a foundation to researchers to enable them to apply appropriate neurophysiological modalities for specific types of IR and IIR research investigations. Discussions on establishing reference tasks and standard data sets and launching a special journal issue on key topics will round out the event.

Assessment of text relevance is an important aspect of human–information interaction. For many search sessions it is essential to achieving the task goal. This work investigates text relevance decision dynamics in a question‐answering task by direct measurement of eye movement using eye‐tracking and brain activity using electroencephalography EEG. The EEG measurements are correlated with the user's goal‐directed attention allocation revealed by their eye movements. In a within‐subject lab experiment ( N = 24), participants read short news stories of varied relevance. Eye movement and EEG features were calculated in three epochs of reading each news story (early, middle, final) and for periods where relevant words were read. Perceived relevance classification models were learned for each epoch. The results show reading epochs where relevant words were processed could be distinguished from other epochs. The classification models show increasing divergence in processing relevant vs. irrelevant documents after the initial epoch. This suggests differences in cognitive processes used to assess texts of varied relevance levels and provides evidence for the potential to detect these differences in information search sessions using eye tracking and EEG.

In this paper, the potential for expanding the set of scientific evidence and insights associated with the users' role during the search process is explored. As it is intended to be a position paper and not a systematic survey, a comprehensive review of literature is not presented here. However, the authors draw on some early stage research, in this emerging area, to describe and explain the generation of neuro-physiological evidence using three types of modalities. The modalities and the associated methods described here, presented in order of increasing complexity, include Eye-tracking, EEG, and fMRI. The paper concludes with a few critical observations regarding the promises and perils of using neuro-physiological approaches in studying search and search behavior.

Query auto completion (QAC) is nearly ubiquitous in modern search systems, however, there are few published studies on how searchers use QAC query suggestions. This study describes the use of QAC by 29 searchers working on eight assigned search topics in a lab setting. We found that our subjects had differing propensities to use QAC, with some searchers rarely using it, while others used QACs for about half their queries. This study extends prior work by examining the use of QAC in the context of whole search sessions across multiple topics, and by comparing search behavior and visual attention for queries that used QAC and those that did not. The study concludes with an exploration of differences in QAC usage and query behavior among searchers, which reveals six possible query entry profiles.

The "Search as Learning" (SAL) workshop is focused on an area within the information retrieval field that is only beginning to emerge: supporting users in their learning whilst interacting with information content.

In this paper, we present a cognitive-economic approach to examining the cost in information search. Unlike previous studies on economic models, we calculated the cost in information search based on participants' eye-tracking data as well as their behavioral data, such as query formulation, search task duration, SERP and web page visits. Using Principal Component Analysis (PCA), we explored a possible latent factor structure of variables representing the cost in information search. Our results indicated that the cost of information seeking could be associated with two distinct aspects of search, exploratory and validation processes.

This short paper presents initial results from a project, in which we investigated differences in how users view relevant and irrelevant Web pages on their visits and revisits. The users' viewing of Web pages was characterized by eye-tracking measures, with a particular attention paid to changes in pupil size. The data was collected in a lab-based experiment, in which users (N=32) conducted assigned information search tasks on Wikipedia. We performed non-parametric tests of significance as well as classification. Our findings demonstrate differences in eye-tracking measures on visits and revisits to relevant and irrelevant pages and thus indicate a feasibility of predicting perceived Web document relevance from eye-tracking data. In particular, relative changes in pupil size differed significantly in almost all conditions. Our work extends results from previous studies to more realistic search scenarios and to Web page visits and revisits.

This Tutorial+Workshop will discuss opportunities and challenges involved in using neuro-physiological tools/techniques (such as fMRI, fNIRS, EEG, eye-tracking, GSR, HR, and facial expressions) and theories in information retrieval. The hybrid format will engage researchers and students at different levels of expertise, from those who are active in this area to those who are interested and want to learn more. The workshop will combine presentations, discussions and tutorial elements and consist of four segments (tutorial, completed research, work-in-progress, closing panel).

While many multidimensional models of relevance have been posited, prior studies have been largely exploratory rather than confirmatory. Lacking a methodological framework to quantify the relationships among factors or measure model fit to observed data, many past models could not be empirically tested or falsified. To enable more positivist experimentation, Xu and Chen [77] proposed a psychometric framework for multidimensional relevance modeling. However, we show their framework exhibits several methodological limitations which could call into question the validity of findings drawn from it. In this work, we identify and address these limitations, scale their methodology via crowdsourcing, and describe quality control methods from psychometrics which stand to benefit crowdsourcing IR studies in general. Methodology we describe for relevance judging is expected to benefit both human-centered and systems-centered IR.

Click through events in search results pages (SERPs) are not reliable implicit indicators of document relevance. A user's task and domain knowledge are key factors in recognition and link selection and the most useful SERP document links may be those that best match the user's domain knowledge. User study participants rated their knowledge of genomics MeSH terms before conducting 2004 TREC Genomics Track tasks. Each participant's document knowledge was represented by their knowledge of the indexing MeSH terms. Results show high, intermediate, and low domain knowledge groups had similar document selection SERP rank distributions. SERP link selection distribution varied when participant knowledge of the available documents was analyzed. High domain knowledge participants usually selected a document with the highest personal knowledge rating. Low domain knowledge participants were reasonably successful at selecting available documents of which they had the most knowledge, while intermediate knowledge participants often failed to do so. This evidence for knowledge effects on SERP link selection may contribute to understanding the potential for personalization of search results ranking based on user domain knowledge.

In this paper, we report findings on how user behaviors vary in tasks with different difficulty levels as well as of different types. Two behavioral signals: document dwell time and number of content pages viewed per query, were found to be able to help the system detect when users are working with difficult tasks.

Abstract The search task and the system both affect the demand on cognitive resources during information search. In some situations the demands may become too high for a person. This article has a three‐fold goal. First, it presents and critiques methods to measure cognitive load. Second, it explores the distribution of load across search task stages. Finally, it seeks to improve our understanding of factors affecting cognitive load levels in information search. To this end, a controlled Web search experiment with 48 participants was conducted. Interaction logs were used to segment search tasks semiautomatically into task stages. Cognitive load was assessed using a new variant of the dual‐task method. Average cognitive load was found to vary by search task stages. It was significantly higher during query formulation and user description of a relevant document as compared to examining search results and viewing individual documents. Semantic information shown next to the search results lists in one of the studied interfaces was found to decrease mental demands during query formulation and examination of the search results list. These findings demonstrate that changes in dynamic cognitive load can be detected within search tasks. Dynamic assessment of cognitive load is of core interest to information science because it enriches our understanding of cognitive demands imposed on people engaged in the search process by a task and the interactive information retrieval system employed.

Abstract We investigated the role of subjective factors in the information search process. Forty‐eight participants each conducted six Web searches in a controlled setting. We examined relationships between subjective factors (happiness levels, satisfaction with and confidence in the search results, feeling lost during search, familiarity with and interest in the search topic, estimation of task difficulty) and objective factors (search behavior, search outcomes, and search‐task characteristics). Data analysis was conducted using a multivariate statistical test (canonical correlations analysis). The findings confirmed existence of several relationships suggested by prior research, including relationships between objective search task difficulty and the perception of task difficulty, and between subjective states and search behaviors and outcomes. One of the original findings suggests that higher happiness levels before and during the search correlate with better feelings after the search, but also correlate with worse search outcomes and lower satisfaction, suggesting that, perhaps, it pays off to feel some “pain” during the search to “gain” quality outcomes.