During a think-aloud study, participants verbalize their thoughts as they complete a specified task. Think-aloud comments provide insights into what a participant is doing and experiencing in the moment. Interactive information retrieval studies have used think-aloud protocols to explore different research questions. However, think-aloud protocols can also influence participants. We report on a qualitative analysis of data collected during a think-aloud study. Participants were asked to learn about a complex topic by searching online and taking notes. After the task, participants were asked whether and how thinking aloud influenced their approach to the task. Open-ended responses revealed 21 (positive and negative) ways in which thinking aloud influenced participants. Additionally, during the study, we measured participants’ working memory (WM) capacity. We did not find that thinking aloud had different influences for low- vs. high-WM participants.

Generative AI (gen AI) tools are rapidly reshaping higher education, influencing research, teaching, learning, and administrative work. This study investigates when, how, and why gen AI is used and not used in a research-intensive university. We conducted in-depth surveys with faculty and staff in mid-2024 (n=102) and mid-2025 (n=101) to capture evolving adoption and perception. Across both years, over 90% of respondents reported intentional use of gen AI, though some described unintentional use or intentional non-use. The most frequently mentioned tools included ChatGPT, Copilot, Gemini, Claude, and Adobe Firefly.

Information credibility is a central concept in information science research and is closely linked to how people evaluate and use information. Credibility perceptions also play an important role in the adoption and continued use of technologies such as generative AI (gen AI). As gen AI becomes increasingly integrated into everyday life, it is essential to understand how people perceive the credibility of information produced by these systems. As part of a larger survey study on the use (and non-use) and perceptions of gen AI in higher education conducted in mid-2024 and mid-2025, we evaluated a set of questionnaire items to assess the perceived credibility of information created by gen AI. In this paper, we present the questionnaire items, descriptive statistics, and correlation matrices from the two surveys, and the results of our exploratory factor analysis examining the underlying structure of the credibility measure. Across both datasets, we identified two distinct factors—output credibility, which refers to the perceived credibility of AI-produced output itself, and relative credibility, which refers to perceptions of AI-produced output relative to human-produced information. We share the instrument and findings to support future refinement and adaptation in studying credibility in the context of gen AI.

Working memory (WM) is involved in high-level cognitive tasks such as comprehension, reasoning, and learning.Search and sensemaking (SSM) is no exception-a wide range of (meta)cognitive activities are involved in the process of making sense of a complex topic by gathering information.Prior studies have found that WM can influence search behaviors, perceptions, and outcomes.However, little work has been done to gain insights into how WM might affect the SSM process.We report on a lab study (𝑁 = 44) in which participants were binned into low-and high-WM groups.During the study, participants were asked to learn about a complex and multifaceted topic by gathering information using a web search engine and taking notes.After the search session, participants were asked to produce a summary of everything they learned.The study investigated four research questions.RQ1 and RQ2 investigate the effects of WM on post-task perceptions and search behaviors.RQ3 investigates the effects of WM on the extent to which participants engaged in specific search, sensemaking, and cognitive activities.To address RQ3, the study used a think-aloud protocol.Search sessions (i.e., recorded actions and think-aloud comments) were then analyzed using qualitative techniques.Finally, RQ4 investigates the effects of WM on learning outcomes.Our RQ4 results found that high-WM participants had better learning outcomes.Our RQ2 and RQ3 results point to possible reasons why.Despite differences for RQ2-RQ4, there were no differences in post-task perceptions (RQ1).

Extensive research exists on user interactions with search engine result pages (SERPs) in desktop and mobile environments. However, relatively little work has focused on understanding how virtual reality (VR) users interact with SERPs in 3D immersive virtual environments (IVEs). Unlike 2D displays with established paradigms (e.g., ranked lists), 3D IVEs lack standardized methods for presenting search results. This work explores how different information arrangements in 3D virtual space impact users' search behaviors and preferences.

People often search for information to acquire procedural knowledge–“how to” knowledge about step-by-step procedures, methods, algorithms, techniques, heuristics, and skills. A procedural search task might involve implementing a solution to a problem, evaluating different approaches to a problem, and brainstorming on the types of problems that can be solved with a specific resource. We report on a study ( N =36) that aimed to better understand how people search for procedural knowledge. Much research has investigated how search task characteristics impact people’s perceptions and behaviors. Along these lines, we manipulated procedural search tasks along two orthogonal dimensions: product and goal. The product dimension relates to the main outcome of the task and the goal dimension relates to task’s success criteria. We manipulated tasks across three product categories and two goal categories. The study investigated four research questions. First, we examined the effects of the product and goal on participants’ (RQ1) pre-task perceptions, (RQ2) post-task perceptions, and (RQ3) search behaviors. Second, regardless of the task product and goal, by analyzing participants’ think-aloud comments and screen activities we closely examined how people search for procedural knowledge. Specifically, we report on (RQ4) important relevance criteria, types of information sought, and challenges.

People often search online for procedural (i.e., “how-to”) knowledge. A procedural search task might involve a do-it-yourself project, cooking a dish, fixing a problem, or learning a new skill. Prior research has studied procedural search tasks from different perspectives: estimating the frequency of procedural searches online, understanding how people acquire procedural knowledge in specific contexts, and developing tools to support procedural search. Less research has aimed at deeply understanding procedural search tasks “in the wild”. To bridge this gap, we conducted a survey (N = 128) on Amazon Mechanical Turk. Participants were asked to recall a recent procedural task for which they searched online. Participants were asked open-ended questions about the task itself and their unique situation (e.g., constraints and needs). Additionally, participants provided webpages they found useful in their searches and described the characteristics of the page that made it useful. Finally, they provided useful pieces of information from each selected page and explained what they gained from the information. Using an inductive coding approach, we analyzed participants’ responses to gain insights about: (1) procedural task characteristics, (2) goals, (3) constraints, (4) contextual factors, (5) relevance criteria, and (6) gains obtained from useful information. Based on our results, we discuss important implications for future research and system design.

User interactions with search engine result pages (SERPs) are well researched in desktop and mobile computing environments. However, relatively little work has focused on how users interact with SERPs presented in 3D immersive virtual environments (IVEs) using virtual reality head-mounted displays (VR HMDs). While 2D displays have well-understood methods to present search results (e.g., ranked lists of 10 blue links), 3D IVEs do not yet have established paradigms for presenting search results.

While current systems are effective in helping searchers resolve simple information needs (e.g., fact-finding), they provide less support for searchers working on complex information-seeking tasks. Complex search tasks involve a wide range of (meta)cognitive activities, including goal-setting, organizing information, drawing inferences, monitoring progress, and revising mental models and search strategies. We report on a lab study ( N = 32) that investigated the influences of a knowledge representation tool called the OrgBox, developed to support searchers with complex tasks. The OrgBox tool was integrated into a custom-built search system and allowed study participants to drag-and-drop textual passages into the tool, organize passages into logical groupings called “boxes”, and make notes on passages and boxes. The OrgBox was compared to a baseline tool (called the Bookmark) that allowed participants to save textual passages, but not organize them nor make notes. Knowledge representation tools such as the OrgBox may provide special benefits for users with different cognitive profiles. In this article, we explore two cognitive abilities: (1) working memory (WM) capacity and (2) switching (SW) ability. Participants in the study were asked to gather information on a complex subject and produce an outline for a hypothetical research article. We investigate the influences of the tool (OrgBox vs. Bookmark) and the participant’s working memory capacity and switching ability on three types of outcomes: (RQ1) search behaviors, (RQ2) post-task perceptions, and (RQ3) the quality of outlines produces by participants.

Prior studies have explored the information-seeking practices of specific professional communities, including lawyers, physicians, engineers, recruiters, and government workers. In this research, we investigate the information-seeking practices of intelligence analysts (IAs) employed by a U.S. government agency. Specifically, we focus on the needs, practices, and challenges related to IAs searching for procedural knowledge using an internal system called the Tradecraft Hub (TC Hub). The TC Hub is a searchable repository of procedural knowledge documents written by agency employees. Procedural knowledge (as opposed to factual and conceptual knowledge) includes knowledge about step-by-step procedures, techniques, methods, tools, technologies, and skills, and is inherently task-oriented. We report on a survey study involving 22 IAs who routinely use the TC Hub. Our survey was designed to address four research questions. In RQ1, we investigate the types of work-related objectives that motivate IAs to search the TC Hub. In RQ2, we investigate the types of information IAs seek when they search the TC Hub. In RQ3, we investigate important relevance criteria used by IAs when judging the usefulness of information. Finally, in RQ4, we investigate the challenges faced by IAs when searching the TC Hub. Based on our findings, we discuss implications for improving and extending searchable knowledge base systems such as the TC Hub that exist in many organizations.

Current search systems are effective in helping users complete simple search tasks (e.g., fact-finding). However, they provide less support for users completing complex search tasks. Complex search tasks involve a diverse set of cognitive and metacognitive activities, such as goal-setting, organizing information, drawing inferences, monitoring progress, and updating mental models. We report on a lab study (N=32) that investigated the uses and influences of a novel knowledge representation tool called the "OrgBox'', developed to support searchers with complex tasks. The OrgBox was integrated into a custom-built search system and allowed participants to save information by drag-and-dropping textual passages into the tool, organize passages into "boxes'', and make notes on passages and boxes. The OrgBox tool was compared to a baseline tool (the "Bookmark'') that allowed participants to save passages, but not organize them nor make notes. We investigate four research questions. In RQ1, we investigate the effects of the knowledge representation tool on participants' post-task perceptions. In RQ2-RQ4, we investigate: (RQ2) how participants used different features of each tool; (RQ3) the perceived benefits and challenges of each tool; and (RQ4) the influences of each tool on the approaches taken by participants to complete the task. To address RQ2-RQ4, we conducted a qualitative analysis of participants' responses during an exit interview. We discuss implications from our results for designing tools to support users with complex search tasks.

Prior work in IR has found that searchers under time constraints may adapt their search processes and perceive their task or their performance differently. In many of these prior studies, the task descriptions implicitly or explicitly conveyed an expectation of the amount of information needed to satisfy the task requirements in terms of number of pages (e.g., find N webpages on topic X) or the time to spend on the task (e.g., search until time is up) rather than allowing the participant to determine how much information was needed. In this lab-based study, we investigated the effects of time constraints on information search and decision-making. Participants completed a series of decision-making tasks with half of the participants receiving a 5-minute time constraint and half given no time guidance. They were asked to make good, specific recommendations for a friend, and they had considerable latitude in deciding how much information they needed. Results showed that participants in the time constraint condition made their decisions faster but there were few significant differences in measures of search behaviors between the time constraint conditions (RQ1). Qualitative analysis indicated that participants adapted their decision task by varying their recommendations in their specificity, justification strength, and contents in both time conditions (RQ2). Finally, we found evidence that the impact of the time constraint on time- and task-related perceptions was moderated by the extent to which participants adapted their decision task (RQ3).

The goal of this study is to advance our understanding of the cognitive mechanisms underlying search-sensemaking by examining the process through the lens of working memory (WM). I aim to extend the existing literature on search and sensemaking by incorporating the role of WM in the process. While WM plays a critical role in complex cognitive tasks, little is known about how it manifests in users' search-sensemaking while they are working with information in a complex information space such as the Web. A mixed-method design will be adopted, consisting of an experimental study and interviews. Adults with different levels of working memory capacity (WMC) will perform a learning assignment task in a lab. The combined data of screen activities and think-aloud comments will be analyzed to investigate the differential patterns in search-sensemaking between high- and low-WMC individuals (RQ1). An evolving working document will be considered an external representation of their knowledge structure and be analyzed to investigate the differential patterns in conceptual changes between high- and low-WMC individuals (RQ2). Finally, a post-session interview will be conducted to identify a range of cognitive strategies used during the task (RQ3). The proposed study will build upon the cognitive process model of sensemaking and the literature on executive functions and WM. The study will help us better understand the cognitive process of search-sensemaking from the perspective of WM. The findings can also inform design decisions for systems that can facilitate search-sensemaking while supporting users with varying levels of WMC.

An important question in interactive information retrieval (IIR) is: How do task characteristics influence users' needs? In this paper, we investigate the effects of cognitive task complexity on the types of information considered useful for a task. We characterize information types from two perspectives. From one perspective, we classify task-related information items based on inherent characteristics (referred to as info-types): factual statements, concepts/definitions, opinionated statements, and insights---tips/advice related to the task domain. From a second perspective, we used Byströ m and J\"a rvelin's framework~\citebystrom1995 to define information types based on how the information might be used to complete the task (referred to as functional roles): (1) to help the task doer understand the task requirements (problem information); (2) to help the task doer strategize on how to approach the task (problem-solving information); and (3) to help the task doer learn about the task domain (domain information). Our results suggest that: (1) cognitive task complexity influences the functional roles of information items deemed useful for the task (RQ1); (2) certain info-types are more (or less) likely to play certain functional roles (RQ2); and task complexity influences the variety of functional roles played by info-types (RQ3).

In interactive information retrieval, an important research question is: How do task characteristics influence users’ needs and behaviors? We report on a laboratory study ( N =32) that investigated the effects of task complexity on the types of information used by participants while searching. Participants completed tasks of four complexity levels and had access to four different types of information provided through a search-assistance tool referred to as the InfoBoxes (IB). The IB tool presented the following types of task-related information ( info-types ) on different tabs: (1) facts, (2) concepts, (3) opinions, and (4) insights. Facts (and opinions) were defined as objective (and subjective) statements relevant to the task. Concepts were defined as important ideas, principles, or entities related to the task. Insights were defined as tips or advice about the task. The study investigated six research questions that considered the effects of task complexity on: (RQ1) participants’ pre-/post-task perceptions about useful info-types; (RQ2) use of different info-types during the task; (RQ3) motivations for engaging with the IB; (RQ4) gains from using it; (RQ5) the search stage participants were in while engaging with the IB; and (RQ6) motivations for sometimes avoiding the IB. Our results suggest that task complexity influenced all six types of outcomes. We discuss implications of our results for designing search assistance tools and systems that favor certain types of content based on task characteristics.

We report on a study that evaluated the effects of working memory and task complexity on participants' perceptions, behaviors, and outcomes. Twenty-four participants performed two search tasks of varying complexity and completed a psychometric test to measure working memory ability. Our results found several important trends. First, task complexity had an effect on participants' perceptions about temporal demand and satisfaction with the time spent on the task. Second, participants with higher working memory exerted more search effort (e.g., issued more queries). Third, participants with higher working memory had better outcomes, particularly during more complex tasks. Finally, while participants with lower working memory exerted less effort (engaged in satisficing behaviors) and had weaker outcomes, working memory did not affect participants' post-task perceptions about workload and satisfaction. We discuss implications of our results for developing search tools to support users with varying levels of working memory.

Prior work has studied how different characteristics of individual users (e.g., personality traits and cognitive abilities) can impact search behaviors and outcomes. We report on a laboratory study ( N = 32) that investigated the effects of three different cognitive abilities (perceptual speed, working memory, and inhibition) in the context of aggregated search. Aggregated search systems combine results from multiple heterogeneous sources (or verticals ) in a unified presentation. Participants in our study interacted with two different aggregated search interfaces (a within-subjects design) that differed based on the extent to which the layout distinguished between results originating from different verticals. The interleaved interface merged results from different verticals in a fairly unconstrained fashion. Conversely, the blocked interface displayed results from the same vertical as a group, displayed each group of vertical results in the same region on the SERP for every query, and used a border around each group of vertical results to help distinguish among results from different sources. We investigated three research questions (RQ1--RQ3). Specifically, we investigated the effects of the interface condition and each cognitive ability on three types of outcomes: (RQ1) participants’ levels of workload, (RQ2) participants’ levels of user engagement, and (RQ3) participants’ search behaviors. Our results found different main and interaction effects. Perceptual speed and inhibition did not significantly affect participants’ workload and user engagement but significantly affected their search behaviors. Specifically, with the interleaved interface, participants with lower perceptual speed had more difficulty finding relevant results on the SERP, and participants with lower inhibitory attention control searched at a slower pace. Working memory did not have a strong effect on participants’ behaviors but had several significant effects on the levels of workload and user engagement reported by participants. Specifically, participants with lower working memory reported higher levels of workload and lower levels of user engagement. We discuss implications of our results for designing aggregated search interfaces that are well suited for users with different cognitive abilities.

An important area of IR research involves understanding how task characteristics influence search behaviors and outcomes. Task complexity is one characteristic that has received considerable attention. One view of task complexity is through the lens of a priori determinability -- the level of uncertainty about task outcomes and processes experienced by the searcher. In this work, we manipulated the determinability of comparative tasks. Our task manipulation involved modifying the scope of the task by specifying exact items and/or exact (objective or subjective) dimensions to consider as part of the task. This paper reports on a within-subject study (N=144) where we investigated how our task manipulation influenced participants' perceptions, levels of engagement, search effort, and choice of search strategies. Our results suggest a complex relationship between task scope, determinability, and different outcome measures. Our most open-ended tasks were perceived to have low determinability (high uncertainty), but were the least challenging for participants due to satisficing. Furthermore, narrowing the scope of tasks by specifying items had a different effect than by specifying dimensions. Specifying items increased the task determinability (lower uncertainty) and made the task easier, while specifying dimensions did not increase the task determinability and made the task more challenging. A qualitative analysis of participants' queries suggests that searching for dimensions is more challenging than for items. Finally, we observed subtle differences between objective and subjective dimensions. We discuss implications for the design of IIR studies and tools to support users.

We report on a crowdsourced study that investigated how two factors influence the way people formulate information requests. Our first factor, medium , considers whether the request is produced using text or voice. Our second factor, target , considers whether the request is intended for a search engine or a human intermediary (i.e., someone who will search on the user’s behalf). In particular, we study how these two factors influence the way people formulate requests in situations where the information need has a specific type of extra-topical dimension (i.e., a type of constraint that is independent from the information need’s topic). We focus on six extra-topical dimensions: (1) domain knowledge, (2) viewpoint, (3) experiential, (4) venue location, (5) source location, and (6) temporal. The extra-topical dimension was manipulated by giving participants carefully constructed search tasks. We analyzed a large number of information requests produced by study participants, and address three research questions. We study the effects of our two factors (medium and target) on (RQ1) participants’ perceptions about their own information requests, (RQ2) the different characteristics of their information requests (e.g., natural language structure, retrieval performance), and (RQ3) participants’ strategies for requesting information when the search task has a specific type of extra-topical dimension. Our results found that both factors influenced participants’ perceptions about their own information requests, the characteristics of participants’ requests, and the strategies adopted by participants to request information matching the extra-topical dimension. Our results have implications for future research on methods that can harness (rather than ignore) extra-topical query terms to retrieve relevant information.