Our research in this paper lies at the intersection of Generative AI (GenAI) and search-as-learning (SAL). GenAI technologies (e.g., ChatGPT) have revolutionized how people search for and interact with information. However, we do not yet fully understand how people use GenAI systems to learn about complex topics. SAL research has studied how different tools can support learning with traditional document retrieval systems. Our research closely relates to SAL work that has investigated the effects of goal-setting on learning during search. We explore the influence of goal-setting on learning during information-seeking sessions with a GenAI system. We report on a between-subjects crowdsourced study (N = 120) in which participants were asked to learn about a complex topic using a GenAI system. The study had four conditions that varied along two factors (a 2 × 2 design). The first factor involved displaying related web results in addition to the GenAI output. The second factor involved giving participants access to the Subgoal Manager (SM), a tool designed to help people develop subgoals and take notes. We investigated the effects of both factors on: (RQ1) perceptions; (RQ2) behaviors; (RQ3) learning and retention; (RQ4) the types of requests issued to the system; and (RQ5) participants’ motivations for engaging (or not engaging) with the related web results. Results found that participants with access to the SM had higher post-task learning outcomes, did less copy/pasting into their notes, perceived the task as more difficult, and requested more examples and support for differentiating concepts from the GenAI system.

The aim of this study is to explore connections among the reasons that lead users to stop a search session, the types of information they found during the session, and the cognitive activities involved in interacting with such information – all in the context of complex tasks involving multiple search sessions. Prior research has primarily examined search-stopping behavior in single search sessions, focusing on instances when users identify needed information or are satisfied with their findings. However, recent insights from the Search as Learning (SAL) community highlight the relationships between user search behavior, the nature of the information they encountered, and various cognitive activities during the information-searching process. We conducted a diary study with 25 participants engaged in real-life tasks that spanned multiple search sessions over time. Our analysis found six predominant reasons users elected to stop a search session during the cross-session search process. We also examined the types of information that participants found and the cognitive activities that they engaged in during search sessions. We found statistically significant associations between the information types found and the session stopping reasons. We did not find significant associations between the cognitive activities and stopping reasons, but did observe that almost half the sessions reached the evaluate level of cognitive activity. Our findings provide insights about search behaviors and help inform the design of tools to support users working on multi-session search tasks.

Abstract Exploratory searches involve significant cognitively demanding aiming at learning and investigation. However, users gain little support from search engines for their cognitive and metacognitive activities (e.g., discovery, synthesis, planning, transformation, monitoring, and reflection) during exploratory searches. To better support the exploratory search process, we designed a new search assistance tool called OrgBox. OrgBox allows users to drag‐and‐drop information they find during searches into “boxes” and “items” that can be created, labeled, and rearranged on a canvas. We conducted a controlled, within‐subjects user study with 24 participants to evaluate the OrgBox versus a baseline tool called the OrgDoc that supported rich‐text features. Our findings show that participants perceived the OrgBox tool to provide more support for grouping and reorganizing information, tracking thought processes, planning and monitoring search and task processes, and gaining a visual overview of the collected information. The usability test reveals users' preferences for simplicity, familiarity, and flexibility of the design of OrgBox, along with technical problems such as delay of response and restrictions of use. Our results have implications for the design of search‐assisting systems that encourage cognitive and metacognitive activities during exploratory search processes.

Users often search for information online across multiple sessions for a specific task. Sometimes cross-session searches may be unanticipated, however, in many cases users know beforehand that their search will spread across multiple sessions. Little is known about why people expect to search across multiple sessions. Based on a qualitative analysis of the interview data collected from a diary study, we identified two types of reasons that motivate people to conduct cross-session searches: (1) task-oriented motivations, (2) cognition-oriented motivations. We also identified a group of reasons that cause challenges for cross-session search. Our findings have implications for the future design of search systems to support cross-session search from different aspects.

As an important skill in many domains, creativity is understood to be a process that includes stages like finding goals, looking up information, exploring, creating ideas, combining ideas, selecting ideas, and executing plans [50]. However, little prior information retrieval research has considered how people use search tools and information resources to support their creative stages.

Prior work in IR has found that searchers under time constraints may adapt their search processes and perceive their task or their performance differently. In many of these prior studies, the task descriptions implicitly or explicitly conveyed an expectation of the amount of information needed to satisfy the task requirements in terms of number of pages (e.g., find N webpages on topic X) or the time to spend on the task (e.g., search until time is up) rather than allowing the participant to determine how much information was needed. In this lab-based study, we investigated the effects of time constraints on information search and decision-making. Participants completed a series of decision-making tasks with half of the participants receiving a 5-minute time constraint and half given no time guidance. They were asked to make good, specific recommendations for a friend, and they had considerable latitude in deciding how much information they needed. Results showed that participants in the time constraint condition made their decisions faster but there were few significant differences in measures of search behaviors between the time constraint conditions (RQ1). Qualitative analysis indicated that participants adapted their decision task by varying their recommendations in their specificity, justification strength, and contents in both time conditions (RQ2). Finally, we found evidence that the impact of the time constraint on time- and task-related perceptions was moderated by the extent to which participants adapted their decision task (RQ3).

Previous studies found that cross-session search tasks are often complex due to the uncertainty of task features, users' perception of the tasks, and the challenges (e.g., time limitation, interruptions) that users encounter during the process. Using Anderson & Krathwohl's taxonomy of learning, we analyzed the complexity of real-life cross-session search tasks reported by 110 Amazon Mechanical Turk workers. We found that cross-session search tasks often require different types of information (i.e., factual and conceptual information, procedure information, opinions, and information for helping metacognition) and are frequently associated with higher-level cognitive activities, including: applying analyzing, evaluating, and creating. Our findings add knowledge about cross-session search, and have insights for future research related to system design and studying cross-session search.

Current search systems provide effective support to users engaged in fact-finding and look-up oriented tasks. However, they provide relatively little support for users engaged in exploratory search tasks that involve cognitive and metacognitive activities such as learning, synthesis, planning, and reflection. We conducted a within-subject user study (N=24) that investigated the effects of a novel knowledge organization tool called the OrgBox, designed to assist users with organizing and synthesizing information, and metacognitive activities. The OrgBox included features to allow users to drag-drop information they found through search into "boxes" that could be created, labelled, and re-arranged. Study participants completed two exploratory search tasks, one with the OrgBox, and one with the OrgDoc, a baseline tool that included features of a rich-text editor (e.g., formatting, bullets) for taking notes.

We report on a survey about people's cross-session search activities in their everyday work and life. We recruited a broad range of participants (N=110) using the Amazon Mechanical Turk service. The survey asked people to describe a recent task in which they searched across multiple sessions, to recall the reasons they started and stopped the most recent search session for the task, and to explain methods they used for reacquainting themselves with the task at the beginning of the most recent session. Across a broad range of tasks reported by our participants, our results show that a majority of the cross-session work tasks involved high levels of cognitive complexity, consultation with additional human information sources (in addition to search engines), and often involved multiple devices (e.g., computer and smartphone). Our analysis of reasons why people stopped and restarted search sessions extends and validates the reasons outlined in Lin and Belkin's model of Multiple Information Seeking Episodes and previous results from MacKay and Watters' study of multi-session search. We also identified methods that searchers use to re-acquaint themselves when restarting multi-session searches, and reasons why they may choose not to use any re-acquainting method. Our results update prior work and provide insights about how search systems can better support cross-session work tasks.

A recent study found that search engines play an important role in supporting creative tasks that people encounter in everyday life. However, little research has been conducted to understand how people use search engines and other information resources to support creative endeavors in their day-to-day work and life. To gain a deeper understanding, we conducted a diary study by asking eleven participants to record diary entries about their own design-related creative projects over a two-week period. In this paper, we report the findings of our qualitative analysis of participants' diary entries and post-diary in-depth interviews. Specifically, we identified six types of information that were essential to design-related creative projects: procedural information, domain information, finished examples, tips/opinions/recommendations, information about specific topics, and inspiring/motivating information. For each type of information, we discuss participants' intents to use the information in their projects. We also describe challenges reported by our participants when they used search engines to address their information needs. Finally, we discuss implications of our findings for future research and the future design of search systems.

In interactive information retrieval, an important research question is: How do task characteristics influence users’ needs and behaviors? We report on a laboratory study ( N =32) that investigated the effects of task complexity on the types of information used by participants while searching. Participants completed tasks of four complexity levels and had access to four different types of information provided through a search-assistance tool referred to as the InfoBoxes (IB). The IB tool presented the following types of task-related information ( info-types ) on different tabs: (1) facts, (2) concepts, (3) opinions, and (4) insights. Facts (and opinions) were defined as objective (and subjective) statements relevant to the task. Concepts were defined as important ideas, principles, or entities related to the task. Insights were defined as tips or advice about the task. The study investigated six research questions that considered the effects of task complexity on: (RQ1) participants’ pre-/post-task perceptions about useful info-types; (RQ2) use of different info-types during the task; (RQ3) motivations for engaging with the IB; (RQ4) gains from using it; (RQ5) the search stage participants were in while engaging with the IB; and (RQ6) motivations for sometimes avoiding the IB. Our results suggest that task complexity influenced all six types of outcomes. We discuss implications of our results for designing search assistance tools and systems that favor certain types of content based on task characteristics.

Recently, there has been interest in integrating information retrieval systems more closely with users' knowledge development processes, especially to support exploratory search. In this work, we investigated how people organize and structure information they discover during exploratory searches. In a lab study, we asked 24 participants to take hand-written notes they could use in the future while they were completing an exploratory search. We then asked participants to organize their findings to share with someone else who wants to explore the topic. Finally, we conducted post-session interviews to gain insights into the types of information saved and how participants organized the information they found. In our qualitative analysis of the notes and interviews, we found that the notes included background information about the topic, key concepts, specific details, useful information sources, and information to help with the broader work task. Notes were primarily structured in lists, and reflected a combination of linear note-taking strategies and grouping by information source or topical themes. Participants changed the content and structure of the shared notes to make them easier to understand and to provide a more thorough overview of the topic. Our findings have implications for the design of search tools to help current searchers organize, structure, and synthesize information and to help future searchers engaged in similar information seeking tasks.

An important area of IR research involves understanding how task characteristics influence search behaviors and outcomes. Task complexity is one characteristic that has received considerable attention. One view of task complexity is through the lens of a priori determinability -- the level of uncertainty about task outcomes and processes experienced by the searcher. In this work, we manipulated the determinability of comparative tasks. Our task manipulation involved modifying the scope of the task by specifying exact items and/or exact (objective or subjective) dimensions to consider as part of the task. This paper reports on a within-subject study (N=144) where we investigated how our task manipulation influenced participants' perceptions, levels of engagement, search effort, and choice of search strategies. Our results suggest a complex relationship between task scope, determinability, and different outcome measures. Our most open-ended tasks were perceived to have low determinability (high uncertainty), but were the least challenging for participants due to satisficing. Furthermore, narrowing the scope of tasks by specifying items had a different effect than by specifying dimensions. Specifying items increased the task determinability (lower uncertainty) and made the task easier, while specifying dimensions did not increase the task determinability and made the task more challenging. A qualitative analysis of participants' queries suggests that searching for dimensions is more challenging than for items. Finally, we observed subtle differences between objective and subjective dimensions. We discuss implications for the design of IIR studies and tools to support users.