Our research in this paper lies at the intersection of Generative AI (GenAI) and search-as-learning (SAL). GenAI technologies (e.g., ChatGPT) have revolutionized how people search for and interact with information. However, we do not yet fully understand how people use GenAI systems to learn about complex topics. SAL research has studied how different tools can support learning with traditional document retrieval systems. Our research closely relates to SAL work that has investigated the effects of goal-setting on learning during search. We explore the influence of goal-setting on learning during information-seeking sessions with a GenAI system. We report on a between-subjects crowdsourced study (N = 120) in which participants were asked to learn about a complex topic using a GenAI system. The study had four conditions that varied along two factors (a 2 × 2 design). The first factor involved displaying related web results in addition to the GenAI output. The second factor involved giving participants access to the Subgoal Manager (SM), a tool designed to help people develop subgoals and take notes. We investigated the effects of both factors on: (RQ1) perceptions; (RQ2) behaviors; (RQ3) learning and retention; (RQ4) the types of requests issued to the system; and (RQ5) participants’ motivations for engaging (or not engaging) with the related web results. Results found that participants with access to the SM had higher post-task learning outcomes, did less copy/pasting into their notes, perceived the task as more difficult, and requested more examples and support for differentiating concepts from the GenAI system.

We present the SRL Perceptions Questionnaire (SPQ), developed to measure perceptions of self-regulated learning (SRL) after information seeking and learning sessions. In a crowd-sourced study (N = 127), participants completed the SPQ after searching to learn about a complex topic. The SPQ asked participants to report their perceptions of particular SRL constructs (e.g., planning, monitoring, strategy use, adapting). A principal component analysis supported a five-factor structure with high reliability (α > =.87). Perceived SRL did not correlate with normalized learning gains, yet pre-task and post-task perceptions showed correlations with several SPQ dimensions. We offer both the SPQ as an instrument for measuring SRL (processes critical to supporting human learning) after information seeking and insights into how perceptions of SRL constructs align with objective learning outcomes, pre-task perceptions, and post-task perceptions while learning during search.

Generative AI (GenAI) technologies such as ChatGPT are changing the ways people interact with information.To illustrate, popular search engines (e.g., Google) have started integrating responses from GenAI tools with the traditional search results.In this paper, we explore the integration of GenAI technology with traditional search in the context of a learning-oriented task.We report on a between-subjects study (𝑁 = 40) in which participants completed a complex, learning-oriented search task.Participants were assigned to one of two conditions.In the SearchOnly condition, participants used a traditional web search system to gather information.In the Search+Chat condition, participants used an experimental system that combined a traditional web search component and an interactive GenAI-based chat component (Chat AI).The study investigated seven research questions.RQ1-RQ3 focused on differences between groups: (RQ1) post-task perceptions, (RQ2) search behaviors, and (RQ3) learning outcomes.To measure learning, participants completed a multiple-choice test before the search task, immediately after, and one week later (to measure retention).RQ4-RQ7 delved deeper into participants' behaviors and experiences in the Search+Chat condition: (RQ4) motivations for (and gains from) engaging with the Chat AI; (RQ5) the phases during which participants engaged with the Chat AI; (RQ6) the types of queries issued to each component; and (RQ7) perceptions about the information returned by each component.

We present a user study (N = 40) that investigated the role of goal-setting on learning during search. To this end, we developed a tool called the Subgoal Manager (SM). The SM was designed to help searchers break apart a learning-oriented search task into smaller subgoals. The tool enabled participants to add, delete, and modify subgoals; take notes with respect to subgoals; and mark subgoals as completed. During the study, participants completed a single learning-oriented search task and were assigned to one of two subgoal conditions. In the Subgoals condition, participants had access to the SM; were instructed to develop at least three subgoals before the search session; and could add, delete, and modify subgoals during the search session. In the NoSubgoals condition, participants were not instructed to set subgoals and were simply provided with a text editor to take notes. We investigate the effects of the subgoal condition on: (RQ1) learning and retention and (RQ2) the extent to which participants engaged in specific self-regulated learning (SRL) processes during the search session. Our results found two important trends. First, participants in the Subgoals condition had better learning outcomes, especially with respect to retention. Second, based on a qualitative analysis of participants’ search sessions, participants in the Subgoals condition engaged in more self-regulated learning (SRL) processes. Combined, our results suggest that goal-setting improves learning during search by encouraging and supporting greater engagement with SRL processes.

Modern search systems are largely designed and optimized for simple navigational or fact-finding tasks, with little support for complex tasks involving comprehension and learning. In response, the search-as-learning research community has undertaken a wide range of research questions focused on understanding how various types of learning outcomes are affected by searcher characteristics, the search task, and the search system. Typically, these views embed learning within a search system. In this paper we take a different view, embedding search within a framework for an end-to-end learning system designed to support learning in a formal educational context. Our central goal is to motivate research questions aligned to advance progress on techniques for active support of comprehension and formal learning. Thus we intentionally set aside goals for informal and surface learning. We argue that to be effective, such a search-centric learning system must model four key components: individual students (searcher factors), the educational domain (topic factors), academic assignments (task factors), and progress toward learning goals (the objective function of the end-to-end system). In modeling these components, our hypothetical system makes inferences about students’ learning histories, knowledge states, comprehension, and the utilities of different types of information resources. We present examples of possible techniques and data sources for each model. We also introduce the novel concept of leveraging school assignments as rich task context. Our intention is not to propose a functional system, but to frame search-as-learning in the context of comprehension and to inspire research questions arising from an end-to-end view of this important research domain.

Search systems are often used to support learning-oriented goals. This trend has given rise to the “search-as-learning” movement, which proposes that search systems should be designed to support learning. To this end, an important research question is: How does a searcher’s type of learning objective (LO) influence their trajectory (or pathway ) toward that objective? We report on a lab study (N = 36) in which participants gathered information to meet a specific type of LO. To characterize LOs and pathways , we leveraged Anderson and Krathwohl’s (A&K’s) taxonomy [ 3 ]. A&K’s taxonomy situates LOs at the intersection of two orthogonal dimensions: (1) cognitive process (CP) (remember, understand, apply, analyze, evaluate, and create) and (2) knowledge type (factual, conceptual, procedural, and metacognitive knowledge). Participants completed learning-oriented search tasks that varied along three CPs (apply, evaluate, and create) and three knowledge types (factual, conceptual, and procedural knowledge). A pathway is defined as a sequence of learning instances (e.g., subgoals) that were also each classified into cells from A&K’s taxonomy. Our study used a think-aloud protocol, and pathways were generated through a qualitative analysis of participants’ think-aloud comments and recorded screen activities. We investigate three research questions. First, in RQ1, we study the impact of the LO on pathway characteristics (e.g., pathway length). Second, in RQ2, we study the impact of the LO on the types of A&K cells traversed along the pathway. Third, in RQ3, we study common and uncommon transitions between A&K cells along pathways conditioned on the knowledge type of the objective. We discuss implications of our results for designing search systems to support learning.

There is a growing body of research in the Search as Learning community that recognizes the need for users to learn during search, but modern search systems have yet to adapt to support this need. Our research proposes three research goals toward addressing the support of user learning during search. Research goal 1 (RG1) introduces a more precise and reliable metric of assessing user learning. Anderson & Krathwohl's 2-dimensional taxonomy is used as a framework to develop learning objectives and assessment questions to measure user learning during search. Additionally, Anderson & Krathwohl's taxonomy is used as a coding scheme to outline the pathways users traverse along the way to a particular learning objective. Research goal 2 (RG2) investigates the prediction of learning objectives using behavioral measures. Finally, research goal 3 (RG3) proposes a search system that presents information relevant to the user based on their current learning sub-goal and scaffolds information based on the pathways they are likely to traverse given a particular learning objective.