Conversational search systems increasingly provide source citations, yet how citation or source presentation formats influence user engagement remains unclear. We conducted a crowdsourcing user experiment with 394 participants comparing four source presentation designs that varied citation visibility and accessibility: collapsible lists, hover cards, footer lists, and aligned sidebars. High-visibility interfaces generated more hovering on sources, though clicking remained infrequent across all conditions. While interface design showed limited effects on user experience and perception measures, it significantly influenced knowledge, interest, and agreement changes. High-visibility interfaces initially reduced knowledge gain and interest, but these positive effects emerged with increasing source usage. The sidebar condition uniquely increased agreement change. Our findings demonstrate that source presentation alone may not enhance engagement and can even reduce it when insufficient sources are provided.

In this study, we conducted semi-structured interviews with 21 IIR researchers to investigate their data reuse practices. This study aims to expand upon current findings by exploring IIR researchers’ information obtaining behaviors regarding data reuse. We identified the information about shared data characteristics that IIR researchers needed when evaluating data reusability, as well as the sources they typically consulted to obtain this information. We consider this work to be an initial step towards revealing IIR researchers’ data reuse practices and find out what the community need to do to promote data reuse. We hope that this study, as well as future research, will inspire more individuals to contribute to the ongoing efforts aimed at designing the standards, infrastructures, and policies, as well as fostering a sustainable culture for data sharing and reuse in this field.

Recent research has explored LLMs as scalable tools for relevance labeling, but studies indicate they are susceptible to priming effects, where prior relevance judgments influence later ones. Although psychological theories link personality traits to such biases, it is unclear whether simulated personalities in LLMs exhibit similar effects. We investigate how Big Five personality profiles in LLMs influence priming in relevance labeling, using multiple LLMs on TREC 2021 and 2022 Deep Learning Track datasets. Our results show that certain profiles, such as High Openness and Low Neuroticism, consistently reduce priming susceptibility. Additionally, the most effective personality in mitigating priming may vary across models and task types. Based on these findings, we propose personality prompting as a method to mitigate threshold priming, connecting psychological evidence with LLM-based evaluation practices.

Characterizing users' judgments and interactions with search engine result pages (SERPs) has been a central theme in Interactive Information Retrieval (IIR) evaluation.In contrast to the perfect rationality assumptions underpinning most existing formal models, people are boundedly rational and are subject to the influence of systematic cognitive biases.To enhance the psychological foundation of user models and better understand the in-situ decisions of boundedly rational users, our between-subject crowdsourcing experiment explored Decoy Effect on users' vulnerability to COVID treatment misinformation, which causes enduring impacts on people's personal health management and wellbeing, and examined the extent to which this effect is moderated by contextual factors and user characteristics.Our results, derived from 540 participants and 2,160 valid SERP evaluation records, indicate that: 1) users' interactions with decoy results may increase their vulnerability to medical misinformation in usefulness and credibility judgments; 2) the size of decoy effect is conditioned by users' prior knowledge and the rank position of decoy results.This research empirically reveals the impact of decoy results on users' context-dependent preferences on ranked search results under varying topics and conditions of medical information evaluation.More broadly, it demonstrates the value of representing and modeling users interacting with information as boundedly rational agents and serves as a step forward towards achieving the goal of truly human-centered IIR.

In the information retrieval (IR) domain, evaluation plays a crucial role in optimizing search experiences and supporting diverse user intents.In the recent LLM era, research has been conducted to automate document relevance labels.These labels have traditionally been assigned by crowd-sourced workers, a process that is both time consuming and costly.This study focuses on LLM-generated usefulness labels, a crucial evaluation metric that considers the user's search intents and task objectives, an aspect where relevance falls short.Our experiment utilizes task-level, query-level, and document-level features along with user search behavior signals, which are essential in defining the usefulness of a document.Our research finds that (i) pre-trained LLMs can generate moderate usefulness labels by understanding the comprehensive search task session, and (ii) pre-trained LLMs perform better judgment in short search sessions when provided with search session contexts.Furthermore, we investigate whether LLMs can capture the unique divergence between relevance and usefulness, along with conducting an ablation study to identify the most critical metrics for accurate usefulness label generation.In conclusion, this work explores LLM-generated usefulness labels by evaluating critical metrics and optimizing for practicality in real-world settings.

Web-based learning is evolving rapidly as traditional search engines are complemented by Large Language Models (LLMs) and other AI technologies. This evolution offers new opportunities, such as automated information synthesis and personalized learning experiences. However, this also presents new challenges, including the need for learners to be aware of potential biases and misinformation in AI-generated content, and to maintain focus and depth in their learning journeys.

Abstract Reusing research data can effectively reduce efforts in data collection and enhance the replicability of evaluation experiments, especially for small laboratories and research teams studying human‐centered systems. Building a sustainable data reuse process and culture relies on frameworks that encompass policies, standards, roles, and responsibilities, all of which must address the diverse needs of data providers, curators, and reusers. This study investigated data reuse practices of experienced researchers in Interactive Information Retrieval (IIR), a field where data reuse has been strongly advocated but still remains a challenge. We conducted 21 semi‐structured in‐depth interviews with IIR researchers from varying demographic backgrounds, institutions, and career stages about their motivations, experiences, and concerns regarding data reuse. We uncovered the rationales, criteria, and strategies they used in reusability assessments, as well as the challenges they faced when attempting to reuse research data in their studies. These empirical findings enrich ongoing discussions about the reusability of user‐generated data and research resources and help promote community‐level data reuse culture and standards in both traditional and emerging IIR research fields.

Abstract Algorithms play a significant role in shaping our experiences of interacting with intelligent information systems but also inherit and amplify data biases, potentially leading to unfair decisions or discriminatory outcomes. This motivates us to investigate users' algorithm literacy, which covers the awareness and knowledge of algorithms and the skills to intervene in the operations of personalization algorithms when interacting with recommendation systems. Since vulnerable groups are more likely to suffer from the negative consequences of algorithmic decision‐making, investigating algorithm literacy among such groups is critical. This study aims to examine older adults' algorithm literacy, who are often considered a vulnerable group and labeled as digital laggards in contemporary information society. The empirical evidence collected from 21 participants in in‐depth interviews and cognitive mapping studies demonstrated that almost all participants are algorithm‐aware to some extent and identified (1) three types of information and sources collected by algorithms in user understanding, (2) two paradigms of how respondents understand personalized recommendations, and (3) two sets of strategies they develop to employ algorithms for improving user experience. The findings shed light on designing human‐centered intelligent information systems for unbiased personalization and developing a more inclusive AI‐assisted society that equally benefits people of all ages.

This study addresses (1) the influence of the decoy effect, a cognitive bias where the presence of an inferior item alters preferences between two options, on users’ search interactions and (2) the measurement of information retrieval systems’ vulnerability to the decoy effect. 1 From the perspective of user behavior, this study investigates the influence of the decoy effect in information retrieval (IR) by examining how decoy results affect users’ interaction on search engine result pages (SERPs), particularly in terms of click-through likelihood, browsing dwell time, and perceived document usefulness. We conducted an experiment based upon regression analysis on user interaction logs from three user study datasets which in total encompass 24 topics, 841 unique search sessions, and 2,685 queries. The findings indicate that decoys significantly increase the likelihood of document clicks and perceived usefulness. To investigate whether the influence of the decoy varies across different levels of task difficulty and user knowledge, we ran an additional experiment on one of the three datasets, which encompasses 6 topics, 166 search sessions and 652 queries. The results indicate that when the task is less challenging, users are more likely to click on a document with a decoy. Additionally, they spend more time on the target document and assign it a higher usefulness score. Furthermore, users with lower knowledge levels about the topic tend to give higher usefulness ratings to the target document. Regarding IR system evaluation, this study provides empirical insights into measuring the vulnerability of text retrieval models to potential decoy effect. An evaluation metric, namely DEcoy Judgement and Assessment VUlnerability (DEJA-VU), is proposed to evaluate the possibility of a retrieval model ranking results in a way that could trigger decoy biases. The experiments on the Text REtrieval Conference (TREC) 19 Deep Learning (DL) passage retrieval task and the TREC 20 DL passage retrieval task demonstrate that ColBERT and SPLADE show higher relevance-oriented retrieval effectiveness while also displaying lower vulnerability to decoy effect. Overall, this work advances the understanding of decoy effect, a well-established concept in cognitive psychology and behavioral economics, in a novel application field (i.e., Information Retrieval). It contributes to modeling users’ search behavior in the context of cognitive biases, as well as assessment of the vulnerability of systems and ranking algorithms to the decoy effect.

Modeling how people interact with search interfaces is core to the field of Interactive Information Retrieval. While various models have been proposed ranging from conceptual (e.g., Belkin’s ASK[12], Berry picking[11], Everyday-life information seeking, etc.) to theoretical (e.g., Information foraging theory[50], Economic theory[4], etc.), more recently there has been a body of working explore how people’s biases and the heuristics that they take influence how they search. This has led to the development of new models of the search process drawing upon Behavioural Economics and Psychology. This half day tutorial will provide a starting point for researchers seeking to learn more about information searching under uncertainty. The tutorial will be structured into two parts. First, we will provide an introduction of the biases and heuristics program put forward by Tversky and Kahneman [59] which assumes that people are not always rational. The second part of the tutorial will provide an overview of the types and space of biases in search [6, 42], before doing a deep dive into several specific examples and the impact of biases on different types of decisions (e.g., health/medical, financial etc.). The tutorial will wrap up with a discussion of some of the practical implication for how we can better design and evaluate IR systems in the light of cognitive biases.

Large language model (LLM) applications, such as ChatGPT, are a powerful tool for online information-seeking (IS) and problem-solving tasks. However, users still face challenges initializing and refining prompts, and their cognitive barriers and biased perceptions further impede task completion. These issues reflect broader challenges identified within the fields of IS and interactive information retrieval (IIR). To address these, our approach integrates task context and user perceptions into human-ChatGPT interactions through prompt engineering. We developed a ChatGPT-like platform integrated with supportive functions, including perception articulation, prompt suggestion, and conversation explanation. Our findings of a user study demonstrate that the supportive functions help users manage expectations, reduce cognitive loads, better refine prompts, and increase user engagement. This research enhances our comprehension of designing proactive and user-centric systems with LLMs. It offers insights into evaluating human-LLM interactions and emphasizes potential challenges for under served users.

Understanding how people interact with search interfaces is core to the field of Interactive Information Retrieval (IIR). While various models have been proposed (e.g., Belkin's ASK, Berry picking, Everyday-life information seeking, Information foraging theory, Economic theory, etc.), they have largely ignored the impact of cognitive biases on search behaviour and performance. A growing body of empirical work exploring how people's cognitive biases influence search and judgments, has led to the development of new models of search that draw upon Behavioural Economics and Psychology. This full day tutorial will provide a starting point for researchers seeking to learn more about information seeking, search and retrieval under uncertainty. The tutorial will be structured into three parts. First, we will provide an introduction of the biases and heuristics program put forward by Tversky and Kahneman [60] (1974) which assumes that people are not always rational. The second part of the tutorial will provide an overview of the types and space of biases in search,[5, 40] before doing a deep dive into several specific examples and the impact of biases on different types of decisions (e.g., health/medical, financial). The third part will focus on a discussion of the practical implication regarding the design and evaluation human-centered IR systems in the light of cognitive biases - where participants will undertake some hands-on exercises.

Abstract Understanding the roles of search gain and cost in users' search decision‐making is a key topic in interactive information retrieval (IIR). While previous research has developed user models based on simulated gains and costs, it is unclear how users' actual perceptions of search gains and costs form and change during search interactions. To address this gap, our study adopted expectation‐confirmation theory (ECT) to investigate users' perceptions of gains and costs. We re‐analyzed data from our previous study, examining how contextual and search features affect users' perceptions and how their expectation‐confirmation states impact their following searches. Our findings include: (1) The point where users' actual dwell time meets their constant expectation may serve as a reference point in evaluating perceived gain and cost; (2) these perceptions are associated with in situ experience represented by usefulness labels, browsing behaviors, and queries; (3) users' current confirmation states affect their perceptions of Web page usefulness in the subsequent query. Our findings demonstrate possible effects of expectation‐confirmation, prospect theory, and information foraging theory, highlighting the complex relationships among gain/cost, expectations, and dwell time at the query level, and the reference‐dependent expectation at the session level. These insights enrich user modeling and evaluation in human‐centered IR.

As artificial intelligence (AI) assisted search and recommender systems have become ubiquitous in workplaces and everyday lives, understanding and accounting for fairness has gained increasing attention in the design and evaluation of such systems. While there is a growing body of computing research on measuring system fairness and biases associated with data and algorithms, the impact of human biases that go beyond traditional machine learning (ML) pipelines still remain understudied. In this Perspective Paper, we seek to develop a two-sided fairness framework that not only characterizes data and algorithmic biases, but also highlights the cognitive and perceptual biases that may exacerbate system biases and lead to unfair decisions. Within the framework, we also analyze the interactions between human and system biases in search and recommendation episodes. Built upon the two-sided framework, our research synthesizes intervention and intelligent nudging strategies applied in cognitive and algorithmic debiasing, and also proposes novel goals and measures for evaluating the performance of systems in addressing and proactively mitigating the risks associated with biases in data, algorithms, and bounded rationality. This paper uniquely integrates the insights regarding human biases and system biases into a cohesive framework and extends the concept of fairness from human-centered perspective. The extended fairness framework better reflects the challenges and opportunities in users’ interactions with search and recommender systems of varying modalities. Adopting the two-sided approach in information system design has the potential to enhancing both the effectiveness in online debiasing and the usefulness to boundedly rational users engaging in information-intensive decision-making.

Previous researches demonstrate that users’ actions in search interaction are associated with relative gains and losses to reference points, known as the reference dependence effect. However, this widely confirmed effect is not represented in most user models underpinning existing search evaluation metrics. In this study, we propose a new evaluation metric framework, namely Reference Dependent Metric (ReDeM), for assessing query-level search by incorporating the effect of reference dependence into the modelling of user search behavior. To test the overall effectiveness of the proposed framework, (1) we evaluate the performance, in terms of correlation with user satisfaction, of ReDeMs built upon different reference points against that of the widely-used metrics on three search datasets; (2) we examine the performance of ReDeMs under different task states, like task difficulty and task urgency; and (3) we analyze the statistical reliability of ReDeMs in terms of discriminative power. Experimental results indicate that: (1) ReDeMs integrated with a proper reference point achieve better correlations with user satisfaction than most of the existing metrics, like Discounted Cumulative Gain (DCG) and Rank-Biased Precision (RBP), even though their parameters have already been well-tuned; (2) ReDeMs reach relatively better performance compared to existing metrics when the task triggers a high-level cognitive load; (3) the discriminative power of ReDeMs is far stronger than Expected Reciprocal Rank (ERR), slightly stronger than Precision and similar to DCG, RBP and INST. To our knowledge, this study is the first to explicitly incorporate the reference dependence effect into the user browsing model and offline evaluation metrics. Our work illustrates a promising approach to leveraging the insights about user biases from cognitive psychology in better evaluating user search experience and enhancing user models.

Abstract Evaluation metrics such as precision, recall and normalized discounted cumulative gain have been widely applied in ad hoc retrieval experiments. They have facilitated the assessment of system performance in various topics over the past decade. However, the effectiveness of such metrics in capturing users’ in-situ search experience, especially in complex search tasks that trigger interactive search sessions, is limited. To address this challenge, it is necessary to adaptively adjust the evaluation strategies of search systems to better respond to users’ changing information needs and evaluation criteria. In this work, we adopt a taxonomy of search task states that a user goes through in different scenarios and moments of search sessions, and perform a meta-evaluation of existing metrics to better understand their effectiveness in measuring user satisfaction. We then built models for predicting task states behind queries based on in-session signals. Furthermore, we constructed and meta-evaluated new state-aware evaluation metrics. Our analysis and experimental evaluation are performed on two datasets collected from a field study and a laboratory study, respectively. Results demonstrate that the effectiveness of individual evaluation metrics varies across task states. Meanwhile, task states can be detected from in-session signals. Our new state-aware evaluation metrics could better reflect in-situ user satisfaction than an extensive list of the widely used measures we analyzed in this work in certain states. Findings of our research can inspire the design and meta-evaluation of user-centered adaptive evaluation metrics, and also shed light on the development of state-aware interactive search systems.

Since its inception, the World Wide Web has become a major information source, consulted for a diversity of informational tasks. With an abundance of information available online, Web search engines have been a main entry point, supporting users in finding suitable Web content for ever more complex information needs. The IWILDS workshop series invites research on complex search activities related to human learning. It provides an interdisciplinary platform for the presentation and discussion of recent research on human learning on the Web, welcoming perspectives from computer & information science, education and psychology.

Promoting diversity in ranking while maintaining the relevance of ranked results is critical for enhancing human-centered search systems. While existing ranking algorithm and diversity IR metrics provide a solid basis for evaluating and improving search result diversification in offline experiments, it misses out possible divergences and temporal changes of users' levels of Diversity Acceptance, which in this work refers to the extent to which users actually prefer to interact with topically diversified search results. To address this gap between offline evaluations and users' expectations, we proposed an intuitive diversity acceptance measure and ran experiments for diversity acceptance prediction and diversity-aware re-ranking based on datasets from both controlled lab and naturalistic settings. Our results demonstrate that: 1) user diversity acceptance change across different query segments and session contexts, and can be predicted from search interaction signals; 2) our diversity-aware re-ranking algorithm utilizing predicted diversity acceptance and estimated relevance labels can effectively minimize the gap between diversity acceptance and result diversity, while maintaining SERP relevance levels. Our research presents an initial attempt on balancing user needs, result diversity, and SERP relevance in sessions and highlights the importance of studying diversity acceptance in promoting effective result diversification.

To support complex search tasks that involve prolonged search sessions and learning goals of varying difficulty, information retrieval (IR) researchers need a more comprehensive understanding of in-situ learning progresses and related factors. Among many cognitive factors associated with learning and searching, we consider interest development as an important dimension because it significantly affects users' learning performances but still remains understudied in interactive IR (IIR). To address this gap, our perspective paper proposes an interest-search-learning (ISL) model to reconceptualize learning in search and characterize the dynamic interplay of interest development, knowledge learning, and searching. Specifically, it achieves three interrelated goals: 1) characterizing the interactions between interest development, learning, and search behaviors; 2) synthesizing and proposing useful measures for capturing in-situ progresses and state variations in interest development and knowledge learning related to search behaviors; 3) identifying new research questions and directions linked to conceptualizing, building, and evaluating learning-centric IR systems. This paper uniquely integrates findings from three research communities (i.e. interest development, learning, IIR) into a cohesive framework and better structures our understanding of the multidimensional cognitive changes in search as learning (SAL). Including the exploration of interest development in SAL research will expand both conceptual and practical visions of AI-assisted learning.

Web search is one of the most ubiquitous online activities and often used as a starting point to learn, i. e., to acquire or extend one's knowledge about certain topics or procedures. When learning by searching the Web, individuals are confronted with an unprecedented amount of information in various forms and varying quality. Thus, successful learning on the Web requires high degrees of self-regulation and should be supported by the adequate design of search, recommendation, and training tools. This creates a highly interdisciplinary research area at the intersection of information retrieval, human-computer interaction, psychology, and educational sciences. Search as Learning (SAL) research examines the relationships between querying, navigation, media consumption behavior, and the learning outcomes during Web search, how they can be measured, predicted, and supported.

In interactive IR (IIR), users often seek to achieve different goals (e.g. exploring a new topic, finding a specific known item) at different search iterations and thus may evaluate system performances differently. Without state-aware approach, it would be extremely difficult to simulate and achieve real-time adaptive search evaluation and recommendation. To address this gap, our work identifies users' task states from interactive search sessions and meta-evaluates a series of online and offline evaluation metrics under varying states based on a user study dataset consisting of 1548 unique query segments from 450 search sessions. Our results indicate that: 1) users' individual task states can be identified and predicted from search behaviors and implicit feedback; 2) the effectiveness of mainstream evaluation measures (measured based upon their respective correlations with user satisfaction) vary significantly across task states. This study demonstrates the implicit heterogeneity in user-oriented IR evaluation and connects studies on complex search tasks with evaluation techniques. It also informs future research on the design of state-specific, adaptive user models and evaluation metrics.

Complex search tasks that involve uncertain solution space and multi-round search iterations are integral to everyday life and information-intensive workplace practices, affecting how people learn, work, and resolve problematic situations. However, current search systems still face plenty of challenges when applied in supporting users engaging in complex search tasks. To address this issue, we seek to explore the dynamic nature of complex search tasks from process-oriented perspective by identifying and predicting implicit task states. Specifically, based upon the Web search logs and user annotation data (regarding information seeking intentions in local search steps, in-situ search problems, and help needed) collected from 132 search sessions in two controlled lab studies, we developed two task state frameworks based on intention state and problem-help state respectively and examined the connection between task states and search behaviors. We report that (1) complex search tasks of different types can be deconstructed and disambiguated based on the associated nonlinear state transition patterns; and (2) the identified task states that cover multiple subtle factors of user cognition can be predicted from search behavioral signals using supervised learning algorithms. This study reveals the way in which complex search tasks are unfolded and manifested in users' search interactions and paves the way for developing state-aware adaptive search supports and system evaluation frameworks.

How users think, behave, and make decisions when interacting with information retrieval (IR) systems is a fundamental research problem in the area of Interactive IR. There is substantial evidence from behavioral economics and decision sciences demonstrating that in the context of decision-making under uncertainty, the carriers of value behind actions are gains and losses defined relative to a reference point, rather than the absolute final outcomes. This Reference Dependence Effect as a systematic cognitive bias was largely ignored by most formal interaction models built upon a series of unrealistic assumptions of user rationality. To address this gap, our work seeks to 1) understand the effects of reference points on search behavior and satisfaction at both query and session levels; 2) apply the knowledge of reference dependence in predicting users' search decisions and variations in level of satisfaction. Based on our experiments on three datasets collected from 1840 task-based search sessions (5225 query segments), we found that: 1) users' search satisfaction and many aspects of search behaviors and decisions are significantly associated with relative gains, losses and the associated reference points; 2) users' judgments of session-level satisfaction are significantly affected by peak and end reference moments; 3) compared to final-outcome-based baselines, models employing gain- and loss-based features often achieve significantly better performances in predicting search decisions and user satisfaction. The adaptation of behavioral economics perspective enables us to keep taking advantage of the collision of interdisciplinary insights in advancing IR research and also increase the explanatory power of formal search models by providing them with a more realistic behavioral and psychological foundation.

Most of the existing IR studies employed final values of search behavior measures in building evaluation metrics. However, according to the theories and empirical evidences from Behavioral Economics studies, in people's evaluations of actions and outcomes, the carriers of the values of different actions are gains and losses defined relative to a reference point, rather than the absolute final assets. Based on this idea, I will first explore how users' levels of search satisfaction are affected by the gains and losses defined relative to the pre-search expectations of system performance or reference levels in a controlled lab study. Then, based on the data collected from a field study, I will test the predicative power of my reference-dependent models (built upon delta-value-based behavioral features given the corresponding reference points) in predicting user satisfaction in naturalistic settings, aiming to examine the extent to which the reference-dependent approach can approximate real users' search evaluations. The findings of this work can help us better understand the subjectivity, bias, and variation in users' evaluation of search experience and thus have implications for user modeling and system recommendations design.

According to the cognitive viewpoint of information retrieval (IR) research, a search task can be conceptualized as a sequence of information seeking intentions which both motivate and are influenced by search behaviors. While the behavioral effects of task features have been thoroughly discussed in a large body of literature, how different information seeking intentions in query segments serve as bridges between task and Web search behavior still remains unexplored. To develop a more comprehensive, multi-level (i.e., task level, intention level, and behavior level) understanding of Web search, the authors analyzed intention and search behavior data collected from 693 query segments generated by 40 participants in a controlled lab setting, seeking to answer two main research questions: 1) from task to intention : how do different task features affect users' information seeking intentions at different stages of a search session? 2) from intention to behavior : How is a user's search behavior associated with their information seeking intentions in the current and next query segments respectively? The results demonstrate that: 1) Task features significantly affected the frequency of occurrence of most of the information seeking intentions, and these effects gradually faded away as search sessions proceeded; 2) The presences of a variety of intentions in both current and subsequent query segments were connected with and detectable by different subsets of behavioral measures. This study contributes to the understanding of the connections between task, intentions in query segments, and search behavior, and thereby has implications for designing system affordances for supporting different intentions and search activities in various task stages and contexts.

Expectation disconfirmation refers to the situation where a user's perceived performance of a system disconfirms her original expectation. Previous information systems studies have demonstrated that expectation disconfirmation can significantly affect a system user's behavior and experience. Inspired by this finding, we go beyond the traditional approach that focuses on the final post-search perception and study the expectation disconfirmation problem in Web search. Our study investigates task difficulty expectation disconfirmation and demonstrates that: (1) unexpectedly difficult task can significantly decrease a user's perceived level of search success and increase the perceived time pressure; (2) the size and direction of task difficulty expectation disconfirmation are significantly associated with Web search behavior; (3) it is possible to predict the state of expectation disconfirmation (especially the negative, unexpectedly difficult cases) based on search behavioral features. This study demonstrates the value of integrating expectation disconfirmation approach with interactive IR research and thus may encourage future researchers to further explore the effects of other aspects of users' expectations and post-search perceptions.

Stage is an essential facet of task. At different stages of search, users' search strategies are often influenced by different search intentions, encountered problems, as well as knowledge states. In previous studies, information seeking and interactive IR researchers have developed and validated some frameworks for describing various task facets and features. However, few studies have explored how to depict and differentiate different stages or states of complex search tasks in a comprehensive, multidimensional manner. The existing theoretical models of search process offer limited contributions to search path evaluation and the design of system recommendations for users at different states. To address this issue at both theoretical and empirical levels, my dissertation aims to construct an explainable framework that can characterize the stages or states of complex search tasks over multiple dimensions and to apply the framework in proactive search path evaluation and recommendation.

We explore the gap between 1) statistically significant relationships between task and browsing behavior and 2) predicting task type from such behaviors. Previous literature has shown relationships between Web browsing behavior and person»s corresponding search task. We find statistically significant browser features for detecting task - comparing the features to previous literature - and apply this knowledge to task classification of search sessions. Even though significant features improve prediction over baselines, it is not by much. We suggest that a more subtle treatment of such features should go beyond statistical significance. In some cases, considering personal patterns may be required for effective prediction.

In this demo, we present Coagmento, a Web-based, open-source tool for information seeking projects that collects information for individuals and groups and helps facilitate collaborative information seeking. Coagmento has been used in information retrieval and human-computer interaction studies to investigate individual and group information seeking behaviors in a lab or a field setting. In this demo, we discuss what Coagmento is, its past uses in prior studies, and its present state. We also discuss current work in progress. With Coagmento recently passing its 10th anniversary, we discuss our intention to make it a tool that is easy to configure for a human information behavior researcher with little programming skill.

One of the emerging and important problems in Interactive Information Retrieval research is predicting search tasks. Given a searcher's behavior during a search session, can the searcher's task be predicted? Which aspects of the task can be predicted, and how quickly and how accurately can the prediction be made? Much past literature has examined relationships between browsing behavior and task type at a statistical level, and recent work is moving towards prediction. While one may think whole session measures are useful for prediction, recent findings on common measures have suggested the contrary. Can less of the session still be useful? We examine the opposite end: the first query. Using multiple data sets for comparison, our results suggest that first query measures can be at least as good as -- and sometimes better than -- whole session measures for certain task type predictions.

This study used wheel activity as an indicator of users' browsing strategy, and explored the effects of various contextual factors on users' browsing patterns. Users' wheel activities were extracted from a search log in a user experiment, in which forty participants with different backgrounds conducted an online search in various contexts. To statistically test the potential effects of contexts on browsing strategy, we calculated three types of scroll-based variables to capture different types of browsing behaviors, and analyzed the effects of contextual factors via OLS regression analysis. Our results revealed that information understanding type tasks might lead to more proportion of revisit browsing range, while higher pressure level and higher pre-familiarity might lead to larger one-time browsing range; however, time constraint did not influence their browsing strategies. Our study also provided methodological implication for future interactive information retrieval studies by testifying and highlighting the usefulness of users' wheel activities in depicting online browsing patterns.

It has been shown that people attempt to accomplish a variety of intentions during the course of an information seeking session, and there is reason to believe that these different information seeking intentions can benefit from system support tailored to each such intention. We address the problem of predicting the presence of such intentions during an information seeking session, through analysis of observable user search behaviors. We present results of a study of 40 participants, each working on two different journalism tasks, which investigated how their search behaviors could indicate their intentions. Using 725 query-segments captured from this study, we demonstrate that information seeking intentions can be predicted with a simple classification model using a linear combination of search behavior features that can be logged with a browser plug-in.