Web-based learning is evolving rapidly as traditional search engines are complemented by Large Language Models (LLMs) and other AI technologies. This evolution offers new opportunities, such as automated information synthesis and personalized learning experiences. However, this also presents new challenges, including the need for learners to be aware of potential biases and misinformation in AI-generated content, and to maintain focus and depth in their learning journeys.

Misinformation is an important topic in the Information Retrieval (IR) context and has implications for both system-centered and user-centered IR. While it has been established that the performance in discerning misinformation is affected by a person’s cognitive load, the variation in cognitive load in judging the veracity of news is less understood. To understand the variation in cognitive load imposed by reading news headlines related to COVID-19 claims, within the context of a fact-checking system, we conducted a within-subject, lab-based, quasi-experiment (N=40) with eye-tracking. Our results suggest that examining true claims imposed a higher cognitive load on participants when news headlines provided incorrect evidence for a claim and were inconsistent with the person’s prior beliefs. In contrast, checking false claims imposed a higher cognitive load when the news headlines provided correct evidence for a claim and were consistent with the participants’ prior beliefs. However, changing beliefs after examining a claim did not have a significant relationship with cognitive load while reading the news headlines. The results illustrate that reading news headlines related to true and false claims in the fact-checking context impose different levels of cognitive load. Our findings suggest that user engagement with tools for discerning misinformation needs to account for the possible variation in the mental effort involved in different information contexts.

We conducted a lab-based eye-tracking study to investigate how interactivity of an AI-powered fact-checking system affects user interactions, such as dwell time, attention, and mental resources involved in using the system. A within-subject experiment was conducted, where participants used an interactive and a non-interactive version of a mock AI fact-checking system, and rated their perceived correctness of COVID-19 related claims. We collected web-page interactions, eye-tracking data, and mental workload using NASA-TLX. We found that the presence of the affordance of interactively manipulating the AI system's prediction parameters affected users’ dwell times, and eye-fixations on AOIs, but not mental workload. In the interactive system, participants spent the most time evaluating claims’ correctness, followed by reading news. This promising result shows a positive role of interactivity in a mixed-initiative AI-powered system.

An important step to foster learning during search is to identify behavioural patterns that distinguish searchers gaining more vs. less knowledge during search. Previous efforts have studied searchers in the short term, typically during a single lab session. We propose a longitudinal study to analyse searching behaviour of students enrolled in a university course, over the span of a few months. Our research aims are to (i) identify how searching behaviour changes over time, as students gain new knowledge on a subject, (ii) whether the perceived-relevance of specific information objects change due to knowledge acquisition, and, (iii) identify if differences exist in search processes for previously seen vs. unseen search tasks. We posit that findings from this research study will be informative for building improved information retrieval systems which are supportive of search as learning.

Collecting participant search logs is an integral part of interactive IR research. Today's existing approaches are either piecemeal solutions, and/or require cumbersome setups. We present YASBIL, a two-component logging solution comprising a browser extension and a WordPress plugin. The browser extension logs the browsing activity in the participants' machines. The WordPress plugin collects the logged data into the researcher's data server. The logging works on any webpage, without the need to own or have knowledge about the HTML structure of the webpage. YASBIL also offers ethical data transparency and security towards participants, by enabling them to view and obtain copies of the logged data, as well as securely upload the data to the researcher's server over an HTTPS connection. We posit that ease of installation and use will make YASBIL especially suitable for remote user-studies, and longitudinal studies in IR.

We propose an image-classification method to predict the perceived-relevance of text documents from eye-movements. An eye-tracking study was conducted where participants read short news articles, and rated them as relevant or irrelevant for answering a trigger question. We encode participants' eye-movement scanpaths as images, and then train a convolutional neural network classifier using these scanpath images. The trained classifier is used to predict participants' perceived-relevance of news articles from the corresponding scanpath images. This method is content-independent, as the classifier does not require knowledge of the screen-content, or the user's information-task. Even with little data, the image classifier can predict perceived-relevance with up to 80% accuracy. When compared to similar eye-tracking studies from the literature, this scanpath image classification method outperforms previously reported metrics by appreciable margins. We also attempt to interpret how the image classifier differentiates between scanpaths on relevant and irrelevant documents.

We investigate the relationship between search behavior, eye -tracking measures, and learning. We conducted a user study where 30 participants performed searches on the web. We measured their verbal knowledge before and after each task in a content-independent manner, by assessing the semantic similarity of their entries to expert vocabulary. We hypothesize that differences in verbal knowledge-change of participants are reflected in their search behaviors and eye-gaze measures related to acquiring information and reading. Our results show that participants with higher change in verbal knowledge differ by reading significantly less, and entering more sophisticated queries, compared to those with lower change in knowledge. However, we do not find significant differences in other search interactions like page visits, and number of queries.