Online digital library searchers who find themselves engaged in multiple concurrent complex search tasks may encounter potentially serendipitous information for one task while pursuing another. This can propel them in unexpected and exciting directions that enhance their knowledge. However, existing search interfaces provide limited support for searchers to handle such serendipitous information encounters, forcing them into an often disruptive choice between continuing their current search or pursuing the new discovery. To address this issue, we introduce Revelio: an academic digital library search interface based on a ‘save now, organize later’ workflow. Revelio provides a low-effort mechanism for deferring potentially serendipitous information encounters, and a semantic similarity approach for subsequently reviewing and saving such search results within a multi-workspace structure. We evaluated Revelio in a between-subjects controlled study with 28 participants using a novel experimental design that creates the opportunity for serendipitous information encounters within a concurrent complex academic search task context. Compared to those who used a baseline search interface, participants who used Revelio were able to effectively defer and subsequently review encountered information, resulting in positive opinions about the interface, increased performance in both prescribed search tasks, and greater perceived knowledge gain. These findings demonstrate the value of search interfaces that explicitly support the deferral and subsequent review of potentially serendipitous information.

Interactive Information Retrieval (IIR) interfaces are typically evaluated using questionnaires that gather post-task subjective measures such as ease of use, usefulness, satisfaction, and user engagement, along with in-task objective measures derived from log analysis.However, a comprehensive evaluation requires a deeper understanding of user behaviour beyond such traditional measures.Integrating eye tracking data with logged feature use and emotional valence provides a multimodal approach to evaluating a search interface at the feature level.To validate this approach, we examined three search interfaces in a controlled laboratory study focused on exploratory search within the context of digital humanities archives.A key benefit of this multimodal approach is that it allows us to evaluate both traditional interaction with the search interface (looking at a feature, using it, and experiencing an emotional response) as well as passive interaction with the search interface (looking at a feature, choosing not to use it but possibly getting information from it, and experiencing an emotional response).Using this approach, we were able to identify specific features of the interfaces that generated positive and negative emotional valence responses when used, as well as features that generated such emotional valence responses when viewed but not used.Such feature-level assessments would be difficult to capture using other means, providing insight into the nature of the searchers' experiences using the search interfaces.

Interactive information retrieval (IIR) interfaces are commonly evaluated using questionnaires that collect post-task subjective measures such as satisfaction, ease of use, usefulness, and user engagement. Although the importance of measuring emotional responses during the search process has been recognized, incorporating this aspect into IIR user studies has been challenging. We have developed a novel method to capture real-time emotional responses based on advances in facial emotion classification approaches. We utilize consumer-grade front-facing cameras to collect emotional responses, which synchronize with the user’s interactions with the search interface. In a controlled laboratory study, the relevance of search results was manipulated to validate the approach’s effectiveness and explore how search results’ relevance impacts users’ emotional responses, post-task evaluations of the search interface, and interactions with search interface features. This enabled us to examine whether we could detect emotional responses, whether recency effects were observed in post-task evaluations, and whether feature use correlated with emotional responses. The study was conducted in the context of exploratory search within an academic digital library. The results of this study demonstrate that both positive and negative emotional responses can be reliably detected during the search process. There is evidence of recency effects in post-task measures, and the study identifies specific interactive features used during the experience of positive and negative emotional responses. This serves as a foundation for the use of emotional responses to supplement post-task survey data when evaluating search interfaces. • Real-time emotion detection in IIR interfaces. • Recency effects observed in post-task subjective measures. • Correlation between emotional responses and post-task evaluations. • Emotional responses variation across search interface features. • Emotions in exploratory search within academic digital libraries.

In academic digital libraries, searchers commonly engage in exploratory search when faced with complex search tasks. An important part of exploratory search is exploratory browsing, where the focus is on search activities associated with discovery, learning, and investigation. However, these critical aspects of exploratory browsing are often not adequately supported by existing digital library search systems. In particular, they are hindered by the inability for searchers to add further information to inform their exploratory browsing style of searching. We address this issue by providing two new features: dynamic tagging of search results and an interactive workspace that allows the searcher to highlight and filter the search results using these tags. We have evaluated this approach compared to a baseline search system in a 32-participant user study. Increases in typical subjective measures were found, along with increases in perceived motivation and ability. Further, the documents saved as part of the exploratory browsing process were of higher precision when using this approach. These results show the value of providing searchers with interactive features that enable an exploratory browsing style of searching, beyond simply entering a query and selecting/saving search results.

Keywords contain important contextual information about search results within academic digital library search interfaces. However, such information tends to be underutilized within modern search interface designs. In prior work, methods for visually linking keywords between search results have been proposed and studied. In this research, we analyze the design space and propose a new approach that aggregates the keywords over all items on the search engine results page (SERP), visually linking them back to their source search result. We have created interactive and static versions of both interfaces, and conducted a controlled laboratory study to assess the impact of the interfaces on measures of utility (efficiency, effectiveness, feature use) and perceived value (usefulness, ease of use, satisfaction, user engagement, knowledge gain, and interest gain). The findings from this research show the merit of using keywords to provide summaries of documents and search result sets, the value of making keywords interactive, and the benefit of using visualization to interactively link information within a search engine results page. The differences between providing the keywords along side each document or aggregated over the entire SERP were minimal, suggesting that it does not matter how the keywords are represented as long as they can be used to interactively reveal relationships among the search results.

When evaluating interactive information retrieval (IIR) interfaces, it is common to collect data using subjective measures such as satisfaction, ease of use, usefulness, and user engagement. However, as these are collected post-task, they serve as surrogate measures for what occurred in the midst of the search activities. Further, such approaches may be subject to recency effects, where the last action in the search process influences the searchers’ opinions about the overall process. With recent improvements in facial emotion classification approaches, we propose that measuring emotional responses may provide a better indication of what is happening throughout search tasks. In this research, we present an approach for collecting real-time emotional responses during a search activity using consumer-grade front-facing cameras and a method of aligning these with search interface feature use. To validate the effectiveness of the approach, we have conducted a controlled laboratory study in which we manipulated the quality of the search results in order to determine if we can detect expected emotional responses, whether search behaviours influencing these emotional responses, and whether recency effects are present in post-task measures. The preliminary results of this study show that our approach is reliable for detecting emotional responses when searchers experience positive and negative emotions throughout the search process, isolate which interactive elements were used when positive and negative emotional responses were experienced, and illustrate how recency effects are present in post-task measures. Our upcoming study will investigate how our approach can be used to evaluate novel search interfaces. We will develop a novel search interface and evaluate it using our approach. Finally, we will create a dashboard to monitor academic literature. Using the same approach, we will demonstrate our approach can extend beyond traditional search interfaces and into more general interface assessment.