Interactive Information Retrieval (IIR) interfaces are typically evaluated using questionnaires that gather post-task subjective measures such as ease of use, usefulness, satisfaction, and user engagement, along with in-task objective measures derived from log analysis.However, a comprehensive evaluation requires a deeper understanding of user behaviour beyond such traditional measures.Integrating eye tracking data with logged feature use and emotional valence provides a multimodal approach to evaluating a search interface at the feature level.To validate this approach, we examined three search interfaces in a controlled laboratory study focused on exploratory search within the context of digital humanities archives.A key benefit of this multimodal approach is that it allows us to evaluate both traditional interaction with the search interface (looking at a feature, using it, and experiencing an emotional response) as well as passive interaction with the search interface (looking at a feature, choosing not to use it but possibly getting information from it, and experiencing an emotional response).Using this approach, we were able to identify specific features of the interfaces that generated positive and negative emotional valence responses when used, as well as features that generated such emotional valence responses when viewed but not used.Such feature-level assessments would be difficult to capture using other means, providing insight into the nature of the searchers' experiences using the search interfaces.

Despite the importance of context in Recommender Systems (RSs) more generally, and its clear applicability in the food domain, most existing research focuses on single contextual factors, and only considers simple extrinsic factors such as location and time. No RSs research has systematically explored the impact of multiple dynamic factors, or investigated the effect of emotion in determining people’s eating, recipe rating and nutritional intake behaviour. To bridge these gaps, we conducted a comprehensive large-scale (n=397) crowdsourced experimental study to uncover the intricate relationship between various simulated contextual factors and users’ subsequent recipe rating and implied nutritional intake behaviour. We further aimed to explore how these contextual factors can be incorporated to improve recommendation performance. Four distinct types of contextual factors were investigated: seasonal, emotional, busyness and physical activity, encompassing a total of seven elements. Our findings show that people’s eating preferences and the likelihood of them choosing to eat healthy recipes vary depending on the simulated context they find themselves in. Moreover, we demonstrate how these contextual features can be used to significantly improve recipe rating prediction performance. Our research has implications for the future development of food RSs, and shows that emotion-aware systems could lead to better healthy food recommendations.

A rapid increase in the use of web-based technologies – and corresponding changes in government and local council policies – in recent years, means that many vital services are now provided solely online. While this has many potential benefits, it can place additional burdens on certain demographic groups, some of whom may become considerably disadvantaged or even disenfranchised. This is particularly problematic for English-as-a Second Language (ESL) speakers, who are often immigrants or refugees and thus have a greater need to access these e-government services, and who may struggle to understand and assess the relevance of complex documents. In this work we investigate the search behaviours and performance of native English speakers and two different groups of ESL speakers when completing e-government tasks, and the effect of document readability/complexity. In contrast with previous work, our results show significant differences between groups of varying language proficiency in terms of objective search performance, time on task, and self-perceived performance and confidence. We also demonstrate that document reading level moderates the effect of language proficiency on objective search performance. The findings contribute to our existing understanding of how English language proficiency affects search for e-government topics, and have important implications for the future development of e-government services to ensure more equitable access and use.

Abstract While it is clear that using a mobile device can interrupt real‐world activities such as walking or driving, the effects of interruptions on mobile device use have been under‐studied. We are particularly interested in how the ambient distraction of walking while using a mobile device, combined with the occurrence of simulated interruptions of different levels of cognitive complexity, affect web search activities. We have established an experimental design to study how the degree of cognitive complexity of simulated interruptions influences both objective and subjective search task performance. In a controlled laboratory study (n = 27), quantitative and qualitative data were collected on mobile search performance, perceptions of the interruptions, and how participants reacted to the interruptions, using a custom mobile eye‐tracking app, a questionnaire, and observations. As expected, more cognitively complex interruptions resulted in increased overall task completion times and higher perceived impacts. Interestingly, the effect on the resumption lag or the actual search performance was not significant, showing the resiliency of people to resume their tasks after an interruption. Implications from this study enhance our understanding of how interruptions objectively and subjectively affect search task performance, motivating the need for providing explicit mobile search support to enable recovery from interruptions.

Abstract The popular outdoor pursuit of backpacking is profoundly changing as the community embraces contemporary information technologies. However, there is little empirical evidence on the adoption and use of consumer electronics by backpackers, nor the implications this has for their habits, practices, and interactions. We investigate long‐distance backpackers' articulations with mobile information technology during the TGO Challenge, a coast‐to‐coast crossing of the Scottish Highlands. By employing mixed methods, we explore how and why backpackers use such technology when planning and undertaking their journeys via a survey (n = 116), pre‐ and post‐challenge interviews with selected TGO participants, and daily in‐field video‐logs. Our results suggest many advantages to using technology in this context, including fluidity of communications and access, while noting that reliance on technology is leading to issues such as increased need for battery power management, and deskilling. The findings highlight implications for the juxtaposition between outdoor recreation, information behavior, and human computer interaction (HCI) and suggest future work in this area.

As technology improves, the use of conversational agents to help users solve information seeking tasks is becoming ever more prevalent. To date we know little about how people behave with such systems, particularly in diverse contexts and for different tasks, their specific needs or how best to support these. By employing a Wizard of Oz (WoZ) methodology and developing a conversational framework, in this work we study how participants (n=28) interact with such a system in an attempt to solve recipe recommendation tasks. Our results are mostly encouraging for the future development of conversational agents in this context, however, they also provide insights into the complexities of building such a system that could convincingly engage with users in productive, human-like conversations.

Improvements in mobile technologies have led to a dramatic change in how and when people access and use information, and is having a profound impact on how users address their daily information needs. Smart phones are rapidly becoming our main method of accessing information and are frequently used to perform "on-the-go'' search tasks. As research into information retrieval continues to evolve, evaluating search behaviour in context is relatively new. Previous research has studied the effects of context through either self-reported diary studies or quantitative log analysis; however, neither approach is able to accurately capture context of use at the time of searching.

More than half of all searches are now submitted on mobile devices, which can (and often are) used in various potentially distracting situations, such as travelling on a noisy train or when walking down a busy street. Research suggests that walking has negative effects on search performance and behaviour and that auditory distractions can impact on user input and affect perception of task duration. In this work we conduct a user study (n=16) using a simulated distracting condition to investigate how auditory distractions change perceived and objective search performance and behaviour. Our results suggest that noisy environments induce stress on users, causing them to feel additional perceived time pressure, leading to a reduced ability to identify task-relevant documents and a compulsion to finish the search task quickly.

While the recent trend of digitisation of government and related services offers many advantages, it could introduce problems for those who are less information literate or who have particular issues searching for and understanding the necessary content. In this study ten participants, who speak English as a second language, were given four search tasks designed to reflect actual information seeking situations. They completed pre- and post-search questionnaires to identify the relevancy of the task, their English language ability and search experience.

Mobile devices are rapidly becoming our main method of accessing the Internet and are frequently used to perform on-the-go search tasks. The use of such devices in situations where attention must be divided, such as when walking, are common and research suggests that this increases cognitive load and, therefore, may have an impact on performance.

By incorporating healthiness into the food recommendation / ranking process we have the potential to improve the eating habits of a growing number of people who use the Internet as a source of food inspiration. In this paper, using insights gained from various data sources, we explore the feasibility of substituting meals that would typically be recommended to users with similar, healthier dishes. First, by analysing a recipe collection sourced from Allrecipes.com, we quantify the potential for finding replacement recipes, which are comparable but have different nutritional characteristics and are nevertheless highly rated by users. Building on this, we present two controlled user studies (n=107, n=111) investigating how people perceive and select recipes. We show participants are unable to reliably identify which recipe contains most fat due to their answers being biased by lack of information, misleading cues and limited nutritional knowledge on their part. By applying machine learning techniques to predict the preferred recipes, good performance can be achieved using low-level image features and recipe meta-data as predictors. Despite not being able to consciously determine which of two recipes contains most fat, on average, participants select the recipe with the most fat as their preference. The importance of image features reveals that recipe choices are often visually driven. A final user study (n=138) investigates to what extent the predictive models can be used to select recipe replacements such that users can be ``nudged'' towards choosing healthier recipes. Our findings have important implications for online food systems.

Smart phones and tablets are rapidly becoming our main method of accessing information and are frequently used to perform on-the-go search tasks. Mobile devices are commonly used in situations where attention must be divided, such as when walking down a street. Research suggests that this increases cognitive load and, therefore, may have an impact on performance. In this work we conducted a laboratory experiment with both device types in which we simulated everyday, common mobile situations that may cause fragmented attention, impact search performance and affect user perception.

The queries submitted by users to search engines often poorly describe their information needs and represent a potential bottleneck in the system. In this paper we investigate to what extent it is possible to aid users in learning how to formulate better queries by providing examples of high-quality queries interactively during a number of search sessions. By means of several controlled user studies we collect quantitative and qualitative evidence that shows: (1) study participants are able to identify and abstract qualities of queries that make them highly effective, (2) after seeing high-quality example queries participants are able to themselves create queries that are highly effective, and, (3) those queries look similar to expert queries as defined in the literature. We conclude by discussing what the findings mean in the context of the design of interactive search systems.

Micro‐blogging services such as Twitter represent constantly evolving, user‐generated sources of information. Previous studies show that users search such content regularly but are often dissatisfied with current search facilities. We argue that an enhanced understanding of the motivations for search would aid the design of improved search systems, better reflecting what people need. Building on previous research, we present qualitative analyses of two sources of data regarding how and why people search Twitter. The first, a diary study (p = 68), provides descriptions of Twitter information needs (n = 117) and important meta‐data from active study participants. The second data set was established by collecting first‐person descriptions of search behavior (n = 388) tweeted by twitter users themselves (p = 381) and complements the first data set by providing similar descriptions from a more plentiful source. The results of our analyses reveal numerous characteristics of Twitter search that differentiate it from more commonly studied search domains, such as web search. The findings also shed light on some of the difficulties users encounter. By highlighting examples that go beyond those previously published, this article adds to the understanding of how and why people search such content. Based on these new insights, we conclude with a discussion of possible design implications for search systems that index micro‐blogging content.

Personalisation is an important area in the field of IR that attempts to adapt ranking algorithms so that the results returned are tuned towards the searcher's interests. In this work we use query logs to build personalised ranking models in which user profiles are constructed based on the representation of clicked documents over a topic space. Instead of employing a human-generated ontology, we use novel latent topic models to determine these topics. Our experiments show that by subtly introducing user profiles as part of the ranking algorithm, rather than by re-ranking an existing list, we can provide personalised ranked lists of documents which improve significantly over a non-personalised baseline. Further examination shows that the performance of the personalised system is particularly good in cases where prior knowledge of the search query is limited.

Distributed events are collections of events taking place within a small area over the same time period and relating to a single topic. There are often a large number of events on offer and the times in which they can be visited are heavily constrained, therefore the task of choosing events to visit and in which order can be very difficult. In this work we investigate how visitors can be assisted by means of a recommender system via 2 large-scale naturalistic studies (n=860 and n=1047). We show that a recommender system can influence users to select events that result in tighter and more compact routes, thus allowing users to spend less time travelling and more time visiting events.

Collaborative filtering systems based on ratings make it easier for users to find content of interest on the Web and as such they constitute an area of much research. In this paper we first present a Bayesian latent variable model for rating prediction that models ratings over each user's latent interests and also each item's latent topics. We describe a Gibbs sampling procedure that can be used to estimate its parameters and show by experiment that it is competitive with the gradient descent SVD methods commonly used in state-of-the-art systems. We then proceed to make an important and novel extension to this model, enhancing it with user-dependent and item-dependant biases to significantly improve rating estimation. We show by experiment on a large set of real ratings data that these models are able to outperform 3 common baselines, including a very competitive and modern SVD-based model. Furthermore we illustrate other advantages of our approach beyond simply its ability to provide more accurate ratings and show that it is able to perform better on the common and important case where the user profile is short.

In this paper we present a longitudinal, naturalistic study of email behavior (n=47) and describe our efforts at isolating re-finding behavior in the logs through various qualitative and quantitative analyses. The presented work underlines the methodological challenges faced with this kind of research, but demonstrates that it is possible to isolate re-finding behavior from email interaction logs with reasonable accuracy. Using the approaches developed we uncover interesting aspects of email re-finding behavior that have so far been impossible to study, such as how various features of email-clients are used in re-finding and the difficulties people encounter when using these. We explain how our findings could influence the design of email-clients and outline our thoughts on how future, more in depth analyses, can build on the work presented here to achieve a fuller understanding of email behavior and the support that people need.

Social tagging systems have recently become very popular as a method of categorising information online and have been used to annotate a wide range of different resources. In such systems users are free to choose whatever keywords or "tags" they wish to annotate each resource, resulting in a highly personalised, unrestricted vocabulary. While this freedom of choice has several notable advantages, it does come at the cost of making searching of these systems more difficult as the vocabulary problem introduced is more pronounced than in a normal information retrieval setting.

We investigate the utility of topic models for the task of personalizing search results based on information present in a large query log. We define generative models that take both the user and the clicked document into account when estimating the probability of query terms. These models can then be used to rank documents by their likelihood given a particular query and user pair.

Ranking of resources in social tagging systems is a difficult problem due to the inherent sparsity of the data and the vocabulary problems introduced by having a completely unrestricted lexicon. In this paper we propose to use hidden topic models as a principled way of reducing the dimensionality of this data to provide more accurate resource rankings with higher recall. We first describe Latent Dirichlet Allocation (LDA) and then show how it can be used to rank resources in a social bookmarking system. We test the LDA tagging model and compare it with 3 non-topic model baselines on a large data sample obtained from the Delicious social bookmarking site. Our evaluations show that our LDA-based method significantly outperforms all of the baselines.