VLDB 2026 Research / reviewers in the wild / expert
Ian Ruthven
dblp:r/IanRuthven
· DBLP profile ↗
70ranked-venue papers in the field
17as first author
13since 2021 · last 2025
0000-0001-6669-5376ORCID · verified
Domains — venue-derived; a paper can count in several
Information Retrieval & Web Search · 66 (16 first)Knowledge Engineering, Semantic Web & Information Systems · 2 (1 first)Database Systems & Data Management · 1Data Mining & Knowledge Discovery · 1
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2025 | Online Information Disclosure and Information Privacy Practices During Significant Life Transitions: A Scoping ReviewabstractSignificant life transitions are difficult and stressful periods, where personal identity and relationships undergo change. Accessing information and social support during these times requires disclosure: we must reveal information about ourselves to gain useful information. Yet, managing disclosure of information online during these periods presents great challenges and burdens to the central actor. We therefore sought to identify the information disclosure practices of people undergoing diverse selected life transitions (leaving the Armed Forces; relationship breakdown; coming out as LGBTQ+; living with cancer) with respect to online privacy and its implications for the design of information access systems. We achieved this via a scoping review, searching across nine databases to identify relevant articles, before carrying out an inductive thematic analysis of the charted data. We surface a key common disclosure strategy of multiple temporal selves, whereby people undergoing life transitions present multiple situated identities reflecting both their past and emerging identities, bound to desired levels of disclosure within specific online spaces/networks. This strategy is transacted via nuanced use of privacy features online, which calls for a high level of digital literacy. Our resulting design recommendations offer opportunities for information systems to support people in: safely partitioning their identity across new, and within existing, online spaces; creating spaces for personal reflection; and having a more informed awareness of the depth of information shared. The need for deeper understanding of the privacy requirements of two under-studied groups - leaving the Armed Forces, and living with cancer – is also highlighted. Ryan Colin Gibson, Ramsay Meiklem, Wendy Moncur, Ian Ruthven |
CHIIR | 4 |
| 2025 | Information avoidance: A critical conceptual review. An Annual Review of Information Science and Technology (ARIST) paperabstractAbstract Information avoidance has long been in the shadow of information seeking. Variously seen as undesired, maladaptive, or even pathological, information avoidance has lacked the sustained attention and conceptualization that has been provided to other information practices. It is also, perhaps uniquely among information practices, often invoked to blame or censure those who engage in it. However, closer examination of information avoidance reveals nuanced and complex patterns of interactions with information, ones that often have positive and beneficial outcomes. We challenge the simplistic tenor of this conversation through this critical conceptual review of information avoidance. Starting from an examination of how information avoidance has been treated within information science and related disciplines, we then draw upon the various terms that have been used to describe a lack of engagement with information to establish seven core characteristics of the concept. We subsequently use this analysis to establish our definition of information avoidance as practices that moderate interaction with information by reducing the intensity of information, restricting control over information, and/or excluding information based on perceived properties. We consider the implications of this definition and its view of information avoidance as a significant information practice on information research. Alison Hicks, Pamela J. McKenzie, Jenny Bronstein, Jette Seiden Hyldegård, Ian Ruthven, Gunilla Widén |
J. Assoc. Inf. Sci. Technol. | 5 |
| 2025 | Right answers to wrong questions: The dysfunctional nature of information needsabstractAbstract People frequently experience difficulties when seeking information to complete tasks. To overcome these difficulties, people require help. Regarding struggles with information needs, past research focuses on unclear information requests, such as ambiguous, under‐specified, and ill‐defined queries, and repairing these by user‐led strategies (e.g., clarification). In an exploratory qualitative study where information clerks were interviewed, we, however, found that well‐formed and seemingly reasonable requests can conceal misconceptions inquirers have (e.g., about what information is required for their current task) and, therefore, interfere with information seeking and task completion, too. Besides being more difficult to identify than unclear requests, such hidden misconceptions also undermine current user‐led repair strategies as they cause inquirers to believe they are making appropriate requests. Understanding misconceptions in information seeking and requests concealing these is, therefore, essential to building more effective information systems. Our study contributes to addressing this task: It is the first to provide empirical insights into how misconceptions can negatively influence information requests, information‐seeking conversations, and task completion. Ultimately, our findings highlight that inquirers' perceived information needs can present an unreliable and even counterproductive basis for task support, implying that researchers and professionals should rethink the prevailing focus on user requests in designing information systems. Melanie A. Kilian, David Elsweiler, Ian Ruthven |
J. Assoc. Inf. Sci. Technol. | 3 |
| 2025 | In search of existential authenticity: Information behaviors during a time of personal transformationabstractAbstract Many people make changes to their lives to provide a greater sense of value, purpose, or coherence to their everyday existence. One goal in making such changes is to improve their sense of existential authenticity, a feeling that their lives are being lived in what they see as meaningful and productive ways. This study looks at the process of creating a sense of existential authenticity from an information perspective. Through interviews with 10 people who have made significant changes to their lives, the study seeks to understand the process for searching for a greater sense of existential authenticity, and the associated information behaviors that occur during this process. The findings reveal a complex and emergent process of information activities leading to a new life and a new information world and open a new area of study for information transitions research. Ian Ruthven |
J. Assoc. Inf. Sci. Technol. | 1 |
| 2024 | The concept of information need and its operationalization in CHIIR researchabstractOne of the most utilized concepts in CHIIR research is ‘information need’, which has been used as part of research designs in both qualitative and quantitative studies, either as an integral element of experiments and evaluations or as an explanatory aspect of information searching. Its definitions and operationalizations are likewise many and they often remain vague hindering the possibility to credibly compare findings. This tutorial will help participants to use the concept more precisely and appropriately in their own research. The tutorial consists of talks by experienced researchers combined with small group and plenary sessions. Katriina Byström, Sanna Kumpulainen, Nils Pharo, Ian Ruthven |
CHIIR | 4 |
| 2024 | Uncharted Territory: Understanding Exploratory Search Behaviours in Literature ReviewsabstractIn the realm of Information Seeking and Retrieval (ISR), searching the literature for relevant references in the context of academic work, such as theses or publications, is widely recognised as an exploratory search task. This task becomes particularly challenging when searchers lack prior knowledge of the subject matter. To help address the growing need for supporting exploratory search endeavours, ISR researchers have developed exploratory search models and interfaces. However, while much attention has been given to conceptualising exploratory searches, little focus has been placed on understanding the specific approaches and behaviours that searchers employ during conducting literature reviews. This paper aims to bridge this gap by conducting semi-structured interviews with 30 Master’s students at the end of a user study. This paper provides comprehensive definitions for the fundamental exploratory characteristics from the recent conceptual model, pinpoints potential factors that could influence these characteristics, introduces new exploratory dimensions, and extends our comprehension of existing ones in the academic context. It also uncovers a spectrum of approaches used in literature searches, shedding light on how individuals rely on specific paper sections to measure their relevance and highlighting essential facets of knowledge acquisition in the context of literature searches. Ayah Soufan, Ian Ruthven, Leif Azzopardi |
CHIIR | 2 |
| 2024 | Information shapingabstractAbstract In this article, I explore the relationships between how humans have evolved to interact with the material world and how we interact with our information worlds. I argue that shaping processes, exemplified by how early humans created stone tools, are core ways to interact with the world that are appropriated to interact with information to create information solutions. To test these claims, I examine existing studies of information use from a shaping perspective. I finish by discussing how this evolutionary perspective to information use can benefit discussions of information behavior. Ian Ruthven |
J. Assoc. Inf. Sci. Technol. | 1 |
| 2024 | Information sculptingabstractAbstract In this paper, I propose sculpting as an analogy for information use. I provide a background to the use of metaphors and analogies in Information Science, propose five analogical mappings between sculpting and information use, and then discuss the benefits to this analogy for thinking about how we use information. Ian Ruthven |
J. Assoc. Inf. Sci. Technol. | 1 |
| 2023 | Exploring older people's challenges on online banking/finance systems: Early findingsabstractUse of technology is a prerequisite to conduct a normal life and people recurrently use online financial services (i.e., any website which involve monetary transactions, for example, Internet banking, online shopping, transport service websites etc.) in everyday contexts. Information interactions are affected by the nature of specific digital services as well as a number of behavioural characteristics of users. Older people aged 65 and over are often digitally excluded from the use of online financial services. We identified some reasons for non-use of financial services through semi-structured interviews with older people and employees who assist older people with digital services. Preliminary findings demonstrated that fear of financial scams, lack of digital skills and lack of help are some of the main factors which inhibit them from using digital financial services efficiently. Dain Thomas, Gobinda G. Chowdhury, Ian Ruthven |
CHIIR | 3 |
| 2022 | Searching the Literature: An Analysis of an Exploratory Search TaskabstractExploratory search is an intuitive concept in interactive information retrieval. While many definitions for Exploratory Search have been proposed, the main dimensions involve high uncertainty with respect to the problem context, the user expertise, and the search process. In this paper, we draw together the different characteristics relating to the three main exploratory dimensions to provide a conceptual model of exploratory search. We build an exploratory search questionnaire using this model. We then use the questionnaire instrument to examine how literature searches are exploratory and what factors influence the exploratory dimensions and characteristics. We provide one of the first detailed investigations into the nature of exploratory literature review searches. Our analysis of the 368 responses reveals that about 84% of the participants described their literature review task as somewhat exploratory or very exploratory in nature. Also, the analysis points to another dimension of the exploratory search, the knowledge gain/change dimension. Furthermore, we investigated how users’ experience influences how people rate the exploratory search characteristics (e.g., whether they find surprising information, learn new concepts and keywords). Ayah Soufan, Ian Ruthven, Leif Azzopardi |
CHIIR | 2 |
| 2022 | An information behavior theory of transitionsabstractAbstract This paper proposes a theory of life transitions focused on information behavior. Through a process of meta‐ethnography, the paper transforms a series of influential theories and models into a theory of transitions for use in Information Science. This paper characterizes the psychological processes involved in transitions as consisting of three main stages, Understanding, Negotiating, and Resolving, each of which have qualitatively different information behaviors and which require different types of information support. The paper discusses the theoretical implications of this theory and proposes ways in which the theory can be used to provide practical support for those undergoing transitions. Ian Ruthven |
J. Assoc. Inf. Sci. Technol. | 1 |
| 2021 | People and places: Bridging the information gaps in refugee integrationabstractAbstract This article discusses the sources of information used by refugees as they navigate integration systems and processes. The study used interviews to examine how refugees and asylum seekers dealt with their information needs, finding that information gaps were bridged through people and places. People included friends, solicitors, and caseworkers, whereas places included service providers, detention centers, and refugee camps. The information needs matrix was used as an analytical tool to examine the operation of sources on refuge‐seekers' integration journeys. Our findings expand understandings of information sources and information grounds. The matrix can be used to enhance host societies' capacity to make appropriate information available and to provide evidence for the implementation of the information needs matrix. Olubukola Oduntan, Ian Ruthven |
J. Assoc. Inf. Sci. Technol. | 2 |
| 2021 | Resonance and the experience of relevanceabstractAbstract In this article, I propose the concept of resonance as a useful one for describing what it means to experience relevance. Based on an extensive interdisciplinary review, I provide a novel framework that presents resonance as a spectrum of experience with a multitude of outcomes ranging from a sense of harmony and coherence to life transformation. I argue that resonance has different properties to the more traditional interpretation of relevance and provides a better system of explanation of what it means to experience relevance. I show how traditional approaches to relevance and resonance work in a complementary fashion and outline how resonance may present distinct new lines of research into relevance theory. Ian Ruthven |
J. Assoc. Inf. Sci. Technol. | 1 |
| 2020 | Information need: Introduction to the special issue
Pia Borlund, Ian Ruthven |
Inf. Process. Manag. | 2 |
| 2019 | Making Meaning: A Focus for Information Interactions ResearchabstractIn this perspectives paper, I discuss meaning-making as an information seeking and interaction enterprise. I present meaning-making as a vital human reaction to significant life changes and present indicative evidence of how people go about gathering information for making meaning within their lives. I discuss some of the various forms of information that can be used for meaning-making, why it is an information seeking task that is different to those we are used to in information seeking research, and motivate meaning-making as a new focus for information seeking and information interactions research. Ian Ruthven |
CHIIR | 1 |
| 2019 | The Information Needs Matrix: A navigational guide for refugee integration
Olubukola Oduntan, Ian Ruthven |
Inf. Process. Manag. | 2 |
| 2019 | The language of information need: Differentiating conscious and formalized information needs
Ian Ruthven |
Inf. Process. Manag. | 1 |
| 2019 | Information behaviors in disadvantaged and dependent circumstances and the role of information intermediariesabstractThis article provides the first empirical study focused exclusively on the information intermediary role in disadvantaged (socioeconomic) and dependent (support) circumstances. We report findings from interviews and focus groups with 49 UK state and voluntary sector professionals providing support to young (<21) mothers from areas of multiple deprivations. We evidence an important information intermediary role with three key contributions to information behaviors in disadvantaged and dependent circumstances. Intermediaries: facilitate information needs recognition, and consider purposeful action within problematic situations; are a key source of information in themselves, and a key integrative connection to other external sources not otherwise accessed; and tailor and personalize information for relevance, and communicate via incremental and recursive cycles that take into account learning needs. We provide parameters for a theory of information intermediary intervention to guide future examination of an important and understudied role; and conceptualize important theoretical relationships between information behavior and social capital, and in particular shared concepts of social integration, and the progressive and integrative intermediary role within. Our findings have significant practical implications for public health policy and digital health strategies, as they evidence an important human information intermediary role among an at‐risk group, with implications for disadvantaged and vulnerable populations more broadly. Steven Buchanan, Cara Jardine, Ian Ruthven |
J. Assoc. Inf. Sci. Technol. | 3 |
| 2018 | Relationships, environment, health and development: The information needs expressed online by young first-time mothersabstractThis study investigates the information needs of young first time mothers through a qualitative content analysis of 266 selected posts to a major online discussion group. Our analysis reveals three main categories of need: needs around how to create a positive environment for a child, needs around a mother's relationships and well‐being and needs around child development and health. We demonstrate the similarities of this scheme to needs uncovered in other studies and how our classification of needs is more comprehensive than those in previous studies. A critical distinction in our results is between two types of need presentation, distinguishing between situational and informational needs. Situational needs are narrative descriptions of a problematic situations whereas informational needs are need statements with a clear request. Distinguishing between these two types of needs sheds new light on how information needs develop. We conclude with a discussion on the implication of our results for young mothers and information providers. Ian Ruthven, Steven Buchanan, Cara Jardine |
J. Assoc. Inf. Sci. Technol. | 1 |
| 2018 | Isolated, overwhelmed, and worried: Young first-time mothers asking for information and support onlineabstractThis study investigates the emotional content of 174 posts from 162 posters to online forums made by young (age 14–21) first‐time mothers to understand what emotions are expressed in these posts and how these emotions interact with the types of posts and the indicators of Information Poverty within the posts. Using textual analyses we provide a classification of emotions within posts across three main themes of interaction emotions, response emotions, and preoccupation emotions and show that many requests for information by young first‐time mothers are motivated by negative emotions. This has implications for how moderators of online news groups respond to online request for help and for understanding how to support vulnerable young parents. Ian Ruthven, Steven Buchanan, Cara Jardine |
J. Assoc. Inf. Sci. Technol. | 1 |
| 2016 | Sharing "happy" informationabstractThis study focuses on the sharing of “happy” information: information that creates a sense of happiness within the individual sharing the information. We explore the range of factors motivating and impacting individuals' happy information‐sharing behavior within a casual leisure context through 30 semistructured interviews. The findings reveal that the factors influencing individuals' happy information‐sharing behavior are numerous, and impact each other. Most individuals considered sharing happy information important to their friendships and relationships. In various contexts the act of sharing happy information was shown to enhance the sharer's happiness. Fiona Tinto, Ian Ruthven |
J. Assoc. Inf. Sci. Technol. | 2 |
| 2014 | You have e-mail, what happens next? Tracking the eyes for genreabstractThis paper reports on an approach to the analysis of form (layout and formatting) during genre recognition recorded using eye tracking. The researchers focused on eight different types of e-mail, such as calls for papers, newsletters and spam, which were chosen to represent different genres. The study involved the collection of oculographic behavior data based on the scanpath duration and scanpath length based metric, to highlight the ways in which people view the features of genres. We found that genre analysis based on purpose and form (layout features, etc.) was an effective means of identifying the characteristics of these e-mails. The research, carried out on a group of 24 participants, highlighted their interaction and interpretation of the e-mail texts and the visual cues or features perceived. In addition, the ocular strategies of scanning and skimming, they employed for the processing of the texts by block, genre and representation were evaluated. Malcolm Clark, Ian Ruthven, Patrik O'Brian Holt, Dawei Song 0001, Stuart N. K. Watt |
Inf. Process. Manag. | 2 |
| 2014 | Using internet groups in situations of information poverty: Topics and information needsabstractThis study explores the use of online newsgroups and discussion groups by people in situations of information poverty. Through a qualitative content analysis of 200 posts across Internet groups, we identify topics and information needs expressed by people who feel they have no other sources of support available to them. We uncover various health, well‐being, social, and identity issues that are not only crucial to the lives of the people posting but which they are unwilling to risk revealing elsewhere—offering evidence that these online environments provide an outlet for the expression of critical and hidden information needs. To enable this analysis, we first describe our method for reliably identifying situations of information poverty in messages posted to these groups and outline our coding approach. Our work contributes to the study of both information seeking within the context of information poverty and the use of Internet groups as sources of information and support, bridging the two by exploring the manifestation of information poverty in this particular online setting. Laura Hasler, Ian Ruthven, Steven Buchanan |
J. Assoc. Inf. Sci. Technol. | 2 |
| 2012 | An eye-tracking approach to the analysis of relevance judgments on the Web: The case of Google search engineabstractEye movement data can provide an in‐depth view of human reasoning and the decision‐making process, and modern information retrieval (IR) research can benefit from the analysis of this type of data. The aim of this research was to examine the relationship between relevance criteria use and visual behavior in the context of predictive relevance judgments. To address this objective, a multimethod research design was employed that involved observation of participants’ eye movements, talk‐aloud protocols, and postsearch interviews. Specifically, the results reported in this article came from the analysis of 281 predictive relevance judgments made by 24 participants using the Google search engine. We present a novel stepwise methodological framework for the analysis of relevance judgments and eye movements on the Web and show new patterns of relevance criteria use during predictive relevance judgment. For example, the findings showed an effect of ranking order and surrogate components (Title, Summary, and URL) on the use of relevance criteria. Also, differences were observed in the cognitive effort spent between very relevant and not relevant judgments. We conclude with the implications of this study for IR research. Panos Balatsoukas, Ian Ruthven |
J. Assoc. Inf. Sci. Technol. | 2 |
| 2011 | Bayesian latent variable models for collaborative item rating predictionabstractCollaborative filtering systems based on ratings make it easier for users to find content of interest on the Web and as such they constitute an area of much research. In this paper we first present a Bayesian latent variable model for rating prediction that models ratings over each user's latent interests and also each item's latent topics. We describe a Gibbs sampling procedure that can be used to estimate its parameters and show by experiment that it is competitive with the gradient descent SVD methods commonly used in state-of-the-art systems. We then proceed to make an important and novel extension to this model, enhancing it with user-dependent and item-dependant biases to significantly improve rating estimation. We show by experiment on a large set of real ratings data that these models are able to outperform 3 common baselines, including a very competitive and modern SVD-based model. Furthermore we illustrate other advantages of our approach beyond simply its ability to provide more accurate ratings and show that it is able to perform better on the common and important case where the user profile is short. Morgan Harvey, Mark J. Carman, Ian Ruthven, Fabio Crestani |
CIKM | 3 |
| 2011 | What Makes Re-finding Information Difficult? A Study of Email Re-finding
David Elsweiler, Mark Baillie, Ian Ruthven |
ECIR | 3 |
| 2011 | Escaping Information Poverty through Internet Newsgroups
Laura Hasler, Ian Ruthven |
ICWSM | 2 |
| 2011 | Improving social bookmark search using personalised latent variable language modelsabstractSocial tagging systems have recently become very popular as a method of categorising information online and have been used to annotate a wide range of different resources. In such systems users are free to choose whatever keywords or "tags" they wish to annotate each resource, resulting in a highly personalised, unrestricted vocabulary. While this freedom of choice has several notable advantages, it does come at the cost of making searching of these systems more difficult as the vocabulary problem introduced is more pronounced than in a normal information retrieval setting. Morgan Harvey, Ian Ruthven, Mark J. Carman |
WSDM | 2 |
| 2010 | Ranking social bookmarks using topic modelsabstractRanking of resources in social tagging systems is a difficult problem due to the inherent sparsity of the data and the vocabulary problems introduced by having a completely unrestricted lexicon. In this paper we propose to use hidden topic models as a principled way of reducing the dimensionality of this data to provide more accurate resource rankings with higher recall. We first describe Latent Dirichlet Allocation (LDA) and then show how it can be used to rank resources in a social bookmarking system. We test the LDA tagging model and compare it with 3 non-topic model baselines on a large data sample obtained from the Delicious social bookmarking site. Our evaluations show that our LDA-based method significantly outperforms all of the baselines. Morgan Harvey, Ian Ruthven, Mark J. Carman |
CIKM | 2 |
| 2010 | Colouring the Dimensions of Relevance
Ulises Cerviño Beresi, Yunhyong Kim, Mark Baillie, Ian Ruthven, Dawei Song 0001 |
ECIR | 4 |
| 2010 | Tripartite Hidden Topic Models for Personalised Tag Suggestion
Morgan Harvey, Mark Baillie, Ian Ruthven, Mark J. Carman |
ECIR | 3 |
| 2010 | Enabling Interactive Query Expansion through Eliciting the Potential Effect of Expansion Terms
Nuzhah Gooda Sahib, Anastasios Tombros, Ian Ruthven |
ECIR | 3 |
| 2009 | Design of an Interface for Interactive Topic Detection and Tracking
Masnizah Mohd, Fabio Crestani, Ian Ruthven |
FQAS | 3 |
| 2008 | A Comparison of Named Entity Patterns from a User Analysis and a System Analysis
Masnizah Mohd, Fabio Crestani, Ian Ruthven |
ECIR | 3 |
| 2008 | Evaluating epistemic uncertainty under incomplete assessments
Mark Baillie, Leif Azzopardi, Ian Ruthven |
Inf. Process. Manag. | 3 |
| 2008 | Introduction to the special issue on evaluating interactive information retrieval systems
Pia Borlund, Ian Ruthven |
Inf. Process. Manag. | 2 |
| 2008 | Contextual factors affecting the utility of surrogates within exploratory search
Ian Ruthven, Mark Baillie, Leif Azzopardi, Ralf Bierig, Emma Nicol, Simon O. Sweeney, Murat Yaciki |
Inf. Process. Manag. | 1 |
| 2008 | Exploring memory in email refindingabstractHuman memory plays an important role in personal information management (PIM). Several scholars have noted that people refind information based on what they remember and it has been shown that people adapt their management strategies to compensate for the limitations of memory. Nevertheless, little is known about what people tend to remember about their personal information and how they use their memories to refind. The aim of this article is to increase our understanding of the role that memory plays in the process of refinding personal information. Concentrating on email re-finding, we report on a user study that investigates what attributes of email messages participants remember when trying to refind. We look at how the attributes change in different scenarios and examine the factors which impact on what is remembered. David Elsweiler, Mark Baillie, Ian Ruthven |
ACM Trans. Inf. Syst. | 3 |
| 2007 | A Retrieval Evaluation Methodology for Incomplete Relevance Assessments
Mark Baillie, Leif Azzopardi, Ian Ruthven |
ECIR | 3 |
| 2007 | Towards task-based personal information management evaluationsabstractPersonal Information Management (PIM) is a rapidly growing area of research concerned with how people store, manage and refind information. A feature of PIM research is that many systems have been designed to assist users manage and refind information, but very few have been evaluated. This has been noted by several scholars and explained by the difficulties involved in performing PIM evaluations. The difficulties include that people re-find information from within unique personal collections; researchers know little about the tasks that cause people to re-find information; and numerous privacy issues concerning personal information. In this paper we aim to facilitate PIM evaluations by addressing each of these difficulties. In the first part, we present a diary study of information re-finding tasks. The study examines the kind of tasks that require users to refind information and produces a taxonomy of refinding tasks for email messages and web pages. In the second part, we propose a task-based evaluation methodology based on our findings and examine the feasibility of the approach using two different methods of task creation. David Elsweiler, Ian Ruthven |
SIGIR | 2 |
| 2007 | Intra-assessor consistency in question answeringabstractIn this paper we investigate the consistency of answer assessment in a complex question answering task examining features of assessor consistency, types of answers and question type. Ian Ruthven, Leif Azzopardi, Mark Baillie, Ralf Bierig, Emma Nicol, Simon O. Sweeney, Murat Yakici |
SIGIR | 1 |
| 2007 | Modelling epistemic uncertainty in ir evaluationabstractModern information retrieval (IR) test collections violate the completeness assumption of the Cranfield paradigm. In order to maximise the available resources, only a sample of documents (i.e. the pool) are judged for relevance by a human assessor(s). The subsequent evaluation protocol does not make any distinctions between assessed or unassesseddocuments, as documents that are not in the pool are assumedto be not relevant for the topic. This is beneficial from a practical point of view, as the relative performance can be compared with confidence if the experimental conditions are fair for all systems. However, given the incompleteness of relevance assessments, two forms of uncertainty emerge during evaluation. The first is Aleatory uncertainty, which refers to variation in system performance across the topic set, which is often addressed through the use of statistical significance tests. The second form of uncertainty is Epistemic, which refers to the amount of knowledge (or ignorance) we have about the estimate of a system's performance. Epistemic uncertainty is a consequence of incompleteness and is not addressed by the current evaluation protocol. In this study, we present a first attempt at modelling both aleatory and epistemic uncertainty associatedwith IR evaluation. We aim to account for both the variability associated with system performance and the amount of knowledge known about the performance estimate. Murat Yakici, Mark Baillie, Ian Ruthven, Fabio Crestani |
SIGIR | 3 |
| 2007 | Introduction to special issue on contextual information retrieval systems
Fabio Crestani, Ian Ruthven |
Inf. Retr. | 2 |
| 2007 | Towards memory supporting personal information management toolsabstractAbstract In this article, the authors discuss reretrieving personal information objects and relate the task to recovering from lapse(s) in memory. They propose that memory lapses impede users from successfully refinding the information they need. Their hypothesis is that by learning more about memory lapses in noncomputing contexts and about how people cope and recover from these lapses, we can better inform the design of personal information management (PIM) tools and improve the user's ability to reaccess and reuse objects. They describe a diary study that investigates the everyday memory problems of 25 people from a wide range of backgrounds. Based on the findings, they present a series of principles that they hypothesize will improve the design of PIM tools. This hypothesis is validated by an evaluation of a tool for managing personal photographs, which was designed with respect to the authors' findings. The evaluation suggests that users' performance when refinding objects can be improved by building personal information management tools to support characteristics of human memory. David Elsweiler, Ian Ruthven |
J. Assoc. Inf. Sci. Technol. | 2 |
| 2006 | The Effects on Topic Familiarity on Online Search Behaviour and Use of Relevance Criteria
Lei Wen, Ian Ruthven, Pia Borlund |
ECIR | 2 |
| 2006 | Examining assessor attributes at HARD 2005abstractNo abstract available. Mark Baillie, Ian Ruthven |
SIGIR | 2 |
| 2006 | An implicit feedback approach for interactive information retrieval
Ryen W. White, Joemon M. Jose, Ian Ruthven |
Inf. Process. Manag. | 3 |
| 2006 | A study of interface support mechanisms for interactive information retrievalabstractAbstract Advances in search technology have meant that search systems can now offer assistance to users beyond simply retrieving a set of documents. For example, search systems are now capable of inferring user interests by observing their interaction, offering suggestions about what terms could be used in a query, or reorganizing search results to make exploration of retrieved material more effective. When providing new search functionality, system designers must decide how the new functionality should be offered to users. One major choice is between (a) offering automatic features that require little human input, but give little human control; or (b) interactive features which allow human control over how the feature is used, but often give little guidance over how the feature should be best used. This article presents a study in which we empirically investigate the issue of control by presenting an experiment in which participants were asked to interact with three experimental systems that vary the degree of control they had in creating queries, indicating which results are relevant in making search decisions. We use our findings to discuss why and how the control users want over search decisions can vary depending on the nature of the decisions and the impact of those decisions on the user's search. Ryen W. White, Ian Ruthven |
J. Assoc. Inf. Sci. Technol. | 2 |
| 2005 | A study of factors affecting the utility of implicit relevance feedbackabstractImplicit relevance feedback (IRF) is the process by which a search system unobtrusively gathers evidence on searcher interests from their interaction with the system. IRF is a new method of gathering information on user interest and, if IRF is to be used in operational IR systems, it is important to establish when it performs well and when it performs poorly. In this paper we investigate how the use and effectiveness of IRF is affected by three factors: search task complexity, the search experience of the user and the stage in the search. Our findings suggest that all three of these factors contribute to the utility of IRF. Ryen W. White, Ian Ruthven, Joemon M. Jose |
SIGIR | 2 |
| 2005 | How users assess Web pages for information seekingabstractAbstract In this article, we investigate the criteria used by online searchers when assessing the relevance of Web pages for information‐seeking tasks. Twenty‐four participants were given three tasks each, and they indicated the features of Web pages that they used when deciding about the usefulness of the pages in relation to the tasks. These tasks were presented within the context of a simulated work‐task situation. We investigated the relative utility of features identified by participants (Web page content, structure, and quality) and how the importance of these features is affected by the type of information‐seeking task performed and the stage of the search. The results of this study provide a set of criteria used by searchers to decide about the utility of Web pages for different types of tasks. Such criteria can have implications for the design of systems that use or recommend Web pages. Anastasios Tombros, Ian Ruthven, Joemon M. Jose |
J. Assoc. Inf. Sci. Technol. | 2 |
| 2005 | Using top-ranking sentences to facilitate effective information accessabstractAbstract Web searchers typically fail to view search results beyond the first page nor fully examine those results presented to them. In this article we describe an approach that encourages a deeper examination of the contents of the document set retrieved in response to a searcher's query. The approach shifts the focus of perusal and interaction away from potentially uninformative document surrogates (such as titles, sentence fragments, and URLs) to actual document content, and uses this content to drive the information seeking process. Current search interfaces assume searchers examine results document‐by‐document. In contrast our approach extracts, ranks, and presents the contents of the top‐ranked document set. We use query‐relevant top‐ranking sentences extracted from the top documents at retrieval time as fine‐grained representations of top‐ranked document content and, when combined in a ranked list, an overview of these documents. The interaction of the searcher provides implicit evidence that is used to reorder the sentences where appropriate. We evaluate our approach in three separate user studies, each applying these sentences in a different way. The findings of these studies show that top‐ranking sentences can facilitate effective information access. Ryen W. White, Joemon M. Jose, Ian Ruthven |
J. Assoc. Inf. Sci. Technol. | 3 |
| 2005 | Evaluating implicit feedback models using searcher simulationsabstractIn this article we describe an evaluation of relevance feedback (RF) algorithms using searcher simulations. Since these algorithms select additional terms for query modification based on inferences made from searcher interaction, not on relevance information searchers explicitly provide (as in traditional RF), we refer to them as implicit feedback models . We introduce six different models that base their decisions on the interactions of searchers and use different approaches to rank query modification terms. The aim of this article is to determine which of these models should be used to assist searchers in the systems we develop. To evaluate these models we used searcher simulations that afforded us more control over the experimental conditions than experiments with human subjects and allowed complex interaction to be modeled without the need for costly human experimentation. The simulation-based evaluation methodology measures how well the models learn the distribution of terms across relevant documents (i.e., learn what information is relevant) and how well they improve search effectiveness (i.e., create effective search queries). Our findings show that an implicit feedback model based on Jeffrey's rule of conditioning outperformed other models under investigation. Ryen W. White, Ian Ruthven, Joemon M. Jose, C. J. van Rijsbergen |
ACM Trans. Inf. Syst. | 2 |
| 2004 | Searcher's Assessments of Task Complexity for Web Searching
David J. Bell, Ian Ruthven |
ECIR | 2 |
| 2004 | A Simulated Study of Implicit Feedback Models
Ryen W. White, Joemon M. Jose, C. J. van Rijsbergen, Ian Ruthven |
ECIR | 4 |
| 2003 | An approach for implicitly detecting information needsabstractSearchers can have problems devising queries that accurately express their, often dynamic, information needs. In this paper we describe an adaptive approach that uses unobtrusive monitoring of interaction to help alleviate such problems and support searchers in their seeking. The approach we propose implicitly selects terms to better represent information needs, gathers evidence on potential changes in these needs, and uses this evidence to tailor the result presentation accordingly. A user evaluation of an interface implementing our approach, presented in [7], shows it can select terms that approximate current information needs and provide evidence to track changes in these needs. Ryen W. White, Joemon M. Jose, Ian Ruthven |
CIKM | 3 |
| 2003 | Re-examining the potential effectiveness of interactive query expansionabstractMuch attention has been paid to the relative effectiveness of interactive query expansion versus automatic query expansion. Although interactive query expansion has the potential to be an effective means of improving a search, in this paper we show that, on average, human searchers are less likely than systems to make good expansion decisions. To enable good expansion decisions, searchers must have adequate instructions on how to use interactive query expansion functionalities. We show that simple instructions on using interactive query expansion do not necessarily help searchers make good expansion decisions and discuss difficulties found in making query expansion decisions. Ian Ruthven |
SIGIR | 1 |
| 2003 | Searchers' criteria For assessing web pagesabstractWe investigate the criteria used by online searchers when assessing the relevance of web pages to information-seeking tasks. Twenty four searchers were given three tasks each, and indicated the features of web pages which they employed when deciding about the usefulness of the pages. These tasks were presented within the context of a simulated work-task situation. The results of this study provide a set of criteria used by searchers to decide about the utility of web pages. Such criteria have implications for the design of systems that use or recommend web pages, as well as to authors of web pages. Anastasios Tombros, Ian Ruthven, Joemon M. Jose |
SIGIR | 2 |
| 2003 | A task-oriented study on the influencing effects of query-biased summarisation in web searching
Ryen W. White, Joemon M. Jose, Ian Ruthven |
Inf. Process. Manag. | 3 |
| 2003 | Incorporating user search behavior into relevance feedbackabstractAbstract In this paper, we present five user experiments on incorporating behavioral information into the relevance feedback process. In particular, we concentrate on ranking terms for query expansion and selecting new terms to add to the user's query. Our experiments are an attempt to widen the evidence used for relevance feedback from simply the relevant documents to include information on how users are searching. We show that this information can lead to more successful relevance feedback techniques. We also show that the presentation of relevance feedback to the user is important in the success of relevance feedback. Ian Ruthven, Mounia Lalmas-Roelleke, C. J. van Rijsbergen |
J. Assoc. Inf. Sci. Technol. | 1 |
| 2002 | Building Bilingual Dictionaries from Parallel Web Documents
Craig J. A. McEwan, Iadh Ounis, Ian Ruthven |
ECIR | 3 |
| 2002 | The Accessibility Dimension for Structured Document Retrieval
Thomas Roelleke, Mounia Lalmas-Roelleke, Gabriella Kazai, Ian Ruthven, Stefan Quicker |
ECIR | 4 |
| 2002 | The Use of Implicit Evidence for Relevance Feedback in Web Retrieval
Ryen W. White, Ian Ruthven, Joemon M. Jose |
ECIR | 2 |
| 2002 | A system using implicit feedback and top ranking sentences to help users find relevant web documentsabstractWe present a web search interface designed to encourage users to interact more fully with the results of a web search. Wrapping around a major commercial search engine, the system combines three main features; real-time query-biased web document summarisation, the presentation of sentences highly relevant to the searcher's query, and evidence captured from searcher interaction with the retrieval results. Ryen W. White, Joemon M. Jose, Ian Ruthven |
SIGIR | 3 |
| 2002 | Finding relevant documents using top ranking sentences: an evaluation of two alternative schemesabstractIn this paper we present an evaluation of techniques that are designed to encourage web searchers to interact more with the results of a web search. Two specific techniques are examined: the presentation of sentences that highly match the searcher's query and the use of implicit evidence. Implicit evidence is evidence captured from the searcher's interaction with the retrieval results and is used to automatically update the display. Our evaluation concentrates on the effectiveness and subject perception of these techniques. The results show, with statistical significance, that the techniques are effective and efficient for information seeking. Ryen W. White, Ian Ruthven, Joemon M. Jose |
SIGIR | 2 |
| 2002 | Combining and selecting characteristics of information useabstractAbstract In this article we report on a series of experiments designed to investigate the combination of term and document weighting functions in information retrieval. We describe a series of weighting functions, each of which is based on how information is used within documents and collections, and use these weighting functions in two types of experiments: one based on combination of evidence for ad hoc retrieval, the other based on selective combination of evidence within a relevance feedback situation. We discuss the difficulties involved in predicting good combinations of evidence for ad hoc retrieval, and suggest the factors that may lead to the success or failure of combination. We also demonstrate how, in a relevance feedback situation, the relevance assessments can provide a good indication of how evidence should be selected for query term weighting. The use of relevance information to guide the combination process is shown to reduce the variability inherent in combination of evidence. Ian Ruthven, Mounia Lalmas-Roelleke, C. J. van Rijsbergen |
J. Assoc. Inf. Sci. Technol. | 1 |
| 2002 | Using Dempster-Shafer's Theory of Evidence to Combine Aspects of Information Use
Ian Ruthven, Mounia Lalmas-Roelleke |
J. Intell. Inf. Syst. | 1 |
| 2001 | Empirical Investigations on Query Modification Using Abductive ExplanationsabstractIn this paper we report on a series of experiments designed to investigate query modification techniques motivated by the area of abductive reasoning. In particular we use the notion of abductive explanation, explanations being a description of data that highlight important features of the data. We describe several methods of creating abductive explanations, exploring term reweighting and query reformulation techniques and demonstrate their suitability for relevance feedback. Ian Ruthven, C. J. van Rijsbergen, Mounia Lalmas-Roelleke |
SIGIR | 1 |
| 2001 | Query-Based Web Page Summarisation: A Task-Oriented EvaluationabstractWe present a system that offers a new way of assessing web document relevance and new approach to the web-based evaluation of such a system. Provisionally named WebDocSum, the system is a query-biased web page summariser that aims to provide an alternative to the short, irrelevant abstracts typical of many web search result lists. Based on an initial evaluation the system appears to be more useful in helping users gauge document relevance than the traditional ranked titles/abstracts approach. Ryen W. White, Joemon M. Jose, Ian Ruthven |
SIGIR | 3 |
| 2000 | Incorporating Aspects of Information Use into Relevance Feedback
Ian Ruthven |
Inf. Retr. | 1 |
| 1999 | A Framework for Investigating the Interaction in Information Retrieval
Mounia Lalmas-Roelleke, Ian Ruthven |
EJC | 2 |