Personalization of information retrieval (PIR) is aimed at tailoring a search toward individual users and user groups by taking account of additional information about users besides their queries. In the past two decades or so, PIR has received extensive attention in both academia and industry. This article surveys the literature of personalization in text retrieval, following a framework for aspects or factors that can be used for personalization. The framework consists of additional information about users that can be explicitly obtained by asking users for their preferences, or implicitly inferred from users' search behaviors. Users' characteristics and contextual factors such as tasks, time, location, etc., can be helpful for personalization. This article also addresses various issues including when to personalize, the evaluation of PIR, privacy, usability, etc. Based on the extensive review, challenges are discussed and directions for future effort are suggested.

It is a frequently seen scenario that when people are not familiar with their search topics, they use a simple keyword search, which leads to a large amount of search results in multiple pages. This makes it difficult for users to pick relevant documents, especially given that they are not knowledgeable of the topics. To explore how systems can better help users find relevant documents from search results, the current research analyzed document selection behaviors of users with different levels of domain knowledge (DK). Data were collected in a laboratory study with 35 participants each searching on four tasks in the genomics domain. The results show that users with high and low DK levels selected different sets of documents to view; those high in DK read more documents and gave higher relevance ratings for the viewed documents than those low in DK did. Users with low DK tended to select documents ranking toward the top of the search result lists, and those with high in DK tended to also select documents ranking down the search result lists. The findings help design search systems that can personalize search results to users with different levels of DK.

We investigated systems and their features that help people use retrieved information to accomplish their information tasks. Participants were 32 college students who first recalled a recently accomplished task, and then worked on a task that they would need to finish. They answered questionnaire questions about what systems and features are helpful for task accomplishment. Our results discovered multiple helpful systems and features, which have implications on designing search systems for better helping accomplish tasks.

Studies have examined how time constraints influence search behaviors; however, no effort has been spent on how time constraints may help predict document usefulness for personalization purposes. This study aims to fill this gap by researching the relationships between time constraints, search behaviors, and usefulness judgments. A controlled lab experiment was conducted with 40 participants searching for four tasks of two types (fact finding and information understanding), under two time conditions (with or without time constraints). Results show that time constraints and usefulness had interaction effects on first dwell time; while usefulness had positive relationship with total dwell time. Results indicate that knowing time constraints helps predict document usefulness from dwell time. The findings provide implications on personalization in information search.

Increasing evidence from psychoanalytic and psychodynamic research suggests that the unconscious influences our daily decisions, and psychologists have suggested that as much as 85-95% of decision making occurs outside our conscious awareness. For information science, it is important to understand how these unconscious processes play a role in information seeking. The current study uses subliminal psychodynamic activation (SPA), through the use of subliminal messages that appeared below the threshold of conscious awareness, to investigate the influence of the unconscious in information searching. Twenty-four college students participated in a controlled laboratory experiment, each searching freely on the Internet for information for three search tasks, with various SPA messages appearing in each search. Participants were systematically assigned to one of the four SPA conditions with various subliminal messages. Users' perceptions of search task topic interest, topic knowledge and task difficulty was examined between different SPA conditions, as well as the change of users' perceptions on topic interest, knowledge, and difficulty before and after searching in different SPA conditions. Findings suggested that users' pre- and post-task ratings of topic interest, knowledge, and difficulty did not show significant differences among the 4 SPA conditions. However, the change in users' perceptions of topic interest and topic knowledge before and after searching showed significant differences when SPA messages were present. Our findings inspire future research in this underexplored field.

As a significant contextual factor in information search, topic knowledge has been gaining increased research attention. We report on a study of the relationship between information searchers' topic knowledge and their search behaviors, and on an attempt to predict searchers' topic knowledge from their behaviors during the search. Data were collected in a controlled laboratory experiment with 32 undergraduate journalism student participants, each searching on 4 tasks of different types. In general, behavioral variables were not found to have significant differences between users with high and low levels of topic knowledge, except the mean first dwell time on search result pages. Several models were built to predict topic knowledge using behavioral variables calculated at 3 different stages of search episodes: the first‐query‐round, the middle point of the search, and the end point. It was found that a model using some search behaviors observed in the first query round led to satisfactory prediction results. The results suggest that early‐session search behaviors can be used to predict users' topic knowledge levels, allowing personalization of search for users with different levels of topic knowledge, especially in order to assist users with low topic knowledge.

Personalization of information retrieval tailors search towards individual users to meet their particular information needs by taking into account information about users and their contexts, often through implicit sources of evidence such as user behaviors. This study looks at users' dwelling behavior on documents and several contextual factors: the stage of users' work tasks, task type, and users' knowledge of task topics, to explore whether or not taking account contextual factors could help infer document usefulness from dwell time. A controlled laboratory experiment was conducted with 24 participants, each coming 3 times to work on 3 subtasks in a general work task. The results show that task stage could help interpret certain types of dwell time as reliable indicators of document usefulness in certain task types, as was topic knowledge, and the latter played a more significant role when both were available. This study contributes to a better understanding of how dwell time can be used as implicit evidence of document usefulness, as well as how contextual factors can help interpret dwell time as an indicator of usefulness. These findings have both theoretical and practical implications for using behaviors and contextual factors in the development of personalization systems.

User domain knowledge affects search behaviors and search success. Predicting a user's knowledge level from implicit evidence such as search behaviors could allow an adaptive information retrieval system to better personalize its interaction with users. This study examines whether user domain knowledge can be predicted from search behaviors by applying a regression modeling analysis method. We identify behavioral features that contribute most to a successful prediction model. A user experiment was conducted with 40 participants searching on task topics in the domain of genomics. Participant domain knowledge level was assessed based on the users' familiarity with and expertise in the search topics and their knowledge of MeSH (Medical Subject Headings) terms in the categories that corresponded to the search topics. Users' search behaviors were captured by logging software, which includes querying behaviors, document selection behaviors, and general task interaction behaviors. Multiple regression analysis was run on the behavioral data using different variable selection methods. Four successful predictive models were identified, each involving a slightly different set of behavioral variables. The models were compared for the best on model fit, significance of the model, and contributions of individual predictors in each model. Each model was validated using the split sampling method. The final model highlights three behavioral variables as domain knowledge level predictors: the number of documents saved, the average query length, and the average ranking position of the documents opened. The results are discussed, study limitations are addressed, and future research directions are suggested.

Knowing, in real time, whether a current searcher in an information retrieval system finds the search task difficult can be valuable for tailoring the system's support for that searcher. This study investigated searcher's behaviors at different stages of the search process; they are: 1) first-round point at the beginning of the search, right before searchers issued their second query; 2) middle point, when searchers proceeded to the middle of the search process, and 3) end point, when searchers finished the whole task. We compared how the behavioral features calculated at these three points were different between difficult and easy search tasks, and identified behavioral features during search sessions that can be used in real-time to predict perceived task difficulty. In addition, we compared the prediction performance at different stages of search process. Our results show that a number of user behavioral measures at all three points differed between easy and difficult tasks. Query interval time, dwell time on viewed documents, and number of viewed documents per query were important predictors of task difficulty. The results also indicate that it is possible to make relatively accurate prediction of task difficulty at the first query round of a search. Our findings can help search systems predict task difficulty which is necessary in personalizing support for the individual searcher.

We report on an investigation of behavioral differences between users in difficult and easy search tasks. Behavioral factors that can be used in real-time to predict task difficulty are identified. User data was collected in a controlled lab experiment (n=38) where each participant completed four search tasks in the genomics domain. We looked at user behaviors that can be obtained by systems at three levels, distinguished by the time point when the measurements can be done. They are: 1) first-round level at the beginning of the search, 2) accumulated level during the search, and 3) whole-session level by the end of the search. Results show that a number of user behaviors at all three levels differed between easy and difficult tasks. Models predicting task difficulty at all three levels were developed and evaluated. A real-time model incorporating first-round and accumulated levels of behaviors (FA) had fairly good prediction performance (accuracy 83%; precision 88%), which is comparable with the model using the whole-session level behaviors which are not real-time (accuracy 75%; precision 92%). We also found that for efficiency purpose, using only a limited number of significant variables (FC_FA) can obtain a prediction accuracy of 75%, with a precision of 88%. Our findings can help search systems predict task difficulty and adapt search results to users.

We report findings on how the user's perception of task difficulty changes before and after searching for information to solve tasks. We found that while in one type of task, the dependent task, this did not change, in another, the parallel task, it did. The findings have implications on designing systems that can provide assistance to users with their search and task solving strategies.

Dwell time as a user behavior has been found in previous studies to be an unreliable predictor of document usefulness, with contextual factors such as the user's task needing to be considered in its interpretation. Task stage has been shown to influence search behaviors including usefulness judgments, as has task type. This paper reports on an investigation of how task stage and task type may help predict usefulness from the time that users spend on retrieved documents, over the course of several information seeking episodes. A 3-stage controlled experiment was conducted with 24 participants, each coming 3 times to work on 3 sub-tasks of a general task, couched either as "parallel" or "dependent" task type. The full task was to write a report on the general topic, with interim documents produced for each sub-task. Results show that task stage can help in inferring document usefulness from decision time, especially in the parallel task. The findings can be used to increase accuracy in predicting document usefulness and accordingly in personalizing search for multi-session tasks.

In this paper, we report findings on how user behaviors vary in tasks with different difficulty levels as well as of different types. Two behavioral signals: document dwell time and number of content pages viewed per query, were found to be able to help the system detect when users are working with difficult tasks.

This study investigates the impact of different search feature designs in DLs on user search experience. The results indicate that the impact is significant in terms of the number of queries issued, search steps, zero-hits pages returned, and search errors.