Personality traits influence how individuals engage, behave, and make decisions during the information-seeking process. However, few studies have linked personality to observable search behaviors. This study aims to characterize personality traits through a multimodal time-series model that integrates eye-tracking data and gaze missingness–periods when the user’s gaze is not captured. This approach is based on the idea that people often look away when they think, signaling disengagement or reflection. We conducted a user study with 25 participants, who used an interactive application on an iPad, allowing them to engage with digital artifacts from a museum. We rely on raw gaze data from an eye tracker, minimizing preprocessing so that behavioral patterns can be preserved without substantial data cleaning. From this perspective, we trained models to predict personality traits using gaze signals. Our results from a five-fold cross-validation study demonstrate strong predictive performance across all five dimensions: Neuroticism (Macro F1 = 77.69%), Conscientiousness (74.52%), Openness (77.52%), Agreeableness (73.09%), and Extraversion (76.69%). The ablation study examines whether the absence of gaze information affects the model performance, demonstrating that incorporating missingness improves multimodal time-series modeling. The full model, which integrates both time-series signals and missingness information, achieves 10-15% higher accuracy and macro F1 scores across all Big Five traits compared to the model without time-series signals and missingness. These findings provide evidence that personality can be inferred from search-related gaze behavior and demonstrate the value of incorporating missing gaze data into time-series multimodal modeling.

In psychological practices, standardized questionnaires serve as essential tools for assessing mental health through structured, clinically-validated questions (i.e., items). While social media platforms offer rich data for mental health screening, computational approaches often bypass these established clinical assessment tools in favor of black-box classification. We propose a novel questionnaire-guided screening framework that bridges psychological practice and computational methods through adaptive Retrieval-Augmented Generation (aRAG). Our approach links unstructured social media content and standardized clinical assessments by retrieving relevant posts for each questionnaire item and using Large Language Models (LLMs) to complete validated psychological instruments. Our findings demonstrate two key advantages of questionnaire-guided screening: First, when completing the Beck Depression Inventory-II (BDI-II), our approach matches or outperforms state-of-the-art performance on Reddit-based benchmarks without requiring training data. Second, we show that guiding LLMs through standardized questionnaires can yield superior results compared to directly prompting them for depression screening, while also providing a more interpretable assessment by linking model outputs to clinically validated diagnostic criteria. Additionally, we show, as a proof-of-concept, how our questionnaire-based methodology can be extended to other mental conditions' screening, highlighting the promising role of LLMs as psychological assessors.

Abstract This paper presents a new approach to dense retrieval across multiple modalities, emphasizing the integration of images and sensor data. Traditional cross-modal retrieval techniques face significant challenges, particularly in processing non-linguistic modalities and creating effective training datasets. To address these issues, we propose a method that uses a shared vector space, optimized with contrastive loss, to enable efficient and accurate retrieval across diverse modalities. A key innovation of our approach is the introduction of a temporal closeness metric, which evaluates the relationship between data points based on their timestamps. This metric helps automatically extract positive and hard negative samples related to the query, improving the training process and enhancing the retrieval model’s performance. We validate our approach using the Lifelog Search Challenge 2024 (LSC’24) dataset, one of the largest multi-modal datasets, including non-linguistic data such as egocentric images, heart rate, and location information. Our evaluation shows that incorporating temporal closeness into the dense retrieval process significantly improves retrieval accuracy and robustness in real-world, multi-modal scenarios. This paper’s contributions include developing a novel dense retrieval framework, introducing the temporal closeness metric, and successfully applying these innovations to a comprehensive multi-modal dataset.

Argument mining has typically been researched for specific corpora belonging to concrete languages and domains independently in each research work. Human argumentation, however, has domain- and language-dependent linguistic features that determine the content and structure of arguments. Also, when deploying argument mining systems in the wild, we might not be able to control some of these features. Therefore, an important aspect that has not been thoroughly investigated in the argument mining literature is the robustness of such systems to variations in language and domain. In this paper, we present a complete analysis across three different languages and three different domains that allow us to have a better understanding on how to leverage the scarce available corpora to design argument mining systems that are more robust to natural language variations.

Deep hashing has been widely used to approximate nearest-neighbor search for image retrieval tasks. Most of them are trained with image-label pairs without any inter-label relationship, which may not make full use of the real-world data. This paper presents deep hashing, named HA2SH, that leverages multiple types of labels with hierarchical structures that an ethnological museum assigns to their artifacts. We experimentally prove that HA2SH can learn to generate hashes that give a better retrieval performance. Our code is available at https://github.com/wbw520/minpaku.

The Third WEPIR 2020 workshop builds on the success of the first two WEPIR meetings held at CHIIR 2018 and CHIIR 2019. WEPIR 2020 again brings together researchers from different backgrounds interested in continuing to explore and advance the evaluation of personalisation in information retrieval. Similar to the first two workshops, WEPIR 2020 has a strong emphasis on active participation by workshop attendees. This was very successfully achieved in the first two workshops by the use of workshop breakout groups exploring topics related to personalisation and information retrieval, and the evaluation of personalisation in information retrieval in general, with subsequent report back to the workshop as a whole. However, a key difference for WEPIR 2020 is that while the first two workshops focused on developing and articulating principles and ideas relating general topics relating to these topics, identified as interesting and important by the attendees at the workshops, WEPIR 2020 focuses breakout discussion on a number of relevant specific use cases of the evaluation of personalisation in information retrieval. A use case is assigned to each breakout group with the plan being to have more than one group work on each use case. The task for each group is to identify specific relevant factors relating to the use case in terms of user activities, data to be collected, ethical issues, and evaluation metrics. Groups will make reports of their discussions to the assembled workshop in the final session with discussion of the alternative solutions relating to the same use case, and contrasting the issues raised by the different use cases. The overall goal of the workshop is to work towards developing a general set of principles and guidelines for addressing the evaluation of specific instances of the use of personalisation in information retrieval tasks. This is consistent with the activities and goals of the first two workshops, but represents a significant progression of the activities towards concrete outcomes of benefit to those exploring personalisation in search, the issues arising in its evaluation, and how researchers might go about tackling this in specific situations.

In this study, we construct a corpus of Japanese local assembly minutes. All speeches in an assembly were transcribed into a local assembly minutes based on the local autonomy law. Therefore, the local assembly minutes form an extremely large amount of text data. Our ultimate objectives were to summarize and present the arguments in the assemblies, and to use the minutes as primary information for arguments in local politics. To achieve this, we structured all statements in assembly minutes. We focused on the structure of the discussion, i.e., the extraction of question and answer pairs. We organized the shared task “QA Lab-PoliInfo” in NTCIR 14. We conducted a “segmentation task” to identify the scope of one question and answer in the minutes as a sub task of the shared task. For the segmentation task, 24 runs from five teams were submitted. Based on the obtained results, the best recall was 1.000, best precision was 0.940, and best F-measure was 0.895.

With the rapid growth of mobile search, it is important to understand how users browse the mobile SERPs and allocate their limited attention to each result. To address this problem, we introduce a two-stage examination model that can separately capture the position bias with a skimming model and the attractiveness bias with an attractiveness model. The effectiveness of the proposed model is validated by using a dataset that contains explicit examination feedbacks from users. We further investigate user»s examination behaviors by analyzing the model parameters learned via EM algorithm. The results reveal some interesting findings such as how the skimming behavior is dependent on the previous examination sequence and what factors are associated with the attractiveness of search results on mobile SERPs.

Academic papers cite other literature for various purposes: such as sharing research motivations, comparing with previous methods or achieved accuracies, and so on. At some annual conference, high impact papers have been selected and honored after about 10 years from the publication to demonstrate both the emphasis research field and great contributions. This paper has proposed the estimation method of high impact papers using characteristics of those citing purposes. To extract the cited structure of scientific literature, we analyze the paper structure based on purpose and identify the citation points in the paper. For evaluating the proposed method, 17 years long proceedings on VLDB (Very Large Data Bases) Conference and "10 Year Award" papers are applied. We believe that the research becomes a foothold for estimating papers that may have a significant impact on the academic field in the future.

The amount of procedural texts, which describe the steps to reach a goal or solve a problem, continues to increase. This paper proposes flow graph generation methods for visualizing procedural texts. Examples of typical procedural texts include cooking texts, operation manuals for appliances/devices, and technical papers and patents that consist of methods or algorithms. The flow graphs of procedural texts allow a process's entire structure to be grasped more simply and for differences or shared parts to be detected. To generate a flow graph, we developed a structure analysis method using a domain-specific dictionary. We applied our proposed methods to an actual recipe and a manual for a computer printer and discussed our results and the remaining problems.

People often conduct exploratory search to explore unfamiliar information space and learn new knowledge. While supporting the highly dynamic and interactive exploratory search is still challenging for the search system, we want to investigate which factors can make the exploratory search successful and satisfying from the user’s perspective. Previous research suggests that domain experts have different search strategies and are more successful in finding domain-specific information, but how the domain expertise level will influence users’ interaction and search outcomes in exploratory search, especially in different knowledge domains, is still unclear. In this work, via a carefully designed user study that involves 30 participants, we investigate the influence of domain expertise levels on the interaction and outcome of exploratory search in three different domains: environment, medicine, and politics. We record participants’ search behaviors, including their explicit feedback and eye fixation sequences, in a laboratory setting. With this dataset, we identify both domain-independent and domain-dependent effects on user behaviors and search outcomes. Our results extend existing research on the effect of domain expertise in search and suggest different strategies for exploiting domain expertise to support exploratory search in different knowledge domains.

The volume of procedural texts, which describe the steps to reach a goal state, continue to increase. This paper proposes structural analysis methods for generating flow graphs from procedural texts. Examples of typical procedural texts include cooking texts, operation manuals for appliances/devices, and technical papers and patents that consist of methods or algorithms. The flow graphs of procedural texts allow the entire structure of a process to be grasped more simply and for differences or shared parts to be detected. To generate a flow graph, we developed a structure analysis method using a domain-specific dictionary. We apply our proposed methods to an actual recipe and a manual for a computer printer and discuss our result and remaining points.

There has been little research on task complexity and difficulty in music information retrieval (MIR), whereas many studies in the text retrieval domain have found that task complexity and difficulty have significant effects on user effectiveness. This study aimed to bridge the gap by exploring i) the relationship between task complexity and difficulty; ii) factors affecting task difficulty; and iii) the relationship between task difficulty, task complexity, and user search behaviors in MIR. An empirical user experiment was conducted with 51 participants and a novel MIR system. The participants searched for 6 topics across 3 complexity levels. The results revealed that i) perceived task difficulty in music search is influenced by task complexity, user background, system affordances, and task uncertainty and enjoyability; and ii) perceived task difficulty in MIR is significantly correlated with effectiveness metrics such as the number of songs found, number of clicks, and task completion time. The findings have implications for the design of music search tasks (in research) or use cases (in system development) as well as future MIR systems that can detect task difficulty based on user effectiveness metrics.

Many people share their daily events and opinions on Twitter. Some tweets are beneficial and others are related to such aspects of a user’s real-life as eating, traffic conditions, and weather. In this paper, we propose an inference method of the real-life aspect distribution of tweets using labeled tweets. Our method infers the aspect probability distributions by a hierarchical estimation framework (HEF), which is hierarchically composed of both unsupervised and supervised machine learning methods. In the first phase, it extracts topics from a sea of tweets using Latent Dirichlet Allocation (LDA). In the second phase, it builds associations between topics and real-life aspects using a small set of labeled tweets. The probability distribution of aspects is inferred using the associations based on the bag of terms extracted from unknown tweets. Our sophisticated experimental evaluations with a large amount of actual tweets demonstrate the high efficiency and robustness of our inference method. Especially in the case of single-label training, HEF showed significantly lower JSD values than other baseline methods, such as Naive Bayes, SVM, and L-LDA.

Twitter reflects events and trends in users' real lives because many of them post tweets related to their experiences. Many studies have succeeded in detecting events such as earthquakes and influenza epidemics, along with real-life information from a large amount of tweets, by assuming users as social sensors. On the other hand, inactive users who don't engage in posting activity, are increasing according as time progresses. To collect a large amount of tweets based on specific users for successful Twitter studies, we have to know the characteristics of users who are active over long periods of time. In this paper, we clarify the characteristics of growth users over a long time to strategically collect a large amount of specific users' tweets. We explore the status of users who were active in 2012, and classify users into three statuses of Dead, Lock, and Alive. Based on the differences between the numbers of tweets in 2012 and 2016, we further classify alive users into three types of Eraser, Slumber, and Growth. We analyze the characteristic feature values observed in each user behavior and provide interesting findings with each status/type based on GMM clustering and point-wise mutual information. Finally, we propose a growth user prediction method by a simple formula consisting of feature values and evaluate the effectiveness. We found that active users more easily dropped out than inactive users, and users who engaged in reciprocal communications by replies and retweets often became Growth type.

The "Search as Learning" (SAL) workshop is focused on an area within the information retrieval field that is only beginning to emerge: supporting users in their learning whilst interacting with information content.

Many Twitter users build various relationships through communication activity such as replies and retweets. For example, friends engage in conversations through replies. Fans unilaterally send many replies to celebrities. In this paper, we focus on such relationships between users. We assume that such relationships are classified into several patterns based on the feature values of communication reciprocity, and the relationships migrate to other ones as time progresses. We clarify the major relationships and transitions by analyzing the pattern frequency and transitions with high probability. From analysis results using a large amount of user pairs that we obtained over a long period, we detected several major and calm relationships.

Many Twitter users post tweets that are related to their particular interests. Users can also collect information by following other users. One approach clarifies user interests by tagging labels based on the users. A user tagging method is important to discover candidate users with similar interests. Typical approaches estimate user interests with terms in tweets and by applying graph theory such as following networks. In contrast, we propose a new user tagging method using the posting time series data of the number of tweets and developed the following hypothesis: Since users have interests, they will post more tweets at the time occurring the events compared with general times. Based on this hypothesis, we extract interests as burst levels from the user and hashtag time series data with Kleinberg's burst enumerating algorithm. We manage the burst levels of users as the term frequency in documents and calculate the hashtag scores for each user by three typical score calculation methods: cosine similarity, Naive Bayes, and TF-IDF. Thus, the proposed method needs no linguistic analysis which requires heavy computational resources. With our sophisticated experimental evaluations with actually active users, we demonstrate the high efficiency of our tagging methods, evaluate them using such information retrieval system evaluation metrics as expected reciprocal rank (ERR) and Q-measure, and clarify the strengths and limitations of each one. Naive Bayes and cosine similarity are especially suitable for user tagging and tag score calculation tasks.

This paper proposes a prototype system called Gaze-Learning-Access-and-Search-Engine 0.1 (GLASE), which can perform image relevance ranking based on gaze data and within-session learning. We developed a search user interface that uses an eye-tracker as an input device and employed a relevance re-ranking algorithm based on the gaze length. The preliminary experimental results showed that using our gaze-driven system reduced the task completion time an average of 13.7% in a search session.

No abstract available.

Community Question Answering (CQA) sites such as Yahoo! Answers have emerged as rich knowledge resources for information seekers. However, answers posted to CQA sites can be irrelevant, incomplete, redundant, incorrect, biased, ill-formed or even abusive. Hence, automatic selection of "good" answers for a given posted question is a practical research problem that will help us manage the quality of accumulated knowledge. One way to evaluate answer selection systems for CQA would be to use the Best Answers (BAs) that are readily available from the CQA sites. However, BAs may be biased, and even if they are not, there may be other good answers besides BAs. To remedy these two problems, we propose system evaluation methods that involve multiple answer assessors and graded-relevance information retrieval metrics. Our main findings from experiments using the NTCIR-8 CQA task data are that, using our evaluation methods, (a) we can detect many substantial differences between systems that would have been overlooked by BA-based evaluation; and (b) we can better identify hard questions (i.e. those that are handled poorly by many systems and therefore require focussed investigation) compared to BAbased evaluation. We therefore argue that our approach is useful for building effective CQA answer selection systems despite the cost of manual answer assessments.

[not available]

Modern information retrieval (IR) test collections have grown in size, but the available manpower for relevance assessments has more or less remained constant. Hence, how to reliably evaluate and compare IR systems using incomplete relevance data, where many documents exist that were never examined by the relevance assessors, is receiving a lot of attention. This article compares the robustness of IR metrics to incomplete relevance assessments, using four different sets of graded-relevance test collections with submitted runs—the TREC 2003 and 2004 robust track data and the NTCIR-6 Japanese and Chinese IR data from the crosslingual task. Following previous work, we artificially reduce the original relevance data to simulate IR evaluation environments with extremely incomplete relevance data. We then investigate the effect of this reduction on discriminative power , which we define as the proportion of system pairs with a statistically significant difference for a given probability of Type I Error, and on Kendall’s rank correlation , which reflects the overall resemblance of two system rankings according to two different metrics or two different relevance data sets. According to these experiments, Q′, nDCG′ and AP′ proposed by Sakai are superior to bpref proposed by Buckley and Voorhees and to Rank-Biased Precision proposed by Moffat and Zobel. We also point out some weaknesses of bpref and Rank-Biased Precision by examining their formal definitions.

introduction Introduction to the NTCIR-6 Special Issue Share on Authors: Noriko Kando National Institute of Informatics National Institute of InformaticsView Profile , Teruko Mitamura Carnegie Mellon University Carnegie Mellon UniversityView Profile , Tetsuya Sakai News Watch, Co. News Watch, Co.View Profile Authors Info & Claims ACM Transactions on Asian Language Information ProcessingVolume 7Issue 2June 2008 Article No.: 4pp 1–3https://doi.org/10.1145/1362782.1362783Online:01 April 2008Publication History 6citation234DownloadsMetricsTotal Citations6Total Downloads234Last 12 Months5Last 6 weeks2 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

We have developed a content construction system for collaborative learning based on a Wiki, a Web- based collaborative document authoring system. Requirements for such a tool are explained in three aspects (usability, content construction, easy and secure operation) and implemented accordingly. Experiment on usability of the system interface suggested improvement from a common Wiki.

We propose a method to rate the of news articles using three clues: (1) commonality of the contents of articles among different news publishers; (2) numerical agreement versus contradiction of numerical values reported in the articles; and (3) objectivity based on subjective speculative phrases and news sources. We tested this method on news stories taken from seven different news sites on the Web. The average agreement between the system-produced credibility and the manual judgments of three human assessors on the 52 sample articles was 69.1%. The limitations of the current approach and future directions are discussed.

We evaluated the interactive retrieval functionality of the Concentric Ring View according to a series of usability studies. This is a ring structure-based graphical user interface, like a planisphere, for image retrieval with multi-faceted metadata. Attribute values for each facet are arranged on a ring, and retrieved images are displayed inside using search keys derived from the attribute values on the bottom part of the rings. By rotating the rings, users can browse retrieved images while adjusting search keys. The first usability test conducted with thirty six participants confirmed that: (i) novice users, even junior high school students, could use this interface; (ii) users could find images better than anticipated; and (iii) the interface was good at choosing the first relevant image, but users could not refine retrieval because they were unable to reuse retrieved results. To solve this problem, we added two functionalities, personal history for reuse and relevance feedback. With these improvements, we named the new version of the interface Concentric Ring View F+. A second usability test with seven participants confirmed the effectiveness of this newer interface.

There are strong expectations for the use of question answering technologies in information access dialogues, such as for information gathering and browsing. In this paper, we empirically examine what kinds of abilities are needed for question answering systems in such situations, and propose a challenge for evaluating those abilities objectively and quantitatively. We also show that existing technologies have the potential to address this challenge. From the empirical study, we found that questions that have values and names as answers account for a majority in realistic information-gathering situations and that those sequences of questions contain a wide range of reference expressions and are sometimes complicated by the inclusion of subdialogues and focus shifts. The challenge proposed is not only novel as an evaluation of the handling of information access dialogues, but also includes several valuable ideas such as categorization and characterization of information access dialogues, and introduces three measures to evaluate various aspects in addressing list-type questions and reference test sets for evaluating context-processing ability in isolation.

article Share on Preface to the special issues on NTCIR-4 Editors: Hiroshi Nakagawa University of Tokyo University of TokyoView Profile , Tatsunori Mori Yokohama National University Yokohama National UniversityView Profile , Noriko Kando National Institute of Informatics National Institute of InformaticsView Profile Authors Info & Claims ACM Transactions on Asian Language Information ProcessingVolume 4Issue 3September 2005 pp 237–242https://doi.org/10.1145/1111667.1111668Published:01 September 2005Publication History 1citation288DownloadsMetricsTotal Citations1Total Downloads288Last 12 Months0Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

This paper describes the Patent Retrieval Task in the Fourth NTCIR Workshop, and the test collections produced in this task. We perform the invalidity search task, in which each participant group searches a patent collection for the patents that can invalidate the demand in an existing claim. We also perform the automatic patent map generation task, in which the patents associated with a specific topic are organized in a multi-dimensional matrix.

Reflecting the rapid growth in the utilization of large test collections for information retrieval since the 1990s, extensive comparative experiments have been performed to explore the effectiveness of various retrieval models. However, most collections were intended for retrieving newspaper articles and technical abstracts. In this paper, we describe the process of producing a test collection for patent retrieval, the NTCIR-3 Patent Retrieval Collection, which includes two years of Japanese patent applications and 31 topics produced by professional patent searchers. We also report experimental results obtained by using this collection to re-examine the effectiveness of existing retrieval models in the context of patent retrieval. The relative superiority among existing retrieval models did not significantly differ depending on the document genre, that is, patents and newspaper articles. Issues related to patent retrieval are also discussed.

This paper gives an overview of the evaluation method used for the Web Retrieval Task in the Third NTCIR Workshop, which is currently in progress. In the Web Retrieval Task, we try to assess the retrieval effectiveness of each Web search engine system using a common data set, and attempt to build a re-usable test collection suitable for evaluating Web search engine systems. With these objectives, we have built 100gigabyte and 10-gigabyte document sets, mainly gathered from the ‘.jp ’ domain. Relevance judgment is performed on the retrieved documents, which are written in Japanese or English.

No abstract available.