This paper proposes a method for learning embedding representations of style-transfer LoRA models based on their internal weight parameters, capturing transformation characteristics without relying on output examples or metadata. To vectorize LoRA models with multiple layers, we extract internal parameters layer-wise, then flatten and reduce dimensionality, representing each model as a sequence of low-dimensional vectors. Using this vector sequence as input, we perform triplet-based metric learning with a Triplet Network composed of three weight-sharing Transformer encoders and an MLP aggregation module. This framework estimates visual similarity between images transformed by different LoRA models. We conduct automatic evaluation of training validity, verify alignment with human relative similarity judgments, and perform retrieval-based ranking assessment. The results demonstrate that the proposed method learns embeddings consistent with human perception and enables stable retrieval of similar LoRA models.

This paper proposes a retrieval algorithm that ranks LoRA adapters for additional merging, given a target image and the LoRA adapter currently used for image generation. In recent image generation workflows, multiple LoRA adapters are often applied simultaneously. However, selecting appropriate adapters to merge still relies on manual inspection of generated samples and associated metadata. We trained a Transformer-based classification model with token embedding to determine whether a given image is generated by a specific pair of LoRA adapters. This trained model can receive a target image, a fixed LoRA adapter in use, and candidate LoRA adapters, and then output a score reflecting the probability that two adapters, when merged, contribute to generating the target image. By ranking candidates based on this score, the proposed method can retrieve the LoRA adapter to be additionally merged. We implemented a retrieval system supporting 100 LoRA adapters, and both automatic evaluation and a user experiment demonstrate the effectiveness of the proposed approach.

In this study, we evaluate the effect of model merging in ad-hoc retrieval tasks. Model merging is a technique that combines the diverse characteristics of multiple models. We hypothesized that applying model merging to domain-specific ad-hoc retrieval tasks could improve retrieval effectiveness. To verify this hypothesis, we merged the weights of a source retrieval model and a domain-specific (non-retrieval) model using a linear interpolation approach. A key advantage of our approach is that it requires no additional fine-tuning of the models. We conducted two experiments each in the medical and Japanese domains. The first compared the merged model with the source retrieval model, and the second compared it with a LoRA fine-tuned model under both full and limited data settings for model construction. The experimental results indicate that model merging has the potential to produce more effective domain-specific retrieval models than the source retrieval model, and may serve as a practical alternative to LoRA fine-tuning, particularly when only a limited amount of data is available.

This paper presents our analysis of neural IR models, particularly focusing on over-penalization for extra information (OPEX) - a phenomenon where addition of a sentence to a document causes an unreasonable decline in the document rank. We found that neural IR models suffered from OPEX, especially when the added sentence is similar to the other sentences in the document. To mitigate OPEX, we propose to apply a window-based scoring approach that segments a document and aggregates scores of the segments to compute the overall document score. We theoretically proved that the window-based scoring approach fully suppressed OPEX in an extreme case where each segment contains only a single sentence, and empirically showed that this approach mitigated OPEX. The code is available at https://github.com/argonism/OPEX .

The problem of inner product search (IPS) is important in many fields. Although maximum inner product search (MIPS) is often considered, its result is usually skewed and static. Users are hence hard to obtain diverse and/or new items by using the MIPS problem. Motivated by this, we formulate a new problem, namely the fair and independent IPS problem. Given a query, a threshold, and an output size k, this problem randomly samples k items from a set of items such that the inner product of the query and item is not less than the threshold. For each item that satisfies the threshold, this problem is fair, because the probability that such an item is outputted is equal to that for each other item. This fairness can yield diversity and novelty, but this problem faces a computational challenge. Some existing (M)IPS techniques can be employed in this problem, but they require O(n) or o(n) time, where n is the dataset size. To scale well to large datasets, we propose a simple yet efficient algorithm that runs in O(log n + k) expected time. We conduct experiments using real datasets, and the results demonstrate that our algorithm is up to 330 times faster than baselines.

Background articles are crucial for readers to grasp the context of news stories fully. However, existing approaches of background article search tend to apply a single ranking method to all types of search topics. In this paper, we focus on exploring search topics on news articles by classifying them into two types:time-sensitive andnon-time-sensitive. To verify whether or not these two types of search topics can benefit from different retrieving methods, we examined a suite of strategies such as document expansion, query rewriting, and semantic re-ranking. Moreover, the relationship between background articles and topics is verified by the two strategies of document expansion (specificity and diversity). The experimental results demonstrate that the optimal usage of the aforementioned strategies is indeed different between the two types of search topics. Furthermore, our in-depth analysis of topics and search results verified that: time-sensitive topics benefit from background articles that can provide more specific knowledge, while non-time-sensitive topics benefit from diversified retrieved documents.

Big data mining is well known to be an important task for data science, because it can provide useful observations and new knowledge hidden in given large datasets. Proximity-based data analysis is particularly utilized in many real-life applications. In such analysis, the distances to k nearest neighbors are usually employed, thus its main bottleneck is derived from data retrieval. Much efforts have been made to improve the efficiency of these analyses. However, they still incur large costs, because they essentially need many data accesses. To avoid this issue, we propose a machine-learning technique that quickly and accurately estimates the k-NN distances (i.e., distances to the k nearest neighbors) of a given query. We train a fully connected neural network model and utilize pivots to achieve accurate estimation. Our model is designed to have useful advantages: it infers distances to the k-NNs at a time, its inference time is O(1) (no data accesses are incurred), but it keeps high accuracy. Our experimental results and case studies on real datasets demonstrate the efficiency and effectiveness of our solution.

Maximum inner product search (or k-MIPS) is a fundamental operation in recommender systems that infer preferable items for users. To support large-scale recommender systems, existing studies designed scalable k-MIPS algorithms. However, these studies do not consider diversity, although recommending diverse items is important to improve user satisfaction. We therefore formulate a new problem, namely diversity-aware k-MIPS. In this problem, users can control the degree of diversity in their recommendation lists through a parameter. However, exactly solving this problem is unfortunately NP-hard, so it is challenging to devise an efficient, effective, and practical algorithm for the diversity-aware k-MIPS problem. This paper overcomes this challenge and proposes IP-Greedy, which incorporates new early termination and skipping techniques into a greedy algorithm. We conduct extensive experiments on real datasets, and the results demonstrate the efficiency and effectiveness of our algorithm. Also, we conduct a case study of the diversity-aware k-MIPS problem on a real dataset. We confirm that this problem can make recommendation lists diverse while preserving high inner products of user and item vectors in the lists.

This paper reveals the characteristics of the reviews that encourage readers to watch the reviewed movie by analyzing large-scale access log data. We assume that some of the reviews that users saw just before they clicked the links to a streaming site contain factors that help users decide whether they watch that movie. Our method used a random forest classifier trained to determine whether a review encouraged a movie-watching behavior. We conducted feature importance-based analysis using three types of features: review itself, item, and reviewer. We analyzed 70,000 user behaviors from Yahoo! Movies (a movie review site in Japan) and Gyao! (a movie streaming site in Japan). Through a cross-validation experiment, the classifier was able to classify encouraging reviews with an F-score of 0.78, and mainly the features about the item contributed to the classification performance. An additional subjects experiment confirmed that these features contribute to the review’s usefulness.

This paper proposes a method for extracting diverse expressions from online movie review texts for a given keyword query. When people watch a movie that makes them cry, they generally do not say “I cried.” Instead, they use such euphemistic language as “I needed a handkerchief” or “My makeup was running.” To enable information retrieval based on audience reactions such as “movies that make me cry” using review texts, a variety of paraphrased expressions must be collected for arbitrary queries. Our proposed method extracts such expressions from review datasets by applying two extensions to Doc2Vec: 1) it changes the granularity of the training sentences to mitigate a lack of context, and 2) it applies query expansion for similarity calculation in advance. We conducted a large-scale experiment using crowdsourcing with 1.29 million actual sentences taken from Yahoo! Movies, Japan. The experimental result revealed that changing the training data granularity and adding the query expansion are both effective to accurately collect more diverse expressions that have a meaning similar to the given query.

This paper proposes a method for ranking movies by keyword queries and examples using machine learning techniques that analyze actual data from the online movie review site. Existing search methods cannot rank movies in “surprising order” for the keyword query “surprising.” People and critics created many “My best surprising movies” rankings on the web. Our proposed method uses a LambdaMART, one of the mainstream Learning to Rank techniques, to learn these personalized rankings and sort the movies through the viewpoint represented by a given query. To accept more complex information needs, we diverted the learning results to a search-by-example algorithm that enables users to input examples, such as “surprising movies like The Usual Suspects or Fight Club.” The experiment using the personal ranking data from the personal content curation service in Yahoo! Movies Japan suggests two findings: direct learning of personal ranking does not improve search performance, and the search-by-example-based application increases user satisfaction.

This paper presents findings from an empirical study of multileaved comparisons, an efficient online evaluation methodology, in a commercial Web service. The most important difference from the previous studies is the number of rankers involved in the online evaluation: we compared 30 rankers for around 90 days by multileaved comparisons. A relatively large number of rankers answered several questions that could not be addressed in the previous work due to a small number of rankers: How much ranking difference is required for rankers to be statistically distinguished? How many impressions are necessary for finding statistically significant differences for correlated rankers? How large difference in offline evaluation can predict significant differences in a multileaved comparison? We answer these questions with the results of the multileaved comparisons, and generalized some of the findings by simulation-based experiments.

To reduce the cost of administrative services, many local governments provide a frequently asked questions (FAQ) page on their websites that lists the questions received from local inhabitants with their official responses. The number of Q&A items posted on the FAQ page, however, will vary depending on the local government. To address this issue, we propose a method for augmenting local government FAQs by using a community-based Q&A (cQA) service. We also propose a new FAQ augmentation task to identify the regional dependence of Q&A to achieve the goal mentioned above. In our experiments, we fine-tuned the bidirectional encoder representations from transformers (BERT) model for this task, using a labeled local-government FAQ dataset. We found that the regional dependence of Q&As can be identified with high accuracy by using both the question and the answer as clues and with fine tuning for the deeper layers in BERT.

Product image search is required to deal with large target image datasets which are frequently updated, and therefore it is not always practical to maintain exhaustive and up-to-date relevance assessments for tuning and evaluating the search engine. Moreover, in similar product image search where the query is also an image, it is difficult to identify the possible search intents behind it and thereby verbalise the relevance criteria for the assessors, especially if graded relevance assessments are required. In this study, we focus on similar product image search within a given product category (e.g., shoes), wherein each image is iconic (i.e., the image clearly shows what the product looks like and basically nothing else), and propose an initial approach to evaluating the task without relying on manual relevance assessments. More specifically, we build a simple probabilistic model that assumes that an image is generated from latent intents representing shape, texture, and colour, which enables us to estimate the relevance score of each image and thereby compute graded relevance measures for any image search engine result page. Through large-scale crowdsourcing experiments, we demonstrate that our proposed measures, InDCG (which is based on per-intent binary relevance) and D-InDCG (which is based on per-intent graded relevance), align reasonably well with human SERP preferences and with human image preferences. Hence, our automatic measures may be useful at least for rough tuning and evaluation of similar product image search.

Question categorization and expert retrieval methods have been crucial for information organization and accessibility in community question & answering (CQA) platforms. Research in this area, however, has dealt with only the text modality. With the increasingly multimodal nature of web content, we focus on extending these methods for CQA questions accompanied by images. Specifically, we leverage the success of representation learning for text and images in the visual question answering (VQA) domain and adapt the underlying concept and architecture for automated category classification and expert retrieval on image-based questions posted on Yahoo! Chiebukuro, the Japanese counterpart of Yahoo! Answers. To the best of our knowledge, this is the first work to tackle the multimodality challenge in CQA, and to adapt VQA models for tasks on a more ecologically valid source of visual questions. Our analysis of the differences between visual QA and community QA data drives our proposal of novel augmentations of an attention method tailored for CQA and use of auxiliary tasks for learning better grounding features. Our final model markedly outperforms the text-only and VQA model baselines for both tasks of classification and expert retrieval on real-world multimodal CQA data.

This paper proposes a new method of supplementing the context of short sentences for the training phase of Doc2Vec. Since CGM (Consumer Generated Media) sites and SNS sites become widespread, the importance of similarity calculation between a given query and a short sentence is increasing. As an example, a search by the query "sad" should find actual expressions such as "I needed a handkerchief" on a movie review site. Doc2Vec is one of the most widely used methods for vectorization of queries and sentences. However, Doc2Vec often exhibits low accuracy if the training data consists of short sentences, because they lack context. We modified Doc2Vec with the hypothesis that other posts for the same topic (i.e. reviews for the same movie in online movie review sites) may share the same background. Our method uses target-topic IDs instead of sentence IDs as the context in the training phase of the Doc2Vec with the PV-DM model; this model estimates the next term from a few previous terms and context. The model trained with item IDs vectorizes a sentence more accurately than a model trained with sentence IDs. We conducted a large-scale experiment using 1.2 million movie review posts and a crowdsourcing-based evaluation. The experimental result demonstrates that our new method achieves higher precision and nDCG than previous Doc2Vec variants and traditional topic modeling methods.

This paper discusses challenges of an online evaluation technique, multileaved comparison, based on the analysis of evaluation results in a community question-answering (cQA) search service. NTCIR-13 OpenLiveQ task offered a shared task in which participants addressed an ad-hoc retrieval task in a cQA service, and evaluated their rankers by multileaved comparison, which combines multiple rankings to generate a single search result page, and simultaneously evaluates the different rankings based on users' clicks on the search result page. Since the number of search result impressions during the evaluation period might not suffice to evaluate a hundred of rankers, we conducted the online evaluation only for rankers that achieved high performance in offline evaluation. The analysis of evaluation results showed that offline and online evaluation results did not fully agree, and a large number of users' clicks were necessary to find a statistically significant difference for every ranker pair. To cope with these problems in large-scale multileaved comparison, we propose a new experimental design that evaluates all the rankers online but intensively tests only the top-k rankers. Simulation-based experiments demonstrated that Copeland counting algorithm could achieve high top-k recall in the top-k identification problem for multileaved comparison.

This study investigates how people carefully search for the Web to obtain credible and accurate information. The goal of this study is to better understand people's attitudes toward careful information seeking via Web search, and the relationship between such attitudes and their daily search behaviors. To this end, we conducted two experiments. We first administrated an online questionnaire to investigate how people's attitudes toward using the strategies for verifying information in the Web search process differ based on various factors such as their credulity toward Web information, individual thinking styles, educational background, and search expertise. We then analyzed their one-year and one-month query logs of a commercial Web search engine to explore how their daily search behaviors are different according to their attitudes. The analysis of the questionnaire and the query logs obtained from ¥subjects participants revealed that (i) the people's attitudes toward using the verification strategies in Web search are positively correlated to their Need for Cognition (NFC), educational background, and search expertise; (ii) people with strong attitudes are likely to click lower-ranked search results than those with intermediate levels of attitude; (iii) people with strong attitudes are more likely to use the terms such as "evidence'' or "truth'' in their queries, possibly to scrutinize the uncertain or incredible information; and (iv) the behavioral differences found in (ii) and (iii) are not identified from the differences in the participants' educational backgrounds. These findings help us explore future directions for a new Web search system that encourages people to be more careful in Web search, and suggest the need for an educational program or training to facilitate the attitudes and skills for using Web search engines to obtain accurate information.

Recent advances in neural networks, along with the growth of rich and diverse community question answering (cQA) data, have enabled researchers to construct robust open-domain question answering (QA) systems. It is often claimed that such state-of-the-art QA systems far outperform traditional IR baselines such as BM25. However, most such studies rely on relatively small data sets, e.g., those extracted from the old TREC QA tracks. Given massive training data plus a separate corpus of Q&A pairs as the target knowledge source, how well would such a system really perform? How fast would it respond? In this demonstration, we provide the attendees of SIGIR 2017 an opportunity to experience a live comparison of two open-domain QA systems, one based on a long short-term memory (LSTM) architecture with over 11 million Yahoo! Chiebukuro (i.e., Japanese Yahoo! Answers) questions and over 27.4 million answers for training, and the other based on BM25. Both systems use the same Q&A knowledge source for answer retrieval. Our core demonstration system is a pair of Japanese monolingual QA systems, but we leverage machine translation for letting the SIGIR attendees enter English questions and compare the Japanese responses from the two systems after translating them into English.

We present one of the world's first attempts to examine the feasibility of multileaving evaluation of document rankings on a large scale commercial community Question Answering (cQA) service. As a natural enhancement of interleaving evaluation, multileaving merges more than two input rankings into one and measures the search user satisfaction of each input ranking on the basis of user clicks on the multileaved ranking. We evaluated the adequateness of two major multileaving methods, team draft multileaving (TDM) and optimized multileaving (OM), proposing their practical implementation for live services. Our experimental results demonstrated that multileaving methods could precisely evaluate the effectiveness of five rankings with different quality by using clicks from real users. Moreover, we concluded that OM is more efficient than TDM by observing that most of the evaluation results with OM converged after showing multileaved rankings around 40,000 times and an in-depth analysis of their characteristics.

Fueled by the increasing popularity of online social networks, social influence analysis has attracted a great deal of research attention in the past decade. The diffusion process is often modeled using influence graphs, and there has been a line of research that involves algorithmic problems in influence graphs. However, the vast size of today's real-world networks raises a serious issue with regard to computational efficiency.

Web searchers often use a Web search engine to find a way or means to achieve his/her goal. For example, a user intending to solve his/her sleeping problem, the query "sleeping pills" may be used. However, there may be another solution to achieve the same goal, such as "have a cup of hot milk" or "stroll before bedtime." The problem is that the user may not be aware that these solutions exist. Thus, he/she will probably choose to take a sleeping pill without considering these solutions. In this study, we define and tackle the alternative action mining problem. In particular, we attempt to develop a method for mining alternative actions for a given query. We define alternative actions as actions which share the same goal and define the alternative action mining problem as similar in the search result diversification. To tackle the problem, we propose leveraging a community Q&A (cQA) corpus for mining alternative actions. We propose a method to compute how well two actions can be alternative actions by using a question-answer structure in a cQA corpus. Our method builds a question-action bipartite graph and recursively computes how well two actions can be alternative actions. We conducted experiments to investigate the effectiveness of our method using two newly built test collections, each containing 50 queries. The experimental results indicated that our proposed method outperformed the query suggestion methods provided by the commercial search engines in terms of D#-nDCG.

A fundamental yet new challenge in information retrieval is the identification of patterns behind search queries. For example, the query "NY restaurant" and "boston hotel" shares the common pattern "LOCATION SERVICE". However, because of the diversity of real queries, existing approaches require data preprocessing by humans or specifying the target query domains, which hinders their applicability.