One of the long-standing questions in search systems is the role of diversity in results. From a product perspective, showing diverse results provides the user with more choice and should lead to an improved experience. However, this intuition is at odds with common machine learning approaches to ranking which directly optimize the relevance of each individual item without a holistic view of the result set. In this paper, we describe our journey in tackling the problem of diversity for Airbnb search, starting from heuristic based approaches and concluding with a novel deep learning solution that produces an embedding of the entire query context by leveraging Recurrent Neural Networks (RNNs). We hope our lessons learned will prove useful to others and motivate further research in this area.

The application of deep learning to search ranking was one of the most impactful product improvements at Airbnb. But what comes next after you launch a deep learning model? In this paper we describe the journey beyond, discussing what we refer to as the ABCs of improving search: A for architecture, ℬ for bias and ℂ for cold start. For architecture, we describe a new ranking neural network, focusing on the process that evolved our existing DNN beyond a fully connected two layer network. On handling positional bias in ranking, we describe a novel approach that led to one of the most significant improvements in tackling inventory that the DNN historically found challenging. To solve cold start, we describe our perspective on the problem and changes we made to improve the treatment of new listings on the platform. We hope ranking teams transitioning to deep learning will find this a practical case study of how to iterate on DNNs.

Data sparsity and cold-start are two major problems in personalized recommendation. They are especially severe in business recommendation, because business transactions are usually completed offline and customers generally do not provide ratings after a transaction. Due to these two problems, matrix factorization (MF) models, which are shown to be effective in many recommendation tasks, are likely to fail on business recommendation tasks, especially for new users and new items. In this paper, we propose an Integrated Bias and Factorization Model (IBFM), which exploits user and business attributes. The user attributes include demographic information, vote information, point-of-interests; the business attributes include check-in information, locations, business names, categories, etc. To handle the cold-start problem, we employ a sampling strategy to generate the latent factor vectors for new users and new businesses based on similar users/businesses. Our methods are evaluated on the data set used in the RecSys 2013 Yelp business rating prediction challenge. Experimental results show that our proposed methods significantly outperform several existing state-of-the-art methods. In particular, the single model IBFM performs the best in this challenge on both public and private leaderboards.

The Android system is getting more and more popular on the mobile devices. Thus, lots of apps have sprung up to facilitate people's daily life. However, many of the apps are released without sufficient testing work, so the users encounter a sudden app crash now and then. This will undoubtedly impact the user's experience and even lead to economic loss. Because current testing tools on Android apps mainly focus on the motion event on the screen, like click event, bugs concerned with data handling module in an app is neglected. In this paper, we propose an automated testing method to fuzz testing the Android apps. The test targets are the Activities which accept outside MIME data. These Activities are picked out by analyzing the Intent-filter tag in the AndroidManifest.xml file. An automated fuzzing tool, DroidFuzzer, is implemented based on the method. Finally, experiments are conducted to prove the effectiveness of it.

As the increasing of popularity of social web, cyber bullying has become a more and more serious issue among children. Bullying causes huge negative effects on children, even suicide. SocialFilter is a realtime system that helps parents and educators track children's messages on Twitter, especially in order to detect whether they have been bullied or bullying others. The aim of the system is 4 I's, identity of bullies, inference of bullying message, influence of bully behavior, and intervention. We solve this problem by using machine learning technique. The current system is tracking tens of thousands of active children users on Twitter and automatically detect bullying messages at real time.

As highly structured documents with rich metadata (such as products, movies, etc.) become increasingly prevalent, searching those documents has become an important IR problem. Unfortunately existing work on document summarization, especially in the context of search, has been mainly focused on unstructured documents, and little attention has been paid to highly structured documents. Due to the different characteristics of structured and unstructured documents, the ideal approaches for document summarization might be different. In this paper, we study the problem of summarizing highly structured documents in a search context. We propose a new summarization approach based on query-specific facet selection. Our approach aims to discover the important facets hidden behind a query using a machine learning approach, and summarizes retrieved documents based on those important facets. In addition, we propose to evaluate summarization approaches based on a utility function that measures how well the summaries assist users in interacting with the search results. Furthermore, we develop a game on Mechanical Turk to evaluate different summarization approaches. The experimental results show that the new summarization approach significantly outperforms two existing ones.

Relevance feedback is an effective approach to improve retrieval quality over the initial query. Typical relevance feedback methods usually select top-ranked documents for relevance judgments, then query expansion or model updating are carried out based on the feedback documents. However, the number of feedback documents is usually limited due to expensive human labeling. Thus relevant documents in the feedback set are hardly representative of all relevant documents and the feedback set is actually biased. As a result, the performance of relevance feedback will get hurt. In this paper, we first show how and where the bias problem exists through experiments. Then we study how the bias can be reduced by utilizing the unlabeled documents. After analyzing the usefulness of a document to relevance feedback, we propose an approach that extends the feedback set with carefully selected unlabeled documents by heuristics. Our experiment results show that the extended feedback set has less bias than the original feedback set and better performance can be achieved when the extended feedback set is used for relevance feedback.

Existing adaptive filtering systems learn user profiles based on users' relevance judgments on documents. In some cases, users have some prior knowledge about what features are important for a document to be relevant. For example, a Spanish speaker may only want news written in Spanish, and thus a relevant document should contain the feature "Language: Spanish"; a researcher working on HIV knows an article with the medical subject "Subject: AIDS" is very likely to be interesting to him/her.

Most existing content-based filtering approaches including Rocchio, Language Models, SVM, Logistic Regression, Neural Networks, etc. learn user profiles independently without capturing the similarity among users. The Bayesian hierarchical models learn user profiles jointly and have the advantage of being able to borrow information from other users through a Bayesian prior. The standard Bayesian hierarchical model assumes all user profiles are generated from the same prior. However, considering the diversity of user interests, this assumption might not be optimal. Besides, most existing content-based filtering approaches implicitly assume that each user profile corresponds to exactly one user interest and fail to capture a user's multiple interests (information needs).

Motivated by the commonly used faceted search interface in e-commerce, this paper investigates interactive relevance feedback mechanism based on faceted document metadata. In this mechanism, the system recommends a group of document facet-value pairs, and lets users select relevant ones to restrict the returned documents. We propose four facet-value pair recommendation approaches and two retrieval models that incorporate user feedback on document facets. Evaluated based on user feedback collected through Amazon Mechanical Turk, our experimental results show that the Boolean filtering approach, which is widely used in faceted search in e-commerce, doesn't work well for text document retrieval, due to the incompleteness (low recall) of metadata assignment in semi-structured text documents. Instead, a soft model performs more effectively. The faceted feedback mechanism can also be combined with document-based relevance feedback and pseudo relevance feedback to further improve the retrieval performance.