Distributed training (DT) has become an important and popular practice for collaborative training of high-quality machine learning (ML) models. The communication efficiency of gradient aggregation has been shown to be the primary performance bottleneck for distributed training today. Advanced programmable switches with in-network computing capabilities provide a promising direction for improving the communication efficiency of DT by offloading some gradient aggregations from the host to switches in the network. In this paper, we propose SPAR to optimize the performance of gradient aggregation under constrained in-network computing capabilities. To improve the aggregation throughput, SPAR jointly optimizes the deployment of in-network aggregation switches and the routing of aggregation requests from workers. We formulate this joint optimization problem as an integer nonlinear programming problem and design an efficient greedy algorithm to compute solutions quickly. The experimental results show that SPAR significantly outper-forms the other state-of-the-art solutions based on in-network aggregation, improving aggregation throughput by up to 3×.

The application of deep learning to search ranking was one of the most impactful product improvements at Airbnb. But what comes next after you launch a deep learning model? In this paper we describe the journey beyond, discussing what we refer to as the ABCs of improving search: A for architecture, ℬ for bias and ℂ for cold start. For architecture, we describe a new ranking neural network, focusing on the process that evolved our existing DNN beyond a fully connected two layer network. On handling positional bias in ranking, we describe a novel approach that led to one of the most significant improvements in tackling inventory that the DNN historically found challenging. To solve cold start, we describe our perspective on the problem and changes we made to improve the treatment of new listings on the platform. We hope ranking teams transitioning to deep learning will find this a practical case study of how to iterate on DNNs.

The application to search ranking is one of the biggest machine learning success stories at Airbnb. Much of the initial gains were driven by a gradient boosted decision tree model. The gains, however, plateaued over time. This paper discusses the work done in applying neural networks in an attempt to break out of that plateau. We present our perspective not with the intention of pushing the frontier of new modeling techniques. Instead, ours is a story of the elements we found useful in applying neural networks to a real life product. Deep learning was steep learning for us. To other teams embarking on similar journeys, we hope an account of our struggles and triumphs will provide some useful pointers. Bon voyage!

The fission matrix acceleration method based on redistributing mesh is proposed in this paper. The mesh used to tally the fission matrix is always fixed in previous fission matrix acceleration methods. When the mesh is redistributed in the process of iteration, the convergence rate of fission matrix acceleration method increases. The basic theory of redistributing mesh method is introduced. We analyze its efficiency and give some suggestions to enhance its stability. Numerical simulations of 1D slab geometries demonstrate that the fission matrix acceleration method based on redistributing mesh is more efficient.

This paper proposes a new supervised semantic edge and gradient extraction approach, which allows the user to roughly scribble over the desired region to extract semantically-dominant and coherent edges in it. Our approach first extracts low-level edge lets (small edge clusters) from the input image as primitives and build a graph upon them, by jointly considering both the geometric and appearance compatibility of edge lets. Given the characteristics of the graph, it cannot be effectively optimized by commonly-used energy minimization tools such as graph cuts. We thus propose an efficient linear algorithm for precise graph optimization, by taking advantage of the special structure of the graph. %Optimal parameter settings of the model are learnt from a dataset. Objective evaluations show that the proposed method significantly outperforms previous semantic edge detection algorithms. Finally, we demonstrate the effectiveness of the system in various image editing tasks.

Fine-grained recognition refers to a subordinate level of recognition, such are recognizing different species of birds, animals or plants. It differs from recognition of basic categories, such as humans, tables, and computers, in that there are global similarities in shape or structure shared within a category, and the differences are in the details of the object parts. We suggest that the key to identifying the fine-grained differences lies in finding the right alignment of image regions that contain the same object parts. We propose a template model for the purpose, which captures common shape patterns of object parts, as well as the co-occurence relation of the shape patterns. Once the image regions are aligned, extracted features are used for classification. Learning of the template model is efficient, and the recognition results we achieve significantly outperform the state-of-the-art algorithms.

We describe a family of global optimization procedures that automatically decompose optimization problems into smaller loosely coupled problems, then combine the solutions of these with message passing algorithms. We show empirically that these methods excel in avoiding local minima and produce better solutions with fewer function evaluations than existing global optimization methods. To develop these methods, we introduce a notion of coupling between variables of optimization that generalizes the notion of coupling that arises from factoring functions into terms that involve small subsets of the variables. It therefore subsumes the notion of independence between random variables in statistics, sparseness of the Hessian in nonlinear optimization, and the generalized distributive law. Despite being more general, this notion of coupling is easier to verify empirically -- making structure estimation easy -- yet it allows us to migrate well-established inference methods on graphical models to the setting of global optimization.

Food recognition is difficult because food items are de-formable objects that exhibit significant variations in appearance. We believe the key to recognizing food is to exploit the spatial relationships between different ingredients (such as meat and bread in a sandwich). We propose a new representation for food items that calculates pairwise statistics between local features computed over a soft pixel-level segmentation of the image into eight ingredient types. We accumulate these statistics in a multi-dimensional histogram, which is then used as a feature vector for a discriminative classifier. Our experiments show that the proposed representation is significantly more accurate at identifying food than existing methods.

In order to induct a Bayesian network from data, researchers proposed a variety of score metrics based on different assumptions. The score metric that performs best is of interest. In this paper, we compared the performance of five score metrics: uniform prior score metric (UPSM), conditional uniform prior score metric (CUPSM), Dirichlet prior score metric (DPSM), likelihood-equivalence Bayesian Dirichlet score metric (BDe), and minimum description length (MDL); resulting from five different assumptions: uniform prior, conditional uniform prior, Dirichlet prior, likelihood equivalence, and MDL. We used a three-node net, a five-node net, and the ALARM net to conduct several comparison experiments. The experimental results show that when they are applied to identify the true network structures, the DPSM yields the best discrimination score and BDe may fail to identify the true network if the equivalent sample size is not set properly. When they are applied to learn a network from data using the K2-like greedy search and the maximum likelihood (ML) parameter estimation, the network inducted by the K2D10, corresponding to the tenth-order DPSM, is most similar to the true network based on the cross-entropy criterion. It is concluded that the tenth-order DPSM is the best score metric and the corresponding K2D10 is the most reliable network learning algorithm.