Distribution shifts are ubiquitous in real-world machine learning applications, posing a challenge to the generalization of models trained on one data distribution to another. We focus on scenarios where data distributions vary across multiple segments of the entire population and only make local assumptions about the differences between training and test (deployment) distributions within each segment. We propose a two-stage multiply robust estimation method to improve model performance on each individual segment for tabular data analysis. The method involves fitting a linear combination of the based models, learned using clusters of training data from multiple segments, followed by a refinement step for each segment. Our method is designed to be implemented with commonly used off-the-shelf machine learning models. We establish theoretical guarantees on the generalization bound of the method on the test risk. With extensive experiments on synthetic and real datasets, we demonstrate that the proposed method substantially improves over existing alternatives in prediction accuracy and robustness on both regression and classification tasks. We also assess its effectiveness on a user city prediction dataset from Meta.

Binary classification models are ubiquitous, and reliably measuring their performance is critical for their proper usage. Ideally, the performance of supervised models is measured using high-quality labeled datasets that are sufficiently large and representative of the population. However, obtaining labels for all segments of the population can be difficult, and model performance typically varies across different segments of the population (e.g., in different countries). In this work, we present a novel methodology to estimate the performance of a binary classifier in segments of the population where labels are unavailable. The main idea is that if two segments are "similar,'' then the performance of the classifier in these two segments would also be "similar.'' Specifically, we define a way to measure similarity between segments, and propose a statistical model that describes the performance of the model in unlabeled segments as a function of the performance in labeled segments. With extensive numerical experiments on synthetic and real-world datasets, we demonstrate that the proposed method substantially improves over existing methods in both estimation accuracy and computational efficiency. We also showcase the application of our method on the Instagram Adult Classifier to improve the geographic coverage and usability of the model.

Misinformation and misbehavior mining on the web (MIS2) workshop is held virtually on August 14, 2021 and is co-located with the ACM SIGKDD 2021 conference. The web has become a breeding ground for misbehavior and misinformation. It is timely and crucial to understand, detect, forecast, and mitigate their harm. MIS2 workshop as an interdisciplinary venue for researchers and practitioners who study the dark side of the web. The workshop program includes a peer-reviewed set of paper presentations and keynote talks, giving the attendees an immersive experience of this research field.

Since its inception, Facebook has become an integral part of the online social community. People rely on Facebook to connect with others and build communities. As a result, it is paramount to protect the integrity of such a large network in a fast and scalable manner. In this paper, we present our efforts to protect various social media entities at Facebook from people who try to abuse our platform. We present a novel Temporal Interaction EmbeddingS (TIES) model that is designed to capture rogue social interactions and flag them for further suitable actions. TIES is a supervised, deep learning, production ready model at Facebook-scale networks. Prior works on integrity problems are mostly focused on capturing either only static or certain dynamic features of social entities. In contrast, TIES can capture both these variant behaviors in a unified model owing to the recent strides made in the domains of graph embedding and deep sequential pattern learning. To show the real-world impact of TIES, we present a few applications especially for preventing spread of misinformation, fake account detection, and reducing ads payment risks in order to enhance Facebook platform's integrity.

Estimating and predicting traffic conditions in arterial networks using probe data has proven to be a substantial challenge. Sparse probe data represent the vast majority of the data available on arterial roads. This paper proposes a probabilistic modeling framework for estimating and predicting arterial travel-time distributions using sparsely observed probe vehicles. We introduce a model based on hydrodynamic traffic theory to learn the density of vehicles on arterial road segments, illustrating the distribution of delay within a road segment. The characterization of this distribution is essentially to use probe vehicles for traffic estimation: Probe vehicles report their location at random locations, and the travel times between location reports must be properly scaled to match the map discretization. A dynamic Bayesian network represents the spatiotemporal dependence on the network and provides a flexible framework to learn traffic dynamics from historical data and to perform real-time estimation with streaming data. The model is evaluated using data from a fleet of 500 probe vehicles in San Francisco, CA, which send Global Positioning System (GPS) data to our server every minute. The numerical experiments analyze the learning and estimation capabilities on a subnetwork with more than 800 links. The sampling rate of the probe vehicles does not provide detailed information about the location where vehicles encountered delay or the reason for any delay (i.e., signal delay, congestion delay, etc.). The model provides an increase in estimation accuracy of 35% when compared with a baseline approach to process probe-vehicle data.