As cloud services are growing and generating high revenues, the cost of downtime in these services is becoming significantly expensive. To reduce loss and service downtime, a critical primary step is to execute incident triage, the process of assigning a service incident to the correct responsible team, in a timely manner. An incorrect assignment risks additional incident reroutings and increases its time to mitigate by 10x. However, automated incident triage in large cloud services faces many challenges: (1) a highly imbalanced incident distribution from a large number of teams, (2) wide variety in formats of input data or data sources, (3) scaling to meet production-grade requirements, and (4) gaining engineers' trust in using machine learning recommendations. To address these challenges, we introduce DeepTriage, an intelligent incident transfer service combining multiple machine learning techniques - gradient boosted classifiers, clustering methods, and deep neural networks - in an ensemble to recommend the responsible team to triage an incident. Experimental results on real incidents in Microsoft Azure show that our service achieves 82.9% F1 score. For highly impacted incidents, DeepTriage achieves F1 score from 76.3% -- 91.3%. We have applied best practices and state-of-the-art frameworks to scale DeepTriage to handle incident routing for all cloud services. DeepTriage has been deployed in Azure since October 2017 and is used by thousands of teams daily.

Machine Learning is transitioning from an art and science into a technology available to every developer. In the near future, every application on every platform will incorporate trained models to encode data-based decisions that would be impossible for developers to author. This presents a significant engineering challenge, since currently data science and modeling are largely decoupled from standard software development processes. This separation makes incorporating machine learning capabilities inside applications unnecessarily costly and difficult, and furthermore discourage developers from embracing ML in first place. In this paper we present ML.NET, a framework developed at Microsoft over the last decade in response to the challenge of making it easy to ship machine learning models in large software applications. We present its architecture, and illuminate the application demands that shaped it. Specifically, we introduce DataView, the core data abstraction of ML.NET which allows it to capture full predictive pipelines efficiently and consistently across training and inference lifecycles. We close the paper with a surprisingly favorable performance study of ML.NET compared to more recent entrants, and a discussion of some lessons learned.

We present WiScape, a framework for measuring and understanding the behavior of wide-area wireless networks, e.g., city-wide or nation-wide cellular data networks using active participation from clients. The goal of WiScape is to provide a coarse-grained view of a wide-area wireless landscape that allows operators and users to understand broad performance characteristics of the network. In this approach a centralized controller instructs clients to collect measurement samples over time and space in an opportunistic manner. To limit the overheads of this measurement framework, WiScape partitions the world into zones, contiguous areas with relatively similar user experiences, and partitions time into zone-specific epochs over which network statistics are relatively stable. For each epoch in each zone, WiScape takes a minimalistic view --- it attempts to collect a small number of measurement samples to adequately characterize the client experience in that zone and epoch, thereby limiting the bandwidth and energy overheads at client devices. For this effort, we have collected ground truth measurements for up to three different commercial cellular wireless networks across (i) an area of more than 155 square kilometer in and around Madison, WI, in the USA, (ii) a road stretch of more than 240 kilometers between Madison and Chicago, and (iii) locations in New Brunswick and Princeton, New Jersey, USA, for a period of more than 1 year. We justify various design choices of WiScape through this data, demonstrate that WiScape can provide an accurate performance characterization of these networks over a wide area (within 4% error for more than 70% of instances) with a low overhead on the clients, and illustrate multiple applications of this framework through a sustained and ongoing measurement study.

Modern network devices employ deep packet inspection to enable sophisticated services such as intrusion detection, traffic shaping, and load balancing. At the heart of such services is a signature matching engine that must match packet payloads to multiple signatures at line rates. However, the recent transition to complex regular-expression based signatures coupled with ever-increasing network speeds has rapidly increased the performance requirements of signature matching. Solutions to meet these requirements range from hardwarecentric ASIC/FPGA implementations to software implementations using high-performance microprocessors. In this paper, we propose a programmable signature matching system prototyped on an Nvidia G80 GPU. We first present a detailed architectural and microarchitectural analysis, showing that signature matching is well suited for SIMD processing because of regular control flow and parallelism available at the packet level. Next, we examine two approaches for matching signatures: standard deterministic finite automata (DFAs) and extended finite automata (XFAs), which use far less memory than DFAs but require specialized auxiliary memory and small amounts of computation in most states. We implement a fully functional prototype on the SIMD-based G80 GPU. This system out-performs a Pentium4 by up to 9X and a Niagara-based 32-threaded system by up to 2.3X and shows that GPUs are a promising candidate for signature matching.