Modern data processing systems require optimization at massive scale, and using machine learning to optimize these systems (ML-for-systems) has shown promising results. Unfortunately, ML-for-systems is subject to over generalizations that do not capture the large variety of workload patterns, and tend to augment the performance of certain subsets in the workload while regressing performance for others. In this paper, we introduce a performance safeguard system, called PerfGuard , that designs pre-production experiments for deploying ML-for-systems. Instead of searching the entire space of query plans (a well-known, intractable problem), we focus on query plan deltas (a significantly smaller space). PerfGuard formalizes these differences, and correlates plan deltas to important feedback signals, like execution cost. We describe the deep learning architecture and the end-to-end pipeline in PerfGuard that could be used with general relational databases. We show that this architecture improves on baseline models, and that our pipeline identifies key query plan components as major contributors to plan disparity. Offline experimentation shows PerfGuard as a promising approach, with many opportunities for future improvement.

Microsoft's internal big data analytics platform is comprised of hundreds of thousands of machines, serving over half a million jobs daily, from thousands of users. The majority of these jobs are recurring and are crucial for the company's operation. Although administrators spend significant effort tuning system performance, some jobs inevitably experience slowdowns, i.e., their execution time degrades over previous runs. Currently, the investigation of such slowdowns is a labor-intensive and error-prone process, which costs Microsoft significant human and machine resources, and which negatively impacts several lines of business.

Automated inspection plays a critical role in many industrial processes, including modern assembly lines. In these processes, components are inspected to ensure adherence to design specifications. Components that are determined to be out-of-specifications are rejected. The benefits of inspection are two-fold. First, defects can be removed early in the process, preventing higher costs incurred in detecting them downstream. Second, the inspection results provide information for manual troubleshooting of root-causes, potentially leading to an improvement in overall quality. However, this form of inspection may also incurs costs if there are false alarms associated with the automated inspection method. Analysis of false alarm costs are rarely addressed in the data mining literature. In this paper, we develop a simple framework to estimate the value of an inspection process, and demonstrate how predictive modeling can be used to increase this value under the right circumstances. In the second part of the paper, we report results from a case-study at a Bosch manufacturing plant, involving tens of thousands of parts and hundreds of quality attributes. A key challenge in this study was the extremely low rate of defects resulting from the operation of a highly-optimized manufacturing process. We show that for such modern assembly lines, machine learning techniques that are robust to class imbalance are particularly well suited. The solution from the case-study yields a positive ROI for the manufacturing plant.