Monitoring online time-series data and providing real-time alerts are crucial for a variety of industry applications. But doing it at a large scale is a challenging task. In this paper, we introduce Virtual-SRE, an anomaly detection framework which utilizes human knowledge and is suitable for large-scale applications. The proposed approach starts without any human labeling. Instead, human knowledge is obtained through feedbacks. A joint data and model optimization process is proposed to ensure that accurate models can be built. This makes it easily applicable to large-scale and heterogeneous tasks while enjoying the benefits of supervised learning. The proposed framework has been deployed to monitor thousands of time-series data streams and obtained favorable results.

Recommending relevant data is vital for helping users to navigate through the ocean of data. We developed a service that learns user preferences through natural user interactions, without asking for user feedbacks, so users are not distracted from their regular workflow. Our approach has few parameters and very low time and space complexities, making it suitable for large scale applications. We demonstrate through experiments how it converges to user preferences and adapts to user behavior changes.

Anomaly detection for time-series data is an important component for monitoring tasks. Due to the massive and diverse types of streaming sources, it is impossible to manually label training data for large-scale applications. So existing solutions are based on unsupervised learning. However, each time-series data has its own domain specific property. Without incorporating human knowledge, unsupervised learning methods cannot match the performance of supervised learning. We introduce an anomaly detection solution which utilizes human knowledge and is suitable for large-scale applications. The proposed approach starts without any human labeling. Instead, human knowledge is obtained through feedbacks. We iteratively refine the training data to ensure that accurate models can be built. This makes it easily applicable to large-scale and heterogeneous tasks while enjoying the benefits of supervised learning.

Bug triage is an important task for software maintenance, especially in the industrial environment, where timely bug fixing is critical for customer experience. This process is usually done manually and often takes significant time. In this paper, we propose a machine-learning-based solution to address the problem efficiently. We argue that in the industrial environment, it is more suitable to assign bugs to software components (then to responsible developers) than to developers directly. Because developers can change their roles in industry, they may not oversee the same software module as before. We also demonstrate experimentally that assigning bugs to components rather than developers leads to much higher accuracy. Our solution is based on text-projection features extracted from bug descriptions. We use a Deep Neural Network to train the classification model. The proposed solution achieves state-of-the-art performance based on extensive experiments using multiple data sets. Moreover, our solution is computationally efficient and runs in near real-time.

In a software life cycle, bugs could happen at any time. Assigning bugs to relevant components/developers is a crucial task for software development. It is also a tough and resource consuming job. First, there are many components in a complex system and it is hard to understand their interactions and identify the root cause. Second, the list of components keeps growing for actively developed products and it is not easy to catch all updates. This task also faces several challenges from the machine learning point of view: 1) the ground truth is mixed with multiple levels of labels; 2) the data are severely imbalanced. 3). concept drift as future bugs are unlikely to come from the same distribution as the historical data. In this paper, we present a machine learning based solution for the bug assignment problem. We build component classifiers using a multi-layer Neural Network, based on features that were learned from data directly. A hierarchical classification framework is proposed to address the mixed label problem and improve the prediction accuracy. We also introduce a recency based sampling procedure to alleviate the data imbalance and concept drift problem. Our solution can easily accommodate new data and handle continuous system development/update.

Imbalanced data poses a serious challenge for many machine learning and data mining applications. It may significantly affect the performance of learning algorithms. In digital marketing applications, events of interest (positive instances for building predictive models) such as click and purchase are rare. A retail website can easily receive a million visits every day, yet only a small percentage of visits lead to purchase. The large amount of raw data and the small percentage of positive instances make it challenging to build decent predictive models in a timely fashion. In this paper, we propose an adaptive sampling strategy to deal with this problem. It efficiently returns high quality training data, ensures system responsiveness and improves predictive performances.