Nowadays, increasing business applications running in the cloud are embracing the microservice architecture. This article presents the challenges and implications of diagnosing root causes of anomalies in large-scale microservice architecture using real incidents in IBM Bluemix. We propose ServiceRank, a novel framework for anomaly detection and root cause identification in the microservice architecture to tackle these challenges. ServiceRank introduces an anomaly detector followed by a root cause analysis module, which detects the suspected abnormal service without pre-defined thresholds. To generalize our approach, we design a causal relationship extraction approach to construct impact graphs for root cause investigation according to specific anomalies. To eliminate cloud design-patterns' impact on anomaly diagnosis, we propose a correlation calibration mechanism in ServiceRank and present a calibration algorithm for the circuit breaker - A typical protection pattern in the microservice architecture. Finally, we design a heuristic investigation algorithm based on the second-order random walk to identify the anomaly's root cause. Experimental results in a simulated environment and the IBM Bluemix platform show that ServiceRank outperforms selected approaches in accuracy and offers fast identification of root cause service when an anomaly occurs. Moreover, we can deploy ServiceRank rapidly and easily in various systems without any pre-defined knowledge.

The emergence of microservice architecture in Cloud systems poses a new challenges for the reliability operation and maintenance. Due to numerous services and diverse types of metrics, it is time-consuming and challenging to identify the root cause of anomaly in large-scale microservice architecture. To solve this issue, this article presents a multi-metric and self-adaptive root cause diagnosis framework, named MS-Rank. MS-Rank decomposes the task into four phases: impact graph construction, random walk diagnosis, result precision evaluation, metrics weight update. Initially, we introduce the concept of implicit metrics and propose a composite impact graph construction algorithm, using multiple types of metrics to discover causal relationships between services. Afterwards, we propose a diagnostic algorithm in which forward, selfward and backward transitions are designed to heuristically identify the root cause services. In addition, we establish a self-adaptive mechanism to update the confidence of different metrics dynamically according to their diagnostic precision. Lastly, we develop a prototype system and integrate MS-Rank into real production system - IBM Cloud. Experimental results show that MS-Rank has a high diagnostic precision and its performance outperforms several selected benchmarks. Through multiple rounds of diagnosis, MS-Rank can optimize itself effectively. MS-Rank can be rapidly deployed in various microservice-based systems and applications, requiring no predefined knowledge. MS-Rank also allows us to introduce expert experiences into its framework to improve the diagnostic efficiency and precision.

This paper presents a self-adaptive root cause diagnosis framework, named MS-Rank, to analyze multiple metrics collected from micro-service architecture. MS-Rank decomposes the task into four phases: impact graph construction, random walk diagnosis, result precision calculation and metrics weight update. First, we introduce a series of basic and implied metrics into MS-Rank, and design an impact graph construction algorithm to discover causal relationship between services during anomalies. Second, we propose a random walk algorithm with forward, selfward and backward transitions to heuristically identify the root cause service. Third, we establish a self-optimizing mechanism to dynamically update the confidence weight of different metrics according to their diagnosis precision. We develop a prototype system and integrate MS-Rank into IBM Cloud, to validate and compare it with selected benchmarks. Experimental results show that MS-Rank offers fast identification and precise diagnosis result. In multiple rounds of diagnosis, MS-Rank optimizes itself effectively.

As more and more systems are migrating to cloud environment, the cloud native system becomes a trend. This paper presents the challenges and implications when diagnosing root causes for cloud native systems by analyzing some real incidents occurred in IBM Bluemix (a large commercial cloud). To tackle these challenges, we propose CloudRanger, a novel system dedicated for cloud native systems. To make our system more general, we propose a dynamic causal relationship analysis approach to construct impact graphs amongst applications without given the topology. A heuristic investigation algorithm based on second-order random walk is proposed to identify the culprit services which are responsible for cloud incidents. Experimental results in both simulation environment and IBM Bluemix platform show that CloudRanger outperforms some state-of-the-art approaches with a 10% improvement in accuracy. It offers a fast identification of culprit services when an anomaly occurs. Moreover, this system can be deployed rapidly and easily in multiple kinds of cloud native systems without any predefined knowledge.

Micro-service architecture is a promising paradigm to develop, deploy and maintain applications using independent and autonomous cloud services. Nowadays, increasingly applications are embracing this model. However, it is difficult and time-consuming to diagnose and identify the actual root cause when anomalies occurs in micro-service architecture due to various factors. This paper introduces a novel framework for anomaly investigation and root cause identification in micro-service architecture. The novelty in our work lies on: (1) Different from existing solutions, in our framework, we propose a frequent pattern mining algorithm on anomaly correlation graph, named FacGraph, to discover root cause services. (2) We leverage breadth first ordered string (BFOS) to reduce the time-consumption of the frequent graph mining (FSM) (3) We further develop a distributed version of FacGraph to improve its paralleled computing efficiency. We evaluate our framework in real production environment IBM Bluemix. Result demonstrate that FacGraph outperforms other methods in diagnosis accuracy and offers a fast identification of root cause service when an anomaly occurs.

Energy efficiency raises significant concerns as it is one of the most promising ways to mitigate climate change. Disaggregation and identification of individual electrical appliances activities are one of the essentials for energy preservation especially for smart buildings. This paper proposes a lightweight electrical appliance activity detection approach for smart building, which leverages a single smart metering device to establish a learning and detection processing for multiple appliances. In this system, data interpolation and transition detection algorithm are proposed to effectively reduce the cost of model training and optimize the detection accuracy. The concept of appliance fingerprint is proposed and a variety of fingerprints, including appliance-based and context-based, are defined to depict fine-grained appliance characteristics. Based on these fingerprints, the paper proposes a multisource fingerprint-weighting KNN (FWKNN) classification algorithm and presents a boosting framework for continuous online learning and detection. A prototype system is implemented and demonstrated in IBM Bluemix PaaS cloud platform. Experimental result and analysis prove that FWKNN outperforms other benchmark methods in detection accuracy.