Modern large-scale service-based systems such as microservice systems have become increasingly complex, making it hard to localize anomalous services when various issues emerge. Traces record the workflows of requests through service instances and have been widely used in anomaly detection and root cause analysis. Existing trace-based approaches widely use statistical methods or learning-based techniques to detect trace anomalies and localize anomalous services. However, these approaches often suffer from the concept drift problem, i.e., the statistical properties of traces change over time in unforeseen ways. In this paper, we propose TraceStream, an anomalous service localization approach based on trace data stream clustering. TraceStream uses data stream clustering to discover potential anomalous trace clusters in evolving trace data and uses spectrum analysis to localize anomalous services based on the clusters. Moreover, TraceStream can effectively incorporate the online feedback of operation engineers based on the trace clusters to improve the accuracy for localizing anomalous services. Our evaluation confirms that TraceStream can effectively detect anomalies and localize anomalous services in an evolving microservice system. It can effectively incorporate human feedback to further improve the performance of anomalous service localization. Moreover, TraceStream is efficient and its efficiency can be further improved by sampling a small portion of traces by cluster.

Due to the large amount and high complexity of trace data, microservice trace analysis tasks such as anomaly detection, fault diagnosis, and tail-based sampling widely adopt machine learning technology. These trace analysis approaches usually use a preprocessing step to map structured features of traces to vector representations in an ad-hoc way. Therefore, they may lose important information such as topological dependencies between service operations. In this paper, we propose TraceCRL, a trace representation learning approach based on contrastive learning and graph neural network, which can incorporate graph structured information in the downstream trace analysis tasks. Given a trace, TraceCRL constructs an operation invocation graph where nodes represent service operations and edges represent operation invocations together with predefined features for invocation status and related metrics. Based on the operation invocation graphs of traces TraceCRL uses a contrastive learning method to train a graph neural network-based model for trace representation. In particular, TraceCRL employs six trace data augmentation strategies to alleviate the problems of class collision and uniformity of representation in contrastive learning. Our experimental studies show that TraceCRL can significantly improve the performance of trace anomaly detection and offline trace sampling. It also confirms the effectiveness of the trace augmentation strategies and the efficiency of TraceCRL.