Abstract Log files can help detect and diagnose erroneous software behaviour, but their utility is limited by the ability of users and developers to sift through large amounts of text. Unsupervised machine learning tools have been developed to automatically find anomalies in logs, but they are usually not designed for situations where a large number of log streams or log files, each with its own characteristics, need to be analyzed and their anomaly scores compared. We propose Ladle, an accurate unsupervised anomaly detection and localization method that can simultaneously learn the characteristics of hundreds of log types and determine which log entries are the most anomalous across these log types. Ladle uses a sentence transformer (a large language model) to embed short overlapping segments of log files and compares new, potentially anomalous, log segments against a collection of reference data. The result of the comparison is re-centered by subtracting a baseline score indicating how much variation tends to occur in each log type, making anomaly scores comparable across log types. Ladle is designed to adapt to data drift and is updated by adding new reference data without the need to retrain the sentence transformer. We demonstrate the accuracy of Ladle on a real-world dataset consisting of logs produced by an endpoint protection platform test suite. We also compare Ladle’s performance on the dataset to that of a state-of-the-art method for single-log anomaly detection, showing that the latter is inadequate for the multi-log task.

Computer vision systems can automatically find and segment anomalies in images even without ever seeing anomalous observations during training. Many methods for such unsupervised anomaly detection (AD) and localization (AL) tasks have been introduced in recent years, but the most accurate methods tend to be computationally heavy. In this paper, we propose Discrepancy Scaling, a method that significantly improves the accuracy of a very fast AD and AL approach called Student-Teacher Feature Pyramid Matching. We show that with Discrepancy Scaling, even a small, mobile-friendly convolutional neural network can perform well on AD and AL tasks.

Sound anomaly detection is a task that aims at identifying unusual or abnormal sounds within audio data. These sounds could be caused by different factors, such as background noise, equipment malfunctions, or unexpected events. Anomaly detection in sound is a well-studied topic, with a lot of research being done in the field. Anomaly localization refers to the process of identifying the specific location or region within a sample where an anomaly (or outlier) occurs. When applied to audio signals, anomaly localization can involve analyzing the spectral content of the sound to detect regions that deviate from the typical or expected pattern.In this study, we present a simple yet effective model based on the Student-Teacher Feature Pyramid Matching Method for locating anomalies in audio data. Utilizing the MIMII dataset by augmenting it with synthetic anomalies, we evaluate the method’s accuracy. Our results demonstrate that the proposed model can accurately locate artificially created anomalies within the spectrograms, both in terms of time and frequency. This approach offers a promising solution for identifying and determining the precise location of anomalies in various audio applications.