status: Published

Reliable anomaly detection under data contamination remains a major challenge in Prognostics and Health Management, especially when degradation processes are gradual and clean training data are unavailable. This paper introduces a weighted iterative refinement framework with autoencoders for contaminated anomaly detection (WIRACAD) to address this problem. The method, which is based on reconstruction residuals, re-weights training samples across iterations in order to progressively reduce the influence of suspected anomalies. This continuous refinement improves the robustness of health indicator learning from contaminated time series. The proposed approach is evaluated on two public benchmarks: the NASA C-MAPSS dataset and the IMS Bearing dataset. Results show consistent improvements in key metrics related to degradation monitoring. In particular, the overal fit score and the monotonicty are improved when compared to baseline autoencoder training and recent refinement-based methods. These findings suggest that iterative sample weighting can enhance unsupervised anomaly detection with autoencoders in settings where data contamination is assumed.

Miscreants use domain names for malicious purposes such as phishing websites or fake webshops. Reactive approaches such as blocklists play an important role in fighting such abuse but have limitations, namely that the domains are typically only included in such a list after abuse has been reported (e.g., there may already be some victims). We propose RegCheck, a system designed to proactively flag suspicious domains at registration time. The core of RegCheck is a machine learning classifier that assesses the risk that the domain name will be used for malicious purposes based on characteristics known at the time of registration. Based on this assessment, it flags some registrations and requires them to undergo additional verification prior to the domain name being activated. The system has been developed collaboratively between SIDN (.nl) and DNS Belgium (.be) and has been deployed as a real-time system at the .be registry since March 2024. Since its deployment, the registry has witnessed a decrease in the number of .be domain name registrations that have been revoked for breaching the terms and conditions, indicating a decline in the number of active malicious registrations.

In many time series classification tasks, each instance is described by multiple channels (i.e., signals). This introduces an additional computational burden as the time and resources to train a classifier increase as more channels become available. This is problematic as some tasks can be described by a huge number of channels. However, not all channels may be necessary as some could be irrelevant or redundant, and including these can (dramatically) increase run time while yielding no benefit in terms of predictive performance. Therefore, it can be useful to automatically select a subset of the channels to include in the analysis. We propose TSelect, a novel scalable and classifier-agnostic approach that automatically selects a relevant and non-redundant subset of the channels for multivariate time series classification (MTSC). Experimentally, we show on a large benchmark suite that TSelect (1) eliminates on average 62% of the channels, (2) significantly improves a classifier’s run time without sacrificing predictive performance, and (3) outperforms the two state-of-the-art channel selectors (ECS and ECP) on the majority of the experiments.

Anomaly detection algorithms attempt to find instances that deviate from the expected behavior. Because this is often tackled as an unsupervised task, anomaly detection models rely on exploiting intuitions about what constitutes anomalous behavior. These typically take the form of data-driven heuristics that measure the anomalousness of each instance. However, the effectiveness of unsupervised detectors are limited by the validity of their intuition. Because these are not universally true, one can improve the detectors' performance by using a semi-supervised approach that exploits a few labeled instances. This paper proposes a novel semi-supervised tree ensemble based anomaly detection framework. We compare our proposed approach to several baselines and show that it achieves comparable performance to state-of-the-art neural networks on six real-world and 14 benchmark datasets.

In the multi-instance learning (MIL) setting instances are grouped together into bags. Labels are provided only for the bags and not on the level of individual instances. A positive bag label means that at least one instance inside the bag is positive, while a negative bag label restricts all the instances in the bag to be negative. MIL data naturally arises in many contexts, such as anomaly detection, where labels are rare and costly, and one often ends up annotating the label for sets of instances. Moreover, in many real-world anomaly detection problems, only positive labels are collected because they usually represent critical events. Such a setting, where only positive labels are provided along with unlabeled data, is called Positive and Unlabeled (PU) learning. Despite being useful for several use cases, there is no work dedicated to learning from positive and unlabeled data in a multi-instance setting for anomaly detection. Therefore, we propose the first method that learns from PU bags in anomaly detection. Our method uses an autoencoder as an underlying anomaly detector. We alter the autoencoder's objective function and propose a new loss that allows it to learn from positive and unlabeled bags of instances. We theoretically analyze this method. Experimentally, we evaluate our method on 30 datasets and show that it performs better than multiple baselines adapted to work in our setting.

Creativity is highly valued in soccer players. It contributes to exciting and unpredictable play, which can help teams to overcome defensive strategies and create scoring opportunities. Consequently, evaluating the creative abilities of players is an important aspect of the player recruitment process. However, there is currently no clear way to measure creativity in soccer. It is not captured by the typical result-based performance indicators, as being creative entails going beyond just doing something useful, to accomplishing something useful but in a unique or atypical way. Therefore in this paper, we define a novel metric to quantify the level of creativity involved in a player's passes. Our Creative Decision Rating (CDR) utilizes machine learning techniques to assess two important factors: the originality of a pass, and its value in terms of increasing the team's chances of scoring a goal. We validated our metric on StatsBomb 360 contextual event stream data of the 2021/22 English Premier League season and show through a number of use cases that it provides another angle on a player's skill, complementing existing player evaluation metrics. Overall, our metric provides a concise method for capturing and quantifying the creativity of soccer players and could have important implications for player recruitment and talent development in the sport.

Sleep stage scoring is an essential component of diagnosing sleep disorders. Unfortunately, it is a time-intensive task that requires clinical experts to annotate an entire night's recording for each patient. Therefore, machine learned models offer the potential to alleviate this burden by automating this task. While learned models achieve acceptable accuracy on curated data, these models still produce highly inaccurate scorings for certain patients when deployed in medical centers. This is because particular subsets of the population may not be adequately represented in the data used to train the model. For example, data are not easily accessible (e.g., a given age group like children) or are hard or impossible to collect (e.g., patients with a rare disease or previously unknown pathology). This creates trust issues as incorrect scorings can have severe consequences such as undetected diseases. To address this, we propose augmenting an existing model with a reject option which enables it to abstain from making predictions if the model is at an elevated risk of making a mistake. We show that traditional rejection frameworks can systematically be too cautious in certain circumstances and abstain even when the model can make good predictions. We propose a solution by considering both the data distribution and the model predictions. We demonstrate the efficacy of our method on a real-world sleep scoring use case. Moreover, we found that our approach leads to improved performance on several publicly available benchmarks.

In-game win probability models, which provide a sports team's likelihood of winning at each point in a game based on historical observations, are becoming increasingly popular. In baseball, basketball and American football, they have become important tools to enhance fan experience, to evaluate in-game decision-making, and to inform coaching decisions. While equally relevant in soccer, the adoption of these models is held back by technical challenges arising from the low-scoring nature of the sport.

Imagine being able to ask questions to a black box model such as "Which adversarial examples exist?", "Does a specific attribute have a disproportionate effect on the model's prediction?" or "What kind of predictions could possibly be made for a partially described example?" This last question is particularly important if your partial description does not correspond to any observed example in your data, as it provides insight into how the model will extrapolate to unseen data. These capabilities would be extremely helpful as they would allow a user to better understand the model's behavior, particularly as it relates to issues such as robustness, fairness, and bias. In this paper, we propose such an approach for an ensemble of trees. Since, in general, this task is intractable we present a strategy that (1) can prune part of the input space given the question asked to simplify the problem; and (2) follows a divide and conquer approach that is incremental and can always return some answers and indicates which parts of the input domains are still uncertain. The usefulness of our approach is shown on a diverse set of use cases.

Given its large applicational potential, time series anomaly detection has become a crucial data mining task. Its goal is to identify periods of a time series where there is a deviation from the expected behavior. Existing approaches focus on analyzing whether the currently observed behavior differs from previously seen, normal behavior. In contrast, this paper tackles the the task where the absence of a previously observed behavior is indicative of an anomaly. In other words, a pattern that is expected to recur in the time series is absent. In real-world use cases, absent patterns can be linked to serious problems. For instance, if a scheduled, regular maintenance operation of a machine does not take place, this can be harmful to the machine at a later time. In this paper, we introduce the task of detecting when a specific pattern is absent in a real-valued time series. We propose a novel technique called FZapPa that can address this task. Empirically, FZapPa outperforms existing anomaly techniques on a benchmark of real-world datasets.

Assessing the impact of the individual actions performed by soccer players during games is a crucial aspect of the player recruitment process. Unfortunately, most traditional metrics fall short in addressing this task as they either focus on rare actions like shots and goals alone or fail to account for the context in which the actions occurred. This paper introduces (1) a new language for describing individual player actions on the pitch and (2) a framework for valuing any type of player action based on its impact on the game outcome while accounting for the context in which the action happened. By aggregating soccer players' action values, their total offensive and defensive contributions to their team can be quantified. We show how our approach considers relevant contextual information that traditional player evaluation metrics ignore and present a number of use cases related to scouting and playing style characterization in the 2016/2017 and 2017/2018 seasons in Europe's top competitions.

Nowadays, all aspects of a production process are continuously monitored and visualized in a dashboard. Equipment is monitored using a variety of sensors, natural resource usage is tracked, and interventions are recorded. In this context, a common task is to identify anomalous behavior from the time series data generated by sensors. As manually analyzing such data is laborious and expensive, automated approaches have the potential to be much more efficient as well as cost effective. While anomaly detection could be posed as a supervised learning problem, typically this is not possible as few or no labeled examples of anomalous behavior are available and it is oftentimes infeasible or undesirable to collect them. Therefore, unsupervised approaches are commonly employed which typically identify anomalies as deviations from normal (i.e., common or frequent) behavior. However, in many real-world settings several types of normal behavior exist that occur less frequently than some anomalous behaviors. In this paper, we propose a novel constrained-clustering-based approach for anomaly detection that works in both an unsupervised and semi-supervised setting. Starting from an unlabeled data set, the approach is able to gradually incorporate expert-provided feedback to improve its performance. We evaluated our approach on real-world water monitoring time series data from supermarkets in collaboration with Colruyt Group, one of Belgiums largest retail companies. Empirically, we found that our approach outperforms the current detection system as well as several other baselines. Our system is currently deployed and used by the company to analyze water usage for 20 stores on a daily basis.

Running is extremely popular and around 10.6 million people run regularly in the United States alone. Unfortunately, estimates indicated that between 29% to 79% of runners sustain an overuse injury every year. One contributing factor to such injuries is excessive fatigue, which can result in alterations in how someone runs that increase the risk for an overuse injury. Thus being able to detect during a running session when excessive fatigue sets in, and hence when these alterations are prone to arise, could be of great practical importance. In this paper, we explore whether we can use machine learning to predict the rating of perceived exertion (RPE), a validated subjective measure of fatigue, from inertial sensor data of individuals running outdoors. We describe how both the subjective target label and the realistic outdoor running environment introduce several interesting data science challenges. We collected a longitudinal dataset of runners, and demonstrate that machine learning can be used to learn accurate models for predicting RPE.

Sports teams are nowadays collecting huge amounts of data from training sessions and matches. The teams are becoming increasingly interested in exploiting these data to gain a competitive advantage over their competitors. One of the most prevalent types of new data is event stream data from matches. These data enable more advanced descriptive analysis as well as the potential to investigate an opponent's tactics in greater depth. Due to the complexity of both the data and game strategy, most tactical analyses are currently performed by humans reviewing video and scouting matches in person. As a result, this is a time-consuming and tedious process. This paper explores the problem of automatic tactics detection from event-stream data collected from professional soccer matches. We highlight several important challenges that these data and this problem setting pose. We describe a data-driven approach for identifying patterns of movement that account for both spatial and temporal information which represent potential offensive tactics. We evaluate our approach on the 2015/2016 season of the English Premier League and are able to identify interesting strategies per team related to goal kicks, corners and set pieces.

Currently, there are many large, automatically constructed knowledge bases (KBs). One interesting task is learning from a knowledge base to generate new knowledge either in the form of inferred facts or rules that define regularities. One challenge for learning is that KBs are necessarily open world: we cannot assume anything about the truth values of tuples not included in the KB. When a KB only contains facts (i.e., true statements), which is typically the case, we lack negative examples, which are often needed by learning algorithms. To address this problem, we propose a novel score function for evaluating the quality of a first-order rule learned from a KB. Our metric attempts to include information about the tuples not in the KB when evaluating the quality of a potential rule. Empirically, we find that our metric results in more precise predictions than previous approaches.

This paper proposes a relational-learning based approach for discovering strategies in volleyball matches based on optical tracking data. In contrast to most existing methods, our approach permits discovering patterns that account for both spatial (that is, partial configurations of the players on the court) and temporal (that is, the order of events and positions) aspects of the game. We analyze both the men's and women's final match from the 2014 FIVB Volleyball World Championships, and are able to identify several interesting and relevant strategies from the matches.

Traditional Markov network structure learning algorithms perform a search for globally useful features. However, these algorithms are often slow and prone to finding local optima due to the large space of possible structures. Ravikumar et al. recently proposed the alternative idea of applying L1 logistic regression to learn a set of pair wise features for each variable, which are then combined into a global model. This paper presents the DTSL algorithm, which uses probabilistic decision trees as the local model. Our approach has two significant advantages: it is more efficient, and it is able to discover features that capture more complex interactions among the variables. Our approach can also be seen as a method for converting a dependency network into a consistent probabilistic model. In an extensive empirical evaluation on 13 datasets, our algorithm obtains comparable accuracy to three standard structure learning algorithms while running 1-4 orders of magnitude faster.