Deep learning (DL) has greatly advanced audio classification, yet the field is limited by the scarcity of large-scale benchmark datasets that have propelled progress in other domains. While AudioSet is a pivotal step to bridge this gap as a universal-domain dataset, its restricted accessibility and limited range of evaluation use cases challenge its role as the sole resource. Therefore, we introduce BirdSet, a large-scale benchmark data set for audio classification focusing on avian bioacoustics. BirdSet surpasses AudioSet with over 6,800 recording hours ($\uparrow17\%$) from nearly 10,000 classes ($\uparrow18\times$) for training and more than 400 hours ($\uparrow7\times$) across eight strongly labeled evaluation datasets. It serves as a versatile resource for use cases such as multi-label classification, covariate shift or self-supervised learning. We benchmark six well-known DL models in multi-label classification across three distinct training scenarios and outline further evaluation use cases in audio classification. We host our dataset on Hugging Face for easy accessibility and offer an extensive codebase to reproduce our results.

Abstract In the field of deep learning based computer vision, the development of deep object detection has led to unique paradigms (e.g., two-stage or set-based) and architectures (e.g., Faster-RCNN or DETR ) which enable outstanding performance on challenging benchmark datasets. Despite this, the trained object detectors typically do not reliably assess uncertainty regarding their own knowledge, and the quality of their probabilistic predictions is usually poor. As these are often used to make subsequent decisions, such inaccurate probabilistic predictions must be avoided. In this work, we investigate the uncertainty calibration properties of different pretrained object detection architectures in a multi-class setting. We propose a framework to ensure a fair, unbiased, and repeatable evaluation and conduct detailed analyses assessing the calibration under distributional changes (e.g., distributional shift and application to out-of-distribution data). Furthermore, by investigating the influence of different detector paradigms, post-processing steps, and suitable choices of metrics, we deliver novel insights into why poor detector calibration emerges. Based on these insights, we are able to improve the calibration of a detector by simply finetuning its last layer.

Training with noisy class labels impairs neural networks’ generalization performance. In this context, mixup is a popular regularization technique to improve training robustness by making memorizing false class labels more difficult. However, mixup neglects that multiple annotators, e.g., crowdworkers, typically provide class labels. Therefore, we propose an extension of mixup, which handles multiple class labels per instance while considering which class label originates from which annotator. Integrated into our multi-annotator classification framework annot-mix, it performs superiorly to eleven (mostly state-of-the-art) approaches in an evaluation study with eleven datasets comprising noisy class labels from either human or simulated annotators. Our code is publicly available through our GitHub repository at https://github.com/ies-research/multi-annotator-machine-learning/tree/annot-mix.

In the increasingly complex industrial system health management domain, accurate prediction of remaining useful life plays an essential role. This paper analyzes the methods to improve the predictive performance of remaining useful life from three aspects: optimizing model structures, augmenting uncertainty estimation in predictions, and transitioning normalization methods. Based on our analysis, we propose a novel model, the Uncertainty Spatial-Temporal Attention Graph Neural Network (USTAGNN), which consists of three primary components: sensor graph construction, a spatio-temporal feature extractor, and a probabilistic prediction module. The feature extractor leverages graph neural networks and temporal convolutional networks as a foundation to extract spatial and temporal features, further enhanced by attention mechanisms, spectral normalization, and residual connections to bolster its distance awareness. Following extensive experimental comparisons, we utilized the parameter-driven dynamic adjacency matrix for sensor graph construction and the deep kernel Gaussian process for precise uncertainty estimation. USTAGNN tries to resolve issues not thoroughly addressed in existing research, such as comparative analyses of sensor graph construction methods, accurate uncertainty estimation, and the model’s generalization under different preprocessing conditions. The proposed model demonstrated state-of-the-art performance on various subsets of the C-MAPSS dataset, achieving up to a 35.9% improvement in prediction score.

Ensuring the reliability of critical industrial systems across various sectors is crucial. It is essential to detect deviations from regular behaviour to mitigate disruptions and preserve infrastructure integrity. However, accurately labelling anomaly datasets is challenging due to their rarity and manual annotation subjectivity. The conventional approach of training separate models for each dataset entity further complicates model development. This paper presents a novel Multi-task Learning framework combining LSTM Autoencoder with temporal attention mechanism (MTL-LATAM) for effective time series anomaly detection. Multitask learning models improve adaptability and generalizability, leading to reduced runtime and compute power while supporting zero-shot evaluation. These models offer flexibility in detecting emerging anomalies. Additionally, we introduce a dynamic thresholding mechanism to incorporate temporal context for anomaly detection and provide visualizations of attention weights to enhance interpretability. The study compares MTL- LATAM, with other multi-task models, evaluates multi-task versus single-task models and assesses the performance of the proposed frame- work in zero-shot learning scenarios. The findings indicate MTL- LATAM’s effectiveness across real-world and open-source datasets, achieving 95% and 97% task synergy. The results underscore the superior performance of multi-task models in zero-shot tasks compared to individual models trained exclusively on their respective datasets.

Human annotators typically provide annotated data for training machine learning models, such as neural networks. Yet, human annotations are subject to noise, impairing generalization performances. Methodological research on approaches counteracting noisy annotations requires corresponding datasets for a meaningful empirical evaluation. Consequently, we introduce a novel benchmark dataset, dopanim, consisting of about 15,750 animal images of 15 classes with ground truth labels. For approximately 10,500 of these images, 20 humans provided over 52,000 annotations with an accuracy of circa 67%. Its key attributes include (1) the challenging task of classifying doppelganger animals, (2) human-estimated likelihoods as annotations, and (3) annotator metadata. We benchmark well-known multi-annotator learning approaches using seven variants of this dataset and outline further evaluation use cases such as learning beyond hard class labels and active learning. Our dataset and a comprehensive codebase are publicly available to emulate the data collection process and to reproduce all empirical results.

In this work, we apply deep active learning to the field of design optimization to reduce the number of computationally expensive numerical simulations widely used in industry and engineering. We are interested in optimizing the design of structural components, where a set of parameters describes the shape. If we can predict the performance based on these parameters and consider only the promising candidates for simulation, there is an enormous potential for saving computing power. We present two query strategies for self-optimization to reduce the computational cost in multi-objective design optimization problems. Our proposed methodology provides an intuitive approach that is easy to apply, offers significant improvements over random sampling, and circumvents the need for uncertainty estimation. We evaluate our strategies on a large dataset from the domain of fluid dynamics and introduce two new evaluation metrics to determine the model's performance. Findings from our evaluation highlights the effectiveness of our query strategies in accelerating design optimization. Furthermore, the introduced method is easily transferable to other self-optimization problems in industry and engineering.

Anomaly detection plays a pivotal role in diverse real-world applications such as cybersecurity, fault detection, network monitoring, predictive maintenance, and highly automated driving. However, obtaining labeled anomalous data can be a formidable challenge, especially when anomalies exhibit temporal evolution. This paper introduces LATAM (Long short-term memory Autoen-coder with Temporal Attention Mechanism) for few-shot anomaly detection, with the aim of enhancing detection performance in scenarios with limited labeled anomaly data. LATAM effectively captures temporal dependencies and emphasizes significant patterns in multivariate time series data. In our investigation, we comprehensively evaluate LATAM against other anomaly detection models, particularly assessing its capability in few-shot learning scenarios where we have minimal examples from the normal class and none from the anomalous class in the training data. Our experimental results, derived from real-world photovoltaic inverter data, highlight LATAM's superiority, showcasing a substantial 27% mean F1 score improvement, even when trained on a mere two-week dataset. Furthermore, LATAM demonstrates remarkable results on the open-source SWaT dataset, achieving a 12% boost in accuracy with only two days of training data. Moreover, we introduce a simple yet effective dynamic thresholding mechanism, further enhancing the anomaly detection capabilities of LATAM. This underscores LATAM's efficacy in addressing the challenges posed by limited labeled anomalies in practical scenarios and it proves valuable for downstream tasks involving temporal representation and time series prediction, extending its utility beyond anomaly detection applications.

Inner-city intersections are among the most critical traffic areas for injury and fatal accidents. Automated vehicles struggle with the complex and hectic everyday life within those areas. Sensor-equipped smart infrastructures, which can cooperate with vehicles, can benefit automated traffic by extending the perception capabilities of drivers and vehicle perception systems. Additionally, they offer the opportunity to gather reproducible and precise data of a holistic scene understanding, including context information as a basis for training algorithms for various applications in automated traffic. Therefore, we introduce the Infrastructural Multi-Person Trajectory and Context Dataset (IMPTC). We use an intelligent public inner-city intersection in Germany with visual sensor technology. A multi-view camera and LiDAR system perceives traffic situations and road users’ behavior. Additional sensors monitor contextual information like weather, lighting, and traffic light signal status. The data acquisition system focuses on Vulnerable Road Users (VRUs) and multi-agent interaction. The resulting dataset consists of eight hours of measurement data. It contains over 2,500 VRU trajectories, including pedestrians, cyclists, e-scooter riders, strollers, and wheelchair users, and over 20,000 vehicle trajectories at different day times, weather conditions, and seasons. In addition, to enable the entire stack of research capabilities, the dataset includes all data, starting from the sensor-, calibration- and detection data until trajectory and context data. The dataset is continuously expanded and is available online for non-commercial research at https://github.com/kav-institute/imptc-dataset.

In this work, we propose an extension of conventional image data by an additional channel in which the associated projection properties are encoded. This addresses the issue of sensor-dependent object representation in projection-based sensors, such as LiDAR, which can lead to distorted physical and geometric properties due to variations in sensor resolution and field of view. To that end, we propose an architecture for processing this data in an instance segmentation framework. We focus specifically on LiDAR as a key sensor modality for machine vision tasks and highly automated driving (HAD). Through an experimental setup in a controlled synthetic environment, we identify a bias on sensor resolution and field of view and demonstrate that our proposed method can reduce said bias for the task of LiDAR instance segmentation. Furthermore, we define our method such that it can be applied to other projection-based sensors, such as cameras. To promote transparency, we make our code and dataset publicly available. This method shows the potential to improve performance and robustness in various machine vision tasks that utilize projection-based sensors.

The development of new electric traction machines is a time-consuming process as it involves intensive testing on motor test benches. Machine-Learning-empowered monitoring offers the opportunity to anticipate costly failures early and hence reduce development time. However, machine learning (ML) for process monitoring requires large amounts of training data, especially as the targeted fault states are scarce and yet diverse in their appearances.Therefore, we propose to use synthetic time series data to leverage the high cost of acquiring training data from experiments in real test benches. In this article, we present a novel scheme to generate synthetic data based on a sub-dimensional time series representation. We introduce a highly flexible model by mapping the data to a latent representation and approximating the latent data distribution by a Gaussian Mixture Model. In addition, we propose the Fréchet InceptionTime Distance (FITD) as a new distance measure to evaluate the generated data. It allows extracting characteristics at different scales by using multiple kernel sizes. In this way, we ensure that the synthesized data contains characteristics similar to those present in the real data. In our experiment, we train two types of fault detectors, one based on real data of a motor test bench and the other based on synthetic data. We also consider employing fault-aware conditional architectures to generate training data for different fault types explicitly. Our final results show that using synthesized data in the training process increases the performance in terms of classification accuracy score (CAS) up to 29%.

Renewable power generation forecasts using machine learning are typically implemented as single-task learning models, where a separate model is trained for each photovoltaic or wind park. In recent years, transfer learning is gaining popularity in these systems, as it can be used to transfer the knowledge gained from source parks to a target park. However, for transferring the knowledge to a target park, there is a need to determine the most similar source park(s) among the existing parks. This similarity determination using historical power measurements is challenging when the target park has limited to no historical data samples. Therefore, we propose a simple multi-task learning architecture that initially learns a common representation of input weather features among the tasks, using a Unified Autoencoder (UAE) and then learns the task specific information utilizing a Task Embedding layer in a Neural Network (TENN). This proposed architecture, UAE-TENN, can be easily extended to new parks with or without historical data. An elaborate performance comparison of single and multi-task learning models is performed on six photovoltaic and wind farm datasets comprising a total of 529 parks. UAE-TENN significantly improves the performance of power forecasting by 10 to 19% for photovoltaic parks and 5 to 22% for wind parks compared to the baseline models. Even in the zero-shot learning scenario, when there is no historical data, we successfully demonstrate that the UAE-TENN improves the forecast accuracy for a new park by 19% for photovoltaic parks.

This article presents a holistic approach for probabilistic cyclist intention detection. By combining probabilities for the current cyclist motion states with a probabilistic ensemble trajectory forecast, we are able to generate reliable estimates of cyclists’ future positions. The probabilities are used as weights in the probabilistic ensemble trajectory forecast. The ensemble consists of specialized models, which produce individual forecasts in the form of Gaussian distributions under the assumption of a certain motion state of the cyclist (e.g. cyclist is starting). By weighting the specialized models, we create forecasts in the form of Gaussian mixtures that define regions within which the cyclists will reside with a certain probability. To evaluate our method, we rate the reliability, sharpness, and positional accuracy of our forecasted distributions. We compare our method to a widely used unimodal approach which produces forecasts in the form of Gaussian distributions and show that our method is able to produce more reliable and sharper outputs while retaining comparable positional accuracy. By comparing two different methods for basic movement detection, one based on the past cyclist trajectory and one using past image sequences, we demonstrate that the results can be further improved by incorporating video information. Our methods are evaluated using a dataset created at a public intersection. The code and the dataset are publicly available.

Abstract In transductive active learning, the goal is to determine the correct labels for an unlabeled, known dataset. Therefore, we can either ask an oracle to provide the right label at some cost or use the prediction of a classifier which we train on the labels acquired so far. In contrast, the commonly used (inductive) active learning aims to select instances for labeling out of the unlabeled set to create a generalized classifier, which will be deployed on unknown data. This article formally defines the transductive setting and shows that it requires new solutions. Additionally, we formalize the theoretically cost-optimal stopping point for the transductive scenario. Building upon the probabilistic active learning framework, we propose a new transductive selection strategy that includes a stopping criterion and show its superiority.

Abstract Support Vector Data Description (SVDD) is a popular one-class classifier for anomaly and novelty detection. But despite its effectiveness, SVDD does not scale well with data size. To avoid prohibitive training times, sampling methods select small subsets of the training data on which SVDD trains a decision boundary hopefully equivalent to the one obtained on the full data set. According to the literature, a good sample should therefore contain so-called boundary observations that SVDD would select as support vectors on the full data set. However, non-boundary observations also are essential to not fragment contiguous inlier regions and avoid poor classification accuracy. Other aspects, such as selecting a sufficiently representative sample, are important as well. But existing sampling methods largely overlook them, resulting in poor classification accuracy. In this article, we study how to select a sample considering these points. Our approach is to frame SVDD sampling as an optimization problem, where constraints guarantee that sampling indeed approximates the original decision boundary. We then propose RAPID, an efficient algorithm to solve this optimization problem. RAPID does not require any tuning of parameters, is easy to implement and scales well to large data sets. We evaluate our approach on real-world and synthetic data. Our evaluation is the most comprehensive one for SVDD sampling so far. Our results show that RAPID outperforms its competitors in classification accuracy, in sample size, and in runtime.

Abstract Stream-based active learning (AL) strategies minimize the labeling effort by querying labels that improve the classifier’s performance the most. So far, these strategies neglect the fact that an oracle or expert requires time to provide a queried label. We show that existing AL methods deteriorate or even fail under the influence of such verification latency. The problem with these methods is that they estimate a label’s utility on the currently available labeled data. However, when this label would arrive, some of the current data may have gotten outdated and new labels have arrived. In this article, we propose to simulate the available data at the time when the label would arrive. Therefore, our method Forgetting and Simulating (FS) forgets outdated information and simulates the delayed labels to get more realistic utility estimates. We assume to know the label’s arrival date a priori and the classifier’s training data to be bounded by a sliding window. Our extensive experiments show that FS improves stream-based AL strategies in settings with both, constant and variable verification latency.

The extension of the power grid by renewable energy generators, e.g., wind power and photovoltaic power, has been accelerated in recent years, particularly in Germany. However, a vast amount of electricity can hardly be fed-in and distributed properly where the grid is slowly extended. This fact causes problems such as waste of resources, overload, and voltage instability. One efficient solution is the regional flexibility market, which requires accurate prediction of power generation and consumption to trade electricity flexibly and optimize electricity flows locally in the grid. Generally, a massive amount of historical samples have to be collected for training data-driven forecasting models, which prolongs the preparation of projects. In this article, we implement deep neural network models using a continual learning-based framework, which can shorten the preparation by warm-up training with a limited amount of samples. The models can detect the changes of a data stream in applications and then update themselves automatically under the trigger condition. We apply the models to a German regional flexibility market for providing continuous power forecasts. The experimental results indicate that the continual-learning models can improve the accuracy of forecasting non-stationary power data. On the other hand, this enhancement can help grid operators to analyze the transmission line load more precisely. This article shows how to establish a more reliable regional energy market by using the continual-learning models.

Systems and functions that rely on machine learning (ML) are the basis of highly automated driving. An essential task of such ML models is to reliably detect and interpret unusual, new, and potentially dangerous situations. The detection of those situations, which we refer to as corner cases, is highly relevant for successfully developing, applying, and validating automotive perception functions in future vehicles where multiple sensor modalities will be used. A complication for the development of corner case detectors is the lack of consistent definitions, terms, and corner case descriptions, especially when taking into account various automotive sensors. In this work, we provide an application-driven view of corner cases in highly automated driving. To achieve this goal, we first consider existing definitions of the general outlier, novelty, anomaly, and out-of-distribution detection to show relations and differences to corner cases. Moreover, we extend an existing camera-focused systematization of corner cases by adding RADAR (radio detection and ranging) and LiDAR (light detection and ranging) sensors. For this, we describe an exemplary toolchain for data acquisition and processing, highlighting the interfaces of corner case detection. We also define a novel level of corner cases, the method layer corner cases, which appear due to uncertainty inherent in the methodology.

Abstract Gathering labeled data to train well-performing machine learning models is one of the critical challenges in many applications. Active learning aims at reducing the labeling costs by an efficient and effective allocation of costly labeling resources. In this article, we propose a decision-theoretic selection strategy that (1) directly optimizes the gain in misclassification error, and (2) uses a Bayesian approach by introducing a conjugate prior distribution to determine the class posterior to deal with uncertainties. By reformulating existing selection strategies within our proposed model, we can explain which aspects are not covered in current state-of-the-art and why this leads to the superior performance of our approach. Extensive experiments on a large variety of datasets and different kernels validate our claims.

Conventional sensor systems record information about directly visible objects, whereas occluded scene components are considered lost in the measurement process. Non-line-of-sight (NLOS) methods try to recover such hidden objects from their indirect reflections - faint signal components, traditionally treated as measurement noise. Existing NLOS approaches struggle to record these low-signal components outside the lab, and do not scale to large-scale outdoor scenes and high-speed motion, typical in automotive scenarios. In particular, optical NLOS capture is fundamentally limited by the quartic intensity falloff of diffuse indirect reflections. In this work, we depart from visible-wavelength approaches and demonstrate detection, classification, and tracking of hidden objects in large-scale dynamic environments using Doppler radars that can be manufactured at low-cost in series production. To untangle noisy indirect and direct reflections, we learn from temporal sequences of Doppler velocity and position measurements, which we fuse in a joint NLOS detection and tracking network over time. We validate the approach on in-the-wild automotive scenes, including sequences of parked cars or house facades as relay surfaces, and demonstrate low-cost, real-time NLOS in dynamic automotive environments.

Radar-based road user detection is an important topic in the context of autonomous driving applications. The resolution of conventional automotive radar sensors results in a sparse data representation which is tough to refine during subsequent signal processing. On the other hand, a new sensor generation is waiting in the wings for its application in this challenging field. In this article, two sensors of different radar generations are evaluated against each other. The evaluation criterion is the performance on moving road user object detection and classification tasks. To this end, two data sets originating from an off-the-shelf radar and a high resolution next generation radar are compared. Special attention is given on how the two data sets are assembled in order to make them comparable. The utilized object detector consists of a clustering algorithm, a feature extraction module, and a recurrent neural network ensemble for classification. For the assessment, all components are evaluated both individually and, for the first time, as a whole. This allows for indicating where overall performance improvements have their origin in the pipeline. Furthermore, the generalization capabilities of both data sets are evaluated and important comparison metrics for automotive radar object detection are discussed. Results show clear benefits of the next generation radar. Interestingly, those benefits do not actually occur due to better performance at the classification stage, but rather because of the vast improvements at the clustering stage.

State-of-the-art, high capacity deep neural networks not only require large amounts of labelled training data, they are also highly susceptible to labelling errors in this data, typically resulting in large efforts and costs and therefore limiting the applicability of deep learning. To alleviate this issue, we propose a novel meta training and labelling scheme that is able to use inexpensive unlabelled data by taking advantage of the generalization power of deep neural networks. We show experimentally that by solely relying on one network architecture and our proposed scheme of combining self-training with pseudo-labels, both label quality and resulting model accuracy, can be improved significantly. Our method achieves state-of-the-art results, while being architecture agnostic and therefore broadly applicable. Compared to other methods dealing with erroneous labels, our approach does neither require another network to be trained, nor does it necessarily need an additional, highly accurate reference label set. Instead of removing samples from a labelled set, our technique uses additional sensor data without the need for manual labelling. Furthermore, our approach can be used for semi-supervised learning.

Classifiers require annotations of instances, i.e., class labels, for training. An annotation process is often costly due to its manual execution through human annotators. Active learning (AL) aims at reducing the annotation costs by selecting instances from which the classifier is expected to learn the most. Many AL strategies assume the availability of a single omniscient annotator. In this article, we overcome this limitation by considering multiple error-prone annotators. We propose the novel AL strategy multi-annotator probabilistic active learning (MaPAL). Due to the nature of learning with error-prone annotators, it must not only select instances but annotators, too. MaPAL builds on a decision-theoretic framework and selects instance-annotator pairs maximizing the classifier's expected performance. Experiments on a variety of data sets demonstrate MaPAL's superior performance compared to five related AL strategies.

Despite the success of deep neural networks (DNN) in many applications, their ability to model uncertainty is still significantly limited. For example, in safety-critical applications such as autonomous driving, it is crucial to obtain a prediction that reflects different types of uncertainty to address life-threatening situations appropriately. In such cases, it is essential to be aware of the risk (i.e., aleatoric uncertainty) and the reliability (i.e., epistemic uncertainty) that comes with a prediction. We present AE-DNN, a model allowing the separation of aleatoric and epistemic uncertainty while maintaining a proper generalization capability. AE-DNN is based on deterministic DNN, which can determine the respective uncertainty measures in a single forward pass. In analyses with synthetic and image data, we show that our method improves the modeling of epistemic uncertainty while providing an intuitively understandable separation of risk and reliability.

Multi-task learning (MTL) provides state-of-the-art results in many applications of computer vision and natural language processing. In contrast to single-task learning (STL), MTL allows for leveraging knowledge between related tasks improving prediction results on all tasks. However, there is a limited number of comparative studies applied to MTL architectures for regression and time series problems taking recent advances of MTL into account. An intriguing, non-linear time-series problem are day ahead forecasts of the expected power generation for renewable power plants. Therefore, the main contribution of this article is a comparative study of the following recent and relevant MTL architectures: Hard parameter sharing (HPS), cross-stitch network (CSN), and sluice network (SN). They are compared to a multi-layer perceptron (MLP) model of similar size in an STL setting. As a additional contribution, we provide a simple, yet practical approach to model task specific information through an embedding layer in an MLP, referred to as task embedding. Further, we contribute a new MTL architecture named emerging relation network (ERN), which can be considered as an extension of the SN. For a solar power dataset, the task embedding achieves the best mean improvement with 8.2%. For two wind and one additional solar dataset, the ERN is the best MTL architecture with improvements up to 11.3%.

In this article, we present an approach to detect basic movements of cyclists in real world traffic situations based on image sequences, optical flow (OF) sequences, and past positions using a multi-stream 3D convolutional neural network (3D-ConvNet) architecture. To resolve occlusions of cyclists by other traffic participants or road structures, we use a wide angle stereo camera system mounted at a heavily frequented public intersection. We created a large dataset consisting of 1,639 video sequences containing cyclists, recorded in real world traffic, resulting in over 1.1 million samples. Through modeling the cyclists' behavior by a state machine of basic cyclist movements, our approach takes every situation into account and is not limited to certain scenarios. We compare our method to an approach solely based on position sequences. Both methods are evaluated taking into account frame wise and scene wise classification results of basic movements, and detection times of basic movement transitions, where our approach outperforms the position based approach by producing more reliable detections with shorter detection times. Our code and parts of our dataset are made publicly available.

An increasing amount of volatile infrastructure, such as solar power plants, wind power plants, battery storages, or electric vehicles, will be added to the distribution grid. Such volatile infrastructure adds strain to the existing infrastructure and may lead to high transmission load on lines and corrupt power grid states. One way to resolve these issues is to establish regional energy markets. Such energy markets try to coordinate energy consumption or production regionally, based on estimates of future power grid states, thus reducing stress on the power grid. For these markets, it is crucial to know possible threatening grid states in advance to allow the market operators to keep the power grid in a healthy grid state with the help of regionally consumed or produced energy. In this work, we propose three different approaches to forecast future grid states with deep neural networks based on a learned numerical weather prediction representation. We evaluate these approaches with experimental results and discuss the remaining challenges that have to be addressed to enable regional energy markets. Furthermore, we provide the dataset to allow other researchers to reimplement our work.

Heterogeneous domain adaptation adapts a machine learning model, here classification model, from a source domain to a target domain to leverage data from both domains. Thereby, supervised heterogeneous domain adaptation expects labeled data from the target domain, while unsupervised heterogeneous domain adaptation does not. In this article, we study the inclusion of active learning to bridge unsupervised and supervised domain adaptation. The active learning approach iteratively queries the most useful instances from the target domain, which are then labeled and used to improve the classification model. Using active learning, the selection of training instances can focus on areas where ambiguity in the source domain resolves in the target domain. Hence, we achieve the same performance with fewer labels. Experiments on real activity recognition data confirm our claims.

Avoiding collisions with vulnerable road users (VRUs) using sensor-based early recognition of critical situations is one of the manifold opportunities provided by the current development in the field of intelligent vehicles. As, especially, pedestrians and cyclists are very agile and have a variety of movement options, modeling their behavior in traffic scenes becomes a challenging task. In this paper, we propose movement models based on machine learning methods, in particular, artificial neural networks, in order to classify the current motion state and to predict the future trajectory of the VRUs. Both model types are also combined to enable the application of specifically trained motion predictors based on a continuously updated pseudo probabilistic state classification. Furthermore, the architecture is used to evaluate motion-specific physical models for starting and stopping and video-based pedestrian motion classification. A comprehensive dataset consisting of a total of 1068 pedestrian and 494 cyclist scenes acquired at an urban intersection is used for optimization, training, and evaluation of the different models. The results show substantially higher classification rates and the ability, through the machine learning approaches, to earlier recognize motion state changes than by the way of interacting multiple model (IMM) Kalman filtering. The trajectory prediction quality has also been improved for all kinds of test scenes, especially when starting and stopping motions are included. Here, 37% and 41% fewer position errors were achieved on average, respectively.

Class assignment (label) is not the only information that can be queried from annotators. Additional feedback, in form of confidences, quantifies the reliability of the provided label. Multiple annotators classifying the same sample with different levels of expertise posses different levels of confidences. In this article, we discuss different types of confidence inspired by real-world application and discuss how they relate to each other. The α-confidence is inspired by humans, the β-confidence works with probabilistic outputs from intelligent machines (e.g., robots), and γ-confidence models non-normalized confidences. We consider uncertain, benevolent annotators, thus the provided labels can be contradictory. To overcome this problem, we propose two fusion strategies that combine the confidences from multiple uncertain annotators to a single one. Numerical and graphical evaluation indicates superior performance of our strategy compared to related strategies, namely, confidence weighted majority vote, c-Certainty, and a maximum likelihood estimation.

Radar-based road user classification is an important yet still challenging task towards autonomous driving applications. The resolution of conventional automotive radar sensors results in a sparse data representation which is tough to recover by subsequent signal processing. In this article, classifier ensembles originating from a one-vs-one binarization paradigm are enriched by one-vs-all correction classifiers. They are utilized to efficiently classify individual traffic participants and also identify hidden object classes which have not been presented to the classifiers during training. For each classifier of the ensemble an individual feature set is determined from a total set of 98 features. Thereby, the overall classification performance can be improved when compared to previous methods and, additionally, novel classes can be identified much more accurately. Furthermore, the proposed structure allows to give new insights in the importance of features for the recognition of individual classes which is crucial for the development of new algorithms and sensor requirements.

In this article, we present an approach to forecast trajectories of vulnerable road users (VRUs) including a numerical quantification of the uncertainty of the forecast. The uncertainty estimates are modeled as normal distributions by means of neural networks. Additionally, we present a method to evaluate the reliability of the forecasted uncertainty estimates, where we utilize quantile-quantile (Q-Q) plots, a graphical method to compare two distributions widely used in statistics. The positional accuracy is evaluated using Euclidean distances, in specific we use the average Euclidean error (AEE) and the average specific AEE (ASAEE). The model is trained and tested using a large dataset of 1311 cyclist trajectories, recorded at an urban intersection in real world traffic. Using this method, we achieve a similar positional accuracy compared to our previous work, where only positions are forecasted. The method is able to produce reliable uncertainty estimates for the motion types start, stop, turn left, and turn rightand produces underconfident uncertainty estimates for the motion types waitand move straight. Since uncertainties are not underestimated, the method can be used as a basis for trajectory planing in automated vehicles.

In near future, vulnerable road users (VRUs) such as cyclists and pedestrians will be equipped with smart devices and wearables which are capable to communicate with intelligent vehicles and other traffic participants. Road users are then able to cooperate on different levels, such as in cooperative intention detection for advanced VRU protection. Smart devices can be used to detect intentions, e.g., an occluded cyclist intending to cross the road, to warn vehicles of VRUs, and prevent potential collisions. This article presents a human activity recognition approach to detect the starting movement of cyclists wearing smart devices. We propose a novel two-stage feature selection procedure using a score specialized for robust starting detection reducing the false positive detections and leading to understandable and interpretable features. The detection is modelled as a classification problem and realized by means of a machine learning classifier. We introduce an auxiliary class, that models starting movements and allows to integrate early movement indicators, i.e., body part movements indicating future behaviour. In this way we improve the robustness and reduce the detection time of the classifier. Our empirical studies with real-world data originating from experiments which involve 49 test subjects and consists of 84 starting motions show that we are able to detect the starting movements early. Our approach reaches an F1-score of 67 % within 0.33 s after the first movement of the bicycle wheel. Investigations concerning the device wearing location show that for devices worn in the trouser pocket the detector has less false detections and detects starting movements faster on average. % compared to reference detector involving all wearing locations. We found that we can further improve the results when we train distinct classifiers for different wearing locations. In this case we reach an F1-score of 94 % with a mean detection time of 0.34 s for the device worn in the trouser pocket.

Machine learning systems learn from data. It is not rare that those data are grouped into categories (also called classes). Thus, one of the goals of a learning algorithm is to find out and understand how to assign new, previously unknown data to the correct class. But, in many cases data is available or can be gathered at low costs, whereas acquiring the corresponding category (or class) involves high costs. This is precisely where active learning (AL) comes into its own: In order to reduce the annotation costs, it allows the learning system to deliberately select the samples which should be annotated. Subsequently, the selected samples are presented to an entity (e.g., humans, simulation systems, etc.), generally addressed under the term oracle, that provides the corresponding classes. Afterwards, the knowledge base of the learner is updated and, depending on a stopping criterion, new labels are queried or not. Such a system is self-aware of its own imperfection, thus, it uses a selection strategy to determine the next most informative sample. Hitherto, it has been shown that AL is a powerful paradigm that evinces the desired results, provided that the oracles are omniscient. But, human oracles are prone to error, so for that reason we can ask ourself: Does AL still work with error prone and uncertain human annotators? In this article we present the results of an active learning case study conducted on 30000 images labeled by two humans and propose a new type of labeling for better solving AL problems with error-prone annotators.

As robots are employed for automating processes in industry, there is a strong demand on robots being able to solve new tasks without costly adaptions. In this article, we present a probabilistic active learning approach for adapting robots to tasks involving object sorting by means of classification according to a human understanding of the problem and its solution. In the beginning, the robot extracts appropriate features of images of the available objects. These features are used by the robot to actively learn to solve the sorting task by asking a human teacher. A novel visualization tool allows to supervise the learning process and to determine when training is complete. Then, the robot is able to successfully sort the objects autonomously. We show our method's superiority compared to other active learning strategies and present results of its application on a real robot to prove the above concept.

Gathering data becomes increasingly simple whereas the labeling of collected instances remains difficult. Active learning provides methods to reduce the labeling effort by intelligent selection of instances. In contrast to building mathematical models or developing heuristics to solve this task, we pursue another approach: We let humans select the instances which should be labeled. Participants are asked to learn to predict the sex of 18 abstract illustrations of bugs as either male or female. This article describes the design, goal and the execution of this study with 14 groups (71 participants). In this exploratory study we analyze humans' balance between exploration and exploitation, the participants' learning behavior, the collaboration within the group as well as the question when to stop querying. The comparison of human performance with baseline active learning algorithms provides promising results which indicate that machine active learning might benefit from incorporating human strategies. Additionally, we provide the complete data and extracted spreadsheets for download.

The classification of individual traffic participants is a complex task, especially for challenging scenarios with multiple road users or under bad weather conditions. Radar sensors provide an - with respect to well established camera systems - orthogonal way of measuring such scenes. In order to gain accurate classification results, 50 different features are extracted from the measurement data and tested on their performance. From these features a suitable subset is chosen and passed to random forest and long short-term memory (LSTM) classifiers to obtain class predictions for the radar input. Moreover, it is shown why data imbalance is an inherent problem in automotive radar classification when the dataset is not sufficiently large. To overcome this issue, classifier binarization is used among other techniques in order to better account for underrepresented classes. A new method to couple the resulting probabilities is proposed and compared to others with great success. Final results show substantial improvements when compared to ordinary multiclass classification.

In this article, we present a novel approach to detect starting motions of cyclists in real world traffic scenarios based on Motion History Images (MHIs). The method uses a deep Convolutional Neural Network (CNN) with a residual network architecture (ResNet), which is commonly used in image classification and detection tasks. By combining MHIs with a ResNet classifier and performing a frame by frame classification of the MHIs, we are able to detect starting motions in image sequences. The detection is performed using a wide angle stereo camera system at an urban intersection. We compare our algorithm to an existing method to detect movement transitions of pedestrians that uses MHIs in combination with a Histograms of Oriented Gradients (HOG) like descriptor and a Support Vector Machine (SVM), which we adapted to cyclists. To train and evaluate the methods a dataset containing MHIs of 394 cyclist starting motions was created. The results show that both methods can be used to detect starting motions of cyclists. Using the SVM approach, we were able to safely detect starting motions 0.506 s on average after the bicycle starts moving with an F1-score of 97.7%. The ResNet approach achieved an F1-score of 100% at an average detection time of 0.144 s. The ResNet approach outperformed the SVM approach in both robustness against false positive detections and detection time.

Organic Computing (OC) and Autonomic Computing (AC) systems are distinct from conventional systems through their ability to self-adapt and to self-organise. However, these properties are just means and not the end. What really makes OC and AC systems useful is their ability to survive in a real world, i.e. to recover from disturbances and attacks from the outside world. This property is called robustness. In this paper, we propose a metric to gauge robustness in order to be able to quantitatively compare the effectiveness of different self-organising and self-adaptive system designs with each other. In the following, we apply this metric to three experimental application scenarios and discuss their usefulness.

In the context of highly automated driving an important aspect is the understanding of Vulnerable Road Users' behavior. In this article we concentrate on starting cyclists at an urban intersection and investigate 104 trajectories of uninstructed cyclists' heads, feet, and bikes. A detailed analysis shows that on average cyclists' heads start moving 0.33 s earlier than the bike. Even 0.31 s before that, about 29 % of the cyclists start with an arm movement. Additionally, we observe two different starting motion patterns. One group accelerates by pushing off the ground with a foot and the other only applies accelerating force to a pedal. Separating the data by different slopes of the road and motion patterns, the analysis shows that motion patterns have more influence on the starting behavior than the slope at the investigated intersection.

The use of expert knowledge is always more or less afflicted with uncertainties for many reasons: Expert knowledge may be imprecise, imperfect, or erroneous, for instance. If we ask several experts to label data (e.g., to assign class labels to given data objects, i.e. samples), we often state that these experts make different, sometimes conflicting statements. The problem of labeling data for classification tasks is a serious one in many technical applications where it is rather easy to gather unlabeled data, but the task of labeling requires substantial effort regarding time and, consequently, money. In this article, we address the problem of combining several, potentially wrong class labels. We assume that we have an ordinal class structure (i.e., three or more classes are arranged such as “light”, “medium-weight”, and “heavy”) and only a few expert statements are available. We propose a novel combination rule, the Extended Imprecise Dirichlet Model Rule (EIDMR) which is based on a k-nearest-neighbor approach and Dirichlet distributions, i.e., second-order distributions for multinomial distributions. In addition, experts may assess the difficulty of the labeling task, which may optionally be considered in the combination. The combination rule EIDMR is compared to others such as a standard Imprecise Dirichlet Model Rule, the Dempster-Shafer Rule, and Murphy's Rule. In our evaluation of EIDMR we first use artificial data where we know the data characteristics and true class labels. Then, we present results of a case study where we classify low-voltage grids with Support Vector Machines (SVM). Here, the task is to assess the expandability of these grids with additional photovoltaic generators (or other distributed generators) by assigning these grids to one of five ordinal classes. It can be shown that the use of our new EIDMR leads to better classifiers in cases where ordinal class labels are used and only a few, uncertain expert statements are available.

Over the past few years extensive research has been conducted to solve classification problems with help of machine learning techniques. However, machine learning is data-driven and obtaining labeled data is often challenging in real applications. Techniques that try to overcome this burden, especially, in the presence of sparsely labeled data, can be found in the field of semi-supervised or active learning, as both make use of unlabeled data. In this paper, a semi-supervised k-nearest neighbor classifier, called Resp-kNN, is proposed for sparsely labeled data. This classifier is based on a probabilistic mixture model and, therefore, combines the advantages of classifiers based on non-parametric density estimate (such as a classical k-nearest neighbor classifier based on Euclidean distance) and classifiers based on parametric density estimates (such as classifiers based on Gaussian mixtures). Experimental results on 21 publicly available benchmark data sets show that Resp-kNN is more robust (regarding the choice of k) and effective for sparsely labeled classification compared to several standard methods.

The prediction of the power generation of wind farms is a non-trivial problem with increasing importance during the last decade due to the rapid increase of wind power generation in the power grid. The prediction task is commonly addressed using numerical weather predictions, statistical methods, or machine learning techniques. Various articles have shown that ensemble techniques for forecasting can yield better results regarding forecasting accuracy than single techniques alone. Typical ensembles make use of a parameter, or data diversity approach to build the models. In this article, we propose a novel ensemble technique using both, cooperative and competitive characteristics of ensembles to gradually adjust the influences of single forecasting algorithms in the ensemble based on their individual strengths using a “coopetitive” weighting formula. The observed quality of the models during training is used to adaptively weigh the models based on the location in the input data space (i.e., depending on the weather situation). We compute the overall weights for a particular weather situation using both, a spatial as well as a global weighting term. The experimental evaluation is performed on a data set consisting of data from 45 wind farms, which is made publicly available. We demonstrate that the technique is among the best performing algorithms compared to other state-of-the-art algorithms and ensembles. Furthermore, the practical applicability of the proposed technique is discussed.

This article presents two methods for predicting the trajectories of cyclists at an intersection and compares them to a Kalman Filter (KF) approach. The first method uses a physical model of cyclists to predict their future position. The second method is based on a polynomial least-squares approximation in combination with a multilayer perceptron artificial neural network and is able to predict the future position of cyclists independent of their motion type such as “Starting”, “Stopping”, “Waiting” or “Passing”. To evaluate the performance of the methods, 566 tracks (394 for training, 172 for testing) of uninstructed cyclists were recorded at a public intersection using a wide angle stereo camera system and laser scanners. Using the tracks as input data, the future trajectory was predicted for a time horizon of 2.5 s. For starting motions the prediction using the physical model leads to 27% more accurate positions than the KF approach for a forecast horizon of 2.5 s. The neural network shows a 34% more accurate result for starting and stopping motions and a similar result for waiting and passing motions.

Power forecasting of renewable energy power plants is a very active research field, as reliable information about the future power generation allow for a safe operation of the power grid and helps to minimize the operational costs of these energy sources. Deep Learning algorithms have shown to be very powerful in forecasting tasks, such as economic time series or speech recognition. Up to now, Deep Learning algorithms have only been applied sparsely for forecasting renewable energy power plants. By using different Deep Learning and Artificial Neural Network algorithms, such as Deep Belief Networks, AutoEncoder, and LSTM, we introduce these powerful algorithms in the field of renewable energy power forecasting. In our experiments, we used combinations of these algorithms to show their forecast strength compared to a standard MLP and a physical forecasting model in the forecasting the energy output of 21 solar power plants. Our results using Deep Learning algorithms show a superior forecasting performance compared to Artificial Neural Networks as well as other reference models such as physical models.

Power forecasting for renewable energy power plants has been a highly active field of research during the past decade. In order to support the operation of the power grid, sophisticated algorithms have to predict the future development of power generation. Algorithms in the class of analog ensembles conduct the process of forecasting by finding historically similar situations (e.g., by comparing weather situations), and merging the historic power generation time series during similar periods to an overall power forecast. However, these algorithms often only use very simple similarity measures, which in turn do not make optimal use of the historic information available. In this article, we propose and compare advanced search strategies for similarity assessment. These strategies include the assessment of forecasting time periods as a whole instead of granular points in time, and joint time windows of historic- and future weather situations. Also, historic power time series are used directly in the comparison strategy. Furthermore, we propose a combined scheme to perform automated feature selection and -weighting for individual weather parameters. We evaluate the proposed technique on a solar farm data set consisting of 21 photovoltaic power plants which is made publicly available. In the evaluation we show that advanced comparison strategies not only offer an advantage over simple strategies, they are even able to outperform other reference techniques, e.g., such based on physical models.

In this article we present three similarity measures based on ontologies that are used to allow data mining in case of very sparse data. With the use of semantic knowledge associated to data sources we can aggregate data from similar sources and use that aggregated data to train a machine learning model. This article focuses on the ontology-based similarity measures that are required to compare data sources. We evaluate our similarity measures using a custom ontology created for fashion specific data and show that it is possible to identify similar data sources and gradually assign them similarity values.

Many efforts have been made during the past decades to investigate the use of orthogonal basis functions in the field of least-squares approximation. Certain bases of orthogonal functions allow for a definition of fast update algorithms for approximations of time series in sliding or growing time windows. In fields such as technical data analytics, temporal data mining, or pattern recognition in time series, appropriate time series representations are needed to measure the similarity of time series or to segment them. This article bridges the gap between mathematical basic research and applications by making fast update techniques for standard polynomials and trigonometric polynomials accessible for time series classification or regression (e.g., forecasting), anomaly or motif detection in time series, etc. This is of utmost importance for online or big data applications. Time series or segments of time series will be represented by features derived from the orthogonal expansion coefficients of the approximating polynomials which capture the essential behavior in the time or spectral domain, i.e. trends and periodic behavior, using standard or trigonometric polynomials. Our experiments show that a reliable computation is possible at a very low runtime compared to a conventional least-squares approach. The algorithms are implemented in Java, C(++), Matlab, and Python and made publicly available.

A person's context data can be used for a multitude of applications, such as energy management or health care. Common context recognition approaches rely on several factors, such as the sensor set, features, or the context modeling algorithm. Discovering the recognition performances of different parameter setting combinations is a complex, time-consuming, and error-prone task. To support the context recognition research, we present the Context Recognition Assistance Tool (CRAT). The Context Recognition Assistance Tool assist by automatically conducting the evaluation for a multitude combination of parameter settings and clearly presents the findings. Using the CRAT, we investigate to what degree five parameters influence the recognition accuracy. To support the research in the field of context recognition, the CRAT is publicly available.

This paper focuses on forecasting of pedestrian's short-time trajectories up to 2.5 s for traffic safety applications. We present a self-learning approach based on artificial neural network movement models and compare it to traditional constant velocity Kalman Filter prediction and extrapolation of polynomials fitted using a least-squares error. Trajectories of uninstructed pedestrians in public traffic at a real urban intersection are acquired by a wide angle stereo camera setup in combination with a 3D head tracking framework. Results using this real-world data show that the artificial neural network significantly improves forecast quality compared to other approaches especially for critical traffic scenes including velocity changes such as starting and stopping. For those velocity changes a reduction of position estimation errors of about 21% compared to the Kalman Filter and to extrapolation of polynomials is obtained. By means of a concrete pedestrian-vehicle scenario we demonstrate the benefit of the proposed approach for an advanced driver assistant system in terms of reaction time.

If knowledge such as classification rules are extracted from sample data in a distributed way, it may be necessary to combine or fuse these rules. In a conventional approach this would typically be done either by combining the classifiers' outputs (e.g., in form of a classifier ensemble) or by combining the sets of classification rules (e.g., by weighting them individually). In this paper, we introduce a new way of fusing classifiers at the level of parameters of classification rules. This technique is based on the use of probabilistic generative classifiers using multinomial distributions for categorical input dimensions and multivariate normal distributions for the continuous ones. That means, we have distributions such as Dirichlet or normal-Wishart distributions over parameters of the classifier. We refer to these distributions as hyperdistributions or second-order distributions. We show that fusing two (or more) classifiers can be done by multiplying the hyperdistributions of the parameters and derive simple formulas for that task. Properties of this new approach are demonstrated with a few experiments. The main advantage of this fusion approach is that the hyperdistributions are retained throughout the fusion process. Thus, the fused components may, for example, be used in subsequent training steps (online training).

Evolvable hardware (EHW) has shown itself to be a promising approach for prosthetic hand controllers. Besides competitive classification performance, EHW classifiers offer self-adaptation, fast training, and a compact implementation. However, EHW classifiers have not yet been sufficiently compared to state-of-the-art conventional classifiers. In this paper, we compare two EHW approaches to four conventional classification techniques:k-nearest-neighbor, decision trees, artificial neural networks, and support vector machines. We provide all classifiers with features extracted from electromyographic signals taken from forearm muscle contractions, and let the algorithms recognize eight to eleven different kinds of hand movements. We investigate classification accuracy on a fixed data set and stability of classification error rates when new data is introduced. For this purpose, we have recorded a short-term data set from three individuals over three consecutive days and a long-term data set from a single individual over three weeks. Experimental results demonstrate that EHW approaches are indeed able to compete with state-of-the-art classifiers in terms of classification performance.

The purpose of this paper is to evaluate and benchmark ensemble methods for time series prediction for daily currency exchange rates using ensemble feedforward neural networks and kernel partial least squares (K-PLS). Best-practice forecasting methods for the US Dollar (USD) per Indian Rupee (IR) are applied for training, validating, and testing the machine learning models. In order to perform the benchmarking evaluation study neural network forecasting methods are first compared on a benchmarked neural network time series prediction method for the Canadian Lynx time series. The K-PLS method is benchmarked in addition with support vector machines (SVM), a similar kernel-based method. Both one-step ahead and a roll-out methods for extended forecast horizons are applied for the currency exchange rates. The paper is novel in the sense that two new ensemble methods are introduced: weight seeding and multiple cross-validation averaging. The paper is also novel in the sense that several new validation indices are proposed that are especially applicable for time series: q2and Q2and the fraction of misses in the exchange rate return space, which is a more relevant metric for currency speculation. As a general conclusion it is found that the USD per IR is quite predictable, while other currencies such as the USD per Euro and the Australian Dollar (AUD) per Euro are not predictable.

Intelligent agents often have the same or similar tasks and sometimes they cooperate to solve a given problem. These agents typically know how to observe their local environment and how to react on certain observations, for instance, and this knowledge may be represented in form of rules. However, many environments are dynamic in the sense that from time to time novel rules are required or old rules become obsolete. In this article we propose and investigate new techniques for knowledge acquisition by novelty detection and reaction as well as obsoleteness detection and reaction that an agent may use for self-adaptation to new situations. For that purpose we consider classifiers based on probabilistic rules. Premises of new rules are learned autonomously while conclusions are either obtained from human experts or from other agents which have learned appropriate rules in the past. By means of knowledge exchange, agents will efficiently be enabled to cope with situations they were not confronted with before. This kind of collaborative intelligence follows the human archetype: Humans are able to learn from each other by communicating learned rules. We demonstrate some properties of the knowledge acquisition techniques using artificial data as well as data from the field of intrusion detection.

In this article, we introduce and investigate 3DS, a novel selection strategy for pool-based active training of a generative classifier, namely CMM (classifier based on a probabilistic mixture model). Such a generative classifier aims at modeling the processes underlying the “generation” of the data. The strategy 3DS considers the distance of samples to the decision boundary, the density in regions where samples are selected, and the diversity of samples in the query set that are chosen for labeling, e.g., by a human domain expert. The combination of the three measures in 3DS is adaptive in the sense that the weights of the distance and the density measure depend on the uniqueness of the classification. With nine benchmark data sets it is shown that 3DS outperforms a random selection strategy (baseline method), a pure closest sampling approach, ITDS (information theoretic diversity sampling), DWUS (density-weighted uncertainty sampling), DUAL (dual strategy for active learning), and PBAC (prototype based active learning) regarding evaluation criteria such as ranked performance based on classification accuracy, number of labeled samples (data utilization), and learning speed assessed by the area under the learning curve.

Alert aggregation is an important subtask of intrusion detection. The goal is to identify and to cluster different alerts-produced by low-level intrusion detection systems, firewalls, etc.-belonging to a specific attack instance which has been initiated by an attacker at a certain point in time. Thus, meta-alerts can be generated for the clusters that contain all the relevant information whereas the amount of data (i.e., alerts) can be reduced substantially. Meta-alerts may then be the basis for reporting to security experts or for communication within a distributed intrusion detection system. We propose a novel technique for online alert aggregation which is based on a dynamic, probabilistic model of the current attack situation. Basically, it can be regarded as a data stream version of a maximum likelihood approach for the estimation of the model parameters. With three benchmark data sets, we demonstrate that it is possible to achieve reduction rates of up to 99.96 percent while the number of missing meta-alerts is extremely low. In addition, meta-alerts are generated with a delay of typically only a few seconds after observing the first alert belonging to a new attack instance.

In this article, we provide a new technique for temporal data mining which is based on classification rules that can easily be understood by human domain experts. Basically, time series are decomposed into short segments, and short-term trends of the time series within the segments (e.g., average, slope, and curvature) are described by means of polynomial models. Then, the classifiers assess short sequences of trends in subsequent segments with their rule premises. The conclusions gradually assign an input to a class. As the classifier is a generative model of the processes from which the time series are assumed to originate, anomalies can be detected, too. Segmentation and piecewise polynomial modeling are done extremely fast in only one pass over the time series. Thus, the approach is applicable to problems with harsh timing constraints. We lay the theoretical foundations for this classifier, including a new distance measure for time series and a new technique to construct a dynamic classifier from a static one, and demonstrate its properties by means of various benchmark time series, for example, Lorenz attractor time series, energy consumption in a building, or ECG data.

The paper presents SwiftSeg, a novel technique for online time series segmentation and piecewise polynomial representation. The segmentation approach is based on a least-squares approximation of time series in sliding and/or growing time windows utilizing a basis of orthogonal polynomials. This allows the definition of fast update steps for the approximating polynomial, where the computational effort depends only on the degree of the approximating polynomial and not on the length of the time window. The coefficients of the orthogonal expansion of the approximating polynomial-obtained by means of the update steps-can be interpreted as optimal (in the least-squares sense) estimators for average, slope, curvature, change of curvature, etc., of the signal in the time window considered. These coefficients, as well as the approximation error, may be used in a very intuitive way to define segmentation criteria. The properties of SwiftSeg are evaluated by means of some artificial and real benchmark time series. It is compared to three different offline and online techniques to assess its accuracy and runtime. It is shown that SwiftSeg-which is suitable for many data streaming applications-offers high accuracy at very low computational costs.

In this paper, a new technique for online signature verification or identification is proposed. The technique integrates a longest common subsequences (LCSS) detection algorithm which measures the similarity of signature time series into a kernel function for support vector machines (SVM). LCSS offers the possibility to consider the local variability of signals such as the time series of pen-tip coordinates on a graphic tablet, forces on a pen, or inclination angles of a pen measured during a signing process. Consequently, the similarity of two signature time series can be determined in a more reliable way than with other measures. A proprietary database with signatures of 153 test persons and the SVC 2004 benchmark database are used to show the properties of the new SVM-LCSS. We investigate its parameterization and compare it to SVM with other kernel functions such as dynamic time warping (DTW). Our experiments show that SVM with the LCSS kernel authenticate persons very reliably and with a performance which is significantly better than that of the best comparing technique, SVM with DTW kernel.

For classification tasks, the application of generative classifiers sometimes has advantages over the use of exclusively discriminative classifiers because loss functions can be considered or rejection criteria can be defined more easily, for instance. We show how a radial basis function (RBF) network with multivariate (elliptical) Gaussian basis functions can be trained in two different ways to obtain a classifier with either a more generative or a more discriminative behavior. Our generative classifier allows a probabilistic interpretation of the external outputs (posterior probability of class membership) and the hidden neurons' activations (posterior probability of a component of the model). For that purpose a variational Bayesian inference approach is applied, which also finds an appropriate number of hidden neurons (i.e., components) ldquoon the flyrdquo. A discriminative classifier is obtained using the resilient propagation training technique. We investigate the properties of the two training techniques in detail by introducing a measure for generative properties of the trained classifiers and by comparing these classifiers on various data sets.

Neural networks are often used to process temporal information, i.e., any kind of information related to time series. In many cases, time series contain short-term and long-term trends or behavior. This paper presents a new approach to capture temporal information with various reference periods simultaneously. A least squares approximation of the time series with orthogonal polynomials will be used to describe short-term trends contained in a signal (average, increase, curvature, etc.). Long-term behavior will be modeled with the tapped delay lines of a time-delay neural network (TDNN). This network takes the coefficients of the orthogonal expansion of the approximating polynomial as inputs such considering short-term and long-term information efficiently. The advantages of the method will be demonstrated by means of artificial data and two real-world application examples, the prediction of the user number in a computer network and online tool wear classification in turning.

The fact that neuroleptics may have a more or less noticeable influence on fine motor skills is well-known. However, up to now there is no system that allows one to measure and to quantify such an impact of neuroleptics or other drugs. This article goes a first step into this direction by demonstrating how the handwriting dynamics of a healthy person can automatically be distinguished from that of a schizophrenic and, therefore, appropriately medicated person. Moreover, it is shown that differences can be detected even for a very simple kind of hand movement. That is, the persons trace a given meander. The handwriting dynamics are measured by means of a pen equipped with force and tilt sensors (biometric smart pen). Then, the parameterized script generator model proposed by Hollerbach is used in order to extract characteristic features from the measured signals, e.g., features describing deviations of measured time series from predicted model time series. These features are then used as inputs of support vector machines that classify whether the handwriting has been provided by a healthy or a diseased person

Experts have important qualitative knowledge about interrelations between more or less abstract concepts in an application area. However, the knowledge of a single expert is typically quite uncertain (e.g., incomplete or imprecise). By fusing the knowledge of several experts it would be possible to obtain more certain and, therefore, more valuable knowledge. Conventional systems for knowledge discovery (KD) and data mining (DM) have the ability to extract valid rules from huge data sets. These rules describe dependencies between attributes and classes in a quantitative way, for instance. By fusing this kind of knowledge with the combined, qualitative knowledge of several experts it would be possible to obtain more comprehensive knowledge about an application area. In this article, we propose a concept for a new KD & DM technique based on computational intelligence: collaborative knowledge discovery (CKD). These techniques combines the uncertain knowledge of several experts using methods based on Dempster-Shafer theory. The combined human knowledge is again fused with automatically extracted, well interpretable knowledge (fuzzy rules embedded in a radial basis function neural network) of a conventional KD system. Thus, a CKD system not only acquires more comprehensive knowledge, but also experience (knowledge about knowledge), meaning that it is able to explain automatically extracted rules to the human experts and to assess the interestingness (e.g., novelty or utility) of these rules. This can be done by adapting inference mechanisms from the field of probabilistic argumentation systems. A CKD system will comprise self-awareness mechanisms (it must know what it knows) as well as environment-awareness mechanisms (it must know what human experts know or what they want to now). In order to reduce the effort for knowledge acquisition, a CKD system must learn (pro-)actively. There are many application areas for such CKD systems, e.g., in the field of technical data mining (quality control, process monitoring, etc.)

Driver assistance systems are the key technology to improve traffic safety and lower the number of deadly accidents. Direct communication between cars will further enhance this field of driver safety. In the context of foresighted driving, Bayesian networks can be used to determine a traffic situation at the current position of a car. Communicating this awareness for the current time and position will help other traffic participants. However, situations change dynamically and cars cannot trust all the information provided by other cars over time. Reasoning with this information is difficult as Bayesian networks cannot use spatial and temporal data in an appropriate way. This article outlines the spatial and temporal problems in predictive driver assistance and demonstrates how they can be solved by considering spatial and temporal influences by applying weighting techniques. The pre-processed information is utilized by a Bayesian network for further refinement. Thus, the proposed approach enables the detection and correct prediction of traffic situations. The approach is evaluated by predicting hazardous rain fields in a car by means of information received from other cars

Knowledge transfer is one of the most important mechanisms of human evolution. The ontogeny of humans enables them to act efficiently in a very dynamic environment. Thus, it would be highly desirable to enable "intelligent" artificial systems to behave in a similar way. This article introduces basic technologies that are needed for that purpose. With these technologies - components of a future knowledge transfer toolbox - it is possible to detect novel concepts that arise in the input space of a classifier or existing classification rules that become obsolete. Then, prototypes of new rules can be created automatically using an on-line clustering mechanism. These prototypes are compared to already existing rules, rated, and eventually accepted or discarded. In case of acceptance, a human expert labels the rules which are then both integrated into the "own" classifier and sent to other classifiers. Thus, knowledge transfer between "intelligent" artificial systems becomes possible and the overall system is provided with a new kind of self-optimization ability

Distributed hash tables are popular third generation P2P protocols which are well understood in theory. These protocols usually assume that every node in the overlay is able to exchange messages with any other overlay node. However, this assumption is not always true for real-world networks, including the PlanetLab or the entire Internet. In these networks, the non-transitive connectivity phenomenon is experienced, in which some overlay nodes are able to exchange messages with a certain node and others are not. This turned out to be a serious problem, particularly for structured P2P overlays. Non-transitive connectivity issues were mainly ignored by P2P research for a long time, but have been intensively discussed recently. This paper suggests a new measure for the degree of non-transitive connectivity and presents a comprehensive, source routing based solution, to overcome non-transitive connectivity problems in distributed hash tables.

Distributed intrusion detection and prevention plays an increasingly important role in securing computer networks. In a distributed intrusion detection system, alerts or high-level meta-alerts are exchanged, aggregated, and correlated in a cooperative fashion to overcome the limitations of conventional intrusion detection systems. Substantial progress has been made, but current systems still suffer from various drawbacks: Most of them only distribute the data collection and not the analysis itself or they rely on a hierarchical or even centralized organization and/or communication architecture. Furthermore, the alerts or meta-alerts are usually aggregated at a pre-defined location and there is no reduction of the vast amount of alerts prior to distribution. Consequently, scalability is limited and any central component in the architecture introduces a "single point of failure ". We propose a completely distributed intrusion detection system based on distributed hash tables to efficiently exchange and aggregate alerts and meta-alerts in a cooperative, self-organizing, and load-balanced way. Independent intrusion detection agents publish their alerts based on a new novelty measure for alerts which prohibits the distribution of already known and hence worthless knowledge. The benefits of our approach are evaluated for a well-known probing attack.

Content pollution is a de facto problem in several currently popular Peer-to-Peer file sharing systems. Previously proposed strategies to fight content pollution include a peer reputation system named Scrubber as well as the alternative Credence object reputation system. This paper builds on previous work into three directions. First, it evaluates the cost-effectiveness of Scrubber and Credence in reducing pollution dissemination when polluters make use of collusion and Sybil attacks. Second, it modifies the Scrubber system to increase its effectiveness, in particular under collusion. Finally, it proposes a hybrid peer and object reputation system that combines the benefits of both strategies. We performed an extensive evaluation of all three systems, for various configurations and pollution mechanisms, as well as collusion and Sybil attacks. Our results show that the new hybrid system is much more effective and robust to malicious actions than any individual strategy, even in very uncooperative and unreliable peer communities.

In many data mining applications that address classification problems, feature and model selection are considered as key tasks. That is, appropriate input features of the classifier must be selected from a given (and often large) set of possible features and structure parameters of the classifier must be adapted with respect to these features and a given data set. This paper describes an evolutionary algorithm (EA) that performs feature and model selection simultaneously for radial basis function (RBF) classifiers. In order to reduce the optimization effort, various techniques are integrated that accelerate and improve the EA significantly: hybrid training of RBF networks, lazy evaluation, consideration of soft constraints by means of penalty terms, and temperature-based adaptive control of the EA. The feasibility and the benefits of the approach are demonstrated by means of four data mining problems: intrusion detection in computer networks, biometric signature verification, customer acquisition with direct marketing methods, and optimization of chemical production processes. It is shown that, compared to earlier EA-based RBF optimization techniques, the runtime is reduced by up to 99% while error rates are lowered by up to 86%, depending on the application. The algorithm is independent of specific applications so that many ideas and solutions can be transferred to other classifier paradigms.

Neural networks are often used for time series processing. Temporal information can, for example, be modeled using the dynamic neural network (DYNN) paradigm which combines delay-elements in feedforward direction with recurrent connections. If networks were applied to process time series, learning typically becomes a very complex and time consuming task. Therefore, training algorithms are needed which are both accurate and fast. In this article six learning algorithms, namely temporal backpropagation through time, resilient propagation, Quasi-Newton, scaled conjugate gradient, backpropagation based on partial Quasi-Newton, and a combination of the latter two algorithms, are presented and applied to DYNN. The various learning algorithms are compared with respect to duration of training, approximation and generalization capability, and convergence speed. Each algorithm is evaluated by means of three real-world application examples, the prediction of the number of users in a computer pool, the prediction of the energy consumption in a building, and the verification of a person using her/his signature. With respect to the experiments conducted here, RProp turns out to be the best algorithm to train DYNN.

With the ongoing growth of the Internet, intrusion detection systems (IDS) play an increasing role in securing communication networks. Particularly, where sensitive and confidential information is stored or transmitted, there is a vital importance of security. In the past few years, soft-computing techniques (especially neural networks) found their way more and more into the research area of intrusion detection and are now an inherent part of it. Although, feature selection is an important task for almost all neural network applications, only very few investigations dealing with any type of automated feature selection are known in the area of intrusion detection. This article sets out an evolutionary algorithm (EA) that performs the tasks of feature selection and architecture optimization for radial basis function (RBF) networks automatically. With the feature selection process proposed, it is possible to reduce the number of input features significantly, which is very important due to the fact that the neural networks can effectively be prevented from overfitting. Furthermore, reducing the number of input features also reduces the time needed for feature extraction as well as the execution time for the trained RBF networks and allows the extraction of well interpretable rules. The results (depending on the attack type, about 2 up to 5 features are needed for attack detection on an average) are demonstrated for seven attack types using the DARPA 1998 intrusion detection evaluation data.

Radial basis function (RBF) networks are used in many applications, e.g. for pattern classification or nonlinear regression. For a given application, parameters of an RBF network such as centers and radii of basis functions or weights must be adapted. Typically, either stochastic, iterative training algorithms (e.g. gradient-based methods such as backpropagation or second-order techniques such as scaled conjugate gradients) or clustering methods in combination with a linear optimization technique (e.g. c-means and singular value decomposition for a linear least-squares problem) are used for this task. The article shows that a combination of the two approaches leads to significant improvements concerning the training time as well as the approximation and generalization properties of the networks. In the particular marketing application investigated here (prediction of customer behavior), the overall training time could be reduced compared to backpropagation and the prediction accuracy could be increased compared to c-means plus singular value decomposition. The article also describes a new idea for the initialization of basis function centers. Basically, this approach is a modification of the standard c-means algorithm that leads to a linear least-squares problem for which solvability can be guaranteed. This idea raises the reliability of the training procedure without additional costs regarding the run time as well as the quality of results.

In many applications neural networks must process or generate time series, and various network paradigms exist for this purpose. Two prominent examples are time-delay neural networks (TDNN), which are known for their noise suppression capability, and NARX (nonlinear autoregressive models with exogenous inputs) networks, which have a powerful modeling ability (at least Turing equivalence). In this article, we suggest a combination of these two approaches, called dynamic neural network (DYNN), which unifies the particular advantages. Efficient training algorithms are needed to adjust the weights of DYNN. Here, we describe an algorithm for the computation of first-order information about the error surface: temporal backpropagation through time (TBPTT). Essentially, this algorithm is a combination of temporal backpropagation (used for TDNN) and backpropagation through time (used for NARX). The first-order information is then utilized to apply the scaled conjugate gradient (SCG) learning algorithm which approximates second-order with first-order information. The benefits of this approach are shown by means of two benchmark data sets: "logistic map" and "building". It is shown that SCG for DYNN is significantly faster and more accurate than other learning algorithms (e.g. TBPTT, resilient propagation, memoryless Quasi-Newton).

Feature selection and architecture optimization are two key tasks in most neural network applications. Appropriate input features must be selected from a given (and often large) set of possible features and architecture parameters of the network such as the number of hidden neurons or learning parameters must be adapted with respect to the selected features and a learning data set. This article sets out an evolutionary algorithm (EA) that performs the tasks simultaneously for radial basis function (RBF) networks. The feasibility and the benefits of this approach are demonstrated in an application in the area of computer security: the detection of attacks (intrusive behavior) in computer networks. The EA, however, is independent from the application example given so that the ideas and solutions may easily be transferred to other applications and even other neural network paradigms. In the application example investigated overall classification rates of about 99.4% (average of eight attack types) can be reached for independent validation data.

In many practical applications of self-organizing maps (SOM, Kohonen Feature Maps), these networks are used for classification tasks. In order to be used for classification, the output neurons have to be assigned to classes which correspond to clusters in the input space (feature space) of the SOM (calibration). Usually, a SOM is calibrated after an unsupervised training, when clusters in the input space are represented by clusters in the weight space. The article presents two new calibration algorithms (one-point-algorithm and many-points-algorithm) which are based on statistical assumptions about the shape of the clusters in the weight space. These clusters are modeled by means of multivariate Gaussian distributions, where the unknown parameters of these distributions and the assignment of output neurons to classes (i. e. an appropriate partition of the output neurons) are determined using a maximum likelihood (ML) estimation. The properties of the two iterative algorithms, monotonic decrease with respect to the optimization criterion chosen and termination, are shown by means of benchmark data (PenDigits: classification of handwritten digits).

Neural networks often must process temporal information, i.e. any kind of information related to a time series. In many of these cases time series contain short-term and long-term information (e.g. trends or periodic behavior). The article presents a new approach which combines hard- and soft-computing techniques to capture information with various reference time windows simultaneously. A least-squares approximation of time series with orthogonal polynomials will be used to infer information about short-term information contained in a signal (average, increase, curvature, etc.). Long-term information will be modeled using the Dynamic Neural Network (DYNN) paradigm,. This network takes the coefficients of the orthogonal expansion of the approximating polynomial as inputs such considering short-term and long-term information efficiently. The advantages of the method will be demonstrated by means of two real-world application examples, the prediction of the user number in a PC-pool and online tool wear classification in turning.

In many neural network applications, the understanding of the network functionality is an important issue. Trained neural networks are often termed as "black boxes" which do not allow to get a deeper insight into the relationships between the input (feature) and output spaces. In the past years some researchers addressed the problem of rule extraction from trained neural networks. This article investigates the properties and results of different rule extraction techniques in a specific application area: The detection of intrusions in computer networks. This application area is chosen because intrusion detection and the development of appropriate intrusion detection systems (IDS) gains more and more importance with the rapidly increasing impact of the Internet.

Feature selection and architecture optimization are two key tasks in many neural network applications. Appropriate input features must be selected from a given (and often large) set of possible features and architecture parameters of the network such as the number of hidden neurons or training parameters must be adapted with respect to the selected features and a data set given. This article describes an evolutionary algorithm (EA) that performs the two tasks simultaneously for radial basis functions (RBF) networks applied to classification problems. In order to reduce the optimization effort significantly these soft-computing techniques are focused with various hard-computing techniques (e.g., clustering, solution of a linear least-squares problem, local search). The feasibility and the benefits of the approach are demonstrated by means of a data mining and knowledge discovery problem in the area of customer relationship management. The algorithm, however, is independent from specific applications such that the ideas and solutions may easily be transferred to other applications and even other neural network paradigms.

Indirect, online tool wear monitoring is one of the most difficult tasks in the context of process monitoring for metal-cutting machining processes. Based on a continuous acquisition of certain process parameters (signals such as cutting forces or acoustic emission) with multi-sensor systems, it is possible to estimate or to classify certain wear parameters. However, despite of intensive scientific research during the past decades, the development of reliable and flexible tool wear monitoring systems is an ongoing attempt. This article introduces a new, hybrid technique for tool wear monitoring in turning which fuses a physical process model (hard computing) with a neural network model (soft computing). The physical model describes the influence of cutting conditions (such as tool geometry or work material) on measured force signals and it is used to normalize these force signals. The neural model establishes a relationship between the normalized force signals and the wear state of the tool. The advantages of this approach are demonstrated by means of experimental results. Moreover, it is shown that the consideration of process parameters, cutting conditions, and wear in one model (either physical or neural) is extremely difficult and that existing hybrid approaches are not adequate. The ideas presented in this article can be transferred to many other process monitoring tasks.

Very often, recurrent neural networks are used to model dynamic, nonlinear relationships. However, particularly in many technical applications recurrent networks do not perform noticeably better than static networks (like e.g. Multilayer Perceptrons) processing only the current input pattern. The main reason for this observation is that the networks have to cope with noisy input and sometimes even noisy output data. Large weights in recurrent connections may cause stability problems and with small weights temporal information cannot be considered in an appropriate way. This paper demonstrates by means of an application example (tool wear monitoring in turning) that in these cases dynamic, non-recurrent paradigms like Time Delay neural networks should be preferred. These networks use tapped delay lines (delay elements in feedforward direction) to take past input information into account. Due to their noise suppression capabilities dynamic, non-recurrent networks can be superior to dynamic, recurrent paradigms like NARX-networks. The paper presents the first use of such a non-recurrent network in the mentioned application area and shows that the results of a wear estimation can be improved significantly.

A framework with a tool-supported high-level specification technique is very important for the development of complex signal processing applications containing software-intensive parts (e.g. hybrid systems in automated production processes) in order to provide safe and reliable systems. In this paper we present the concept of a framework, which is an object-oriented CASE-tool offering a graphical specification ability to model and validate a given application and to control its execution. A variety of people having different programming skills is able to use this visual specification technique effectively. Especially users not being interested in implementation details can specify their application on a high abstraction level by connecting reusable and reliable components (modules representing basic algorithms). As a result, complex signal graphs representing the dataflow between the modules are created. The tool supports this software specification technique by automatic type checking for the connections between modules and by changeable module parameters. On the other hand it is easy for software engineers to integrate additional signal processing algorithms into the framework thus building suitable module libraries without considering a specific high-level application.

Wear monitoring systems often use neural networks for a sensor fusion with multiple input patterns. Systems for a continuous online supervision of wear have to process pattern sequences. Therefore recurrent neural networks have been investigated in the past. However, in most cases where only noisy input or even noisy output patterns are available for a supervised learning, success is not forthcoming. That is, recurrent networks don't perform noticeably better than non-recurrent networks processing only the current input pattern like multilayer perceptrons. This paper demonstrates on the basis of an application example (online tool wear monitoring in turning) that results can be improved significantly with special non-recurrent feedforward networks. The approach uses time-delay neural networks which consider the position of a single pattern in a pattern sequence by means of delay elements at the synapses. In the mentioned application example, the average error in the estimation of a characteristic wear parameter could be reduced by about 24.2% compared with multilayer perceptrons.

Most of the existing Digital Signal Processors (DSPs) are optimized for a fast and efficient computation of the Fast Fourier Transform (FFT). However, there are only two floating-point DSPs available, which perform the butterfly operation of a FFT in 4 processor cycles, but no fixed-point DSP is designed that way. The new 24 bit fixed-point DSP DAISY, which is able to execute the butterfly in 4 cycles even using a two-stage pipelined multiplier, is described in this paper. With this pipelined multiplication it is possible to reduce the processor cycle time significantly.

One of the most important tasks of automatic tool monitoring systems for CNC-lathes is the supervision of a tool's wear. Considering the state of wear and the actual working process (e.g. rough or finish turning) it is possible to exchange a tool (or only the insert) just in time, which offers significant economic advantages. This paper presents a new method to estimate two wear parameters by means of artificial neural networks (multilayer perceptrons or time-delay neural networks). The input parameters of the networks are process-specific parameters (like the feed rate or the depth of cut) and characteristic coefficients extracted from signals measured with a multi-sensor system in the tool holder.