Classifier ensembles are mainly used for classification problems exhibiting complex data characteristics, e.g., a high class imbalance. They consist of a set of classifiers and a decision fusion method that combines the predictions of the classifiers. However, creating an accurate ensemble is a challenging optimization problem, since the individual classifiers and the decision fusion method must be selected from a huge search space that consists of numerous classification and fusion algorithms. Thereby, the selected classifiers have to be both accurate and diverse, so that they complement each other and make correct predictions on different data subsets. In addition, the selected decision fusion method has to accurately combine the predictions of individual classifiers into a consensus decision. Current literature focuses on AutoML approaches that reduce the complexity of the optimization problem by omitting the optimization of the classifier diversity and of the decision fusion. In this paper, we propose Auto-CEn as a novel ensemble approach based on AutoML that efficiently solves the optimization problem by selecting a set of accurate and diverse classifiers for the ensemble and by optimizing the decision fusion. In our extensive evaluation on 20 real-world datasets, we show that Auto-CEn outperforms several state-of-the-art baselines and that this improvement is mainly attributed to its novel contributions of diversity-based classifier selection and decision fusion optimization.

Combining two or more classifiers into an ensemble and fusing the individual classifier decisions to a consensus decision can improve the accuracy for a classification problem. The classification improvement of the fusion result depends on numerous factors, such as the data set, the combination scenario, the decision fusion algorithm, as well as the prediction accuracies and diversity of the multiple classifiers to be combined. Due to these factors, the best decision fusion algorithm for a given decision fusion problem cannot be generally determined in advance. In order to support the user in combining classifiers and to achieve the best possible fusion result, we propose the PUSION (Python Universal fuSION) framework, a novel generic and automated framework for decision fusion of classifiers. The framework includes 14 decision fusion algorithms and covers a total of eight different combination scenarios for both multi-class and multi-label classification problems. The introduced concept of AutoFusion detects the combination scenario for a given use case, automatically selects the applicable decision fusion algorithms and returns the decision fusion algorithm that leads to the best fusion result. The framework is evaluated with two real-world case studies in the field of fault diagnosis. In both case studies, the consensus decision of multiple classifiers and heterogeneous fault diagnosis methods significantly increased the overall classification accuracy. Our evaluation results show that our framework is of practical relevance and reliably finds the best performing decision fusion algorithm for a given combination task.

Abstract Real-world data of multi-class classification tasks often show complex data characteristics that lead to a reduced classification performance. Major analytical challenges are a high degree of multi-class imbalance within data and a heterogeneous feature space, which increases the number and complexity of class patterns. Existing solutions to classification or data pre-processing only address one of these two challenges in isolation. We propose a novel classification approach that explicitly addresses both challenges of multi-class imbalance and heterogeneous feature space together. As main contribution, this approach exploits domain knowledge in terms of a taxonomy to systematically prepare the training data. Based on an experimental evaluation on both real-world data and several synthetically generated data sets, we show that our approach outperforms any other classification technique in terms of accuracy. Furthermore, it entails considerable practical benefits in real-world use cases, e.g., it reduces rework required in the area of product quality control.

The adoption of machine learning (ML) in organizations is characterized by the use of multiple ML software components. Citizen data scientists face practical requirements when building ML systems, which go beyond the known challenges of ML, e.g., data engineering or parameter optimization. They are expected to quickly identify ML system options that strike a suitable trade-off across multiple performance criteria. These options also need to be understandable for non-technical users. Addressing these practical requirements represents a problem for citizen data scientists with limited ML experience. This calls for a method to help them identify suitable ML software combinations. Related work, e.g., AutoML systems, are not responsive enough or cannot balance different performance criteria. In this paper, we introduce AssistML, a novel concept to recommend ML solutions, i.e., software systems with ML models, for predictive use cases. AssistML uses metadata of existing ML solutions to quickly identify and explain options for a new use case. We implement the approach and evaluate it with two exemplary use cases. Results show that AssistML proposes ML solutions that are in line with users' performance preferences in seconds.

Classification techniques are increasingly adopted for quality control in manufacturing, e.g., to help domain experts identify the cause of quality issues of defective products. However, real-world data often imply a set of analytical challenges, which lead to a reduced classification performance. Major challenges are a high degree of multi-class imbalance within data and a heterogeneous feature space that arises from the variety of underlying products. This paper considers such a challenging use case in the area of End-of-Line testing, i.e., the final functional test of complex products. Existing solutions to classification or data pre-processing only address individual analytical challenges in isolation. We propose a novel classification system that explicitly addresses both challenges of multi-class imbalance and a heterogeneous feature space together. As main contribution, this system exploits domain knowledge to systematically prepare the training data. Based on an experimental evaluation on real-world data, we show that our classification system outperforms any other classification technique in terms of accuracy. Furthermore, we can reduce the amount of rework required to solve a quality issue of a product.

Machine learning approaches may support various use cases in the manufacturing industry. However, these approaches often do not address the inherent characteristics of the real manufacturing data at hand. In fact, real data impose analytical challenges that have a strong influence on the performance and suitability of machine learning methods. This paper considers such a challenging use case in the area of End-of-Line testing, i.e., the final functional check of complex products after the whole assembly line. Here, classification approaches may be used to support quality engineers in identifying faulty components of defective products. For this, we discuss relevant data sources and their characteristics, and we derive the resulting analytical challenges. We have identified a set of sophisticated data-driven methods that may be suitable to our use case at first glance, e.g., methods based on ensemble learning or sampling. The major contribution of this paper is a thorough comparative study of these methods to identify whether they are able to cope with the analytical challenges. This comprises the discussion of both fundamental theoretical aspects and major results of detailed experiments we have performed on the real data of our use case.

Today, a multitude of highly-connected applications and information systems hold, consume and produce huge amounts of heterogeneous data. The overall amount of data is even expected to dramatically increase in the future. In order to conduct, e.g., data analysis, visualizations or other value-adding scenarios, it is necessary to integrate specific, relevant parts of data into a common source. Due to oftentimes changing environments and dynamic requests, this integration has to support ad-hoc and flexible data processing capabilities. Furthermore, an iterative and explorative trial-and-error integration based on different data sources has to be possible. To cope with these requirements, several data mashup platforms have been developed in the past. However, existing solutions are mostly non-extensible, monolithic systems or applications with many limitations regarding the mentioned requirements. In this paper, we introduce an approach that copes with these issues (i) by the introduction of patterns to enable decoupling from implementation details, (ii) by a cloud-ready approach to enable availability and scalability, and (iii) by a high degree of flexibility and extensibility that enables the integration of heterogeneous data as well as dynamic (un-)tethering of data sources. We evaluate our approach using runtime measurements of our prototypical implementation.

Scientific workflows have to deal with highly heterogeneous data environments. In particular, they have to carry out complex data provisioning tasks that filter and transform heterogeneous input data in such a way that underlying tools or services can ingest them. This results in a high complexity of workflow design. Scientists often want to design their workflows on their own, but usually do not have the necessary skills to cope with this complexity. Therefore, we have developed a pattern-based approach to workflow design, thereby mainly focusing on workflows that realize numeric simulations [4]. This approach removes the burden from scientists to specify low-level details of data provisioning. In this demonstration, we apply a prototype implementation of our approach to various use cases and show how it makes simulation workflow design tailor-made for scientists.

Scientific workflows often have to process huge data sets in a multiplicity of data formats. For that purpose, they typically embed complex data provisioning tasks that transform these heterogeneous data into formats the underlying tools or services can handle. This results in an increased complexity of workflow design. As scientists typically design their scientific workflows on their own, this complexity hinders them to concentrate on their core issue, namely the experiments, analyses, or simulations they conduct. In this paper, we present the core idea of a pattern-based approach to alleviate the design of scientific workflows. This approach is particularly targeted at the needs of scientists. We exemplify and assess the pattern-based design approach by applying it to a complex scientific workflow realizing a real-world simulation of structure changes in bones.