Classifier ensembles are mainly used for classification problems exhibiting complex data characteristics, e.g., a high class imbalance. They consist of a set of classifiers and a decision fusion method that combines the predictions of the classifiers. However, creating an accurate ensemble is a challenging optimization problem, since the individual classifiers and the decision fusion method must be selected from a huge search space that consists of numerous classification and fusion algorithms. Thereby, the selected classifiers have to be both accurate and diverse, so that they complement each other and make correct predictions on different data subsets. In addition, the selected decision fusion method has to accurately combine the predictions of individual classifiers into a consensus decision. Current literature focuses on AutoML approaches that reduce the complexity of the optimization problem by omitting the optimization of the classifier diversity and of the decision fusion. In this paper, we propose Auto-CEn as a novel ensemble approach based on AutoML that efficiently solves the optimization problem by selecting a set of accurate and diverse classifiers for the ensemble and by optimizing the decision fusion. In our extensive evaluation on 20 real-world datasets, we show that Auto-CEn outperforms several state-of-the-art baselines and that this improvement is mainly attributed to its novel contributions of diversity-based classifier selection and decision fusion optimization.

Efficient clustering algorithms, such as k -Means, are often used in practice because they scale well for large datasets. However, they are only able to detect simple data characteristics. Ensemble clustering can overcome this limitation by combining multiple results of efficient algorithms. However, analysts face several challenges when applying ensemble clustering, i. e., analysts struggle to (a) efficiently generate an ensemble and (b) combine the ensemble using a suitable consensus function with a corresponding hyperparameter setting. In this paper, we propose EffEns, an efficient ensemble clustering approach to address these challenges. Our approach relies on meta-learning to learn about dataset characteristics and the correlation between generated base clusterings and the performance of consensus functions. We apply the learned knowledge to generate appropriate ensembles and select a suitable consensus function to combine their results. Further, we use a state-of-the-art optimization technique to tune the hyperparameters of the selected consensus function. Our comprehensive evaluation on synthetic and real-world datasets demonstrates that EffEns significantly outperforms state-of-the-art approaches w.r.t. accuracy and runtime.

Analysts often struggle with the combined algorithm selection and hyperparameter optimization problem, a.k.a. CASH problem in literature. Typically, they execute several algorithms with varying hyperparameter settings to find configurations that show valuable results. Efficiently finding these configurations is a major challenge. In clustering analyses, analysts face the additional challenge to select a cluster validity index that allows them to evaluate clustering results in a purely unsupervised fashion. Many different cluster validity indices exist and each one has its benefits depending on the dataset characteristics. While experienced analysts might address these challenges using their domain knowledge and experience, especially novice analysts struggle with them. In this paper, we propose a new meta-learning approach to address these challenges. Our approach uses knowledge from past clustering evaluations to apply strategies that experienced analysts would exploit. In particular, we use meta-learning to (a) select a suitable clustering validity index, (b) efficiently select well-performing clustering algorithm and hyperparameter configurations, and (c) reduce the search space to suitable clustering algorithms. In the evaluation, we show that our approach significantly outperforms state-of-the-art approaches regarding accuracy and runtime.

Abstract Real-world data of multi-class classification tasks often show complex data characteristics that lead to a reduced classification performance. Major analytical challenges are a high degree of multi-class imbalance within data and a heterogeneous feature space, which increases the number and complexity of class patterns. Existing solutions to classification or data pre-processing only address one of these two challenges in isolation. We propose a novel classification approach that explicitly addresses both challenges of multi-class imbalance and heterogeneous feature space together. As main contribution, this approach exploits domain knowledge in terms of a taxonomy to systematically prepare the training data. Based on an experimental evaluation on both real-world data and several synthetically generated data sets, we show that our approach outperforms any other classification technique in terms of accuracy. Furthermore, it entails considerable practical benefits in real-world use cases, e.g., it reduces rework required in the area of product quality control.

Abstract Clustering is a fundamental primitive in manifold applications. In order to achieve valuable results in exploratory clustering analyses, parameters of the clustering algorithm have to be set appropriately, which is a tremendous pitfall. We observe multiple challenges for large-scale exploration processes. On the one hand, they require specific methods to efficiently explore large parameter search spaces. On the other hand, they often exhibit large runtimes, in particular when large datasets are analyzed using clustering algorithms with super-polynomial runtimes, which repeatedly need to be executed within exploratory clustering analyses. We address these challenges as follows: First, we present LOG-Means and show that it provides estimates for the number of clusters in sublinear time regarding the defined search space, i.e., provably requiring less executions of a clustering algorithm than existing methods. Second, we demonstrate how to exploit fundamental characteristics of exploratory clustering analyses in order to significantly accelerate the (repetitive) execution of clustering algorithms on large datasets. Third, we show how these challenges can be tackled at the same time. To the best of our knowledge, this is the first work which simultaneously addresses the above-mentioned challenges. In our comprehensive evaluation, we unveil that our proposed methods significantly outperform state-of-the-art methods, thus especially supporting novice analysts for exploratory clustering analyses in large-scale exploration processes.

\n Data analysis is a highly iterative process. In order to achieve\n valuable analysis results, analysts typically execute many\n configurations, i.e., algorithms and their hyperparameter settings,\n based on their domain knowledge. While experienced analysts may be\n able to define small search spaces for promising configurations,\n especially novice analysts define large search spaces due to their\n lack of domain knowledge. In the worst case, they perform an\n exhaustive search throughout the whole search space, resulting in\n infeasible runtimes. Recent advances in the research area of AutoML\n address this challenge by supporting novice analysts in the combined\n algorithm selection and hyperparameter optimization (CASH) problem\n for supervised learning tasks. However, no such systems exist for\n unsupervised learning tasks, such as the prevalent task of\n clustering analysis. In this work, we present our novel AutoML4Clust\n approach, which efficiently supports novice analysts regarding CASH\n for clustering analyses. To the best of our knowledge, this is the\n first thoroughly elaborated approach in this area. Our comprehensive\n evaluation unveils that AutoML4Clust significantly outperforms\n several existing approaches, as it achieves considerable speedups\n for the CASH problem, while still achieving very valuable clustering\n results.\n