In certain subfields of machine learning, such as those involving homomorphic encryption or quantum computing, it is crucial to estimate the numerical precision of data used for training without compromising model quality.This paper introduces a method for precisely quantifying the loss of accuracy in distance-based classifiers, such as Generalized Learning Vector Quantization, when operating on quantized data represented by bounded integer sets.Our approach employs conditional entropy to measure the information loss induced by quantization, which closely correlates with the model's mean performance.

Training of machine learning models for classification frequently yields several different solutions although the performance remains approximately the same, i.e. one observes many close-to-optimum solutions with only marginal performance differences which are, however, qualitatively well-distinguishable.This behaviour is known as the Rashomon effect and may be dedicated to the stochastic in the training process, different learning strategies or various initial settings.Hence, model explanations may become difficult and have to be related to a given configuration.Therefore, stable and plausible explanations are required based on the evaluation of the Rashomon set.Yet, the consistency of the resulting explanations remained largely unexplored so far.Here we propose to evaluate the Rashomon set qualitatively by means of a cluster analysis based on the determination of the feature importance.Feature importance of a model gives insights about the decision making process and, hence, provides an appropriate criterion to distinguish model decision realizations.Clustering of them reveal stable and plausible classification strategies and, hence, contribute to reliable explanations.

Counterfactuals are useful objects to explain decisions of machine learning classifiers.In the best case, counterfactuals can provide to derive causal inference structure realized by the model.Yet, counterfactual generation in general is known as a constrained optimization problem.In this contribution we demonstrate that counterfactuals can be determined geometric-analytically in case of prototype based classifiers.For this we only require that nearest prototype classification is based on norms induced by an inner product, which has to be applied for consistency also to evaluate the deviation between a given sample and a desired counterfactual class.* M.K. and L.R. are supported by

Machine learning classifiers adjust implicitly or explicitly the importance of the data features to solve a given classification task.This feature weighting often does not imply feature sparseness, which, however, may be important for interpretability and model evaluation.This contribution proposes how to force feature sparseness in combination with feature relevance for prototype-based classification learning to obtain reliable and interpretable classification decisions.

Evaluating domination reliability-a network reliability measure related to service networks-is a computationally expensive task, due to its proven NP-hardness.To address this challenge, we propose an interpretable prototype-based classification approach that predicts domination reliability levels from selected graph features using Generalized Matrix Learning Vector Quantization (GMLVQ) with a particular focus on how these graph features influence the predicted reliability levels.The interpretability is enhanced by a physically motivated visualization of an associated threshold graph, which is derived from the learned relevance matrix.

Learning prototypes in an unsupervised manner that respects the data density and topology is crucial for tasks such as clustering, representation learning, and visualization of high-dimensional datasets.In this paper, we propose a generalization of the Neural Gas algorithm to Riemannian manifolds, leveraging geodesic distances for prototype adaptation.The approach additionally generates a prototype neighborhood structure, enabling faithful approximation of both geometry and topology of data distributed on Riemannian manifolds.We demonstrate its effectiveness on real-world datasets from manifolds such as SO(n), S n ++ and Gr(n, k) and compare our approach to Riemannian versions of other related methods such as K-Means, K-Medoids and a Riemannian Self-Organizing Map.

Developing fair classification models is a crucial aspect of machine learning research. However, unintended distortion in training data - biased data - can lead to discriminatory decisions. In this paper, we developed a workflow for detecting and mitigating bias in data using a shallow, interpretable machine learning models: the Generalized Matrix Learning Vector Quantization. We extent the approach by a relevance-based analysis to identify and reduce bias in the data. Combining similarity metric adaptation and relevance-based analysis, we can develop fair classification models that minimize the influence of bias in the data. Our results demonstrate that this method is effective in reducing bias in classification models and therefore supports fair decision-making.

Privacy-Preserving Machine Learning has become an important field in the age of Big Data and AI Hype . Methods like Differential Privacy and Homomorphic Encryption (HE) became key ideas to preserve privacy and to counter well-known attacks. Yet, practice shows, that both approaches are not without pitfalls. HE suffers under severe computational overhead, which makes the training of larger Machine Learning models as encrypted circuit not feasible. However, training shallow or sparse networks, like Prototype-Based Models may be realized. In this work, we present a proof-of-concept for the realization of Learning Vector Quantization - 1 (LVQ-1) - a shallow Nearest Prototype Classifier (NPC) - as an encrypted circuit by using TFHE as the encryption scheme. Our results indicate, that the feasibility is influenced by the dimensionality of the dataset and its respective encoding, but also that both, feasibility and performance, depend on the chosen distance function. Beyond our practical work, we provide an overview of TFHE and how LVQ may violate privacy.

Prototype-based classification learning methods are known to be inherently interpretable. However, this paradigm suffers from major limitations compared to deep models, such as lower performance. This led to the development of the so-called deep Prototype-Based Networks (PBNs), also known as prototypical parts models. In this work, we analyze these models with respect to different properties, including interpretability. In particular, we focus on the Classification-by-Components (CBC) approach, which uses a probabilistic model to ensure interpretability and can be used as a shallow or deep architecture. We show that this model has several shortcomings, like creating contradicting explanations. Based on these findings, we propose an extension of CBC that solves these issues. Moreover, we prove that this extension has robustness guarantees and derive a loss that optimizes robustness. Additionally, our analysis shows that most (deep) PBNs are related to (deep) RBF classifiers, which implies that our robustness guarantees generalize to shallow RBF classifiers. The empirical evaluation demonstrates that our deep PBN yields state-of-the-art classification accuracy on different benchmarks while resolving the interpretability shortcomings of other approaches. Further, our shallow PBN variant outperforms other shallow PBNs while being inherently interpretable and exhibiting provable robustness guarantees.

With federated learning scenarios gaining popularity to outsource computational heavy tasks or to increase generalizability of machine learning models, there is also a rise of research in terms of the security and privacy of the respective data used for these tasks.While differential privacy is studied well for Learning Vector Quantization, we want to present steps towards Homomorphic Encryption.In this regard, we will show theoretically how LVQ-1 can be adapted to be compatible with the TFHE encryption scheme and present experimental results.

We propose a bias detection and mitigating scheme for data in the context of classification tasks based on learning vector quantizers (LVQ) as classifier.For this purpose generalized LVQ endowed with an advanced matrix adaptation scheme is used for bias detection.The bias removal from data is realized applying a nullspace data projection using the adjusted matrix.The usefulness of the approach is demonstrated and illustrated in terms of two real world datasets.* M.K. is

An interpretable approach to classification learning using cross-entropy loss is the Probabilistic Learning Vector Quantizer (PLVQ) as a robust prototype-based classifier.We propose a variant of the PLVQ, that allows the integration of domain knowledge.This strategy is becoming increasingly popular as a means of developing intelligent models that can enhance performance and gain acceptance from domain experts.In this paper, we put forth the idea of incorporating externally known class relations as supplementary information.We present theoretical aspects of the model and demonstrate its capabilities through numerical experiments.

We present a new approach for the assessment of the reliability of coherent systems by using a prototype-based classification method.More specifically, reliability levels for consecutive k-out-of-n systems, which serve as a model for a particular type of networks, are classified using Generalized Matrix Learning Vector Quantization, which provides useful information about the impact of the input probabilities on the classified reliability levels.Our approach is not limited to reliability analysis, but is generally applicable for estimating the probability of the union of any finite family of events, based on their individual and pairwise probabilities.

Knowledge integration into machine learning systems is a promising and successful strategy to achieve more plausible and consistent results.The plausibility is accompanied by better model interpretability due to the adjustment of the machine learning system to the domain specic requirements and restrictions.Further, informed machine learning can be seen as a particular task specic regularization of the model leading to better learning convergence and frequently also requiring a lower amount of training data.This short introduction paper addresses some recent aspects, how domain knowledge can be integrated into learning systems on dierent levels ranging from informed feature extraction to domain adjusted structure and model architecture.* M.K. is supported by the IAI-

In this work, we will consider how privacy for vector quantization models can be broken in a federated learning environment.We show how a potential attacker can expose data from the prototype updates without needing to know about the specific model used by exploiting the transparency of vector quantization.Finally, a 1-user environment example based on GLVQ will be shown.* R. S. is supported by grants of the

For an autonomous vehicle to drive safely and efficiently, it is important to have a good understanding of the surrounding environment. Having early information about other road users can enable the vehicle to anticipate their behavior and take appropriate actions to avoid dangerous situations. Humans often use light reflections caused by oncoming vehicles at night to anticipate their appearance before they are directly visible. This problem of provident vehicle detection at night has already been studied in rural land road scenarios. However, urban scenarios are more complex, as the number of light sources is higher and the light reflections are more complex. In this paper, we therefore present the PVDN-urban dataset, which is the first dataset to study provident vehicle detection at night in urban scenarios. The dataset contains detailed annotations of light reflections caused by oncoming vehicles at night in urban scenarios. Also, it provides bounding box annotations for all vehicles, to make the dataset usable also for conventional vehicle detection tasks under low-light conditions. We provide an in-depth analysis of the dataset, an efficient annotation method for light reflections, as well as baseline results using state-of-the-art semantic segmentation models. With that, we provide the basis to further study provident vehicle detection at night also for complex urban scenarios. Furthermore, we provide a dataset for the development of algorithms for general vehicle detection at night in urban scenarios.

In machine learning, data often comes from different sources, but combining them can introduce extraneous variation that affects both generalization and interpretability. For example, we investigate the classification of neurodegenerative diseases using FDG-PET data collected from multiple neuroimaging centers. However, data collected at different centers introduces unwanted variation due to differences in scanners, scanning protocols, and processing methods. To address this issue, we propose a two-step approach to limit the influence of center-dependent variation on the classification of healthy controls and early vs. late-stage Parkinson's disease patients. First, we train a Generalized Matrix Learning Vector Quantization (GMLVQ) model on healthy control data to identify a "relevance space" that distinguishes between centers. Second, we use this space to construct a correction matrix that restricts a second GMLVQ system's training on the diagnostic problem. We evaluate the effectiveness of this approach on the real-world multi-center datasets and simulated artificial dataset. Our results demonstrate that the approach produces machine learning systems with reduced bias - being more specific due to eliminating information related to center differences during the training process - and more informative relevance profiles that can be interpreted by medical experts. This method can be adapted to similar problems outside the neuroimaging domain, as long as an appropriate "relevance space" can be identified to construct the correction matrix.

This paper is an effort to parameterize Information Bottle-neck Theory to become a supervised classifier.We introduce a parametrization by means of Learning Vector Quantization.With this new approach, one can find suitable components that are necessary for an accurate, yet efficient, classification.A balance between compression and representation is made by means of a specially designed objective function.

Quantum Computing Research proposed strategies to solve binary optimization problems.Application on current and near-term generation Hardware is possible.Even if computational benefits of the strategies are yet to be shown, we want to explore connections to prototype learning schemes.We examine cost functions for vector quantization based on data point selection and how they can be transformed into a common quadratic unconstrained binary optimization formulation (QUBO).There are different approaches for solving QUBO problems using quantum computer or quantum annealer hardware.We look at their current limits and how they might change.

Artificial Intelligence (AI), a discipline with decades of history, is living its golden era due to striking developments that solve problems that were unthinkable just a few years ago, like generative models of text, images and video.The broad range of AI applications has also arrived to Physics, providing solutions to bottleneck situations, e.g., numerical methods that could not solve certain problems or took an extremely long time, optimization of quantum experimentation, or qubit control.Besides, Quantum Computing has become extremely popular for speeding up AI calculations, especially in the case of data-driven AI, i.e., Machine Learning (ML).The term Quantum ML is already known and deals with learning in quantum computers or quantum annealers, quantum versions of classical ML models and different learning approaches for quantum measurement and control.Quantum AI (QAI) tries to take a step forward in order to come up with disruptive concepts, such as, human-quantum-computer interfaces, sentiment analysis in quantum computers or explainability of quantum computing calculations, to name a few.This special session includes five high-quality papers on relevant topics, like quantum reinforcement learning, parallelization of quantum calculations, quantum feature selection and quantum vector quantization, thus capturing the richness and variability of approaches within QAI.

We introduce an efficient approach for supervised landmark selection in sparse Nyström approximation of kernel matrices.Our method converts structured non-vectorial input data such as graphs or text into a vectorial dissimilarity representation, enabling class-informed landmark identification through prototype-based learning.Experimental results show competitive approximation quality compared to existing strategies and demonstrate the positive effect of integrating class information into the selection process of Nyström landmarks making our approach an efficient and versatile solution for large-scale kernel learning.

Approximation problems, and thus regression problems, have been widely considered as machine learning problems.A popular model to tackle such tasks are radial-basis-function networks (RBFN) and variants thereof.However, due to the global approximation scheme, RBFN, when trained in a supervised manner without additional constraints, may lack local representation.To this end, we propose approaches that aim to preserve locality in terms of the regression problem by using the Neural Gas algorithm.The models are tested on different data sets and compared to the supervised RBFN approach.* R. S. is supported by

The encounter of large amounts of biological sequence data generated during the last decades and the algorithmic and hardware improvements have offered the possibility to apply machine learning techniques in bioinformatics. While the machine learning community is aware of the necessity to rigorously distinguish data transformation from data comparison and adopt reasonable combinations thereof, this awareness is often lacking in the field of comparative sequence analysis. With realization of the disadvantages of alignments for sequence comparison, some typical applications use more and more so-called alignment-free approaches. In light of this development, we present a conceptual framework for alignment-free sequence comparison, which highlights the delineation of: 1) the sequence data transformation comprising of adequate mathematical sequence coding and feature generation, from 2) the subsequent (dis-)similarity evaluation of the transformed data by means of problem-specific but mathematically consistent proximity measures. We consider coding to be an information-loss free data transformation in order to get an appropriate representation, whereas feature generation is inevitably information-lossy with the intention to extract just the task-relevant information. This distinction sheds light on the plethora of methods available and assists in identifying suitable methods in machine learning and data analysis to compare the sequences under these premises.

We present an approach for generating vectorial representations of graphs for machine learning applications based on a sensoric response principle and multiple graph kernels.The sensor perspective reduces the graph kernel computations significantly.Thus, multiple kernel (relevance) learning can be realized using the interpretable generalized matrix learning vector quantization (GMLVQ) classifier.Results obtained in small molecule classification serve as proof of concept.* K.S.B and M.K. are supported by a grant of the European Social

A short introduction to the application of informationtheoretic and machine learning methods to biomolecular and medical data is provided as the motivating material that supports special session dedicated to this topic at ESANN 2022.In particular, we highlight current developments of foundation such as interpretability and model certainty.Further, we emphasize how theoretic models provide a natural framework to deal with heterogeneous and complex data structures as frequently occurring in biomedical research.

One-class-classification remains an important problem in machine learning, which is related to data representation and outlier detection, but different from them in several aspects. In the present contribution we propose an one-class-classifier based on a prototype vector quantization model. We modeled a corresponding cost function to account for aspects of representation learning and to appropriately evaluate the one-class classifier. The prototype-based model ensures a local representation of the target class. After this introduction, we obtain an interpretable one-class classifier model. We demonstrate the capabilities of the approach by applying the classifier to illustrative toy data examples as well as on real data in a medical context.

We present an approach to discriminate SARS-CoV-2 virus types based on their RNA sequence descriptions avoiding a sequence alignment. For that purpose, sequences are preprocessed by feature extraction and the resulting feature vectors are analyzed by prototype-based classification to remain interpretable. In particular, we propose to use variants of learning vector quantization (LVQ) based on dissimilarity measures for RNA sequence data. The respective matrix LVQ provides additional knowledge about the classification decisions like discriminant feature correlations and, additionally, can be equipped with easy to realize reject options for uncertain data. Those options provide self-controlled evidence, i.e., the model refuses to make a classification decision if the model evidence for the presented data is not sufficient. This model is first trained using a GISAID dataset with given virus types detected according to the molecular differences in coronavirus populations by phylogenetic tree clustering. In a second step, we apply the trained model to another but unlabeled SARS-CoV-2 virus dataset. For these data, we can either assign a virus type to the sequences or reject atypical samples. Those rejected sequences allow to speculate about new virus types with respect to nucleotide base mutations in the viral sequences. Moreover, this rejection analysis improves model robustness. Last but not least, the presented approach has lower computational complexity compared to methods based on (multiple) sequence alignment. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s00521-021-06018-2.

Abstract Prototype-based models like the Generalized Learning Vector Quantization (GLVQ) belong to the class of interpretable classifiers. Moreover, quantum-inspired methods get more and more into focus in machine learning due to its potential efficient computing. Further, its interesting mathematical perspectives offer new ideas for alternative learning scenarios. This paper proposes a quantum computing-inspired variant of the prototype-based GLVQ for classification learning. We start considering kernelized GLVQ with real- and complex-valued kernels and their respective feature mapping. Thereafter, we explain how quantum space ideas could be integrated into a GLVQ using quantum bit vector space in the quantum state space $${\mathcal {H}}^{n}$$ H n and show the relations to kernelized GLVQ. In particular, we explain the related feature mapping of data into the quantum state space $${\mathcal {H}}^{n}$$ H n . A key feature for this approach is that $${\mathcal {H}}^{n}$$ H n is an Hilbert space with particular inner product properties, which finally restrict the prototype adaptations to be unitary transformations. The resulting approach is denoted as Qu-GLVQ. We provide the mathematical framework and give exemplary numerical results.

In this paper we present a realization of the informationbottleneck-paradigm by means of an improved counter propagation network.It combines an unsupervised vector quantizer for data compression with a subsequent supervised learning vector quantization model.The approach is mathematically justified and yields an interpretable model for classification under the constraint of data compression, which is not longer independently learned from the classification task.* M.K., M.

Machine learning-based systems are now part of a wide array of real-world applications seamlessly embedded in the social realm.In the wake of this realisation, strict legal regulations for these systems are currently being developed, addressing some of the risks they may pose.This is the coming of age of the interpretability and explainability problems in machine learning-based data analysis, which can no longer be seen just as an academic research problem.In this tutorial, associated to ESANN 2021 special session on "Interpretable Models in Machine Learning and Explainable Artificial Intelligence", we discuss explainable and interpretable machine learning as post-hoc and ante-hoc strategies to address these problems and highlight several aspects related to them, including their assessment.The contributions accepted for the session are then presented in this context.* A.V. is supported by Spanish

Learning Vector Quantizers (LVQ) and its cost-functionbased variant called Generalized Learning Vector Quanitzation (GLVQ) are powerful, yet simple and interpretable classification models.Even though GLVQ is an effective tool for classifying vectorial data, it cannot handle raw sequence data of potentially different lengths.Usually, this problem is solved by manually engineering fixed-length features or by employing recurrent networks.Therefore, a natural idea is to incorporate recurrent units for data processing into the GLVQ network structure.The processed data can then be compared in a latent space for classification decisions.We demonstrate the ability of this approach on illustrative classification problems.* M.K. and J.R. are

MOTIVATION: Viruses are the most abundant biological entities and constitute a large reservoir of genetic diversity. In recent years, knowledge about them has increased significantly as a result of dynamic development in life sciences and rapid technological progress. This knowledge is scattered across various data repositories, making a comprehensive analysis of viral data difficult. RESULTS: In response to the need for gathering a comprehensive knowledge of viruses and viral sequences, we developed Virxicon, a lexicon of all experimentally acquired sequences for RNA and DNA viruses. The ability to quickly obtain data for entire viral groups, searching sequences by levels of taxonomic hierarchy-according to the Baltimore classification and ICTV taxonomy-and tracking the distribution of viral data and its growth over time are unique features of our database compared to the other tools. AVAILABILITYAND IMPLEMENTATION: Virxicon is a publicly available resource, updated weekly. It has an intuitive web interface and can be freely accessed at http://virxicon.cs.put.poznan.pl/.

Methods for adversarial robustness certification aim to provide an upper bound on the test error of a classifier under adversarial manipulation of its input. Current certification methods are computationally expensive and limited to attacks that optimize the manipulation with respect to a norm. We overcome these limitations by investigating the robustness properties of Nearest Prototype Classifiers (NPCs) like learning vector quantization and large margin nearest neighbor. For this purpose, we study the hypothesis margin. We prove that if NPCs use a dissimilarity measure induced by a seminorm, the hypothesis margin is a tight lower bound on the size of adversarial attacks and can be calculated in constant time—this provides the first adversarial robustness certificate calculable in reasonable time. Finally, we show that each NPC trained by a triplet loss maximizes the hypothesis margin and is therefore optimized for adversarial robustness. In the presented evaluation, we demonstrate that NPCs optimized for adversarial robustness are competitive with state-of-the-art methods and set a new benchmark with respect to computational complexity for robustness certification.

Abstract In this contribution, we consider the classification of time series and similar functional data which can be represented in complex Fourier and wavelet coefficient space. We apply versions of learning vector quantization (LVQ) which are suitable for complex-valued data, based on the so-called Wirtinger calculus. It allows for the formulation of gradient-based update rules in the framework of cost-function-based generalized matrix relevance LVQ (GMLVQ). Alternatively, we consider the concatenation of real and imaginary parts of Fourier coefficients in a real-valued feature vector and the classification of time-domain representations by means of conventional GMLVQ. In addition, we consider the application of the method in combination with wavelet-space features to heartbeat classification.

Abstract Neural networks are state-of-the-art classification approaches but are generally difficult to interpret. This issue can be partly alleviated by constructing a precise decision process within the neural network. In this work, a network architecture, denoted as Classification-By-Components network (CBC), is proposed. It is restricted to follow an intuitive reasoning based decision process inspired by Biederman's recognition-by-components theory from cognitive psychology. The network is trained to learn and detect generic components that characterize objects. In parallel, a class-wise reasoning strategy based on these components is learned to solve the classification problem. In contrast to other work on reasoning, we propose three different types of reasoning: positive, negative, and indefinite. These three types together form a probability space to provide a probabilistic classifier. The decomposition of objects into generic components combined with the probabilistic reasoning provides by design a clear interpretation of the classification decision process. The evaluation of the approach on MNIST shows that CBCs are viable classifiers. Additionally, we demonstrate that the inherent interpretability offers a profound understanding of the classification behavior such that we can explain the success of an adversarial attack. The method's scalability is successfully tested using the ImageNet dataset.

The paper deals with realizations of transfer learning for classification, i. e. the adaptation of a classifier model to a changed data distribution. This change could be a data drift or a more complex transformation. We propose to model those data changes by manifolds describing continuous transformations of the data. This description can be seen as a generalization of function based transfer models considered so far. The manifold description of the transfer function allows either to adjust the classifier model to the changed data distribution or a back-transformation of those data to the original data space. To get the approach feasible, the manifold is approximated by the affine part of the Taylor expansion of the manifold structure. Moreover, the affine approximation shows mathematical correspondences to tangent metric leaning, which was developed for handling of data with drifts in classification methods. The paper provides the mathematical background for manifold based transfer data learning. Further, the approach is exemplarily applied for the generalized learning vector quantization classifier. This classifier is a prominent method which frequently achieves a high performance and a robust behavior while the classifier complexity is low compared to more sophisticated approaches like deep learning architectures or support vector machines. Moreover, the good interpretability of learning vector quantization classifiers also contributes to an intuitive practical understanding of transfer learning.

We propose a learning vector quantization algorithm variant for prototype-based classification learning with adaptive tangent distance learning. Tangent distances were developed to achieve dissimilarity measures invariant with respect to transformations and distortions like rotation, noise, etc.. Usually, these tangent distances are predefined in applications or are estimated in preprocessing. We introduce in this paper a generalized learning vector quantizer (GLVQ) with an online adaptation scheme for tangent distances. The adaptation takes place as a stochastic gradient descent learning accompanying the usual online prototype learning. In this way, class discriminative tangents are learned contributing to a better classification performance. Further, the resulting update schemes can be seen as a special type of local matrix learning in GLVQ. In this paper, we provide the full mathematical theory behind the derived tangent distance adaptation rule and demonstrate the classification ability of the resulting GLVQ model in comparison to state-of-the-art tangent distance based classifiers in the field.

We present a theoretical analysis of Learning Vector Quantization (LVQ) with adaptive distance measures. Specifically, we consider generalized Euclidean distances which are parameterized in terms of a quadratic matrix of adaptive relevance parameters. Winner-takes-all prescriptions based on the heuristic LVQ1 are in the center of our interest. We derive and study stationarity conditions and show, among other results, that stationary prototypes can be written as linear combinations of the training data apart from irrelevant contributions in the null-space of the relevance matrix. The investigation of the metrics updates reveals that relevance matrices become singular with only one or very few non-zero eigenvalues. Implications of this property are discussed and, furthermore, the effect of preventing singularity by introducing an appropriate penalty term is studied. Theoretical findings are confirmed in terms of illustrative example data sets.

Classification and decision systems in data analysis are mostly based on accuracy optimization. This criterion is only a conditional informative value if the data are imbalanced or false positive/negative decisions cause different costs. Therefore more sophisticated statistical quality measures are favored in medicine, like precision, recall etc‥ Otherwise, most classification approaches in machine learning are designed for accuracy optimization. In this paper we consider variants of learning vector quantizers (LVQs) explicitly optimizing those advanced statistical quality measures while keeping the basic intuitive ingredients of these classifiers, which are the prototype based principle and the Hebbian learning. In particular we focus in this contribution particularly to precision and recall as important measures for use in medical applications. We investigate these problems in terms of precision-recall curves as well as receiver-operating characteristic (ROC) curves well-known in statistical classification and test analysis. With the underlying more general framework, we provide a principled alternatives traditional classifiers, such that a closer connection to statistical classification analysis can be drawn.

Linear data transformations are essential operations in many machine learning algorithms, helping to make such models more flexible or to emphasize certain data directions. In particular for high dimensional data sets linear transformations are not necessarily uniquely determined, though, and alternative parameterizations exist which do not change the mapping of the training data. Thus, regularization is required to make the model robust to noise and more interpretable for the user. In this contribution, we characterize the group of transformations which leave a linear mapping invariant for a given finite data set, and we discuss the consequences on the interpretability of the models. We propose an intuitive regularization mechanism to avoid problems in under-determined configurations, and we test the approach in two machine learning models.

In the present paper we investigate the analysis of functional magnetic resonance image (fMRI) data based on voxel response analysis. All voxels in local spatial area (volume) of a considered voxel form its so-called searchlight. The searchlight for a presented task is taken as a complex pattern. Task dependent discriminant analysis of voxel is then performed by assessment of the discrimination behavior of the respective searchlight pattern for a given task. Classification analysis of these patterns is usually done using linear support vector machines (linSVMs) as a machine learning approach or another statistical classifier like linear discriminant classifier. The test classification accuracy determining the task sensitivity is interpreted as the discrimination ability of the related voxel. However, frequently, the number of voxels contributing to a searchlight is much larger than the number of available pattern samples in classification learning, i.e. the dimensionality of patterns is higher than the number of samples. Therefore, the respective underlying mathematical classification problem has not an unique solution such that a certain solution obtained by the machine learning classifier contains arbitrary (random) components. For this situation, the generalization ability of the classifier may drop down. We propose in this paper another data processing approach to reduce this problem. In particular, we reformulate the classification problem within the searchlight. Doing so, we avoid the dimensionality problem: We obtain a mathematically well-defined classification problem, such that generalization ability of a trained classifier is kept high. Hence, a better stability of the task discrimination is obtained. Additionally, we propose the utilization of generalized learning vector quantizers as an alternative machine learning classifier system compared to SVMs, to improve further the stability of the classifier model due to decreased model complexity.

This issue marks the first anniversary issue of IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS after it changed its name from IEEE TRANSACTIONS ON NEURAL NETWORKS. I am happy to report that we had a great year! The number of new submissions in a year exceeded 1,000 for the first time in the history of TNN/TNNLS. IEEE TNN had a very successful development for 22 years from 1990 to 2011, and we have good reasons to believe that IEEE TNNLS will have many more years of successful growth.

In the present paper we investigate the application of differentiable kernel for generalized matrix learning vector quantization as an alternative kernel-based classifier, which additionally provides classification dependent data visualization. We show that the concept of differentiable kernels allows a prototype description in the data space but equipped with the kernel metric. Moreover, using the visualization properties of the original matrix learning vector quantization we are able to optimize the class visualization by inherent visualization mapping learning also in this new kernel-metric data space.

The contribution describes the application of the Team 'Computational Intelligence Group' from the University of Applied Sciences Mittweida (Germany) to the ICMLA Face Recognition Challenge 2012. In particular we explain the data preprocessing and feature extraction, which was applied before classification learning. Further we give details about the used classification algorithm - the enhanced generalized matrix learning vector quantization model (eGMLVQ). We provide information about the results as well as observed classification properties detected by the learning algorithm.

We suggest and investigate the use of Generalized Matrix Relevance Learning (GMLVQ) in the context of discriminative visualization. This prototype-based, supervised learning scheme parameterizes an adaptive distance measure in terms of a matrix of relevance factors. By means of a few benchmark problems, we demonstrate that the training process yields low rank matrices which can be used efficiently for the discriminative visualization of labeled data. Comparison with well known standard methods illustrate the flexibility and discriminative power of the novel approach. The mathematical analysis of GMLVQ shows that the corresponding stationarity condition can be formulated as an eigenvalue problem with one or several strongly dominating eigenvectors. We also study the inclusion of a penalty term which enforces non-singularity of the relevance matrix and can be used to control the role of higher order eigenvalues, efficiently.

One aspect of self-organizing systems is their desired ability to be self-learning, i.e., to be able to adapt dynamically to conditions in their environment. This quality is awkward especially if it comes to applications in security or safety-sensitive areas. Here a step towards more trustful systems could be taken by providing transparency of the processes of a system. An important means of giving feedback to an operator is the visualization of the internal processes of a system. In this position paper we address the problem of visualizing dynamic processes especially in self-learning systems. We take an existing self-learning system from the field of computer vision as an example from which we derive questions of general interest such as possible options to visualize the flow of information in a dynamic learning system or the visualization of symbolic data. As a side effect the visualization of learning processes may provide a better understanding of underlying principles of learning in general, i.e, also in biological systems. That may also facilitate improved designs of future self-learning systems.

In this paper, we consider the magnification behavior of neural maps using several (parametrized) divergences as dissimilarity measure instead of the Euclidean distance. We show experimentally that optimal magnification, i.e. information optimum data coding by the prototypes, can be achieved for properly chosen divergence parameters. Thereby, the divergences considered here represent all main classes of divergences. Hence, we can conclude that information optimal vector quantization can be processed independently from the divergence class by appropriate parameter setting.

Prototype based classifiers are effective algorithms in modeling classification problems and have been applied in multiple domains. While many supervised learning algorithms have been successfully extended to kernels to improve the discrimination power by means of the kernel concept, prototype based classifiers are typically still used with Euclidean distance measures. Kernelized variants of prototype based classifiers are currently too complex to be applied for larger data sets. Here we propose an extension of Kernelized Generalized Learning Vector Quantization (KGLVQ) employing a sparsity and approximation technique to reduce the learning complexity. We provide generalization error bounds and experimental results on real world data, showing that the extended approach is comparable to SVM on different public data.

We discuss the use of divergences in dissimilarity-based classification. Divergences can be employed whenever vectorial data consists of non-negative, potentially normalized features. This is, for instance, the case in spectral data or histograms. In particular, we introduce and study divergence based learning vector quantization (DLVQ). We derive cost function based DLVQ schemes for the family of γ‐divergences which includes the well-known Kullback–Leibler divergence and the so-called Cauchy–Schwarz divergence as special cases. The corresponding training schemes are applied to two different real world data sets. The first one, a benchmark data set (Wisconsin Breast Cancer) is available in the public domain. In the second problem, color histograms of leaf images are used to detect the presence of cassava mosaic disease in cassava plants. We compare the use of standard Euclidean distances with DLVQ for different parameter settings. We show that DLVQ can yield superior classification accuracies and Receiver Operating Characteristics.

Supervised and unsupervised vector quantization methods for classification and clustering traditionally use dissimilarities, frequently taken as Euclidean distances. In this article, we investigate the applicability of divergences instead, focusing on online learning. We deduce the mathematical fundamentals for its utilization in gradient-based online vector quantization algorithms. It bears on the generalized derivatives of the divergences known as Fréchet derivatives in functional analysis, which reduces in finite-dimensional problems to partial derivatives in a natural way. We demonstrate the application of this methodology for widely applied supervised and unsupervised online vector quantization schemes, including self-organizing maps, neural gas, and learning vector quantization. Additionally, principles for hyperparameter optimization and relevance learning for parameterized divergences in the case of supervised vector quantization are given to achieve improved classification accuracy.

In this paper, we present a regularization technique to extend recently proposed matrix learning schemes in learning vector quantization (LVQ). These learning algorithms extend the concept of adaptive distance measures in LVQ to the use of relevance matrices. In general, metric learning can display a tendency towards oversimplification in the course of training. An overly pronounced elimination of dimensions in feature space can have negative effects on the performance and may lead to instabilities in the training. We focus on matrix learning in generalized LVQ (GLVQ). Extending the cost function by an appropriate regularization term prevents the unfavorable behavior and can help to improve the generalization ability. The approach is first tested and illustrated in terms of artificial model data. Furthermore, we apply the scheme to benchmark classification data sets from the UCI Repository of Machine Learning. We demonstrate the usefulness of regularization also in the case of rank limited relevance matrices, i.e., matrix learning with an implicit, low-dimensional representation of the data.

In this paper, we develop a Tanimoto metric variant of the evolving tree for the analysis of mass spectrometric data of animal fur. The evolving tree is an extension of self-organizing maps developed to analyze hierarchical clustering problems. Together with the Tanimoto similarity measure, which is intended to work with taxonomic structured data, the evolving tree is well suited for the identification of animal hair based on mass spectrometry fingerprints. Results show a suitable hierarchical clustering of the test data and also a good retrieval capability with a logarithmic number of comparisons.

Nuclear magnetic resonance spectroscopy is a technique for the analysis of complex biochemical materials. Thereby the identification of known sub-patterns is important. These measurements require an accurate preprocessing and analysis to meet clinical standards. Here we present a method for an appropriate sparse encoding of NMR spectral data combined with a fuzzy classification system allowing the identification of sub-patterns including mixtures thereof. The method is evaluated in contrast to an alternative approach using simulated metabolic spectra.

In the present contribution we propose two recently developed classification algorithms for the analysis of mass-spectrometric data-the supervised neural gas and the fuzzy-labeled self-organizing map. The algorithms are inherently regularizing, which is recommended, for these spectral data because of its high dimensionality and the sparseness for specific problems. The algorithms are both prototype-based such that the principle of characteristic representants is realized. This leads to an easy interpretation of the generated classifcation model. Further, the fuzzy-labeled self-organizing map is able to process uncertainty in data, and classification results can be obtained as fuzzy decisions. Moreover, this fuzzy classification together with the property of topographic mapping offers the possibility of class similarity detection, which can be used for class visualization. We demonstrate the power of both methods for two exemplary examples: the classification of bacteria (listeria types) and neoplastic and non-neoplastic cell populations in breast cancer tissue sections.

We present a general framework for association learning in self-organizing maps (SOMs), which can be specified for the utilization for supervised fuzzy classification. In this way, we obtain a prototype based fuzzy classification model (FLSOM), which can be easily interpreted and visualized due to the fundamental properties of SOMs. Moreover, the provided extension gives the ability to detect class similarities. We apply this approch to classification and class similarity detection for mass spectrometric data in case of cancer disease and obtain comparable results. We demonstrate that the FLSOM-based class similarity detection leads to clinically expected class similarities. Finally, this approach can be taken a semi-supervised learning approach in a twofold sense: association learning is influenced by two terms an unsupervised and a supervised learning term. Further, if no association is given for a data point, only the unsupervised learning amount is applied.

K-means clustering combines a variety of striking properties because of which it is widely used in applications: training is intuitive and simple, the final classifier represents classes by geometrically meaningful prototypes, and the algorithm is quite powerful compared to more complex alternative clustering algorithms. In this contribution, we focus on extensions which incorporate additional information into the clustering algorithm to achieve a better accuracy: neighborhood cooperation from neural gas, (possibly fuzzy) label information of input data, and general problem-adapted distances instead of the standard Euclidean metric. These extensions can be formulated in a simple general framework by means of a cost function. We demonstrate the ability of these variants on several representative clustering problems from computational biology.

In this paper, we examine the scope of validity of the explicit self-organizing map (SOM) magnification control scheme of Bauer et al. (1996) on data for which the theory does not guarantee success, namely data that are n-dimensional, n > or =2, and whose components in the different dimensions are not statistically independent. The Bauer et al. algorithm is very attractive for the possibility of faithful representation of the probability density function (pdf) of a data manifold, or for discovery of rare events, among other properties. Since theoretically unsupported data of higher dimensionality and higher complexity would benefit most from the power of explicit magnification control, we conduct systematic simulations on "forbidden" data. For the unsupported n=2 cases that we investigate, the simulations show that even though the magnification exponent alpha achieved achieved by magnification control is not the same as the desired alpha desired, alpha achieved systematically follows alpha desired with a slowly increasing positive offset. We show that for simple synthetic higher dimensional data information, theoretically optimum pdf matching (alpha achieved = 1) can be achieved, and that negative magnification has the desired effect of improving the detectability of rare classes. In addition, we further study theoretically unsupported cases with real data.

We extend the self-organizing map in the variant as proposed by Heskes to a supervised fuzzy classification method. This leads to a robust classifier where efficient learning with fuzzy labeled or partially contradictory data is possible. Further, the integration of labeling into the location of prototypes in a self-organizing map leads to a visualization of those parts of the data relevant for the classification. The method is incorporated in a clinical proteomics toolkit dedicated for biomarker search which allows the necessary preprocessing and further data analysis with additional visualizations.

We consider different ways to control the magnification in self-organizing maps (SOM) and neural gas (NG). Starting from early approaches of magnification control in vector quantization, we then concentrate on different approaches for SOM and NG. We show that three structurally similar approaches can be applied to both algorithms that are localized learning, concave-convex learning, and winner-relaxing learning. Thereby, the approach of concave-convex learning in SOM is extended to a more general description, whereas the concave-convex learning for NG is new. In general, the control mechanisms generate only slightly different behavior comparing both neural algorithms. However, we emphasize that the NG results are valid for any data dimension, whereas in the SOM case, the results hold only for the one-dimensional case.

We extend soft nearest neighbor classification to fuzzy classification with adaptive class labels. The adaptation follows a gradient descent on a cost function. Further, it is applicable for general distance measures, in particular task specific choices and relevance learning for metric adaptation can be done. The performance of the algorithm is shown on synthetic as well as on real life data taken from proteomic research.

The paper deals with the application of the novel generalized relevance learning vector quantization based classification algorithm in comparison to the unified maximum separability analysis as a special variant of support vector machine algorithms. The algorithms are compared and their performance on real life data, taken from clinical studies, is demonstrated. It is shown that the vector quantization classifier gives competitive results in comparison to the considered support vector machine algorithm and shows the recently theoretical proven equivalence in classification capability of both paradigms.

We introduce a multiple subpopulation approach for parallel evolutionary algorithms the migration scheme of which follows a SOM-like dynamics. We successfully apply this approach to clustering in VLSI-design. The advantages of the approach are shown which consist in a reduced communication overhead between the subpopulations preserving a non-vanishing information flow.

Neural maps project data from an input space onto a neuron position in a (often lower dimensional) output space grid in a neighborhood preserving way, with neighboring neurons in the output space responding to neighboring data points in the input space. A map-learning algorithm can achieve an optimal neighborhood preservation only, if the output space topology roughly matches the effective structure of the data in the input space. We here present a growth algorithm, called the GSOM or growing self-organizing map, which enhances a widespread map self-organization process, Kohonen's self-organizing feature map (SOFM), by an adaptation of the output space grid during learning. The GSOM restricts the output space structure to the shape of a general hypercubical shape, with the overall dimensionality of the grid and its extensions along the different directions being subject of the adaptation. This constraint meets the demands of many larger information processing systems, of which the neural map can be a part. We apply our GSOM-algorithm to three examples, two of which involve real world data. Using recently developed methods for measuring the degree of neighborhood preservation in neural maps, we find the GSOM-algorithm to produce maps which preserve neighborhoods in a nearly optimal fashion.

The neighborhood preservation of self-organizing feature maps like the Kohonen map is an important property which is exploited in many applications. However, if a dimensional conflict arises this property is lost. Various qualitative and quantitative approaches are known for measuring the degree of topology preservation. They are based on using the locations of the synaptic weight vectors. These approaches, however, may fail in case of nonlinear data manifolds. To overcome this problem, in this paper we present an approach which uses what we call the induced receptive fields for determining the degree of topology preservation. We first introduce a precise definition of topology preservation and then propose a tool for measuring it, the topographic function. The topographic function vanishes if and only if the map is topology preserving. We demonstrate the power of this tool for various examples of data manifolds.