This paper has three intertwined goals. The first is to introduce a new similarity measure for scatter plots. It uses Delaunay triangulations to compare two scatter plots regarding their relative positioning of clusters. The second is to apply this measure for the robustness assessment of a recent deep neural network (DNN) approach to dimensionality reduction (DR) for data visualization. It uses a nonlinear generalization of Fisher's linear discriminant analysis (LDA) as the encoder network of a deep autoencoder (DAE). The DAE's decoder network acts as a regularizer. The third goal is to look at different variants of the DNN: ones that promise robustness and ones that promise high classification accuracies. This is to study the trade-off between these two objectives - our results support the recent claim that robustness may be at odds with accuracy; however, results that are balanced regarding both objectives are achievable. We see a restricted Boltzmann machine (RBM) pretraining and the DAE based regularization as important building blocks for achieving balanced results. As a means of assessing the robustness of DR methods, we propose a measure that is based on our similarity measure for scatter plots. The robustness measure comes with a superimposition view of Delaunay triangulations that enables a fast comparison of results from multiple DR methods.

We elaborate on the robustness assessment of a deep neural network (DNN) approach to dimensionality reduction for data visualization. The proposed DNN seeks to improve the class separability and compactness in a low-dimensional feature space, which is a natural strategy to obtain well-clustered visualizations. It consists of a DNN-based nonlinear generalization of Fisher's linear discriminant analysis and a DNN-based regularizer. Regarding data visualization, a well-regularized DNN guarantees to learn sufficiently similar data visualizations for different sets of samples that represent the data approximately equally well. Such a robustness against fluctuations in the data is essential for many real-world applications. Our results show that the combined DNN is considerably more robust than the generalized discriminant analysis alone. We further support this conclusion by examining feature representations from four comparative approaches. As a means of measuring the structural dissimilarity between different feature representations, we propose a hierarchical cluster analysis.

Deriving mathematical models for the simulation of dynamic processes is costly and time-consuming. This paper examines the possibilities of deep neural networks (DNNs) as a means to facilitate and accelerate this step in development. DNNs are machine learning models that have become a state-of-the-art solution to a wide range of data analysis and pattern recognition tasks. Unlike mathematical modeling approaches, DNN approaches require little to no domain-specific knowledge. Given a sufficient amount of data, a model of the complex nonlinear input-to-output relations of a dynamic system can be learned autonomously. To validate this DNN based modeling approach, we use the example of a gas-fired absorption heat pump. The DNN is learned based on several measurement series recorded during a hardware-in-the-loop (HiL) simulation of the heat pump. A mathematical reference model of the heat pump that was tested in the same HiL environment is used for a comparison of a mathematical and a DNN based modeling approach. Our results show that DNNs can yield models that are comparable to the reference model. The presented methodology covers the data preprocessing, the learning of the models and their validation. It can be easily transferred to more complex dynamic processes.

We present an approach to feature extraction that is a generalization of the classical linear discriminant analysis (LDA) on the basis of deep neural networks (DNNs). As for LDA, discriminative features generated from independent Gaussian class conditionals are assumed. This modeling has the advantages that the intrinsic dimensionality of the feature space is bounded by the number of classes and that the optimal discriminant function is linear. Unfortunately, linear transformations are insufficient to extract optimal discriminative features from arbitrarily distributed raw measurements. The generalized discriminant analysis (GerDA) proposed in this paper uses nonlinear transformations that are learnt by DNNs in a semisupervised fashion. We show that the feature extraction based on our approach displays excellent performance on real-world recognition and detection tasks, such as handwritten digit recognition and face detection. In a series of experiments, we evaluate GerDA features with respect to dimensionality reduction, visualization, classification, and detection. Moreover, we show that GerDA DNNs can preprocess truly high-dimensional input data to low-dimensional representations that facilitate accurate predictions even if simple linear predictors or measures of similarity are used.

Deep Neural Networks (DNNs) denote multilayer artificial neural networks with more than one hidden layer and millions of free parameters. We propose a Generalized Discriminant Analysis (GerDA) based on DNNs to learn discriminative features of low dimension optimized with respect to a fast classification from a large set of acoustic features for emotion recognition. On nine frequently used emotional speech corpora, we compare the performance of GerDA features and their subsequent linear classification with previously reported benchmarks obtained using the same set of acoustic features classified by Support Vector Machines (SVMs). Our results impressively show that low-dimensional GerDA features capture hidden information from the acoustic features leading to a significantly raised unweighted average recall and considerably raised weighted average recall.

Constructing a recognition system based on raw measurements for different objects usually requires expert knowledge of domain specific data preprocessing, feature extraction, and classifier design. We seek to simplify this process in a way that can be applied without any knowledge about the data domain and the specific properties of different classification algorithms. That is, a recognition system should be simple to construct and simple to operate in practical applications. For this, we have developed a nonlinear feature extractor for high-dimensional complex patterns, using Deep Neural Networks (DNN). Trained partly supervised and unsupervised, the DNN effectively implements a nonlinear discriminant analysis based on a Fisher criterion in a feature space of very low dimensions. Our experiments show that the automatically extracted features work very well with simple linear discriminants, while the recognition rates improve only minimally if more sophisticated classification algorithms like Support Vector Machines (SVM) are used instead.

We propose a framework for optimizing Deep Neural Networks (DNN) with the objective of learning low-dimensional discriminative features from high-dimensional complex patterns. In a two-stage process that effectively implements a Nonlinear Discriminant Analysis (NDA), we first pretrain a DNN using stochastic optimization, partly supervised and unsupervised. This stage involves layer-wise training and stacking of single Restricted Boltzmann Machines (RBM). The second stage performs fine-tuning of the DNN using a modified back-propagation algorithm that directly optimizes a Fisher criterion in the feature space spanned by the units of the last hidden-layer of the network. Our experimental results show that the features learned by a DNN using the proposed framework greatly facilitate classification, even when the discriminative features constitute a substantial dimension reduction.

For image contours approximated by conic curve segments. The authors examine how to detect efficiently, given geometric relationships between conic segments. They present a method that can serve as a general tool for model-driven matching of conic curve segments.>

CARTRACK is a computer vision system that can reliably detect, track, and measure vehicle rears in images from a video camera in a following car. The system exploits the symmetry property typical for the rear of most vehicles on normal roads. The authors present two novel methods for detecting mirror symmetry in images, one based directly on the intensity values and another one based on a discrete representation of local orientation. CARTRACK has been used for realtime experiments with test vehicles of Volkswagen and Daimler-Benz.>

A stereo obstacle detection system has been developed for automatically guided vehicles that operate on flat (factory) floors. The system does not attempt to reconstruct the 3D environment visually but simply tries to detect obstacles on the floor in the vehicle's path. The approach to stereo image processing uses inverse perspective mappings to facilitate matching of the binocular field of vision against the expected 3D structure of the environment. Assuming a known relative camera model, a geometrical image transformation is computed which essentially compensates the stereo disparities for the image points of the floor. After the mapping operation the images are compared and local mismatches are interpreted as possible obstacle locations. The system has been successfully tested in a factory environment. The implementation runs on standard microprocessor hardware in real time.>