Selecting a minimal feature set that is maximally informative about a target variable is a central task in machine learning and statistics. Information theory provides a powerful framework for formulating feature selection algorithms—yet, a rigorous, information-theoretic definition of feature relevancy, which accounts for feature interactions such as redundant and synergistic contributions, is still missing. We argue that this lack is inherent to classical information theory which does not provide measures to decompose the information a set of variables provides about a target into unique, redundant, and synergistic contributions. Such a decomposition has been introduced only recently by the partial information decomposition (PID) framework. Using PID, we clarify why feature selection is a conceptually difficult problem when approached using information theory and provide a novel definition of feature relevancy and redundancy in PID terms. From this definition, we show that the conditional mutual information (CMI) maximizes relevancy while minimizing redundancy and propose an iterative, CMI-based algorithm for practical feature selection. We demonstrate the power of our CMI-based algorithm in comparison to the unconditional mutual information on benchmark examples and provide corresponding PID estimates to highlight how PID allows to quantify information contribution of features and their interactions in feature-selection problems.

Nature relies on highly distributed computation for the processing of information in nervous systems across the entire animal kingdom. Such distributed computation can be more easily understood if decomposed into the three elementary components of information processing, i.e. storage, transfer and modification, and rigorous information theoretic measures for these components exist. However, the distributed computation is often also linked to neural dynamics exhibiting distinct rhythms. Thus, it would be beneficial to associate the above components of information processing with distinct rhythmic processes where possible. Here we focus on the storage of information in neural dynamics and introduce a novel spectrally-resolved measure of active information storage (AIS). Drawing on intracortical recordings of neural activity in ferrets under anesthesia before and after loss of consciousness (LOC) we show that anesthesia- related modulation of AIS is highly specific to different frequency bands and that these frequency-specific effects differ across cortical layers and brain regions. We found that in the high/low gamma band the effects of anesthesia result in AIS modulation only in the supergranular layers, while in the alpha/beta band the strongest decrease in AIS can be seen at infragranular layers. Finally, we show that the increase of spectral power at multiple frequencies, in particular at alpha and delta bands in frontal areas, that is often observed during LOC ('anteriorization') also impacts local information processing-but in a frequency specific way: Increases in isoflurane concentration induced a decrease in AIS in the alpha frequencies, while they increased AIS in the delta frequency range < 2Hz. Thus, the analysis of spectrally-resolved AIS provides valuable additional insights into changes in cortical information processing under anaesthesia.

Studies investigating neural information processing often implicitly ask both, which processing strategy out of several alternatives is used and how this strategy is implemented in neural dynamics. A prime example are studies on predictive coding. These often ask whether confirmed predictions about inputs or prediction errors between internal predictions and inputs are passed on in a hierarchical neural system-while at the same time looking for the neural correlates of coding for errors and predictions. If we do not know exactly what a neural system predicts at any given moment, this results in a circular analysis-as has been criticized correctly. To circumvent such circular analysis, we propose to express information processing strategies (such as predictive coding) by local information-theoretic quantities, such that they can be estimated directly from neural data. We demonstrate our approach by investigating two opposing accounts of predictive coding-like processing strategies, where we quantify the building blocks of predictive coding, namely predictability of inputs and transfer of information, by local active information storage and local transfer entropy. We define testable hypotheses on the relationship of both quantities, allowing us to identify which of the assumed strategies was used. We demonstrate our approach on spiking data collected from the retinogeniculate synapse of the cat (N = 16). Applying our local information dynamics framework, we are able to show that the synapse codes for predictable rather than surprising input. To support our findings, we estimate quantities applied in the partial information decomposition framework, which allow to differentiate whether the transferred information is primarily bottom-up sensory input or information transferred conditionally on the current state of the synapse. Supporting our local information-theoretic results, we find that the synapse preferentially transfers bottom-up information.

Large-scale, high-quality data sets are central to the development of advanced machine learning techniques that increase the effectiveness of existing optimization methods or even inspire novel ones. Especially in the engineering domain, such high-quality data sets are rare due to confidentiality concerns and generation costs, be it computational or manual efforts. We, therefore, introduce the OSU-Honda Automobile Hood Dataset (CarHoods10k), an industry-grade 3-D vehicle hood data set of over 10 000 shapes along with mechanical performance data that were validated against real-world hood designs by industry experts. CarHoods10k offers researchers and practitioners the unique opportunity to develop novel methods on realistic data with relevance to real-world vehicle design. To illustrate central use cases, we first apply methods from geometric deep learning to learn a compact latent representation for design space exploration. Second, we use machine learning models to predict mechanical hood performance from the learned latent representation. We thus demonstrate the effectiveness of machine learning for building metamodels, which are used in design optimization whenever possible to replace costly engineering simulations. Third, we integrate CarHoods10k in a topology optimization approach based on evolutionary algorithms to demonstrate its capability to search for high-performing structures, while maintaining manufacturability constraints.

Methods for learning and compressing high-dimensional data allow designers to generate novel and low-dimensional design representations for shape optimization problems. By using compact design spaces, global optimization algorithms require less function evaluations to characterize the problem landscape. Furthermore, data-driven representations are often domain-agnostic and independent of the user expertise, and thus potentially capture more relevant design features than a human designer would suggest. However, more factors than the dimensionality play a role in the efficiency of design representations. In this paper, we perform a comparative analysis of design representations for 3D shape optimization problems obtained with principal component analysis, kernel-principal component analysis and a 3D point cloud autoencoder, which we apply on a benchmark data set of computer aided engineering car models. We evaluate the shape-generative capabilities of these methods and show that we can modify the geometries more locally with the autoencoder than with the remaining methods. In a vehicle aerodynamic optimization framework, we verify that this property of the autoencoder representation improves the optimization performance by enabling potentially complementary degrees of freedom for the optimizer. With our study, we provide insights on the qualitative properties and quantifiable measures on the efficiency of deep neural networks as shape generative models for engineering optimization problems, as well as analyses of geometric representations for engineering optimization with evolutionary algorithms.

Modern product design in the engineering domain is increasingly driven by computational analysis including finite-element based simulation, computational optimization, and modern data analysis techniques such as machine learning. To apply these methods, suitable data representations for components under development as well as for related design criteria have to be found. While a component’s geometry is typically represented by a polygon surface mesh, it is often not clear how to parametrize critical design properties in order to enable efficient computational analysis. In the present work, we propose a novel methodology to obtain a parameterization of a component’s plastic deformation behavior under stress, which is an important design criterion in many application domains, for example, when optimizing the crash behavior in the automotive context. Existing parameterizations limit computational analysis to relatively simple deformations and typically require extensive input by an expert, making the design process time intensive and costly. Hence, we propose a way to derive a compact descriptor of deformation behavior that is based on spectral mesh processing and enables a low-dimensional representation of also complex deformations. We demonstrate the descriptor’s ability to represent relevant deformation behavior by applying it in a nearest-neighbor search to identify similar simulation results in a filtering task. The proposed descriptor provides a novel approach to the parametrization of geometric deformation behavior and enables the use of state-of-the-art data analysis techniques such as machine learning to engineering tasks concerned with plastic deformation behavior.

In order to reduce the dimensionality of 3D point cloud representations, autoencoder architectures generate increasingly abstract, compressed features of the input data. Visualizing these features is central to understanding the learning process, however, while successful visualization techniques exist for neural networks applied to computer vision tasks, similar methods for geometric, especially non-Euclidean, input data are currently lacking. Hence, we propose a first-of-kind method to project the features learned by point cloud autoencoders into a 3D-space augmented with color maps. Our proposal explores the properties of 1D-convolutions, used in state-of-the art point cloud autoencoder architectures to handle the input data, which leads to an intuitive interpretation of the visualized features. Furthermore, we tackle the search for relevant co-activations in the feature space by clustering the input data in the latent space, where we explore the correspondence between network features and geometric characteristics of typical shapes of the clusters. We tested our approach with experiments on a benchmark data set, and with three different configurations of a point cloud autoencoder, where we show that the features learned by the autoencoder correlate with the occupancy of the input space by the training data.

Information transfer, measured by transfer entropy, is a key component of distributed computation. It is therefore important to understand the pattern of information transfer in order to unravel the distributed computational algorithms of a system. Since in many natural systems distributed computation is thought to rely on rhythmic processes a frequency resolved measure of information transfer is highly desirable. Here, we present a novel algorithm, and its efficient implementation, to identify separately frequencies sending and receiving information in a network. Our approach relies on the invertible maximum overlap discrete wavelet transform (MODWT) for the creation of surrogate data in the computation of transfer entropy and entirely avoids filtering of the original signals. The approach thereby avoids well-known problems due to phase shifts or the ineffectiveness of filtering in the information theoretic setting. We also show that measuring frequency-resolved information transfer is a partial information decomposition problem that cannot be fully resolved to date and discuss the implications of this issue. Last, we evaluate the performance of our algorithm on simulated data and apply it to human magnetoencephalography (MEG) recordings and to local field potential recordings in the ferret. In human MEG we demonstrate top-down information flow in temporal cortex from very high frequencies (above 100Hz) to both similarly high frequencies and to frequencies around 20Hz, i.e. a complex spectral configuration of cortical information transmission that has not been described before. In the ferret we show that the prefrontal cortex sends information at low frequencies (4-8 Hz) to early visual cortex (V1), while V1 receives the information at high frequencies (> 125 Hz).

The disruption of coupling between brain areas has been suggested as the mechanism underlying loss of consciousness in anesthesia. This hypothesis has been tested previously by measuring the information transfer between brain areas, and by taking reduced information transfer as a proxy for decoupling. Yet, information transfer is a function of the amount of information available in the information source-such that transfer decreases even for unchanged coupling when less source information is available. Therefore, we reconsidered past interpretations of reduced information transfer as a sign of decoupling, and asked whether impaired local information processing leads to a loss of information transfer. An important prediction of this alternative hypothesis is that changes in locally available information (signal entropy) should be at least as pronounced as changes in information transfer. We tested this prediction by recording local field potentials in two ferrets after administration of isoflurane in concentrations of 0.0%, 0.5%, and 1.0%. We found strong decreases in the source entropy under isoflurane in area V1 and the prefrontal cortex (PFC)-as predicted by our alternative hypothesis. The decrease in source entropy was stronger in PFC compared to V1. Information transfer between V1 and PFC was reduced bidirectionally, but with a stronger decrease from PFC to V1. This links the stronger decrease in information transfer to the stronger decrease in source entropy-suggesting reduced source entropy reduces information transfer. This conclusion fits the observation that the synaptic targets of isoflurane are located in local cortical circuits rather than on the synapses formed by interareal axonal projections. Thus, changes in information transfer under isoflurane seem to be a consequence of changes in local processing more than of decoupling between brain areas. We suggest that source entropy changes must be considered whenever interpreting changes in information transfer as decoupling.