Evolutionary algorithms have shown substantial progress in recent years, especially in neural architecture search applications, or neuroevolution. Despite their effectiveness, analyzing and understanding the evolutionary paths these algorithms traverse to reach solutions remains challenging. Often these algorithms involve distributed computing strategies, which can include subpopulations or islands, and they explore massive or even unbounded search spaces which can include both weights and architecture, in both continuous and non-continuous domains. Manually examining individual solutions to understand the evolutionary dynamics is often infeasible due to large population sizes, large genome sizes, and high generation counts. This work introduces a new methodology for visualizing neuroevolution population dynamics called genetic distance projections, along with a novel neural network based method for generating these representations. We evaluate this methodology empirically and find it performs better than other traditional methods in generating these representations. We further validate the usefulness of these visualizations using case studies from EXAMM, a long standing neuroevolution algorithm, in which one case study even led to finding and fixing a bug in EXAMM's algorithm.

Parameter prediction is essential for many applications, facilitating insightful interpretation and decision-making. However, in many real life domains, such as power systems, medicine, and engineering, it can be very expensive to acquire ground truth labels for certain datasets as they may require extensive and expensive laboratory testing. In this work, we introduce a semi-supervised learning approach based on topological projections in self-organizing maps (SOMs), which significantly reduces the required number of labeled data points to perform parameter prediction, effectively exploiting information contained in large unlabeled datasets. While few-shot learning has seen significant advances in recent years, the majority of existing approaches focus on classification tasks, making our regression-based method particularly novel for continuous parameter estimation problems. Our proposed method first trains SOMs on unlabeled data followed by only using a minimal number of available labeled data points to assign targets to key best matching units (BMU). The values estimated for newly-encountered data points are computed utilizing the average of the N closest labeled data points in the SOM’s U-matrix in tandem with a topological shortest path distance calculation scheme. The effectiveness of our approach has been validated through practical application in power engineering, specifically for estimating critical coal property values in coal power plants, where traditional laboratory testing is both time-consuming and costly. Our results indicate that the proposed minimally supervised model significantly outperforms traditional regression techniques, including linear and polynomial regression, Gaussian process regression, K-nearest neighbors, as well as deep neural network models and related clustering schemes.

Identifying phases of flight is important in the field of general aviation, as knowing which phase of flight data is collected from aircraft flight data recorders can aid in the more effective detection of safety or hazardous events. General aviation flight data for phase of flight identification is usually per-second data, comes on a large scale, and is class imbalanced. It is expensive to manually label the data and training classification models usually faces class imbalance problems. This work investigates the use of a novel method for minimally supervised self-organizing maps (MS-SOMs) which utilize nearest neighbor majority votes in the SOM U-matrix for class estimation. Results show that the proposed method can reach or exceed a naive SOM approach which utilized a full data file of labeled data, with only 30 labeled datapoints per class. Additionally, the minimally supervised SOM is significantly more robust to the class imbalance of the phase of flight data. These results highlight how little labeled data is required for effective phase of flight identification.

Transfer learning entails taking an artificial neural network (ANN) that is trained on a source dataset and adapting it to a new target dataset. While this has been shown to be quite powerful, its use has generally been restricted by architectural constraints. Previously, in order to reuse and adapt an ANN's internal weights and structure, the underlying topology of the ANN being transferred across tasks must remain mostly the same while a new output layer is attached, discarding the old output layer's weights. This work introduces network-aware adaptive structure transfer learning (N-ASTL), an advancement over prior efforts to remove this restriction. N-ASTL utilizes statistical information related to the source network's topology and weight distribution in order to inform how new input and output neurons are to be integrated into the existing structure. Results show improvements over prior state-of-the-art, including the ability to transfer in challenging real-world datasets not previously possible and improved generalization over RNNs without transfer.