This paper presents a policy for selecting the most informative individuals in a teacher-learner type coevolution. We propose the use of the surprisal of the mean, based on Shannon information theory, which best disambiguates a collection of arbitrary and competing models based solely on their predictions. This policy is demonstrated within an iterative coevolutionary framework consisting of symbolic regression for model inference and a genetic algorithm for optimal experiment design. Complex symbolic expressions are reliably inferred using fewer than 32 observations. The policy requires 21% fewer experiments for model inference compared to the baselines and is particularly effective in the presence of noise corruption, local information content as well as high dimensional systems. Furthermore, the policy was applied in a real-world setting to model concrete compression strength, where it was able to achieve 96.1% of the passive machine learning baseline performance with only 16.6% of the data.

Markerless pose inference of arbitrary subjects is a primary problem for a variety of applications, including robot vision and teaching by demonstration. Unsupervised kinematic pose inference is an ideal method for these applications as it provides a robust, training-free approach with minimal reliance on prior information. However, these methods have been considered intractable for complex models. This paper presents a general framework for inferring poses from a single depth image given an arbitrary kinematic structure without prior training. A co-evolutionary algorithm, consisting of pose and predictor populations, is applied to overcome the traditional limitations in kinematic pose inference. Evaluated on test sets of 256 synthetic and 52 real images, our algorithm shows consistent pose inference for 34 and 78 degree of freedom models with point clouds containing over 40,000 points, even in cases of significant self-occlusion. Compared to various baselines, the co-evolutionary algorithm provides at least a 3.5-fold increase in pose accuracy and a two-fold reduction in computational effort for articulated models.

Despite the range of applications and successes of evolutionary algorithms, expensive fitness computations often form a critical performance bottleneck. A preferred method of reducing the computational overhead is to coevolve rank predictors, providing a coarse and lightweight fitness approximation that has proven to drastically increase performance. However, the majority of previous work on rank predictor coevolution focused solely on improving the predictor heuristics while strategies to select the equally important trainer population is often an afterthought. Four different strategies are presented and benchmarked on a symbolic regression problem using hundreds of test problems with varying complexities. Of the four strategies, updating the trainer population with the solution of the highest rank variance is found to be significantly superior, resulting in a four to ten fold reduction in computational effort for similar convergence rates over the remaining strategies.

Despite the popularity and success of neural networks in research, the number of resulting commercial or industrial applications has been limited. A primary cause for this lack of adoption is that neural networks are usually implemented as software running on general-purpose processors. Hence, a hardware implementation that can exploit the inherent parallelism in neural networks is desired. This paper investigates how the restricted Boltzmann machine (RBM), which is a popular type of neural network, can be mapped to a high-performance hardware architecture on field-programmable gate array (FPGA) platforms. The proposed modular framework is designed to reduce the time complexity of the computations through heavily customized hardware engines. A method to partition large RBMs into smaller congruent components is also presented, allowing the distribution of one RBM across multiple FPGA resources. The framework is tested on a platform of four Xilinx Virtex II-Pro XC2VP70 FPGAs running at 100 MHz through a variety of different configurations. The maximum performance was obtained by instantiating an RBM of 256 × 256 nodes distributed across four FPGAs, which resulted in a computational speed of 3.13 billion connection-updates-per-second and a speedup of 145-fold over an optimized C program running on a 2.8-GHz Intel processor.

Despite the popularity and success of neural networks in research, the number of resulting commercial or industrial applications have been limited. A primary cause of this lack of adoption is due to the fact that neural networks are usually implemented as software running on general-purpose processors. Algorithms to implement a neural network in software are typically O(n2) problems -- as a result, neural networks are unable to provide the performance and scalability required in non-academic settings.

Although there are many neural network FPGA architectures, there is no framework for designing large, high-performance neural networks suitable for the real world. In this paper, we present two concepts to support a multi-FPGA architecture for stochastic restricted Boltzmann machines (RBM), a popular type of neural network. First, a hardware core, called the kth stage piecewise linear interpolator, is used to implement a high-precision, pipelined function generator. The interpolator increases the resolution of a look up table implementation, guaranteeing an additional bit of precision for every pipeline stage. This function generator is used to implement a sigmoid function required in stochastic node selection. Next, a partitioning algorithm is used to efficiently divide a RBM amongst multiple FPGAs. The partitioning algorithm optimizes performance by minimizing the inter-FPGA communication. The architecture is tested on the Berkeley Emulation Engine 2 running at 100 MHz. One board supports a RBM of 256 times 256 nodes, and results in a computational speed of 1.85 billion connection-updatesper- second and a speed-up of 85-fold over an optimized C program running on a 2.8 GHz Intel processor.

This paper presents several challenges and solutions in designing an efficient Message Passing Interface (MPI) implementation for embedded FPGA applications. Popular MPI implementations are designed for general-purpose computers which have significantly different properties and trade-offs than embedded platforms. Our work focuses on two types of interactions that are not present in typical MPI implementations. First, a number of improvements designed to accelerate software-hardware interactions are introduced, including a Direct Memory Access (DMA) engine with MPI functionality; the use of non-interrupting, non-blocking messages; and a proposed function, called MPI_Coalesce, to reduce the function call overhead from a series of sequential messages. These improvements resulted in a speed-up of 5-fold compared to an embedded software-only MPI implementation. Next, a novel dataflow message passing model is presented for hardware-hardware interactions to overcome the limitations of atomic messages, allowing hardware engines to communicate and compute simultaneously. This dataflow model provides a natural method for hardware designers to build high performance, MPI systems. Finally, two hardware cores, Tee cores and message watchdog timers, are introduced to provide a transparent method of debugging hardware MPI designs.