Abstract Reservoir Computing is a computing model ideal for performing computation on varied physical substrates. However, these physical reservoirs can be difficult to scale up. We propose joining various reservoirs together as an approach to solving this problem, simulating physical reservoirs with Echo State Networks (ESNs). We investigate various methods of combining ESNs to form larger reservoirs, including a method that we dub Restricted ESNs . We provide a notation for describing Restricted ESNs, and use it to benchmark a standard ESN against restricted ones. We investigate two methods to keep the weight matrix density consistent when comparing a Restricted ESN to a standard one, which we call overall consistency and patch consistency . We benchmark restricted ESNs on NARMA10 and the sunspot prediction benchmark, and find that restricted ESNs perform similarly to standard ones. We present some application scenarios in which restricted ESNs may offer advantages over standard ESNs. We then test restricted ESNs on a version of the multi-timescale Multiple Superimposed Sines tasks, in order to establish a baseline performance that can be improved upon in further work. We conclude that we can scale up reservoir performance by linking small homogeneous subreservoirs together without significant loss in performance over a single large reservoir, justifying future work on using heterogeneous subreservoirs for greater flexibility.

The Reservoir Computing (RC) paradigm is a supervised machine learning approach that makes use of the inherent processing capacity of dynamical systems. Using the system's transient response to an external input, delayed chaotic systems offer rich dynamics for information processing, and have therefore been recognised as ideal systems for reservoir computing. A distinctive feature of delay-line reservoirs is their single-input/single-output structure, which makes them efficient for physical implementation. However, this also presents a significant limitation to multi-input tasks, as the sequence of information in the time-multiplexed input stream is not obvious. Here, we propose enhancing the input masking process used in delay-feedback RCs to mix multiple inputs in the time domain. We investigate two approaches: ‘interleaved’ and ‘sequential’, of injecting multi-input signals into a delay-line reservoir without modifying its topology. Further, we propose a novel task for RC, which inherently requires multiple inputs, to evaluate our approach: the control of a forced Van der Pol oscillator system. We use the trained reservoir as a controller to regulate the nonlinear dynamics of the Van der Pol system by constraining its trajectory to a circle. We find that, with careful choice of model parameters and offset masking scheme, the ‘sequential’ method outperforms the ‘interleaved’ method on this task.

The delay-feedback reservoir is a branch of reservoir computing that has allowed for a more hardware friendly implementation by reducing the typically large number of inputs and outputs within neural networks to a single physical input and output. This is achieved by using a single non-linear node and time-multiplexing the input signal with a masking signal, to create many “virtual neurons” within a reservoir; emulating a larger spatial neural network. While the relationship of the number of virtual nodes, masking frequency, and overall delay length is well known, the effect that the timescale has on computational performance is not widely investigated or understood; usually because working with specific substrates implies the reservoir is confined to its inherent timescales. Hence, there is currently no general methodology for tuning delay-feedback reservoir systems to specific applications. Here, we create a parameterisable hardware-realistic computational model in order to simulate a delay-feedback reservoir operating at different timescales, and explore the effect this has on the connectivity between the virtual nodes and the computational performance of the system. Our results show that the timescale has indeed a significant effect on computational tasks with a long-term dependency on previous input stimuli. We then show that it is possible to emulate larger virtual node networks within smaller ones with little loss in performance and an increase in efficiency. This is an essential step towards understanding the function of the timescale within a delay-feedback reservoir, and a methodology to enable systematic tailoring of delay-feedback reservoirs to specific computational tasks across different timescales.

In order to improve the classification accuracy, state-of-the-art CNNs usually involve large and complex convolutional layers. However, for certain applications, e.g. Internet of Things (IoT), where such CNNs are to be implemented on resource-constrained platforms, the CNN architectures have to be small and efficient. To deal with this problem, reducing the resource consumption in convolutional layers has become one of the most significant efforts when designing CNNs. In this work, a multi-objective optimisation approach is proposed to trade-off between the amount of computation and network accuracy by using Multi-Objective Evolutionary Algorithms (MOEAs). The size and number of convolution kernels used are proportional to computational resource consumption of CNNs. Therefore, this paper considers reducing the computational resource consumption by reducing the size of kernels in convolutional layers. In addition to kernel size unconventional kernel shapes are investigated and results show these clearly improve performance over using traditionally square convolution kernels alone. The main contributions of this paper are therefore a methodology to significantly reduce computational cost of CNNs, based on unconventional kernel shapes, and provide different trade-offs for specific use cases. The experimental results further demonstrate that the proposed method achieves large improvements in resource consumption with no significant reduction in network performance. Compared with the benchmark CNN, the best trade-off architecture shows a reduction in multiplications of up to 4.06X and with slight increase in classification accuracy on CIFAR-10 dataset.

Fabrication issues throttle VLSI designs with pessimistic design constraints and speed-grade device binning necessary to avoid failure of devices. We propose that a on chip monitoring system (a Nervous System) can reduce this margin by automatically sensing and reacting to failures and environmental changes at runtime. We demonstrate that pessimistic margins in the FPGA tools allow our test circuit to be overclocked by twice the maximum design tool frequency and run at 50 °C above its maximum operating temperature without error. The Configurable Intelligence Array is introduced as a low-overhead intelligence platform and used for a prototype neural circuit that can close the loop between a timing-fault detector and a programmable Phase Locked Loop (PLL) oscillator.

Abstract We explore the effect of connectivity and topology on the dynamical behaviour of Reservoir Computers. At present, considerable effort is taken to design and hand-craft physical reservoir computers. Both structure and physical complexity are often pivotal to task performance, however, assessing their overall importance is challenging. Using a recently developed framework, we evaluate and compare the dynamical freedom (referring to quality) of neural network structures, as an analogy for physical systems. The results quantify how structure affects the behavioural range of networks. It demonstrates how high quality reached by more complex structures is often also achievable in simpler structures with greater network size. Alternatively, quality is often improved in smaller networks by adding greater connection complexity. This work demonstrates the benefits of using dynamical behaviour to assess the quality of computing substrates, rather than evaluation through benchmark tasks that often provide a narrow and biased insight into the computing quality of physical systems.

To tackle the complexity of state-of-the-art electronic systems, silicon foundries continuously shrink the technology nodes and electronic design automation (EDA) vendors offer hierarchical design flows to decompose systems into smaller blocks. However, such a staged design methodology consists of various levels of abstraction, where margins will be accumulated and result in degradation of the overall design quality. This limits the full use of capabilities of both the process technology and EDA tools. In this work, a study of drive granularity of standard cells is performed and an interpolation method is proposed for drive option expansion within original cell libraries. These aim to investigate how industrial synthesis tools deal with the drive strength selection using different granularity sets. In addition, a fully-automated, multi-objective (MO) EDA digital flow is introduced for power, performance, area (PPA) optimisation based on drive strength refinement. This population-based search method better handles the increased difficulty of cell selection when using larger logic libraries, producing better optimised solutions than standard tool flow in this case. The achieved experimental results demonstrate how the improved drive granularity cells overall enhance the quality of designs and how a significant improvement in trading off PPA is achieved by the MOEDA flow.

Large-scale distributed computing architectures such as, e.g. systems on chip or many-core devices, offer ad-vantages over monolithic or centralised single-core systems in terms of speed, power/thermal performance and fault tolerance. However, these are not implicit properties of such systems and runtime management at software or hardware level is required to unlock these features. Biological systems naturally present such properties and are also adaptive and scalable. To consider how these can be similarly achieved in hardware may be beneficial. We present Social Insect behaviours as a suitable model for enabling autonomous runtime management (RTM) in many-core architectures. The emergent properties sought to establish are self-organisation of task mapping and system-level fault tolerance. For example, large social insect colonies accomplish a wide range of tasks to build and maintain the colony. Many thousands of individuals, each possessing relatively little intelligence, contribute without any centralised control. Hence, it would seem that social insects have evolved a scalable approach to task allocation, load balancing and robustness that can be applied to large many-core computing systems. Based on this, a self-optimising and adaptive, yet fundamentally scalable, design approach for many-core systems based on the emergent behaviours of social-insect colonies are developed. Experiments capture decision-making processes of each colony member to exhibit such high-level behaviours and embed these decision engines within the routers of the many-core system.

This paper describes a novel multi-reconfigurable architecture, which allows variability-aware design, rapid prototyping and post-fabrication optimisation of digital systems. This is achieved by exploiting reconfiguration at both the digital function level and the transistor level. A runtime variability map of the architecture, created using ring oscillators, is presented.

Intrinsic device variability has become a significant problem in deep sub-micron technology nodes. The stochastic variations in device performance, which are a result of structural irregularities at the atomic scale, can impact both the yield and reliability of a circuit design. In this paper we describe a novel multi-reconfigurable FPGA architecture, the programmable analogue and digital array (PAnDA), which can tackle this problem by allowing post-fabrication reconfiguration of the effective transistor gate widths in a circuit. We demonstrate the advantages of this architecture by creating a frequency variability map of the array using ring oscillators in order to ascertain the location of any frequency outliers. We then show that it is possible, using an evolutionary algorithm, to select alternative transistor configurations which minimise the difference in frequency between one of these outliers and the chips median frequency of operation. Such methods can be used to increase system performance and reliability by presenting an array with more uniform performance characteristics.

The Reservoir Computing (RC) framework is said to have the potential to transfer onto any input-driven dynamical system, provided two properties are present: (i) a fading memory, and (ii) input separability. A typical reservoir consists of a fixed network of recurrently connected processing units; however recent hardware implementations have shown reservoirs are not ultimately bound by this architecture. Previously, we have demonstrated how the RC framework can be applied to randomly-formed carbon nanotube composites to solve computational tasks. Here, we apply the RC framework to an evolvable substrate and compare performance to an already established in materia training technique, referred to as evolution in materia. The results show that by adding the programmable reservoir layer, reservoir computing in materia can significantly outperform the original evolution in materia implementation. This suggests the RC framework offers improved performance, even across non-temporal tasks, when combined with the evolution in materia technique.

Robot interaction planning is a computationally expensive process which rarely makes use of previous experiences in a deliberative manner. This paper addresses this issue by examining dimensionality reduction techniques to allow comparison of objects in a robot's environment based on the way they react to robot manipulation. We compare a number of techniques which can map objects from an observation space, which may contain thousands of dimensions, to a lower dimensionality space - the embedding space - which allows objects to be compared in an efficient manner, making knowledge transfer between similar objects more computationally tractable.

The growing impact of the network on the overall power consumption of many-core systems introduces a need for mechanisms that reduce the power required for data communication without significantly impacting performance. This paper proposes a low-overhead mechanism for frequency control of individual channels in a Network-on-Chip system. The proposed mechanism is low-overhead, distributed and easy to tune for varying traffic, which are crucial aspects in a many-core context. This is primarily achieved by the asymmetry of the controller, which defines different responses for increases and decreases in performance requirements, allowing a channel to operate at the minimum required frequency without adversely affecting stability, necessary for subsequent voltage scaling.

A novel hierarchical fault-tolerance methodology for reconfigurable devices is presented. A bespoke multi-reconfigurable FPGA architecture, the programmable analogue and digital array (PAnDA), is introduced allowing fine-grained reconfiguration beyond any other FPGA architecture currently in existence. Fault blind circuit repair strategies, which require no specific information of the nature or location of faults, are developed, exploiting architectural features of PAnDA. Two fault recovery techniques, stochastic and deterministic strategies, are proposed and results of each, as well as a comparison of the two, are presented. Both approaches are based on creating algorithms performing fine-grained hierarchical partial reconfiguration on faulty circuits in order to repair them. While the stochastic approach provides insights into feasibility of the method, the deterministic approach aims to generate optimal repair strategies for generic faults induced into a specific circuit. It is shown that both techniques successfully repair the benchmark circuits used after random faults are induced in random circuit locations, and the deterministic strategies are shown to operate efficiently and effectively after optimisation for a specific use case. The methods are shown to be generally applicable to any circuit on PAnDA, and to be straightforwardly customisable for any FPGA fabric providing some regularity and symmetry in its structure.

Large social insect colonies require a wide range of important tasks to be undertaken to build and maintain the colony. Fortunately, in most nests there are many thousands of workers available to offer their assistance to ensure the expansion and survival of the colony. However, there is a crucial equilibrium between the number of workers performing each task that must not only be maintained but must also continuously adapt to sudden changes in environment and colony need. What is most fascinating is that social insects can sustain this balance without any centralised control and with colony members that have relatively little intelligence when considered on their own. Due to this simplicity and evident scalability it would seem that social insects have evolved an interesting scalable approach to task allocation that could be applied to very large many-core systems. To investigate this we have explored biological models of task allocation in ant colonies and applied this to a 36-core Network on Chip. This paper not only shows that effective decentralised task allocation is achieved, but also that such a scheme can adapt to faults and alter its behaviour to meet soft real-time constraints. Therefore, it is established that social insect inspired intelligence models offer a suitable metaphor and development direction for tackling the challenges introduced by dark silicon and in-field faults in a decentralised and adaptive fashion.

In this paper, we propose a novel technique based on multiobjective genetic algorithms (MOGA) to solve the circuit clustering problem in field-programmable gate array (FPGA) computer aided design (CAD) flow. As the circuit clustering result is subsequently used by the circuit place-and-route process at the post-synthesis stage, the clustering quality significantly affects the routing cost and the utilisation of FPGA. Among clustering metrics, reducing the global interconnects (global nets) between configurable logic blocks (CLBs) can be viewed as the most efficient precondition to improve the clustering quality. Since the circuit has many connection properties, it is difficult to identify the proper component combinations for sub-circuits that optimise interconnect, and it is even more complicated when CLBs pose constraints. In our technique, we break the circuit clustering task into two phases. We firstly apply the MOGA to identify the best component combinations to FPGA CLBs while respecting constraints, and a second MOGA takes clustering results from the first phase and performs further optimisation. Our proposed method is tested using the MCNC-20 benchmark. The results show that the proposed technique produces better results than state-of-art algorithms in reducing global nets. The overall improvement is up to 4.33% and 14.24% in reducing the cluster number and the global net number compared to the FPGA circuit packer, iRAC, which is considered to be the best clustering method to reduce global nets.

Implementing Dynamic Voltage and Frequency Scaling (DVFS) is a non-trivial task on FPGAs and requires knowledge about the feasible voltage and frequency (VF) ranges as a first step. The VF feasible ranges depend not only on the size of the critical path in the design but also on the inter- and intra-die variability on the FPGA die. Moreover, the variations in the configuration of the FPGA highly affect feasible VF ranges. Therefore, it is crucial to characterise feasibility by studying the relationship between feasible VF regions and these sources of variability in FPGAs. In this paper we employ a self-checking multiplier which uses residue codes and DVFS implemented on the programmable logic component of a Xilinx Zynq ZC702 device as an error-detection circuit to study these feasible regions. Results show that, as expected, feasible VF ranges vary with FPGA configuration. More interestingly, significant variation of the feasible VF regions is found for different dies. These results highlight the necessity of dynamic self-testing as a part of an adaptive DVFS implementation on FPGAs. Employing the techniques presented in this work enables the implementation of efficient adaptive on-line DVFS on programmable logic while ensuring reliability.

The modeling and simulation of semiconductor devices is a difficult and computationally intensive task. However the expense of fabrication and testing means that accurate modeling and simulation are crucial to the continued progress of the industry. To create these models and then perform the simulations requires parameters from accurate physical models to be obtained and then more abstract models created that can perform more complex circuit simulations. Device models (motifs) are created as a mitigation technique for improvement the circuit performance and as technology advances to help with the effects of transistor variability. In order to explore the characteristics of new device motifs on circuit designs, obtaining accurate and reliable device models becomes the first problem for designers. In this paper a Two Step Evolution Strategy (2SES) is proposed for device parameter model extraction. The proposed 2SES approach automatically extracts a set of parameters with respect to a specified device model. Compared with conventional mathematical extraction approach, 2SES is an efficient and accurate method to solve the parameter extraction problem and simultaneously addresses the fact of the mathematical extraction having the complexity of Multi-objective optimization. Compared with single step ES extract result, it is shown that the two-step ES extraction process continues improving generations by adjusting the optimisation parameters. Finally, an application of a new device motif on circuit design is given at end of the paper and compared against a standard device.

This paper explores the potential for transistor level fault tolerance on a new Programmable Analogue and Digital Array (PAnDA) architecture1. In particular, this architecture features Combinatorial Configurable Analogue Blocks (CCABs) that can implement a number of combinatorial functions similar to FPGAs. In addition, PAnDA allows one to reconfigure features of the underlying analogue layer. In PAnDA-EINS, the functions that the CCAB can implement are predefined through the use of a routing block. This paper is a study of whether removing this routing block and allowing direct control of the transistors provides benefits for fault tolerance. Experiments are conducted in two stages. In the first stage, a logic function is evolved on a CCAB and then optimised using a GA. A fault is then injected into the substrate, breaking the logic function. The second stage of the experiment consists of evolving the logic function again on the faulty substrate. The results of these experiments show that the removal of the routing block from the CCAB is beneficial for fault tolerance.

Field programmable gate arrays (FPGAs) are widely used in applications where online reconfigurable signal processing is required. Speed and function density of FPGAs are increasing as transistor sizes shrink to the nanoscale. As these transistors reduce in size intrinsic variability becomes more of a problem and to reliably create electronic designs according to specification time consuming statistical simulations become necessary; and even with accurate models and statistical simulation, the fabrication yield will decrease as every physical instance of a design behaves differently. This paper describes an adaptive, evolvable architecture that allows for correction and optimization of circuits directly in hardware using bioinspired techniques. Similar to FPGAs, the programmable analog and digital array (PAnDA) architecture introduced provides a digital configuration layer for circuit design. Accessing additional configuration options of the underlying analog layer enables continuous adjustment of circuit characteristics at runtime, which enables dynamic optimization of the mapped design's performance. Moreover, the yield of devices can be improved postfabrication via reconfiguration of the analog layer, which can overcome faults induced due to variability and process defects. Since optimization goals are generic, i.e., not restricted to reducing stochastic variability, power consumption or increasing speed, the same mechanisms can also enhance the device's fault tolerant abilities in the case of component degradation and failures during its lifetime or when exposed to hazardous environments.

Biological genomes have evolved over a period of millions of years and comprise thousands of genes, even for the simplest organisms. However, in nature, only 1-2% of the genes play an active role in creating and maintaining the organism, while the majority are evolutionary fossils. This raises the question of whether a considerably larger number of (partly redundant) genes are required in order to effectively build a functional developmental system, of which, in the final system only a fraction is required for the latter to function. This paper investigates different approaches to creating artificial developmental systems (ADSs) based on variable length gene regulatory networks (GRNs). The GRNs are optimized using an evolutionary algorithm (EA). A comparison is made between the different variable length representations and fixed length representations. It is shown that variable length GRNs can achieve both reducing computational effort during optimization and increasing speed and compactness of the resulting ADS, despite the higher complexity of the encoding required. The results may also improve the understanding of how to effectively model GRN based developmental systems. Taking results of all experiments into account makes it possible to create an overall ranking of the different patterns used as a testbench in terms of their complexity. This ranking may aid to compare related work against. In addition, this allows a detailed assessment of the ADS used and enables the identification of missing mechanisms.

Multicellular organisms in biology possess invaluable characteristics, which artificial systems in engineering lack, such as adaptivity and robustness. Modeling biologically inspired multicellular developmental systems in evolutionary computation have been considered for a number of years, and there exist many developmental simulations that capture multicellular characteristics of biological organisms. However, the relative importance of many mechanisms in such models is still poorly understood. This paper undertakes a detailed investigation of the importance of many mechanisms and parameters on the organizational behavior of a gene regulatory network based artificial developmental system via classical pattern matching experiments. The work leads to an improved understanding of artificial multicellular development, which will assist in its utilization in the application of evolutionary computation. It may also provide a better understanding of mechanisms of biological development.

A novel approach to image compression using a gene regulatory network (GRN) based artificial developmental system (ADS) is introduced. The proposed algorithm exploits the fact that a series of complex organisms (≡ developmental states) can be represented via a GRN description and the indices of the developmental steps in which they occur. Organisms are interpreted as tiles of an image at each developmental step which results in the (re-)construction of an image during the developmental process. It is shown that GRNs are suitable for image compression and achieve higher compression rates than JPEG when optimised for a particular image. It is also shown that the same GRN has the potential to encode multiple images, each represented by a different series of numbers of developmental steps.

Many researchers have encountered the problem that the evolution of electronic circuits becomes exponentially more difficult when problems with an increasing number of outputs are tackled. Although this is an issue in both intrinsic and extrinsic evolution experiments, overcoming this problem is particularly challenging in the case of evolvable hardware, where logic and routing resources are constrained according to the given architecture. Consequently, the success of experiments also depends on how the inputs and outputs are interfaced to the evolvable hardware. Various approaches have been made to solve the multiple output problem: partitioning the task with respect to the input or output space, incremental evolution of sub-tasks or resource allocation. However, in most cases, the proposed methods can only be applied in the case of extrinsic evolution. In this paper, we have accordingly, focused on scaling problem of increasing numbers of outputs when logic circuits are intrinsically evolved. We raise a number of questions: how big is the performance drop when increasing the number of outputs? Can the resources of evolvable hardware be structured in a suitable way to overcome the complexity imposed by multiple outputs, without including knowledge about the problem domain? Can available resources in hardware still be efficiently used when pre-structured? In order to answer these questions, different structural implementations are investigated. We have looked at these issues in solving three problems: 4-bit parity, 2-bit adder and 2-bit multiplier.

A model for intrinsic artificial development is introduced in this paper. The proposed model features a novel mechanism where growth emerges, rather than being triggered by a single action. Different types of cell signalling ensure that breaking symmetries is rather the norm than an exception, and gene activity is regulated on two layers: first, by the proteins that are produced by the gene regulatory network (GRN). Second, through structural feedback by second messenger molecules, which are not directly produced through gene expression, but are produced by sensor proteins, which take the cell's structure into account. The latter feedback mechanism is a novel approach, intended to enable adaptivity and environment coupling in real-world applications. The model is implemented in hardware, and is designed to run autonomously in resource limited embedded systems. Initial experiments are carried out to measure long-term stability, dynamics, adaptivity and scalability of the new approach. Furthermore the ability of the GRN to produce patterns of different symmetries is examined.

The design of a new biologically inspired artificial developmental system is described in this paper. In general, developmental systems converge slower and are more computationally expensive than direct evolution. However, the performance trends of development indicate that the full benefit of development will arise with larger and more complex problems that exhibit some sort of regularity in their structure: thus, the aim is to evolve larger electronic systems through the modularity allowed by development. The hope is that the proposed artificial developmental system will exhibit adaptivity and fault tolerance in the future. The cell signalling and the system of Gene Regulatory Networks present in biological organisms are modelled in our developmental system, and tailored for tackling real world problems on electronic hardware. For the first time, a Gene Regulatory Network system is successfully shown to develop the complete circuit structure of a desired digital circuit without the help of another mechanism or any problem specific structuring. Experiments are presented that show the modular behaviour of the developmental system, as well as its ability to solve non-modular circuit problems.

This work is part of a project that aims to develop and operate integrated evolvable hardware systems using unconstrained evolution. Experiments are carried out on an evolvable hardware platform featuring both combinatorial and registered logic as well as sequential feedback loops. In order to be able to accurately assess the transient output of the system and at the same time speed up evolution, new fitness evaluation methods are introduced. These bitwise and hierarchical fitness evaluation methods are adapted and further developed specifically for hardware implementation. It is shown that the newly developed approaches are particularly powerful in coping with two important issues: computational ambiguities, which generally occur when evaluating binary strings, and transient effects resulting from measuring hardware output. On two combinatorial problems it is shown that the new fitness functions improve the performance of evolution and allow stable solutions to be found more reliably. The experiments are carried out with a recently developed hardware platform called reconfigurable integrated system array (RISA).