Neuromorphic computing achieves highly energy-efficient computations while adapting to environmental changes. Processing time series data by spiking neural networks can further reduce the power consumption of neuromorphic computing devices because most of the energy in the network is consumed only when the neuron generates and transmits a spike. In this study, we designed fully analog two-variable spiking neuron and spiking neural network circuits, taking advantage of the physical properties of transistors as analog devices. The energy consumption of the circuit for generating a spike was 22.7 fJ/spike when the MOS transistors were operating in the subthreshold region. The proposed circuits were implemented on an analog very large-scale integrated (VLSI) circuit chip in a$0.18~\mu $m CMOS process. The circuits exhibit complex spike dynamics even under subthreshold operation according to chip measurements. We demonstrated that the spike sequence generated by the spiking neural network circuit was successfully applied to spoken digit recognition tasks via a reservoir computing framework with 14.4 fJ/SOP efficiency. These results provide important insights into edge AI applications of SNN-based neuromorphic hardware.

Edge computing requires low-power, real-time processing of complex information. Spiking neural networks are highly expected to be applied to edge computing due to their efficient computing properties. Here, we design a mixed-signal LSI consisting of an analog spiking neural network and digital inference circuits for edge device. The two-variable spiking neuron circuits operate in an analog manner using the physical properties of transistors and are connected through synaptic circuits to form a network. The neural network successfully operates and exhibits complex nonlinear behavior in response to external inputs. The power consumption of the spike generation is as low as tens of femtojoules per spike because the transistors in the circuits operate in the subthreshold region, which is enough to be used as edge computing devices. In addition, a digital circuit was designed to perform real-time inference using the spiking sequences from the analog spiking neural network. The result showed that the mixed-signal LSI consisting of the analog spiking neural network and the digital inference circuits can be applied to the spoken digit classification task in real time. The proposed system has the potential to be used as ultra-low-power neuromorphic hardware in practical applications.

A spatio-temporal learning (STL) rule was developed based on the results of physiological experiments in the hippocampus. In contrast to the general Hebbian learning rule, the STL rule updates synaptic weight values through the internal states of neurons, depending on the coincidence of input patterns. This allows the STL rule to learn slight differences in the contextual structure of the spatio-temporal patterns. In the original STL network (STLnet) model, the synaptic weight values take both positive and negative. In this study, considering physiological knowledge and hardware implementation, we propose a modified STLnet model in which synapses are separated into excitatory and inhibitory that take on only positive and negative values, respectively. We then analyze the learning performance of the STLnet models. We confirm that the modified STLnet model has comparable learning performance to the original STLnet model. For the hardware implementation of the modified STLnet, we determine the optimal ratio of excitatory to inhibitory synapses. This study establishes important design guidelines for synaptic weights of the STLnet model.

The continuous-time/value extended spatiotemporal context learning and memory network (eSTCLMN) model shows promise for low-power edge AI hardware equipped with quick one-shot learning capability, due to its suitability for implementation in fully analog asynchronous event-driven spiking integrated circuits. However, designing analog integrated circuits requires pre-optimization of circuit parameters, which is challenging with software simulations of large differential equation system involving asynchronous spikes. To address this challenge, we discretize the eSTCLMN model in time and value, enabling its implementation in event-driven mixed analog/digital circuits. In this modified model, synaptic circuits operate asynchronously with their clocks, while neurons are built with fully continuous-time/value analog circuits. Thus, the proposed approximate eST-CLMN model is represented by a set of differential and difference equations. Moreover, we introduce an asynchronous event-driven mixed analog/digital proof-of-concept (POC) hardware system for the approximate eSTCLMN model. A small-size test circuit board for the POC system is fabricated using commercially available components. Experimental results on the test circuit board demonstrate the validity of the major components of the proposed circuitry for the approximate eSTCLMN model. Finally, we propose the hardware configuration of a medium-size eSTCLMN model circuit system.

A compact two-variable spiking neuron integrated circuit was proposed. Measurements of the circuit confirmed various types of spike responses with ultra-low power consumption. In this study, the two-variable spiking neuron circuit is modified to generate chaotic spike trains suitable for complex spatiotemporal information processing. The chaotic behavior is confirmed with time waveforms, return maps, bifurcation diagrams, and diversity index of inter-spike intervals through circuit simulations and experiments.

Edge computing requires the processing of real-time and personalized information with low power consumption. Neuromorphic devices are promising candidates for applications related to edge computing. Rate neurons, which are typically used in neuromorphic hardware, persistently consume power regardless of their outputs. To further reduce the power consumption of neuromorphic devices, spiking neurons are more suitable because they are event-driven, and information is transferred only when the neuron fires. Herein, we propose a two-variable spiking neuron circuit that operates in a fully analog manner by utilizing the physical properties of transistors as analog devices. By operating in the subthreshold region of the MOS transistor, the energy required to produce a spike is approximately tens of fJ/spike. Furthermore, the analog neuron can exhibit complex spike dynamics, such as chattering, as confirmed using post-layout simulations. The simulations indicated that a neural network comprising the proposed neuron circuits operates successfully and exhibits complex nonlinear behavior. These results provide a basis for dedicated hardware spiking neuron circuits, which could be used as ultra-low-power neuromorphic hardware in various applications, such as realizing liquid-state machines for processing time-series signals.

We have developed a method of secret-key exchange assisted by a secure hash algorithm for a stream cipher based on the augmented Lorenz map as a high-dimensional chaotic map. Two legitimate users are assumed to possess chaotic oscillators subject to the original Lorenz equations to exchange randomized chaotic signals. The oscillators achieve perfect synchronization and generate a sequence of binary numbers to be shared as the secret key. The users, as well as an eavesdropper, cannot estimate the synchronization error because of the randomization of chaotic signals. Nevertheless, only the legitimate users have high confidence in sharing the secret key because of the dynamical stability of the synchronization process. The users can confirm the sharing of the secret key by exchanging the hash values of the keys that are generated by a secure hash algorithm based on the logistic map. We discuss the performance of our method with reference to the results of numerical experiments.

Simple structure and robust property of a reservoir neural network are preferable for hardware implementation of a high performance learning system, especially for time-series data processing. One of salient feature of the reservoir network is reproducibility or consistency of its responses to the same or similar inputs. This is usually guaranteed through the echo state property of the network by properly choosing synaptic weights among reservoir neurons. Another important feature is a variety of dynamics in the reservoir, which makes the reservoir to process complex time-varying input signals. One way to increase the variety of dynamics is introducing chaotic behavior by destabilize the reservoir network by changing weight values. However, this will violate the echo state property, therefore, chaotic dynamics are usually avoided in the reservoir computing.In this paper, we propose a method to introduce high-dimensional chaotic dynamics into the reservoir network, but keeping its consistency. To achieve this, we use a chaotic neural network model in the reservoir network, while keeping the weight matrix in the reservoir network to satisfy the echo state property criteria. In order to show the consistency of the chaotic neural network reservoir, preliminary results for chaotic time-series predictions through the chaotic neural network reservoir are illustrated. In addition, we discuss the application of the chaotic neural network reservoir to a self-aware hardware system.

An echo state network with a reservoir consisting of 200 tanh neurons is applied to the short-term prediction of a chaotic time series generated using the augmented Lorenz equations as a hyperchaotic flow model. The predictive performance is examined in terms of the Kolmogorov-Sinai entropy and the Kaplan - Yorke dimension of a chaotic attractor in comparison with those for chaotic flow models having a single positive Lyapunov exponent. We discuss the predictive performance of the reservoir in terms of a universal simulator of chaotic attractors on the basis of Ueda's view of chaos, i.e., random transitions between unstable periodic orbits in a chaotic attractor.

We propose a control strategy of chaotic dynamics to stabilize periodic orbits in nonlinear discrete-time dynamical systems (maps) and apply it to a chaotic neuron map model not only numerically but also experimentally by analog circuit implementation. The control method is based on an adaptive feedback adjustment of a control parameter of the system, which uses typical bifurcation structures of nonlinear dynamical systems. We can observe a clear fractal structure in the sets composed of controlled states with respect to different initial conditions. We also discuss possible applications of the controlled system as an analog-valued memory with high-capacity.

We propose a multi-hysteresis voltage controlled current source (multi-hysteresis VCCS). The multi-hysteresis VCCS consists of multiple single-hysteresis VCCSs in parallel. The multi-hysteresis VCCS can exhibit various kinds of i - v characteristics. In addition, we introduce a chaotic oscillator by applying the multi-hysteresis VCCSs. The proposed oscillator is suitable for the IC implementation. A fully-differential multiscroll chaotic oscillator circuit is designed. The SPICE simulation results confirm the multi-hysteresis characteristics and various chaotic attractors.

In this paper, we improve performance of a chaotic neuro-computer in solving quadratic assignment problems (QAPs) by adopting an output-coding which constructs a feasible solution from analog internal-states of neurons at each iteration. Through measurements from the chaotic neuro-computer hardware, we show that we constantly obtain the optimum solution for size-10 QAPs. Furthermore, chaotic search dynamics through chaotic itinerancy is confirmed from time evolutions of a cost function and an energy function. Moreover, we observe internal states of arbitrary three neurons in a network to extract useful information on network dynamics that is effective in solving the QAPs.

We construct the dynamical associative memory on the switched-capacitor (SC) 400-neuron chaotic neuro-computer prototype. We observe a variety of associative dynamics from the prototype. The chaotic behavior of the dynamical association comes from complexity in real number. The analog SC chaotic neurons used in the system can handle real numbers through their continuous variables, therefore, they would faithfully reproduce the chaotic behavior. In construct, digital computers cannot handle almost all real numbers. In this respect, we analyze the measured results from the hardware system in comparison with those from computer simulations. In the computer simulation, we take into account characteristics of the analog circuit and noise.

The quadratic assignment problem (QAP) is one of the nondeterministic polynomial (NP)-hard combinatorial optimization problems. One of the heuristic algorithms for the QAP is the tabu-search. The exponential tabu-search has been implemented on a neural network, and further it has been extended to be driven by chaotic dynamics based on a chaotic neural network for efficient search. Moreover, chaotic dynamics has also been exploited to avoid the local minima problem. We propose a chaos driven tabu-search neural network hardware system with switched-current chaotic neuron ICs. We build a mixed analog/digital system for the size-10 QAP.

Effectiveness of chaotic search dynamics on solving combinatorial optimization problems including the quadratic assignment problem (QAP) has been shown with numerical simulations with chaotic neural networks. The powerful searching ability of the chaotic neuro-dynamics comes from complexity in real number. Therefore, it is essential to implement the chaotic search dynamics with devices that can handle real numbers such as analog electrical circuits. We construct a chaotic neuro-computer system consisting of 100 switched-capacitor (SC) chaotic neuron circuits connected via 10000 digital synapse circuits. We solve the QAPs with the prototype neuro-computer hardware system. As a result, we verify good performance of the analog massively parallel processing with chaotic neuro-dynamics in solving the QAPs.

We propose a mixed analog/digital system architecture of a chaos driven exponential tabu search for quadratic assignment problems. We construct a small size evaluation system using switched-capacitor chaotic neuron ICs and programmable logic devices. The experimental results from the system verify the validity and hardware compatibility of the proposed architecture.

We propose an integrated neuron circuit for an asynchronous pulse neural network model. This circuit is suitable for an implementation of a wide range of spatio-temporal coding networks since the circuit can function as both a coincidence detector and an integrator of the input pulses by properly setting bias voltage. We fabricate a prototype chip for the proposed circuit using MOSIS HP/Agilient 0.5 /spl mu/m CMOS semiconductor process. The experimental measurements from the chip confirm that the integrated neuron circuit qualitatively replicates the behavior of the model. Especially, coincidence detection of input pulses in a short-time window, and further, complex behavior including chaos in the internal state value and in the interspike intervals of the output pulses are illustrated.

We propose a neuron-synapse integrated circuit (IC) chip-set for large-scale chaotic neural networks. We use switched-capacitor (SC) circuit techniques to implement a three-internal-state transiently-chaotic neural network model. The SC chaotic neuron chip faithfully reproduces complex chaotic dynamics in real numbers through continuous state variables of the analog circuitry. We can digitally control most of the model parameters by means of programmable capacitive arrays embedded in the SC chaotic neuron chip. Since the output of the neuron is transfered into a digital pulse according to the all-or-nothing property of an axon, we design a synapse chip with digital circuits. We propose a memory-based synapse circuit architecture to achieve a rapid calculation of a vast number of weighted summations. Both of the SC neuron and the digital synapse circuits have been fabricated as IC forms. We have tested these IC chips extensively, and confirmed the functions and performance of the chip-set. The proposed neuron-synapse IC chip-set makes it possible to construct a scalable and reconfigurable large-scale chaotic neural network with 10000 neurons and 10000/sup 2/ synaptic connections.

We propose a simple speech compression algorithm using subband division and subadaptive piecewise linear quantization for a voice-mail system. Voice mail is an audio equivalent of sending letters. The main differences are that computer networks deliver the mail and that electronic recording is used instead of letters. Although speech data are stored in a semiconductor memory device, its capacity and the available network capacity are limited. Therefore, it is necessary to compress the data as much as possible. However, there are two conditions to be satisfied. The first is that the reconstructed datum must be understood correctly. Second, we need to identify the sender. Signals with a rate of 64 kbit/s are compressed at a ratio of about 1/9 using the proposed subband division and subadaptive piecewise linear quantization.

A dynamical associative memory network is experimentally constructed using integrated switched-capacitor chaotic neurons. Dynamical retrieval characteristics of memories, which correspond to the chaotic itinerancy, are observed. Network responses with different external stimuli are also investigated.

IC implementation of chaotic neuron using switched capacitor circuit technique is described. Various measured results are presented and interpreted. A new phenomenon called "transient chaos", which is essential for chaotic simulated annealing, is also demonstrated.>

A simple method for designing a hierarchical structure transversal filter (HSTF) with a linear phase is described. The HSTF can provide an output equivalent to a higher-order FIR (finite impulse response) digital filter due to using a number of lower-order FIR digital subfilters. Designing of the HSTF is required only to determine a low-order FIR subfilter provided that a multiple mapping function is given. Therefore, this method can save a lot of computation in the design of an FIR filter with a long length. This structure is also suitable for parallel processing concerned with the subfilters and an LSI implementation.>