Human capabilities in understanding visual relations are far superior to those of AI systems, especially for previously unseen objects. For example, while AI systems struggle to determine whether two such objects are visually the same or different, humans can do so with ease. Active vision theories postulate that the learning of visual relations is grounded in actions that we take to fixate objects and their parts by moving our eyes. In particular, the low-dimensional spatial information about the corresponding eye movements is hypothesized to facilitate the representation of relations between different image parts. Inspired by these theories, we develop a system equipped with a novel Glimpse-based Active Perception (GAP) that sequentially glimpses at the most salient regions of the input image and processes them at high resolution. Importantly, our system leverages the locations stemming from the glimpsing actions, along with the visual content around them, to represent relations between different parts of the image. The results suggest that the GAP is essential for extracting visual relations that go beyond the immediate visual content. Our approach reaches state-of-the-art performance on several visual reasoning tasks being more sample-efficient, and generalizing better to out-of-distribution visual inputs than prior models.

This demonstration implements efficient speech recognition through the use of biologically-inspired units implemented on the recently introduced Artificial Intelligence Unit (AIU Spyre). Speech recognition models approach human-level accuracy, but are significantly more power-hungry compared to the human brain. Efficient biological processes can be incorporated into deep learning models substantially reducing the computational cost and inference time. Simultaneously, novel chips, such as the AIU Spyre are designed from the ground up for energy-efficient execution of artificial neural networks. We demonstrate the scalability and efficiency of our neuro-inspired speech model through an implementation on the AIU Spyre.

LCN

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an exact mapping from a network with Rectified Linear Units (ReLUs) to an SNN that fires exactly one spike per neuron. For our constructive proof, we assume that an arbitrary multi-layer ReLU network with or without convolutional layers, batch normalization and max pooling layers was trained to high performance on some training set. Furthermore, we assume that we have access to a representative example of input data used during training and to the exact parameters (weights and biases) of the trained ReLU network. The mapping from deep ReLU networks to SNNs causes zero percent drop in accuracy on CIFAR10, CIFAR100 and the ImageNet-like data sets Places365 and PASS. More generally our work shows that an arbitrary deep ReLU network can be replaced by an energy-efficient single-spike neural network without any loss of performance.

Biological neural networks are equipped with an inherent capability to continuously adapt through online learning. This aspect remains in stark contrast to learning with error backpropagation through time (BPTT) that involves offline computation of the gradients due to the need to unroll the network through time. Here, we present an alternative online learning algorithm ic framework for deep recurrent neural networks (RNNs) and spiking neural networks (SNNs), called online spatio-temporal learning (OSTL). It is based on insights from biology and proposes the clear separation of spatial and temporal gradient components. For shallow SNNs, OSTL is gradient equivalent to BPTT enabling for the first time online training of SNNs with BPTT-equivalent gradients. In addition, the proposed formulation unveils a class of SNN architectures trainable online at low time complexity. Moreover, we extend OSTL to a generic form, applicable to a wide range of network architectures, including networks comprising long short-term memory (LSTM) and gated recurrent units (GRUs). We demonstrate the operation of our algorithm ic framework on various tasks from language modeling to speech recognition and obtain results on par with the BPTT baselines.

Automatic speech recognition systems (ASR), such as the recurrent neural network transducer (RNN-T), have reached close to human-like performance and are deployed in commercial applications. However, their core operations depart from the powerful biological counterpart, the human brain. On the other hand, the current developments in biologically-inspired ASR models lag behind in terms of accuracy and focus primarily on small-scale applications. In this work, we revisit the incorporation of biologically-plausible models into deep learning and enhance their capabilities, by taking inspiration from the brain’s diverse neural and synaptic dynamics. In particular, we propose novel deep learning units by introducing neural connectivity concepts emulating the axo-somatic and the axo-axonic synapses and integrate them into the RNN-T architecture. We demonstrate for the first time that such a model can yield performance levels competitive to the state-of-the-art. Moreover, our implementation has a significantly reduced computational cost and a lower latency.

Developing energy-saving neural network models is a topic of rapidly increasing interest in the artificial intelligence community. Spiking neural networks (SNNs) are biologically inspired models that strive to leverage the energy efficiency stemming from a long process of evolution under limited resources. In this paper we propose a SNN model where each neuron integrates piecewise linear postsynaptic potentials caused by input spikes and a positive bias, and spikes maximally once. Transformation of such a network into the ANN domain yields an approximation of a standard ReLU network, leading to a facilitated training based on backpropagation and an adaptation of the batch normalization. With backpropagation-trained weights, SNN inference offers a sparse-signal and low-latency classification, which can be readily adapted for a stream of input patterns, lending itself to an efficient hardware implementation. The supervised classification of MNIST and Fashion-MNIST datasets, using this approach, provides accuracy close to that of an ANN and surpassing other single-spike SNNs.

Neuromorphic computing takes inspiration from how the brain works to design power- and area-efficient hardware architectures for learning systems. Recently, unsupervised feature learning neuromorphic architectures have been presented, including a concept of synaptic competition that promotes the engagement of the synapses in the learning beyond weight storage. However, it is common to train these neuromorphic systems following the classic machine learning assumption of i.i.d. dataset sampling, which may not hold for real world inputs. In this paper, we propose a more realistic dataset sampling technique and apply it for online learning in a neuromorphic system using phase-change memristors as synapses and implementing synaptic competition. Furthermore, we propose a novel formulation of synaptic competition that captures orthogonal features, alternatively to independent components. We experimentally demonstrate the operation of the system for a non-i.i.d. stream and compare the performance to the models of lateral inhibition and dendritic inhibition. The obtained results demonstrate online feature learning capabilities of the proposed system and robustness to non-i.i.d. inputs.

Neuromorphic systems provide biologically inspired methods of computing, alternative to the classical von Neumann approach. In these systems, computation is performed by a network of spiking neurons controlled by the values of their synaptic weights, which are updated in the process of learning. Providing efficient synaptic learning rules, such as spike-timing-dependent plasticity (STDP), is a challenging task. These rules need to primarily use local information, but simultaneously develop a knowledge representation that is useful in the global context. From the implementation viewpoint, they also need to be suited for particular hardware technology. In this work, we propose a system with spiking neurons and synapses realized using phase-change devices. We design in a bottom-up manner an architecture for pattern learning and feature extraction. Experimental results from a prototype hardware platform demonstrate the capabilities of the proposed neuromorphic system.

Neuromorphic systems increasingly attract research interest owing to their ability to provide biologically inspired methods of computing, alternative to the classic von Neumann architecture. In these systems, computing relies on spike-based communication between neurons, and memory is represented by evolving states of the synaptic interconnections. In this work, we first demonstrate how spike-timing-dependent plasticity (STDP) based synapses can be realized using the crystal-growth dynamics of phase-change memristors. Then, we present a novel learning architecture comprising an integrate-and-fire neuron and an array of phase-change synapses that is capable of detecting temporal correlations in parallel input streams. We demonstrate a continuous re-learning operation on a sequence of binary 20×20 pixel images in the presence of significant background noise. Experimental results using an array of phase-change cells as synaptic elements confirm the functionality and performance of the proposed learning architecture.

In this paper, impulsive control is applied to a class of linear feedback systems and studied both theoretically and experimentally, with a particular focus on the usage in nanopositioning. By using impulsive control, improvements in tracking performance and tolerance to measurement noise can be achieved which are beyond the limits of conventional linear feedback.

Phase-change memory (PCM) has emerged as one among the most promising technologies for next-generation non-volatile solid-state memory. Multilevel storage, namely storage of non-binary information in a memory cell, is a key factor for reducing the total cost-per-bit and thus increasing the competiveness of PCM technology in the nonvolatile memory market. In this paper, we present a family of advanced programming schemes for multilevel storage in PCM. The proposed schemes are based on iterative write-and-verify algorithms that exploit the unique programming characteristics of PCM in order to achieve significant improvements in resistance-level packing density, robustness to cell variability, programming latency, energy- per-bit and cell storage capacity. Experimental results from PCM test-arrays are presented to validate the proposed programming schemes. In addition, the reliability issues of multilevel PCM in terms of resistance drift and read noise are discussed.

Probe-storage devices employ large arrays of probes to write/read data in parallel in some storage medium, and combine ultra-high density, low access times, and low power consumption. A particular probe-storage technique utilizes thermomechanical means to store and retrieve information in thin polymer films. In this paper, a system-level channel model for the thermomechanical probe-storage channel is presented. Each of the components of the proposed model is derived by extensive characterization of experimentally obtained readback signals from probe recording tests. Moreover, detection techniques that are actually utilized in a probe-storage prototype implementation are described, followed by coding techniques for added reliability in the presence of particles or other impurities of the storage medium. In addition to low-complexity coding constructs, a concatenated coding scheme with an outer LDPC and inner modulation code is considered, in order to establish a benchmark for overall system performance. A novel methodology for joint decoding of outer LDPC and inner (d,k) modulation codes is developed. Furthermore, an optimal soft decoder for the modulation code is proposed, based on a modification of the decoder metrics to accurately account for the probe storage channel output statistics. Experimental results are used throughout the paper to validate the channel model and identify its relevant parameters, as well as to verify the system performance obtained by simulations.

MEMS-based scanning-probe data storage devices are emerging as potential ultra-high-density, low-access-time, and low-power alternatives to conventional data storage. Thermomechanical probe-based storage on thin polymer films is arguably the most advanced scanning-probe data storage scheme. The performance evaluation of a small-scale storage device prototype based on this concept is presented. The emphasis is on understanding the timing jitter in the read-back signals. Experiments are performed that confirm that the primary source of timing-jitter is the nanometer-scale perturbations of the micro-scanner while positioning the recording medium relative to the read/write transducers. Analytical estimates of these micro-scanner perturbations are obtained. An extensive performance evaluation, using the experimentally identified channel and medium-noise spectral characteristics, is conducted to study the impact of the microscanner perturbations on the performance of the storage device.

Scanning-probe data storage is emerging as a viable alternative to conventional data storage, offering ultra-high density, low access times, and low power consumption. One probe-storage technique utilizes a thermomechanical means to store and retrieve information in thin polymer films. We describe the readback signal path and characterize the thermomechanical-based probe-storage recording channel. It is shown that this channel exhibits a particular nonlinear behavior at high storage densities or high recording power, that is, the energy per unit time used to write a bit of information. A simple model is proposed that accurately captures the characteristics of this nonlinearity. Experimental results from single-probe recording setups are used to verify the validity of this model and identify its relevant parameters.

100BASE-T has enhanced the performance of Ethernet networks from 10 to 100 Mbps. However, an Ethernet network under high offered load can produce unpredictable and excessive packet delays due to the capture effect introduced by the binary exponential backoff (BEB) algorithm. A solution that efficiently overcomes this problem is the PACE algorithm. This paper presents a performance analysis of several schemes of an algorithm based on PACE technology. We developed a mathematical model of the system and we used the equilibrium point analysis (EPA) technique in order to analyze the above model. It is shown that the proposed schemes improve the performance of the network both in terms of throughput and average packet delay.