Multiplicative stochasticity such as Dropout improves the robustness and gener- alizability deep neural networks. Here, we further demonstrate that always-on multiplicative stochasticity combined with simple threshold neurons provide a suf- ficient substrate for deep learning machines. We call such models Neural Sampling Machines (NSM). We find that the probability of activation of the NSM exhibits a self-normalizing property that mirrors Weight Normalization, a previously studied mechanism that fulfills many of the features of Batch Normalization in an online fashion. The normalization of activities during training speeds up convergence by preventing internal covariate shift caused by changes in the distribution of inputs. The always-on stochasticity of the NSM confers the following advantages: the network is identical in the inference and learning phases, making the NSM a suitable substrate for continual learning, it can exploit stochasticity inherent to a physical substrate such as analog non-volatile memories for in memory computing, and it is suitable for Monte Carlo sampling, while requiring almost exclusively addition and comparison operations. We demonstrate NSMs on standard classification benchmarks (MNIST and CIFAR) and event-based classification benchmarks (N-MNIST and DVS Gestures). Our results show that NSMs perform comparably or better than conventional artificial neural networks with the same architecture.

As we approach the end of the silicon road map, alternative computing models that can solve at-scale problems in the data-centric world are becoming important. This is accompanied by the realization that binary abstraction and Boolean logic, which have been the foundations of modern computing revolution, fall short of the desired performance and power efficiency. In particular, hard computing problems relevant to pattern matching, image and signal processing, optimizations, and neuromorphic applications require alternative approaches. In this paper, we review recent advances in oscillatory dynamical system-based models of computing and their implementations. We show that simple configurations of oscillators connected using simple electrical circuits can result in interesting phase and frequency dynamics of such coupled oscillatory systems. Such networks can be controlled, programmed, and observed to solve computationally hard problems. Although our discussion in this paper is limited to insulator-to-metal transition devices and spin-torque oscillators, the general philosophy of such a computing paradigm of “let physics do the computing” can be translated to other mediums as well, including micromechanical and optical systems. We present an overview of the mathematical treatments necessary to understand the time evolution of these systems and highlight the recent experimental results in this area that suggest the potential of such computational models.

In this paper, we consider devices, circuits, and systems comprised of transistors with integrated ferroelectrics. Said structures are actively being considered by various semiconductor manufacturers as they can address a large and unique design space. Transistors with integrated ferroelectrics could (i) enable a better switch (i.e., offer steeper subthreshold swings), (ii) are CMOS compatible, (iii) have multiple operating modes (i.e., I-V characteristics can also enable compact, 1-transistor, non-volatile storage elements, as well as analog synaptic behavior), and (iv) have been experimentally demonstrated (i.e., with respect to all of the aforementioned operating modes). These device-level characteristics offer unique opportunities at the circuit, architectural, and system-level, and are considered here from device, circuit/architecture, and foundry-level perspectives.

This paper reviews an analog circuit system of capacitively coupled relaxation oscillators whose time evolution can be used to solve the graph coloring problem. These oscillators consist of a series combination of an insulator-metal-transition (IMT) device and a resistance. Such circuits were also demonstrated experimentally using VO2(Vanadium Dioxide) as the phase transition material. The time evolution of circuit dynamics depend on eigenvectors of the adjacency matrix in the same way as is used by spectral algorithms for graph coloring. As such, a coupled network of such oscillators with piecewise linear dynamics have steady state phases which can be used to approximate the minimum vertex coloring of a graph.

In this paper we review recent work on computational paradigms involving coupled relaxation oscillators built using metal-insulator-transition (MIT) devices. Such oscillators made using MIT devices based on Vanadium-Dioxide thin films are very compact and can be realized in hardware. Networks of such oscillators have interesting phase and frequency dynamics which can be programmed to solve computationally hard problems.