Lower operating voltages and faster clock frequencies in advanced fabrication processes increase the circuit delay sensitivity to voltage, temperature, and process variations and modeling approximations. Uncorrelated delay variations along data and clock paths cause timing violations. In this paper, we propose a method for correcting timing violations by in-circuit tuning of clock latencies after fabrication. We introduce adaptive delay sequential elements (ADSEs) that use charge storage on pMOS floating gates to tune the clock latencies of timing critical flip-flops. ADSEs facilitate in-circuit optimization of clock latencies under varying operating conditions. ADSE tuned clock latencies are nonvolatile and can be repeatedly adjusted after fabrication using only electrical signals. We present examples of implicit and explicit pulsed ADSEs and their tuning operations. Our experiments with fabricated prototypes show that ADSEs can tune their clock latencies with picosecond resolution over one-half of the clock period. Our experiments also show that ADSE sensitivities to supply voltage, temperature, noise, and transistor mismatch are comparable to nonadaptive sequential elements. We present experimental data that show ADSE tuned delays change only 15% after ten years at 125degC. We propose a method for selective tuning of embedded ADSEs and demonstrate its application in a fabricated prototype. ADSEs can selectively replace timing-critical flip-flops of a circuit with negligible area impact

This paper introduces adaptive delay sequential elements (ADSEs). ADSEs are registers that use nonvolatile, floating-gate transistors to tune their internal clock delays. We propose ADSEs for correcting timing violations and optimizing circuit performance. We present an ADSE circuit example, system architecture, and tuning methodology. We present experimental results that demonstrate the correct operation of our example circuit and discuss the die-area impact of using ADSEs. Our experiments also show that voltage and temperature sensitivity of ADSEs are comparable to non-adaptive flip-flops.

Device mismatch in VLSI degrades the accuracy of analog arithmetic circuits and lowers the learning performance of large-scale neural net- works implemented in this technology. We show compact, low-power on-chip calibration techniques that compensate for device mismatch. Our techniques enable large-scale analog VLSI neural networks with learn- ing performance on the order of 10 bits. We demonstrate our techniques on a 64-synapse linear perceptron learning with the Least-Mean-Squares (LMS) algorithm, and fabricated in a 0.35m CMOS process.

This paper introduces the Field-Programmable Learning Array, a new paradigm for rapid prototyping of learning primitives and machine- learning algorithms in silicon. The FPLA is a mixed-signal counterpart to the all-digital Field-Programmable Gate Array in that it enables rapid prototyping of algorithms in hardware. Unlike the FPGA, the FPLA is targeted directly for machine learning by providing local, parallel, on- line analog learning using floating-gate MOS synapse transistors. We present a prototype FPLA chip comprising an array of reconfigurable computational blocks and local interconnect. We demonstrate the via- bility of this architecture by mapping several learning circuits onto the prototype chip.

We present the bump mixture model, a statistical model for analog data where the probabilistic semantics, inference, and learning rules derive from low-level transistor behavior. The bump mixture model relies on translinear circuits to perform probabilistic infer- ence, and floating-gate devices to perform adaptation. This system is low power, asynchronous, and fully parallel, and supports vari- ous on-chip learning algorithms. In addition, the mixture model can perform several tasks such as probability estimation, vector quanti- zation, classification, and clustering. We tested a fabricated system on clustering, quantization, and classification of handwritten digits and show performance comparable to the E-M algorithm on mix- tures of Gaussians.

Local long-term adaptation is a well-known feature of the synaptic junctions in nerve tissue. Neuroscientists have demonstrated that biology uses local adaptation both to tune the performance of neural circuits and for long-term learning. Many researchers believe it is key to the intelligent behavior and the efficiency of biological organizms. Although engineers use adaptation in feedback circuits and in software neural networks, they do not use local adaptation in integrated circuits to the same extent that biology does in nerve tissue. A primary reason is that locally adaptive circuits have proved difficult to implement in silicon. We describe complementary metal-oxide-semiconductor (CMOS) devices called synapse transistors that facilitate local long-term adaptation in silicon. We show that synapse transistors enable self-tuning analog circuits in digital CMOS, facilitating mixed-signal systems-on-a-chip. We also show that synapse transistors enable silicon circuits that learn autonomously, promising sophisticated learning algorithms in CMOS.

Competitive learning is a general technique for training clustering and classification networks. We have developed an 11-transistor silicon circuit, that we term an automaximizing bump circuit, that uses silicon physics to naturally implement a similarity computation, local adaptation, simultaneous adaptation and computation and nonvolatile storage. This circuit is an ideal building block for constructing competitive-learning networks. We illustrate the adaptive nature of the automaximizing bump in two ways. First, we demonstrate a silicon competitive-learning circuit that clusters one-dimensional (1-D) data. We then illustrate a general architecture based on the automaximizing bump circuit; we show the effectiveness of this architecture, via software simulation, on a general clustering task. We corroborate our analysis with experimental data from circuits fabricated in a 0.35-mum CMOS process.

Atmospheric scientists need to observe fluctuations in the ionosphere, both to probe the underlying atmospheric physics and to remove the effects of these fluctuations from other measurements. We have built an FPGA-based, pipelined array processor that allows us to make these observations in real-time, using passive radar techniques. Our array processor time-multiplexes 16 multiply-accumulators across 1536 radar ranges, performing a pipelined correlation and integration of the radar signal for each range. A DSP-based postprocessor generates real-time range-Doppler profiles of the ionospheric targets.

A Hidden-Articulator Markov Model (HAMM) is a Hidden Markov Model (HMM) in which each state represents an articulatory configuration. Articulatory knowledge, known to be useful for speech recognition [4], is represented by specifying a mapping of phonemes to articulatory configurations; vocal tract dynamics are represented via transitions between articulatory configurations. In previous work [13], we extended the articulatory-feature model introduced by Erler [7] by using diphone units and a new technique for model initialization. By comparing it with a purely random model, we showed that the HAMM can take advantage of articulatory knowledge. In this paper, we extend that work in three ways. First, we decrease the number of parameters, making it comparable in size to standard HMMs. Second, we evaluate our model in noisy contexts, verifying that articulatory knowledge can provide benefits in adverse acoustic conditions. Third, we use a corpus of sideby -side speech and articulator tra...

Competitive learning is a technique for training classification and clustering networks. We have designed and fabricated an 11- transistor primitive, that we term an automaximizing bump circuit, that implements competitive learning dynamics. The circuit per(cid:173) forms a similarity computation, affords nonvolatile storage, and implements simultaneous local adaptation and computation. We show that our primitive is suitable for implementing competitive learning in VLSI, and demonstrate its effectiveness in a standard clustering task.

We describe the design, fabrication, characterization, and modeling of an array of single transistor synapses. The single transistor synapses simultaneously perform long term weight storage, compute the product of the input and floating gate value, and update the weight value according to a hebbian or a backpropagation learning rule. The charge on the floating gate is decreased by hot electron injection with high selectivity for a particular synapse. The charge on the floating gate is increased by electron tunneling, which results in high selectivity between rows, but much lower selectivity between columns along a row. When the steady state source current is used as the representation of the weight value, both the incrementing and decrementing functions are proportional to a power of the source current.

We describe single-transistor silicon synapses that compute, learn, and provide non-volatile memory retention. The single transistor synapses simultaneously perform long term weight storage, com(cid:173) pute the product of the input and the weight value, and update the weight value according to a Hebbian or a backpropagation learning rule. Memory is accomplished via charge storage on polysilicon floating gates, providing long-term retention without refresh. The synapses efficiently use the physics of silicon to perform weight up(cid:173) dates; the weight value is increased using tunneling and the weight value decreases using hot electron injection. The small size and low power operation of single transistor synapses allows the devel(cid:173) opment of dense synaptic arrays. We describe the design, fabri(cid:173) cation, characterization, and modeling of an array of single tran(cid:173) sistor synapses. When the steady state source current is used as the representation of the weight value, both the incrementing and decrementing functions are proportional to a power of the source current. The synaptic array was fabricated in the standard 21'm double - poly, analog process available from MOSIS.

We present a silicon model of an axon which shows promise as a building block for pulse-based neural computations involving cor(cid:173) relations of pulses across both space and time. The circuit shares a number of features with its biological counterpart including an excitation threshold, a brief refractory period after pulse comple(cid:173) tion, pulse amplitude restoration, and pulse width restoration. We provide a simple explanation of circuit operation and present data from a chip fabricated in a standard 2Jlm CMOS process through the MOS Implementation Service (MOSIS). We emphasize the ne(cid:173) cessity of the restoration of the width of the pulse in time for stable propagation in axons.