A novel spike-based computation architecture has been developed which represents synaptic weights in time. An analog chip with 32 neurons, 1024 synapses and an AER block has been fabricated in 0.5μm technology. A digital implementation of the architecture having 6,144 neurons and 100,352 synapses on an FPGA is also described. A digital controller for routing spikes can processes up to 34 million synapses per second. The architecture is called the time machine as it operates on timing events and uses time to store weights. The time machine is general enough for implementing many spike-based algorithms yet provides flexibility and configurability.

We evaluate the performance of an asynchronous analog-to-digital converter known as the Integrate and Fire (I&F) circuit. The reconstruction algorithm for the sampler is presented along with the analytical results for the signal-to-noise ratio (SNR) of the reconstructed signal. We present the power vs. SNR tradeoff, based on the measurements from the chip fabricated in AMI 0.6 µm technology. The figure-of-merit (FOM) of the I&F sampler is 0.6 pJ/conversion, which is comparable to the reported FOM for other synchronous and asynchronous ADCs, even though the I&F is implemented in a much older technology as compared to the other ADCs.

We present the integrate-and-fire (IF) model as a time encoding scheme and show that it corresponds to a special case of the asynchronous Σ-Δ modulator (ASDM). We depart from typical rate based encoding used by conventional ASDM towards time based encoding. We show that for certain families of signals, such as neural recordings, the IF sampler accurately encodes the regions of interest at sub-Nyquist data rates. Nevertheless, the encoder requires nonlinear recovery algorithms, instead of the typical low pass filter.

A higher-order model to determine audibility of audio signals is presented. Previous models have been energy based (second order) and adequate only for stationary, narrow-band signals. Music, speech and other audio signals are nonstationary and wideband so traditional energy models poorly predict the audibility of these sounds. The predictions from the higher-order model are compared to actual subjective listening tests to show that the higher-order, wide-band technique outperforms previous models.

The nature of spike coding of auditory signals is studied by comparing mean firing rate codes with conventional approaches in speech recognition tasks. Mean firing rate spike representations are problematic since most auditory nerve fibers are saturated at typical conversation levels. However, these problems are eased somewhat when it is considered that there are other nerve fibers (e.g. low spontaneous firing rate fibers) that could efficiently encode the information at each channel of the cochlea. We show that window-based, mean firing rate features can be used with a crude cochlea model to achieve the same level of performance as a conventional MFCC-based approach. These results assume that there are sufficient neurons available at each channel to average the randomness of the stochastic spike trains. Furthermore, we argue that the mean firing rate features could be augmented with timing cues for further performance improvement.

Humans are able to process speech and other sounds effectively in adverse environments, hearing through noise, reverberation, and interference from other speakers. To date, machines have been unable to match human performance. One profound difference between biological and engineering systems comes at the input stage. In machines, an acoustic signal is typically chopped into short equally spaced segments in time. In biological systems, the cochlea outputs asynchronous spikes that respond in real-time to acoustic inputs. In this paper we describe a spiking cochlea implementation and recent experiments in both speaker and speech recognition that use spikes as input.

We present a low-bandwidth analog circuit for implementing an adaptive biphasic leaky integrate-and-fire neuron. This neuron circuit is targeted for signal compression in neural recording applications. Unlike other adaptive neuron circuits, this adaptive integrate-and-fire neuron supports signal reconstruction with known threshold voltages. Matlab simulations show promising bandwidth reduction comparing to an integrate-and-fire neuron without the adaptive feature. We quantify the circuit performance in terms of the tradeoff between signal reconstruction accuracy and bandwidth.

We have developed an adaptively biased integrate and fire (I&F) circuit to reduce the power consumption of our ADC replacement approach. Unlike conventional circuits this I&F consumes no standby power. Power consumed is input signal dependent and increases exponentially as the input approaches the set threshold value. We describe the circuit model and present an intuitive as well as mathematical model for the power savings. Results from the chip fabricated in AMI 0.6 mum technology are also presented.

A biologically plausible algorithm is proposed for phoneme recognition that makes use of spikes for computation. The prototype system is demonstrated on voiced phonemes and shows competitive performance with state-of-the-art systems on a vowel dataset in the presence of noise. Using novel phase locking cues, the algorithm performs surprisingly well down to SNR values as low as 5 dB, where the performance of the baseline algorithm used for comparison drops considerably. These results suggest, not only a possible explanation for the extreme noise robustness of human recognition, but also a novel technique for information processing for speech signals with computers.

A neuronal recording system for brain-machine interfaces (BMI) based on asynchronous biphasic pulse coding is described. It demonstrates the first step in the development of a completely implanted wireless solution with a fully integrated circuit architecture. A recording experiment comparing, in parallel, a commercial recording system (Tucker-Davis technology (TDT)) and the UF's custom solution (FWIRE) is set up to compare performance. The novel aspect of the UF system is that the analog signal is represented by an asynchronous pulse train, which provides a low-power, low-bandwidth, noise-resistant means for coding and transmission. Taking advantage of neural firing features, the pulse-based approach uses only 3 K pulses/second to record a 25 kHz bandwidth signal from a hardware neural simulator.

We present a system consisting of a spiking cochlea chip and real-time event-based processing software that is able to discriminate between two sets of sounds based on their periodicity content. The periodicity measurements are computed from the spike timing information of asynchronous output spikes from the binaural spiking-cochlea chip. The chip consists of a matched pair of silicon cochlea with an address event interface for the output. Each section of the cochlea is modeled by a second-order low-pass filter followed by a simplified Inner Hair Cell circuit and a Spiking Neuron circuit. We show discrimination results using the periodicity measure for 2 classes of sound and preliminary localization results based on a discriminated sound.

Today's implanted neural systems are bound by tight constraints on power and communication bandwidth. Most conventional ADC-based approaches fall into two categories. Either they transmit all of the information at the Nyquist rate but are ultimately limited to only a handful of channels due to communication bandwidth constraints. Or they perform spike detection on the front-end which allows a scale up to 100 or more channels but prevents the use of spike sorting on the backend. Spike sorting is an important step that provides a labeling to multiple neurons on each channel and further improves the accuracy of spike detection. In this paper we describe the pulse-based approach used in the FWIRE (Florida wireless implantable recording electrodes) project. A hardware spiking neuron on each channel is configured either to transmit pulses for full reconstruction on the back-end, or to transmit dramatically fewer pulses but still allow for spike sorting on the back-end. Spike sorting results show that the pulse-based spike sorting accuracy is competitive with conventional methods used in daily practice.

There has been significant interest in encoding information into asynchronous timing events or spikes using models of biological neurons. It has been shown that these neurons can act as lossless encoders and software algorithms can reconstruct the encoded information. We present here, a real time implementation of a reconstruction algorithm on a TI TMS320C6713 DSK board. This implementation reconstructs the signal from a spike train which was encoded by a fabricated neuron chip. A signal to error ratio of 32 dB with an average firing rate of 8 KHz is obtained. We discuss the practical implementation, challenges and possible future directions.

We have developed the time derivative neuron to reduce the bandwidth needed to encode continuous-time input signals using spiking neuron models. Unlike conventional spiking neuron models, the time derivative neuron generates more spikes in regions of high change and is essentially invariant to DC shifts of the input. We describe the model and develop an efficient reconstruction algorithm to demonstrate its reconstruction accuracy and transmission bandwidth. The idea has both digital and analog hardware realizations and we show a SPICE transistor- level simulation of an efficient analog CMOS implementation.

We propose a duplex theory of spike coding in the early stages of the auditory system based on the intensity and noise levels of the acoustic stimuli. According to this concept, at low intensity levels, where auditory nerve firings cannot generate a high enough synchrony among neuron ensembles, rate coding is more likely favored against phase-locking via synchrony coding. To the contrary, at conversational intensity levels, phase synchrony coding is preferred due to its superior and highly noise robust performance. The theory is supported by both evidence from biology, as well as from experimental simulations using biologically plausible models of the entire processing chain from spike generation to recognition.

Neurons are point process systems, in the sense that the inputs and output which are spike trains can be treated as point processes. System identification of a point process system has been studied mostly with single input. However, multiple input is required in many applications such as liquid state machines or neural prosthetics. We propose a simple multiple-input spike based adaptive filter which is based on an integrate-and-fire neuron model. The optimal closed solution is derived, and the performance is analyzed with respect to noise in various parameters and measurement.

This paper reviews on-going efforts towards the development of the Florida wireless implantable recording electrodes (FWIRE). The FWIRE microsystem platform is a fully implantable flexible substrate microelectrode array that employs state-of-the-art integrate-and-fire (IF) signal representation and wireless interface circuitry for recording neural activity from behaving rodents. The modular nature of the implantable neural recording electrode allows future enhancements to be seamlessly added to improve functionality including but not limited to rechargeability through inductive coupling, custom microelectrode arrays, higher capacity batteries, and more advanced integrated circuit technologies. This paper concentrates on custom integrated circuits such as neural interfacing amplifiers, baseband signal processing, wireless data and power interfaces, and battery management system.

We propose SHAPE, a novel pitch estimation algorithm (PEA) that builds upon previous PEAs, addressing and solving some of their limitations. The SHAPE algorithm estimates the pitch using a smooth function to compute the average peak to valley distance at harmonic locations. This is done by performing an integral transform over the square-root magnitude of the spectrum using a truncated decaying cosine as a kernel. SHAPE was designed to both match human perception particularly for inharmonic sounds and to be competitive with numerous other PEAs as demonstrated on a common database.

A novel precompensation algorithm for a pulse width modulation (PWM)-based digital audio amplifier has been developed for portable applications. The straight line segment PWM (SLSPWM) algorithm results in both low-power and low-complexity devices. The division-free version of this algorithm modifies the algorithm by removing the division using a polynomial approximation. Both SLSPWM and division-free SLSPWM feature 16-bit quality performance with fewer calculations compared with previous works.

The echo state network (ESN) is a recurrent neural network proposed by Herbert Jaeger with a simplified training routine. Previously, we have demonstrated the noise-robust performance of the predictive ESN classifier in automatic speech recognition experiments in which the network was trained to predict the next frame of speech features. Classification performance was limited because the predictive models lacked discriminability, so we changed the model output to a one-of-many output encoding scheme and trained the model discriminatively. Performance was compared to a hidden Markov model (HMM) in small-vocabulary ASR experiments with additive noise. Accuracy of 50% was achieved by a discriminative ESN classifier at -8.5 dB SNR, compared to 0.4 dB SNR for a predictive ESN classifier and 6.6 dB SNR for an HMM. With discriminative training, a larger reservoir was employed for the discriminative ESN classifier compared to the predictive ESN classifier which resulted a larger memory depth and more noise-robust performance.

We propose a noise robust feature extraction technique for speech signals using phase synchrony. The front-end employs a psychoacoustic cochlea model with inner hair cells to transform speech into a parallel stream of spike trains as observed in the auditory nerve fibers. The degree of phase synchrony among nerve fibers with similar characteristic frequencies is calculated to yield a feature vector which shows little degradation in response to increasing levels of noise. As a benchmark, the feature set is used in a biologically plausible model with a spike-based, liquid state machine classifier for a simple acoustic classification task. Though applied to a simplified domain, the results indicate a superior performance when compared to a conventional speech recognition system, especially at very low signal-to-noise ratios.

Artificial neural networks have been shown to perform well in automatic speech recognition (ASR) tasks, although their complexity and excessive computational costs have limited their use. Recently, a recurrent neural network with simplified training, the echo state network (ESN), was introduced by Jaeger and shown to outperform conventional methods in time series prediction experiments. We created the predictive ESN classifier by combining the ESN with a state machine framework. In small-vocabulary ASR experiments, we compared the noise-robust performance of the predictive ESN classifier with a hidden Markov model (HMM) as a function of model size and signal-to-noise ratio (SNR). The predictive ESN classifier outperformed an HMM by 8-dB SNR, and both models achieved maximum noise-robust accuracy for architectures with more states and fewer kernels per state. Using ten trials of random sets of training/validation/test speakers, accuracy for the predictive ESN classifier, averaged between 0 and 20 dB SNR, was 81plusmn3%, compared to 61plusmn2% for an HMM. The closed-form regression training for the ESN significantly reduced the computational cost of the network, and the reservoir of the ESN created a high-dimensional representation of the input with memory which led to increased noise-robust classification.

Automatic speech processing research has produced many advances in the analysis of time series. Knowledge of the production and perception of speech has guided the design of many useful algorithms, and automatic speech recognition has been at the forefront of the machine learning paradigm. In contrast to the advances made in automatic speech processing, analysis of other bioacoustic signals, such as those from dolphins and bats, has lagged behind. In this paper, we demonstrate how techniques from automatic speech processing can significantly impact bioacoustic analysis, using echolocating bats as our model animal. Compared to conventional techniques, machine learning methods reduced detection and species classification error rates by an order of magnitude. Furthermore, the signal-to-noise ratio of an audible monitoring signal was improved by 12 dB using techniques from noise-robust feature extraction and speech synthesis. The work demonstrates the impact that speech research can have across disciplines

An asynchronous biphasic pulse train signal representation mechanism is proposed for low bandwidth, low power sensor applications. We prove that this novel pulse coding method provides lossless encoding theoretically. We also present a CMOS circuit realization using the AMI 0.6 mum process. Cadence simulation results show that the reconstructed signal has 8 effective bits of resolution with less than 100 muW of total power consumption

The echo state network (ESN) has been recently proposed as an alternative recurrent neural network model. An ESN consists of a reservoir of conventional processing elements, which are recurrently interconnected with untrained random weights, and a readout layer, which is trained using linear regression methods. The key advantage of the ESN is the ability to model systems without the need to train the recurrent weights. In this paper, we use an ESN to model the production of speech signals in a classification experiment using isolated utterances of the English digits "zero" through "nine." One prediction model for each digit was trained using frame-based speech features (cepstral coefficients) from all train utterances, and the readout layer consisted of several linear regressors which were trained to target different portions of the time series using a dynamic programming algorithm (Viterbi). Each novel test utterance was classified with the label from the digit model with the minimum mean squared prediction error. Using a corpus of 4130 isolated digits from 8 male and 8 female speakers, the highest classification accuracy attained with an ESN was 100.0% (99.1%) on the train (test) set, compared to 100% (94.7%) for a hidden Markov model (HMM). HMM performance increased to 100.0% (99.8%) when context features (first- and second-order temporal derivatives) were appended to the cepstral coefficients. The ESN offers an attractive alternative to the HMM because of the ESN's simple train procedure, low computational requirements, and inherent ability to model the dynamics of the signal under study

In this paper an architecture, signal reconstruction algorithm and first-ever implementation of an asynchronous delta-sigma converter are presented. The signal reconstruction algorithm can mathematically perfectly reconstruct the original signal only using timing events. A prototype circuit designed and fabricated in a standard 0.5/spl mu/m CMOS process with a 5V power supply is presented. The tests show that an 8-bit resolution with 6kHz signal bandwidth and only 715 /spl mu/W power consumption is possible.

A warped filter is presented as a new speech enhancement method to adjust formant bandwidths on a critical band scale. The warped filter enhances perceived loudness without adding signal energy by exploiting the psychoacoustic nature of the auditory system. The critical band concept in auditory theory states that when the energy in a signal remains constant, loudness increases when the energy spreads beyond a critical bandwidth. A warped filter is proposed and developed to elevate the perceived loudness of clean speech by applying nonlinear bandwidth expansion to the formant regions of vowels in accordance with the critical band scale. The filter has been inspired and motivated by the biological representation of loudness in the peripheral auditory system and the critical band concept of hearing.

Throughout the years digital signal processing has been encroaching more and more into what had previously been exclusive analog circuit domains. Here we briefly summarize various lines of research in CMOS imaging sensing technology paying special attention to the gradual transition from analog to digital processing. The transition from continuous-time continuous-value pixels to digital (discrete-time binary-value) pixels could be viewed as the natural final step in the evolution of these imaging circuits in order to achieve wide dynamic range with acceptable signal to noise ratio. After data is readoff a digital pixel imager chip, digital signal processing is required to reconstruct the image for display.

Many speech recognition systems use mel-frequency cepstral coefficient (mfcc) feature extraction as a front end. In the algorithm, a speech spectrum passes through a filter bank of mel-spaced triangular filters, and the filter output energies are log-compressed and transformed to the cepstral domain by the OCT. The spacing of filter bank center frequencies mimics the known warped-frequency characteristics of the human auditory system, yet the bandwidths of these filters is not chosen through biological inspiration. Instead they are set by aligning endpoints of the triangle, which is itself an arbitrary shape. It is surprising that for such a popular speech recognition front end, proper analysis or optimization of the filter bandwidths has not been performed. With complex cochlear models, realistic filter shapes that more closely approximate critical bands are used. And these filters, compared to the filters used in mfcc, are considerably wider and overlap with neighboring filters more. We have extended this filter characteristic to the mfcc algorithm and found that the increased filter bandwidth improves recognition performance in clean speech and provides added noise robustness as well.

This paper reviews the problem of translating signals into symbols preserving maximally the information contained in the signal time structure. In this context, we motivate the use of nonconvergent dynamics for the signal to symbol translator. We then describe a biologically realistic model of the olfactory system proposed by W. Freeman (1975) that has locally stable dynamics but is globally chaotic. We show how we can discretize Freeman's model using digital signal processing techniques, providing an alternative to the more conventional Runge-Kutta integration. This analysis leads to a direct mixed-signal (analog amplitude/discrete time) implementation of the dynamical building block that simplifies the implementation of the interconnect. We present results of simulations and measurements obtained from a fabricated analog VLSI chip.

Psychophysical experiments inspire a more complete analysis of the effect of quantization on a modified version of the histogram indexing method of object recognition. We derive an equation that describes how the amount of quantization and number of features kept affects the recognition accuracy. The equation shows that quantization from 2/sup 24/ colors to 15 colors has a negligible effect on accuracy. A simulation shows that large numbers of objects cause a corresponding decrease in accuracy, but that keeping more features can increase the accuracy even for massive quantization. An object recognition experiment with real data shows dramatically better results when quantization is used, indicating that massive color quantization can provide some invariance to lighting conditions.

We have combined the standard front-end filter bank with a feature extraction process to produce a model that requires dramatically less hardware (or software). This new model, based on the frequency-sampled ratio spectrum, can be interpreted as a small set of constant-Q filters whose center frequencies adapt to locations of high signal energy, The resulting feature vectors are shown to outperform several competing techniques for phoneme recognition, Results from fabricated CMOS analog VLSI circuits illustrate a hardware efficient method to sample the ratio spectrum.

This paper presents a low power MOS-VLSI implementation of an oscillator proposed by Freeman (1975) to model a very important component of the olfactory cortex. The model dynamics has time constants in the order of 1/220 (s). To accomplish the long time constants a new filtering technique recently proposed was utilized. All the blocks involving information processing were designed to operate below threshold (weak inversion).

In adaptive signal processing, the least-mean squares (LMS) algorithm has long been used in signal enhancement and noise cancellation but it cannot overcome the difficulty caused by the signal leakage into the reference input. Hence we have to explore more general statistical properties about the observed signals. This view corresponds to a statistical modeling of the signals using statistical measures such as a loss function, which is different from the mutual information. This paper proposes a new loss function based on generalized Gaussian distribution family, and derives new simple adaptive learning rules. Our separator based on the new generalized "anti-Hebbian rules" is also justified by the simulation on both artificial and real data with good performance.

Using clues from neurobiological adaptation, we have developed the constant-statistics (CS) algorithm for nonuniformity correction of infrared focal point arrays (IRFPAs) and other imaging arrays. The CS model provides an efficient implementation that can also eliminate much of the ghosting artifact that plagues all scene-based nonuniformity correction (NUC) algorithms. The CS algorithm with deghosting is demonstrated on synthetic and real infrared (IR) sequences and shown to improve the overall accuracy of the correction procedure.

Generalized smoothing networks have been developed which enforce smoothness constraints for any arbitrary level of derivative of the input data. Furthermore, discontinuities of any order of derivative can be detected by providing for continuous line processes, which selectively inhibit smoothing. Second- and higher-order networks are required for many problems in early vision; first-order networks are often unsatisfactory. Examples in surface interpolation, edge detection, and image segmentation are shown. Solution of these types of problems typically takes a prohibitive amount of time, even on supercomputers. A significant advantage of these proposed networks is that they can be mapped directly to analog VLSI hardware.>

The authors present an operational perception system for cross-country navigation which has been verified in both simulated and real-world environments. Range data from a laser range scanner is transformed into an alternate representation called the Cartesian elevation map (CEM). A detailed vehicle model operates on the CEM to produce traversability information along selected trajectories. This information supports a real-time reflexive planning system. The authors successfully demonstrated their obstacle detection and avoidance algorithms on board the autonomous land vehicle at Martin Marietta Aerospace Corporation in Denver. They also propose enhancements to their current operational system.>

A description is given of the first cross-country map and sensor-based autonomous operation of a robotic vehicle. Experiments on the autonomous land vehicle (ALV) in natural terrain were performed. An overview of the software architecture is provided, and details of the perception and planning techniques are presented. Two key experiments in which the vehicle avoided known and unknown obstacles in its path are described.>

This paper describes an object recognition algorithm both on a sequential machine and on a single instruction multiple data (SIMD) parallel processor such as the MIT connection machine. The problem, in the way it is presently formulated on a sequential machine, is essentially a propagation of constraints through a tree of possibilities in an attempt to prune the tree to a small number of leaves. The tree can become excessively large, however, and so implementations on massively parallel machines are sought in order to speed up the problem. Two fast parallel algorithms are described here, a static algorithm and a dynamic algorithm. The static algorithm reformulates the problem by assigning every leaf in the completely expanded unpruned tree to a separate processor in the connection machine. Then pruning is done in nearly constant time by broadcasting constraints to the entire SIMD array. This parallel version is shown to run three to four orders of magnitude faster than the sequential version. For large recognition problems which would exceed the capacity of the machine, a dynamic algorithm is described which performs a series of loading and pruning steps, dynamically allocating and deallocating processors through the use of the connection machine's global router communications mechanism.