Based on the predictive coding theory in neuro- science, we designed a bi-directional and recur- rent neural net, namely deep predictive coding networks (PCN), that has feedforward, feedback, and recurrent connections. Feedback connections from a higher layer carry the prediction of its lower-layer representation; feedforward connec- tions carry the prediction errors to its higher-layer. Given image input, PCN runs recursive cycles of bottom-up and top-down computation to update its internal representations and reduce the differ- ence between bottom-up input and top-down pre- diction at every layer. After multiple cycles of recursive updating, the representation is used for image classification. With benchmark datasets (CIFAR-10/100, SVHN, and MNIST), PCN was found to always outperform its feedforward-only counterpart: a model without any mechanism for recurrent dynamics, and its performance tended to improve given more cycles of computation over time. In short, PCN reuses a single architecture to recursively run bottom-up and top-down pro- cesses to refine its representation towards more accurate and definitive object recognition.

Deep Neural Networks (DNNs) are the algorithm of choice for image processing applications. DNNs present highly parallel workloads that lead to the emergence of custom hardware accelerators. Deep Learning (DL) models specialized in different tasks require a programmable custom hardware and a compiler/mapper to efficiently translate different DNNs into an efficient dataflow in the accelerator. The goal of this paper is to present a compiler for running DNNs on Snowflake, which is a programmable hardware accelerator that targets DNNs. The compiler correctly generates instructions for various DL models: AlexNet, VGG, ResNet and LightCNN9. Snowflake, with a varying number of processing units, was implemented on FPGA to measure the compiler and Snowflake performance properties upon scaling up. The system achieves 70 frames/s and 4.5 GB/s of off-chip memory bandwidth for AlexNet without linear layers on Xilinx’s Zynq-SoC XC7Z045 FPGA.

Inspired by "predictive coding" - a theory in neuroscience, we develop a bi-directional and dynamic neural network with local recurrent processing, namely predictive coding network (PCN). Unlike feedforward-only convolutional neural networks, PCN includes both feedback connections, which carry top-down predictions, and feedforward connections, which carry bottom-up errors of prediction. Feedback and feedforward connections enable adjacent layers to interact locally and recurrently to refine representations towards minimization of layer-wise prediction errors. When unfolded over time, the recurrent processing gives rise to an increasingly deeper hierarchy of non-linear transformation, allowing a shallow network to dynamically extend itself into an arbitrarily deep network. We train and test PCN for image classification with SVHN, CIFAR and ImageNet datasets. Despite notably fewer layers and parameters, PCN achieves competitive performance compared to classical and state-of-the-art models. Further analysis shows that the internal representations in PCN converge over time and yield increasingly better accuracy in object recognition. Errors of top-down prediction also reveal visual saliency or bottom-up attention.

Since the emergence of Deep Neural Networks (DNNs) as a prominent technique in the field of computer vision, the ImageNet classification challenge has played a major role in advancing the state-of-the-art. While accuracy figures have steadily increased, the resource utilization of winning models has not been properly taken into account. In this work, we present a comprehensive analysis of important metrics in practical applications: accuracy, memory footprint, parameters, operations count, inference time and power consumption. Key findings are: (1) power consumption is independent of batch size and architecture; (2) accuracy and inference time are in a hyperbolic relationship; (3) energy constraint is an upper bound on the maximum achievable accuracy and model complexity; (4) the number of operations is a reliable estimate of the inference time. We believe our analysis provides a compelling set of information that helps design and engineer efficient DNNs.

Recurrent Neural Networks (RNNs) have the ability to retain memory and learn from data sequences, which are fundamental for real-time applications. RNN computations offer limited data reuse, which leads to high data traffic. This translates into high off-chip memory bandwidth or large internal storage requirement to achieve high performance. Exploiting parallelism in RNN computations are bounded by this two limiting factors, among other constraints present in embedded systems. Therefore, balance between internally stored data and off-chip memory data transfer is necessary to overlap computation time with data transfer latency. In this paper, we present three hardware accelerators for RNN on Xilinx's Zynq SoC FPGA to present how to overcome challenges involved in developing RNN accelerators. Each design uses different strategies to achieve high performance and scalability. Each co-processor was tested with a character level language model. The latest design called DeepRnn, achieves up to 23 X better performance per power than Tegra X1 development board for this application.

Deep learning is becoming increasingly popular for a wide variety of applications including object detection, classification, semantic segmentation and natural language processing. Convolutional neural networks (CNNs) are a type of deep neural network that achieve high accuracy for these tasks. CNNs are hierarchical mathematical models comprising billions of operations to produce an output. The high computational complexity combined with the inherent parallelism in these models makes them an excellent target for custom accelerators. In this work we present Snowflake, a scalable, efficient low-power accelerator that is agnostic to CNN architectures. Our design is able to achieve an average computational efficiency of 91% which is significantly higher than comparable architectures. We implemented Snowflake on a Xilinx Zynq XC7Z045 APSoC. On this platform, Snowflake is capable of achieving 128 G-ops/s while consuming 9.48 W of power. Snowflake achieves a throughput and energy efficiency of 98 frames per second and 10.3 frames per joule, respectively, on AlexNet and 34 frames per second and 3.6 frames per joule on GoogLeNet.

Pixel-wise semantic segmentation for visual scene understanding not only needs to be accurate, but also efficient in order to find any use in real-time application. Existing algorithms even though are accurate but they do not focus on utilizing the parameters of neural network efficiently. As a result they are huge in terms of parameters and number of operations; hence slow too. In this paper, we propose a novel deep neural network architecture which allows it to learn without any significant increase in number of parameters. Our network uses only 11.5 million parameters and 21.2 GFLOPs for processing an image of resolution 3 × 640 × 360. It gives state-of-the-art performance on CamVid and comparable results on Cityscapes dataset. We also compare our networks processing time on NVIDIA GPU and embedded system device with existing state-of-the-art architectures for different image resolutions.

Deep convolutional neural networks (DCNNs) have become a very powerful tool in visual perception. DCNNs have applications in autonomous robots, security systems, mobile phones, and automobiles, where high throughput of the feedforward evaluation phase and power efficiency are important. Because of this increased usage, many field-programmable gate array (FPGA)-based accelerators have been proposed. In this paper, we present an optimized streaming method for DCNNs' hardware accelerator on an embedded platform. The streaming method acts as a compiler, transforming a high-level representation of DCNNs into operation codes to execute applications in a hardware accelerator. The proposed method utilizes maximum computational resources available based on a novel-scheduled routing topology that combines data reuse and data concatenation. It is tested with a hardware accelerator implemented on the Xilinx Kintex-7 XC7K325T FPGA. The system fully explores weight-level and node-level parallelizations of DCNNs and achieves a peak performance of 247 G-ops while consuming less than 4 W of power. We test our system with applications on object classification and object detection in real-world scenarios. Our results indicate high-performance efficiency, outperforming all other presented platforms while running these applications.

An updated image sensor to be used in a miniature microscope system for recording brain activity over a wide cortical area (4–9 mm2) is presented. The image sensor is a 100 × 100 pixel array fabricated in bulk CMOS process. The sensor also contains a 10-bit ADC and two input channels to measure other physiological signals concurrently with the optical data. Another feature of the image sensor is the ability to observe dF over sequential frames in hardware, in order to reduce the amount of data that must be read out of the image sensor at high speeds. The speed and noise performance of the image sensor is suitable for voltage-sensitive optical probes in small systems for awake and freely moving animals.

We present a non-coherent wireless communication testing platform for ultra wide-band impulse radio (UWB-IR) with frequency-shift-keying (FSK) on-off-keying (OOK). Both transmitter and receiver platform will be present with a control graphic user interface (GUI) supported by Spartan-3 FPGAs. The transmitter chip is integrated while the receiver is build from off-the-shelf components. Monopole antennas are used in the system. The demonstration shows a transformative wireless communication technology for low power low cost short range applications.

This paper proposes an algorithm for feedforward categorization of objects and, in particular, human postures in real-time video sequences from address-event temporal-difference image sensors. The system employs an innovative combination of event based hardware and bio-inspired software architecture. An event-based temporal difference image sensor is used to provide input video sequences, while a software module extracts size and position invariant line features inspired by models of the primate visual cortex. The detected line features are organized into vectorial segments. After feature extraction, a modified line segment Hausdorff distance classifier combined with on-the-fly cluster-based size and position invariant categorization. The system can achieve about 90 percent average success rate in the categorization of human postures, while using only a small number of training samples. Compared to state-of-the-art bio-inspired categorization methods, the proposed algorithm requires less hardware resource, reduces the computation complexity by at least five times, and is an ideal candidate for hardware implementation with event-based circuits.

In this paper we present a low power temporal-difference image sensor with wireless communication capability designed specifically for imaging sensor networks. The event-based image sensor features a 64 × 64 pixel array and can also report standard analog intensity images. An ultra-wide-band radio channel allows to transmit digital temporal difference images wirelessly to a receiver with high rates and reduced power consumption. The sensor can wake up the radio when it detects a specific number of pixels intensity modulation, so that only significant frames are communicated. The prototype chip was implemented using a 2-poly 3-metal AMIS 0.5 μ m CMOS process. Power consumption is 0.9 mW for the sensor and 15 mW for radio transmission to distance of 4 m with rates of 1.3 Mbps and 160 fps.

People rarely put in their papers the things that didn't work, the mistakes they made, and how they found out what went wrong. Such confessions can help others learn how to avoid similar mistakes. Twenty-six confessions were collected to form the bulk of this paper. Themes that arise are errors that result from not understanding the limitations of simulation tools in modeling physical reality, chip verification errors that result from lack of clear communication between designers, and projects that are considered in their own isolated environment of technical challenges rather than the broader context of their environment or application.

We present a micro-chip implementation of a low current measurement system for biomedical applications using capacitive feedback that exhibits 190 fA of RMS noise in a 1 kHz bandwidth. The sampling rate is selectable up to 100 kHz. When measuring the amplifier noise with a 10 GΩ resistor and a 47 pF capacitor at the input, typical of cell membrane capacitance in DNA and patch clamp experiments, the measured RMS noise was 2.44 pA on a 50 pA signal in a 10 kHz bandwidth. Two channels were implemented on 630 × 440 μm2using a 0.5-μm CMOS process. Each channel consumes 1.5 mW of power from a 3.3 V supply. We used our device to measure the characteristics of an artificial lipid bilayer similar to the ones used in DNA sequencing experiments via nanopores.

Clock-less synchronization and data recovery can be achieved with FSK-OOK modulation and demodulation in non-coherent UWB impulse radio (UWB-IR) system. In this work we report a 30 Mbps non-coherent FSK-OOK UWB-IR transmitter. The chip is fabricated with 0.25 μm Silicon on Sapphire (SOS) process. The chip size is 1 mm by 1 mm with pad. The power consumption with 2.5 V supply is 30 mW. A graphic user interface was developed to control the power and frequency of the transmitter as well as the data rate of the test-bench.

We demonstrate a low power temporal-difference image sensor with wireless communication capability designed specifically for imaging sensor networks. The event-based image sensor features a 64×64 pixel array and can also report standard analog intensity images. An ultra-wide-band (UWB) radio channel allows to transmit digital temporal difference images wirelessly to a receiver with high rates and reduced power consumption. The sensor wakes up when it detects enough scene changes and only communicates meaningful frames. A complete demo platform, for both the wireless sensor (transmitter) and a receiver was developed. Power consumption is 0.9 mW for the sensor and 15 mW for radio transmission to 4 m with rates of 1.3 Mbps and 160 fps.

In this paper we present a low power temporal-difference image sensor with wireless communication capability designed specifically for imaging sensor networks. The event-based image sensor features a 64 × 64 pixel array and can also report standard analog intensity images. An ultra-wideband (UWB) radio channel allows to transmit digital temporal difference images wirelessly to a receiver with high rates and reduced power consumption. The sensor wakes up when it detects enough scene changes and only communicates meaningful frames. Power consumption is 0.9 mW for the sensor and 15 mW for radio transmission to 4 m with rates of 1.3 Mbps and 160 fps.

The four chips presented in the special session on "Activity-driven, event-based vision sensors" quickly output compressed digital data in the form of events. These sensors reduce redundancy and latency and increase dynamic range compared with conventional imagers. The digital sensor output is easily interfaced to conventional digital post processing, where it reduces the latency and cost of post processing compared to imagers. The asynchronous data could spawn a new area of DSP that breaks from conventional Nyquist rate signal processing. This paper reviews the rationale and history of this event-based approach, introduces sensor functionalities, and gives an overview of the papers in this session. The paper concludes with a brief discussion on open questions.

In this paper we present a scalable hardware architecture to implement large-scale convolutional neural networks and state-of-the-art multi-layered artificial vision systems. This system is fully digital and is a modular vision engine with the goal of performing real-time detection, recognition and segmentation of mega-pixel images. We present a performance comparison between a software, FPGA and ASIC implementation that shows a speed up in custom hardware implementations.

Long term monitoring of neuronal activity in awake behaving subjects can provide fundamental information about brain dynamics for both neuroscience and neuroengineering applications. Recent advances in VLSI systems has focused on designing wireless neural recording systems which can be mounted on animals and acquire neural signals in real time. These advances provide an unparalleled opportunity to study phenomenon such as neural plasticity in both a basic science setting (learning and memory), and also a clinical setting (injury and recovery). Here we present an integrated VLSI system for wireless telemetry of the entire spectrum of neural signals, spikes, local field potentials, electrocorticograms (ECoG) and electroencephalograms (EEG). The system integrates two custom designed VLSI chips, a 16 channel neural interface which can amplify, filter and digitize neural data up to 16 kS/sec and 12 bits and a low power ultra-wideband (UWB) chip which can transmit data at rates up to 14 Mbps. The entire system which includes these VLSI circuits, a digital interface board and a battery, is small, 1.2 × 1.2 × 2.6 in3, and light weight, 33 grams, so it can be chronically mounted on a rat. The system consumes 32.8 mA at 3.3V and can record for 6 hours running from the 200 mAh coin cell battery. Bench-top and in vitro characterization of the system showed comparable performance to the wired recording system.

We present a custom image sensor capable of reporting intensity, spatial contrast, and temporal difference images at the pixel level. The smart pixel is composed of only 11 transistors, allowing tight integration of different functionalities in a 16×21 μm2pixel area. The sensor array is 128×128 pixels, with a fill factor of 42%, and operates at 800 fps and 13M events/s with a power consumption of 1.2mW.

In this paper, we report on the noise analysis of low current measurement systems for biomedical applications. We analyzed resistive feedback, capacitive feedback and current conveyor circuits for low current measurement systems. Detailed noise analysis are presented and matched with measurement data using a 0.5-μm fabrication process. Based on the theoretical analysis and the measurements, the capacitive feedback system provides better noise performance for the measurement of low current. The capacitive feedback is capable of measuring 700fA RMS at 10KHz sampling rate, whereas the resistive feedback provides 4pA and the current conveyor provides 600pA. This paper provides design guidelines to maximize the measurement performance of low current for biomedical instrumentation.

We present a 4-channel bio-potential recording system using asynchronous delta analog-to-digital converter. The system is designed to reduce the amount of data in neuro-physiological sensors. The circuit includes low-power low-noise amplifiers with asynchronous delta modulators. The analog front end offers a gain of 53 dB within bandwidth of 200 Hz-20 kHz. In this asynchronous data converter, the minimum input-referred delta resolution is 4μV. The input-referred rms noise is 2μV. T h e system was fabricated with AMI 0.5μm CMOS technology. The chip size is 1.5mm by 1.5mm. The power consumption of the 4-channel system with 3.3V supply is 118.8μW in static state and 501.6μW with 240kbps data conversion rate. A graphic user interface was developed to monitor the transmitted signal in real time.

We present the first, fully integrated, multi-channel implementation of a patch-clamp measurement system. The system was implemented in a 0.5 μm silicon-on-sapphire process. The system can record two simultaneous cell membrane currents up to ±20 μA with a rms noise of 8 pA in a 10 kHz bandwidth. The system can compensate for the capacitance and resistance of the pipette electrode, up to 10 pF and 100 MΩ respectively. The system die size is 3 × 3 mm and the power consumption is 5 mW per channel at 3.3 V.

This paper proposes a new approach to recognize human postures in realtime video sequences. The algorithm employs temporal difference imaging between video sequences as input and then decompose the contour of the active object into vectorial line segments. A scheme based on simplified line segment Hausdorff distance combined with projection histograms is proposed to achieve size and position invariance recognition. Consistent with the hierarchical model of the human visual system, sub-sampling techniques are used to represent the object by line segments at multiple resolution levels. The whole classification is described as a coarse to fine procedure. An average realtime recognition rate of 88% is achieved in the experiment. Compared to conventional convolution method, the proposed algorithm reduces the computation cycles by 10 - 100 times. This work sets the foundation for size and position invariant object recognition for the implementation of event-based vision systems.

We demonstrate a realtime human postures recognition platform. The algorithm employs temporal difference imaging between video sequences as input and then decompose the contour of the active object into vectorial line segments. A scheme based on simplified line segment Hausdorff distance combined with projection histograms is proposed to achieve size and position invariance recognition. Inspired by the hierarchical model of human visual system, the whole classification is described as a coarse to fine procedure. 88% average realtime recognition rate is achieved in the experiment.

An image sensor to be used in a miniature microscope system for recording brain activity over a wide cortical area (4-9 mm2) is presented. The image sensor is a 32 times 32 pixel array fabricated in bulk CMOS process. Each pixel has a 74 mum times 34 mum photodiode. The SNR performance of the image sensor is comparable to current state-of-the-art voltage-sensitive dye imaging sensors. The power consumption and weight of the proposed microscope is much lower than other voltage-sensitive dye imaging systems and can be used on awake and freely moving animals.

We present a low-power pulse-based amplifier and data conversion circuit for recording bio-potentials. The circuit is designed for data reduction in neurophysiological sensor and communication systems. A low-power low-noise AC amplifier with an asynchronous delta A/D converter is implemented. The asynchronous delta modulation has the advantage of data reduction, clock-less and no quantization noise. The amplifier gain is 53 dB (bandwidth of 200 Hz-20 kHz) with DC variation rejection. The input-referred rms noise is 2 muV. The delta amplitude in the A/D converter is 0.1 V at 3.3 V voltage supply. The system is fabricated with AMI 0.5 mum CMOS technology. The chip size is 1191 mum by 713 mum. The power consumption with 3.3 V supply is 75.9 muW in static state and 194.7 muW with 60 kbps data conversion rate. A graphic user interface was developed to monitor the transmitted signal in real time.

We present a low-power high-speed ultra-wideband (UWB) pulse radio transmission test platform. The transmitter circuit is fabricated on the Peregrine semiconductors 0.5 micron silicon-on-sapphire (SOS) CMOS process and packaged in a DIP16 package. 12.5 Mbps data rate of wireless transmission is achieved. The power consumption of the transmitter is 10 mW when the pulse rate is 90 MHz. The core circuit size is 70 mum by 130 mum. The transmission distance is in excess of 4 m. The circuit is a transformative technology candidate for high-data rate and low power wireless biomedical applications.

In this paper we report a 64×64 active pixel image sensor (APS), Mototrigger, which targets low-power image sensing for sensor networks. The image sensor computes the temporal-difference between two continuous light-intensity frames and communicates them in address-event format. The image sensor is capable of computing 30 temporal-difference frame/s and consumes 1.2mW with a 3V power supply. Light-intensity images are also available as an output at a speed of 60 frame/s. The pixel occupies an area of 29×28μm2 with a fill factor of 23%. The die size is 3mm by 3mm, including pads and test structures. The image sensor is fabricated in a 0.5μm bulk CMOS process.

In this paper we describe an address-event vision system designed to detect accidental falls in elderly home care applications. The system raises an alarm when a fall hazard is detected. We use an asynchronous temporal contrast vision sensor which features sub-millisecond temporal resolution. A lightweight algorithm computes an instantaneous motion vector and reports fall events. We are able to distinguish fall events from normal human behavior, such as walking, crouching down, and sitting down. Our system is robust to the monitored person’s spatial position in a room and presence of pets.

We present a back-illuminated 32 × 32 pixel image sensor in 0.5-μm silicon-on-sapphire process. The imager performs “snap-shot” image acquisition and analog readout at a continuous rate of thousands of frames/s and consumes as little as 250 μW. Each pixel consists of a phototransistor and a memory capacitor in 40 μm × 40 μm with a fill factor of 43%. The image sensor is suited for hyper-spectral imaging at high speeds.

We present a back-illuminated 32 x 32 pixel SOI image sensor chip in 0.5-mum silicon-on-sapphire process capable of ultraviolet imaging. The imager performs "snap-shot" image acquisition and analog readout at a continuous rate of a thousand frames/s and consumes as little as 650 muW. Each pixel consists of a photodiode and a memory capacitor in 40 mum x 40 mum with a fill factor of 43%. The image sensor is suited for hyper-spectral imaging at high speeds.

A low-power tunable transmitter for ultra-wide band (UWB) radio was designed and fabricated. The design is based on a ring oscillator VCO, to produce short pulses. Both the pulse duration and frequency is controllable by two voltage biases. The transmitter can be used in both pulse-amplitude modulation (PAM) and pulse-position modulation (PPM). The circuit was fabricated on a 0.5mum silicon-on-sapphire CMOS process, and has been experimentally characterized with a transmitter and receiver module. The UWB transmitter produces monocycle pulses of 800 ps duration. The power consumption is 75muW when the pulse frequency is 3 MHz. The core pulse generator occupies 0.0091mm2of silicon area.

We present a fully-integrated implementation of a patch-clamp measurement system. The system was implemented in a 0.5mum silicon-on-sapphire process and is capable of recording cell membrane currents up to plusmn20nA, with a rms noise of 5pA at 10kHz bandwidth. The system can compensate for the capacitance and resistance of the planar electrode, up to 20pF and 4MOmega respectively. The die size is 1150 by 700mum. The power consumption is 3.3mW at 3.3V. The integrated patch- clamp system will be used to fabricate high-throughput planar patch-clamp systems.

The paper explores an algorithm and methodology for realtime video compression and communication over sensor network. Video is encoded using the address-event representation (AER) and using custom image sensors capable of detecting intensity-differences (motion) information. The work focuses on keeping a constant low bit-rate over sensor network channels based on ZigBee radios, so that the motion information in an image can degrade gracefully when the resolution or the image content is varied over time. The paper shows that a 320times240 video stream can be compressed up to 50 times while being intelligible for interpreting motion patterns. This compression requires zero computation when the image is encoded with AER. The work will be applied to remote monitoring of home-care patients and surveillance

We report on the design and measurement results of three-dimensional (3D) photo-detectors in a 0.18μm silicon-on-insulator technology. The photodiodes respond to light in the range of l-200,000lux with currents of 2fA to 300pA and can be arranged in a 3D stack. The phototransistors respond to light intensities of 5-200,000lux with currents from 50fA to 2.3μA. We also report spectral data obtained from the photodiode and show that a photodiode stack can be used to extract color information without the use of color filters. The data in this paper is essential to the design of advanced imaging arrays and color sensors in 3D SOI processes.

In this paper we present a feature-extracting image sensor targeted to wireless image sensor networks. The image sensor was designed and fabricated on a 3D integrated 0.18μm silicon-on-insulator CMOS process. The image sensor can simultaneously capture an intensity image and extract image features, which include temporal illumination differentiations and contours. By taking advantage of massively parallel, vertical connectivity, the image sensor can perform more analog computations at higher speed and higher communication efficiency. The top layer is covered by photodiodes and the pixel fill factor is 95%.

The paper report on the design and test of a low-voltage temperature sensor designed for MEMS power-harvesting systems. The core of the sensor is a bandgap voltage reference circuit operating with a supply voltage in the range of 1-1.5V. The prototype was fabricated on a conventional 0.5μm Silicon-on-Sapphire (SOS) process. The sensor design consumes 15μA of current at 1V. The internal reference voltage is 550mV. The temperature sensor has a digital square wave output whose frequency is proportional to temperature. A linear model of the dependency of output frequency with temperature has a conversion factor of 1.6kHz/°C. The output is also independent of supply voltage in the range of 1-1.5V. Measured results and targeted applications for the proposed circuit were reported.

We fabricated an integrated patch-clamp amplifier capable of recording nano-amperes of current in whole-cell patch recording. Patch-clamp amplifiers presently consist of a head-stage and a large controller box that contains many hundreds of components. Our design compresses the function of the head-stage and the controller box into one chip that can be mounted near the patch-clamp electrode and can be controlled through a high-speed serial bus paving the way for higher densities in parallel patch-clamp systems. We report on the electrical measurements from the fabricated device.

We designed, fabricated and tested four kinds of nonvolatile memories in a commercially-available 0.5μm Silicon-on-sapphire CMOS process. The device tested are PMOS and NMOS with MOS- and MIM-based floating gates. We report on the test results of all four devices and demonstrate that MIM-based NMOS floating gate cells can be use to achieve a threshold shift of 0.4V and no significant decay of threshold voltage after small retentions tests of more than 100s.

We present a multichip, mixed-signal VLSI system for spike-based vision processing. The system consists of an 80 x 60 pixel neuromorphic retina and a 4800 neuron silicon cortex with 4,194,304 synapses. Its functionality is illustrated with experimental data on multiple components of an attention-based hierarchical model of cortical object recognition, including feature coding, salience detection, and foveation. This model exploits arbitrary and reconfigurable connectivity between cells in the multichip architecture, achieved by asynchronously routing neural spike events within and between chips according to a memory-based look-up table. Synaptic parameters, including conductance and reversal potential, are also stored in memory and are used to dynamically configure synapse circuits within the silicon neurons.

Although imaging is an information-rich sensing modality, the use of cameras in sensor networks is very often prohibited by factors such as power, computation cost, storage, communication bandwidth and privacy. In this paper we consider information selective and privacy-preserving address-event imagers for sensor networks. Instead of providing full images with a high degree of redundancy, our efforts in the design of these imagers specialize on selecting a handful of features from a scene and outputting these features in address-event representation. In this paper we present our initial results in modeling and evaluating address-event sensors in the context of sensor networks. Using three different platforms that we have developed, we illustrate how to model address-event cameras and how to build an emulator using these models. We also present a lightweight classification scheme to illustrate the computational advantages of address-event sensors. The paper concludes with an evaluation of the classification algorithm and a feasibility study of using COTS components to emulate address-event inside a sensor network.

We fabricated a 3D-integrated multi-chip sensor and actuator and demonstrated the ability of communication with a floating die and no galvanic connection. The prototype was fabricated on a conventional 0.5mum silicon-on-sapphire (SOS) process. We designed a heater and a temperature sensor module with digital output based on a bandgap voltage reference. We used capacitive coupling to provide both intra-die communication of the digital temperature readings and also energy-harvesting by means of a charge pump. The non-galvanically interconnected prototype is an enabling technology for three-dimensional VLSI fabrication, 3D CMOS, wafer stacking and packaging

We designed and fabricated a 4-channel digital isolation amplifier in a 0.5mum silicon-on-sapphire technology. The isolation device was fabricated on a single die, taking advantage of the isolative properties of the sapphire substrate. The individual isolation channels can operate in excess of 40Mbps using digital phase-shift-keying modulation. Modulation of the input signal is used to increase immunity to errors at low input data rates. The device can tolerate ground bounces of 1V/mus and isolate more than 800V. The device uses N+1 capacitors for TV channels as opposed to 2N of previous implementations, thus minimizing the coupling silicon area and increasing reliability. Typical applications are in harsh industrial environments, transportation, medical and life-critical systems

We describe a sensor network based on smart imager sensors able to extract events of interest from a scene. We present a framework towards the use of image sensors for realizing sensor networks that can interpret behaviors in physical space. An array of heterogeneous event-based CMOS imagers is employed at front-end, to convert light intensity, motion and other vision primitives into pulse-density modulated stream of address-events. The wireless nodes are Yale XYZ motes employing high data rate communication and large processing capabilities. Energy-aware communication is implemented at the sensor level by employing an event-based readout with pixel-level filtering capability. This paper provides an overview of the research challenges followed by a brief description of our current research efforts towards creating a functional experimental system for investigating these intelligent networks

We designed and fabricated an 8-bit analog-to-digital converter (ADC) in a 0.5/spl mu/m silicon-on-sapphire CMOS technology. The ultra-low power and low voltage design of the ADC was targeted for wireless sensor interfaces and energy-scavenging systems. The ADC is capable of 4.5MHz operation, producing 409kS/s, consuming 3/spl mu/Watt at 1.5V power supply. It uses an active area of 615/spl times/782/spl mu/m/sup 2/ . The acceptable input range is 0.97V at 1.5V power supply.

We fabricated an integrated patch-clamp amplifier capable of recording from pico- to tens of micro-amperes of current. The high-dynamic range of seven decades and the pico-ampere sensitivity of the instrument was designed for whole-cell patch-clamp recordings. The prototype was fabricated on a 0.5/spl mu/m silicon-on-sapphire process. The device employs an integrating headstage with a frequency-modulated output pulse ranging from 3Hz to 10MHz. A digital interface produces a 16bit output conversion of the input currents. We report on electrical measurements from the fabricated device, and measurements conducted on cells in a typical patch-clamp experiment.

We report on dark current measurements from lateral, 100nm thick, PIN photodiodes fabricated in the Peregrine Semiconductor, silicon on sapphire (SOS) CMOS technology. We compare interdigitated photodiode geometries with edgeless structures that do not have active device regions adjacent to LOCOS. We also compare two methods for device design. One employs a polysilicon gate to block the implant in the intrinsic region of the device while the second utilizes a specific mask layer in the technology called an SDBlock mask. Our results suggests that the dark current is primarily a function of the junction width. Furthermore, polysilicon gate devices have lower dark currents than SDBlock structures. Finally, we perform noise measurements and extract flicker noise parameters for the two methods and find that polysilicon gate structures have greater levels of flicker noise than SDblock devices.

We discuss an imaging architecture for sensor network applications, that employs a 32 times 32 address-event representation (AER) imager. At the sensor level, pixels convert light intensity into a pulse density modulated stream of address events. Two different types of COTS wireless radio nodes are used, along with two separate approaches to wireless data transmission - one as a train of AER addresses, and the other as a histogram of the active pixels. Information transmitted in the limited-bandwidth network yields effective means for detection and partial recognition of the object even at very low bit rates, yielding a maximum experimental frame-rate of close to 6fps

The use of low-cost imagers in sensor network applications is imminent. Small low-power imagers have many potential applications in visual confirmation of events, target recognition, security and entertainment. To make the autonomous operation of imager sensors in such applications possible, smart imager sensors should be able to calibrate their views with respect to other imagers in the network and should become aware of the locations of other non-imager sensors. Such calibration will allow cameras with multiple vantage points and sensors distributed in physical space to jointly undertake collaborative tasks.

We have constructed a system that uses an array of 9,600 spiking sili- con neurons, a fast microcontroller, and digital memory, to implement a reconfigurable network of integrate-and-fire neurons. The system is designed for rapid prototyping of spiking neural networks that require high-throughput communication with external address-event hardware. Arbitrary network topologies can be implemented by selectively rout- ing address-events to specific internal or external targets according to a memory-based projective field mapping. The utility and versatility of the system is demonstrated by configuring it as a three-stage network that accepts input from an address-event imager, detects salient regions of the image, and performs spatial acuity modulation around a high-resolution fovea that is centered on the location of highest salience.

We examine channel access algorithms and circuits for intra and inter chip communication channels. Classical access techniques such as arbitration, scanning, ALOHA, and priority encoding are compared by assessing throughput, latency, and power consumption. Our results provide guidance in the design of bio-inspired networks of processors, for efficient transmission of information with limited power consumption and reduced latency.

Gaze-contingent variable resolution display techniques allocate computational resources for image generation preferentially to the area around the center of gaze where visual sensitivity to detail is the greatest. Although these techniques are computationally efficient, their behavioral consequences with realistic tasks and materials are not well understood. The behavior of human observers performing visual search of natural scenes using gaze-contingent variable resolution displays is examined. A two-region display was used where a high-resolution region was centered on the instantaneous center of gaze, and the surrounding region was presented in a lower resolution. The radius of the central high-resolution region was varied from 1 to 15 degrees while the total amount of computational resources required to generate the visual display was kept constant. Measures of reaction time, accuracy, and fixation duration suggest that task performance is comparable to that seen for uniform resolution displays when the central region size is approximately 5 degrees.