We present a novel event-based stereo matching algorithm that exploits the asynchronous visual events from a pair of silicon retinas. Unlike conventional frame-based cameras, recent artificial retinas transmit their outputs as a continuous stream of asynchronous temporal events, in a manner similar to the output cells of the biological retina. Our algorithm uses the timing information carried by this representation in addressing the stereo-matching problem on moving objects. Using the high temporal resolution of the acquired data stream for the dynamic vision sensor, we show that matching on the timing of the visual events provides a new solution to the real-time computation of 3-D objects when combined with geometric constraints using the distance to the epipolar lines. The proposed algorithm is able to filter out incorrect matches and to accurately reconstruct the depth of moving objects despite the low spatial resolution of the sensor. This brief sets up the principles for further event-based vision processing and demonstrates the importance of dynamic information and spike timing in processing asynchronous streams of visual events.

A fully configurable bias current reference is described. The output of the current reference is a gate voltage which produces a desired current. For each daisy-chained bias, 32 bits of configuration are divided into 22 bits of bias current, 6 bits of active-mirror buffer current, and 4 bits of other configuration. Configuration of each bias allows specifying the type of transistor (nfet or pfet), whether the bias is enabled or weakly pulled to the rail, whether the bias is for a cascode, and whether the bias transistor uses a shifted source (SS) voltage for sub-off-current biasing. In addition, the current reference integrates a pair of voltage regulators that generate stable voltage sources near the rails, suitable for the SS current references. Measurements from fabricated current references built in 180 nm CMOS show that the reference achieves at least 110 dB (22-bit) dynamic range and reaches 160dB when power-rail gate biasing is included. Generated bias currents reach at least 30x smaller current than the transistor off-current. Each current reference occupies an area of 620 × 50 um2. The design kit schematics and layout are open-sourced.

Balancing a normal pencil on its tip requires rapid feedback control with latencies on the order of milliseconds. This demonstration shows how a pair of spike-based silicon retina dynamic vision sensors (DVS) is used to provide fast visual feedback for controlling an actuated table to balance an ordinary pencil. Two DVSs view the pencil from right angles. Movements of the pencil cause spike address-events (AEs) to be emitted from the DVSs. These AEs are transmitted to a PC over USB interfaces and are processed procedurally in real time. The PC updates its estimate of the pencil's location and angle in 3d space upon each incoming AE, applying a novel tracking method based on spike-driven fitting to a model of the vertical shape of the pencil. A PD-controller adjusts X-Y-position and velocity of the table to maintain the pencil balanced upright. The controller also minimizes the deviation of the pencil's base from the center of the table. The actuated table is built using ordinary high-speed hobby servos which have been modified to obtain feedback from linear position encoders via a microcontroller. Our system can balance any small, thin object such as a pencil, pen, chop-stick, or rod for many minutes. Balancing is only possible when incoming AEs are processed as they arrive from the sensors, typically at intervals below millisecond ranges. Controlling at normal image sensor sample rates (e.g. 60 Hz) results in too long latencies for a stable control loop.

Balancing a normal pencil on its tip requires rapid feedback control with latencies on the order of milliseconds. This demonstration shows how a pair of spike-based silicon retina dynamic vision sensors (DVS) is used to provide fast visual feedback for controlling an actuated table to balance an ordinary pencil. Two DVSs view the pencil from right angles. Movements of the pencil cause spike address-events (AEs) to be emitted from the DVSs. These AEs are transmitted to a PC over USB interfaces and are processed procedurally in real time. The PC updates its estimate of the pencil's location and angle in 3d space upon each incoming AE, applying a novel tracking method based on spike-driven fitting to a model of the vertical shape of the pencil. A PD-controller adjusts X-Y-position and velocity of the table to maintain the pencil balanced upright. The controller also minimizes the deviation of the pencil's base from the center of the table. The actuated table is built using ordinary high-speed hobby servos which have been modified to obtain feedback from linear position encoders via a microcontroller. Our system can balance any small, thin object such as a pencil, pen, chop-stick, or rod for many minutes. Balancing is only possible when incoming AEs are processed as they arrive from the sensors, typically at intervals below millisecond ranges. Controlling at normal image sensor sample rates (e.g. 60 Hz) results in too long latencies for a stable control loop.

This paper describes CAVIAR, a massively parallel hardware implementation of a spike-based sensing-processing-learning-actuating system inspired by the physiology of the nervous system. CAVIAR uses the asychronous address-event representation (AER) communication framework and was developed in the context of a European Union funded project. It has four custom mixed-signal AER chips, five custom digital AER interface components, 45k neurons (spiking cells), up to 5M synapses, performs 12G synaptic operations per second, and achieves millisecond object recognition and tracking latencies.

This paper proposes a simple focal plane pattern detector architecture using a novel pixel sensor based on the dichromatic vertacolor structure. Additionally, the sensor transfers dichromatic intensity values using a self-timed time-to- first-spike scheme, which provides high dynamic range imaging. The intensity information is transmitted using the address event representation protocol. The spectral information is sampled automatically at each intensity reading in a ratioed way that maintains high dynamic range. A test chip consisting of 20 pixels has been fabricated in 1.5 um 2P 2M CMOS and characterized. The combined pattern detector/ imager core consumes 45 uA at 5 V supply voltage.

In this paper we describe an address-event vision system designed to detect accidental falls in elderly home care applications. The system raises an alarm when a fall hazard is detected. We use an asynchronous temporal contrast vision sensor which features sub-millisecond temporal resolution. A lightweight algorithm computes an instantaneous motion vector and reports fall events. We are able to distinguish fall events from normal human behavior, such as walking, crouching down, and sitting down. Our system is robust to the monitored person’s spatial position in a room and presence of pets.

Fast sensory-motor processing is challenging when using traditional frame-based cameras and computers. Here the authors show how a hybrid neuromorphic-procedural system consisting of an address-event silicon retina, a computer, and a servo motor can be used to implement a fast sensory-motor reactive controller to track and block balls shot at a goal. The system consists of a 128times128 retina that asynchronously reports scene reflectance changes, a laptop PC, and a servo motor controller. Components are interconnected by USB. The retina looks down onto the field in front of the goal. Moving objects are tracked by an event-driven cluster tracker algorithm that detects the ball as the nearest object that is approaching the goal. The ball's position and velocity are used to control the servo motor. Running under Windows XP, the reaction latency is 2.8plusmn0.5 ms at a CPU load of1 million events per second (Meps), although fast balls only create ~30 keps. This system demonstrates the advantages of hybrid event-based sensory motor processing

The paper presents a spike-based saccadic recognition system that uses a temporal-derivative silicon retina on a pan-tilt unit and an aVLSI multi-neuron classifier with a time-to-first-spike output coding. By using the spike information during the last 150 ms of a saccadic movement, we generate a reliable, sparse stimulus representation of image patches. The paper describes a novel classification scheme where the retinal spikes during this time influence the time-to-first spike of classifier neurons which receive the same constant input current. The preferred pattern of the neuron is stored in the synaptic connectivity between the retina and the classifier neuron. The authors demonstrates the robustness and real-time performance of this recognition scheme on a saccadic system which uses analog VLSI components.

The growing interest in pulse-mode processing by neural networks is encouraging the development of hardware implementations of massively parallel, distributed networks of integrate-and-fire (I&F) neurons. We have developed a reconfigurable multi-chip neuronal system for modeling feature selectivity and applied it to oriented visual stimuli. Our system comprises a temporally differentiating imager and a VLSI competitive network of neurons which use an asynchronous address event representation (AER) for communication. Here we describe the overall system, and present experimental data demonstrating the effect of recurrent connectivity on the pulse-based orientation selectivity

This paper describes an on-chip programmable bias current generator, intended for mixed signal chips requiring a wide ranging set of currents. The individual generators share a master current reference. A serial digital interface to the chip controls the biases by bits loaded into a 24-bit shift register. These bits control the steering of current from a current splitter. The summed current splitter output is actively mirrored to a broadcasted bias voltage. Measurements from an implementation in 0.35/spl mu/ 4M-2P CMOS show a total range of bias current of over 6 decades (>120 dB) ranging from a few times the off-current up to the master reference current. For currents larger than the minimum, the generator has resolution spanning nearly its full 24 bit range (144dB), e.g. for a master current of 10 /spl mu/A, any bias current can be varied by as little as 0.5 pA with the caveat that the code is not guaranteed monotonic. Each bias occupies an area of 0.026 mm/sup 2/, which is about 65% of the bonding pad that it replaces. Measured variation in generated currents is <10% in strong inversion and about 20-30% in weak inversion.

We present a 100dB dynamic range 2times64 pixel dual-line optical sensor with asynchronous event-based readout. Each individual pixel of the sensor operates autonomously and responds with <100mus latency to relative intensity changes. It operates largely independent of overall scene illumination, directly encodes object reflectance, and greatly reduces redundancy while preserving precise timing information. The line sensor was fabricated using a 0.35mum standard CMOS technology. Results of the measurement performance of the sensor are presented. The intended application area is precision timing measurement under variable lighting conditions

A 5-layer neuromorphic vision processor whose components communicate spike events asychronously using the address-event- representation (AER) is demonstrated. The system includes a retina chip, two convolution chips, a 2D winner-take-all chip, a delay line chip, a learning classifier chip, and a set of PCBs for computer interfacing and address space remappings. The components use a mixture of analog and digital computation and will learn to classify trajectories of a moving object. A complete experimental setup and measurements results are shown.