A silicon retina is an intelligent vision sensor that can execute real-time image preprocessing by using a parallel analog circuit that mimics the structure of the neuronal circuits in the vertebrate retina. For enhancing the sensor's robustness to changes in illumination in a practical environment, we have designed and fabricated a silicon retina on the basis of a computational model of brightness constancy. The chip has a wide-dynamic-range and shows a constant response against changes in the illumination intensity. The photosensor in the present chip approximates logarithmic illumination-to-voltage transfer characteristics as a result of the application of a time-modulated reset voltage technique. Two types of image processing, namely, Laplacian-Gaussian-like spatial filtering and computing the frame difference, are carried out by using resistive networks and sample/hold circuits in the chip. As a result of these processings, the chip exhibits brightness constancy over a wide range of illumination. The chip is fabricated by using the 0.25- μm complementary metal-oxide semiconductor image sensor technology. The number of pixels is 64 × 64, and the power consumption is 32 mW at the frame rate of 30 fps. We show that our chip not only has a wide-dynamic-range but also shows a constant response to the changes in illumination.

We describe a binocular vision system that emulates disparity computation in the neuronal circuit of the primary visual cortex (V1). The system consists of two sets of silicon retinas and simple cell chips that correspond to the binocular vision and field programmable gate array (FPGA) circuit. This arrangement mimics the hierarchical architecture of the visual system of the brain. The silicon retina is an analog very large scale integrated (aVLSI) circuit and possesses a Laplacian-Gaussian-like spatial filter similar to the receptive field of the vertebrate retina. The simple cell chip generates a Gabor-like spatial filter similar to the orientation-selective receptive field of the simple cell in V1 by aggregating several pixels of the silicon retina. The FPGA receives the outputs from the two simple cell chips corresponding to binocular inputs from the left and right eyes and calculates the binocular disparity in real-time based on the disparity energy model. The system provides output images tuned to five different disparities in parallel. The disparity map is obtained by comparing these disparity energy outputs. Due to the combination of the parallel and analog computation of the aVLSIs and the pixel-wise computation with hard-wired digital circuits, the present system can efficiently compute the binocular disparity using compact hardware and low power dissipation in real-time.

We designed a mixed analog-digital neuromorphic vision system that can replicate the response of complex cells in the primary visual cortex (V1). Using the system, the binocular disparity energy maps were calculated in natural scenes in real time. Because of its compact hardware and low power dissipation, the neuromorphic vision system developed in the present study is suitable to robotic vision. More interestingly, it provides insights to explore the visual function of the neuronal network of the brain, visualizing neural images inferred from physiological experiments.

A real-time vision sensor for collision avoidance was designed. To respond selectively to approaching objects on direct collision course, the sensor employs an algorithm inspired by the visual nervous system in a locust, which can avoid a collision robustly by using visual information. We implemented the architecture of the locust nervous system with a compact hardware system which contains mixed analogdigital integrated circuits consisting of an analog resistive network and field-programmable gate array (FPGA) circuits. The response properties of the system were examined by using simulated movie images, and the system was tested also in realworld situations by loading it on a motorized car. The system was confirmed to respond selectively to colliding objects even in complicated real-world situations.

In this research, a dialog environment between human and virtual environment has been constructed. At preset, if a person wants to interact with virtual environment, special device such a data glove is required, but it makes difficult for general users to manipulate virtual objects. When we cannot manipulate objects directly, it is natural that we ask someone with a privilege to do the operation in place of us. In the case, it is convenient if methods used in daily life are allowed. That is, the method to be proposed here is based on the integration between verbal information through the utterances and non-verbal information by the gestures such as finger pointing. The experimental results have proved the effectiveness of this approach in terms of facilitating man-machine interaction and communication. The environment constructed in this research allows a user to communicate by talking and showing gestures to a personified agent in virtual environment. A user can use his/her finger to point at a virtual object and ask him to manipulate it.

A high resolution neuromorphic multi-chip vision system was fabricated to emulate the orientation selective response of the simple cell in the primary visual cortex. The vision system consists of two types of analog chips: a silicon retina and a simple cell chip. The center-surround concentric receptive fields of the silicon retina are aggregated in the simple cell chip, mimicking the hierarchical architecture in the visual system of the brain. Both chips have 100 times 100 pixels. Using the orientation-selective outputs obtained from the multi-chip system, texture segregation was conducted based on a computational model inspired by neurophysiology. The texture image was filtered by the two orthogonally oriented receptive fields of the multi-chip system and the filtered images were combined to segregate the area of different texture orientation with the aid of field programmable gate array (FPGA). The present multi-chip system is useful to emulate and verify computational models for texture segregation of the cortical cells

The neuromorphic silicon retina is a novel analog very large scale integrated circuit that emulates the structure and the function of the retinal neuronal circuit. We fabricated a neuromorphic silicon retina, in which sample/hold circuits were embedded to generate fluctuation-suppressed outputs in the previous study [1]. The applications of this silicon retina, however, are limited because of a low spatial resolution and computational variability. In this paper, we have fabricated a multichip silicon retina in which the functional network circuits are divided into two chips: the photoreceptor network chip (P chip) and the horizontal cell network chip (H chip). The output images of the P chip are transferred to the H chip with analog voltages through the line-parallel transfer bus. The sample/hold circuits embedded in the P and H chips compensate for the pattern noise generated on the circuits, including the analog communication pathway. Using the multichip silicon retina together with an off-chip differential amplifier, spatial filtering of the image with an odd- and an even-symmetric orientation selective receptive fields was carried out in real time. The analog data transfer method in the present multichip silicon retina is useful to design analog neuromorphic multichip systems that mimic the hierarchical structure of neuronal networks in the visual system.

A high resolution neuromorphic aVLSI was fabricated to emulate the orientation selective response of the simple cell in the primary visual cortex. The aVLSI circuits consist of two analog chips: a silicon retina and an orientation chip. The center-surround concentric receptive fields of the silicon retina are aggregated in the orientation chip, mimicking the hierarchical architecture in the visual system of the brain. Both chips have 100 /spl times/ 100 pixels and therefore, this multi-chip system is applicable to robotic vision. Using the orientation-selective outputs obtained from the multi-chip system, a texture segregation was conducted based on a similar algorithm of the energy computation. The texture image was filtered by the two orthogonally oriented receptive fields of multi-chip system and the filtered images were combined to segregate the area of different texture orientation with the aid of a PC. The study demonstrated that the orientation-selective multi-chip system developed is useful to emulate the texture segregation employing a fundamental architecture to generate the simple cell response in the primary visual cortex and is applicable to robotic vision.

In this paper, we designed and fabricated a multichip neuromorphic analog very large scale integrated (aVLSI) system, which emulates the orientation selective response of the simple cell in the primary visual cortex. The system consists of a silicon retina and an orientation chip. An image, which is filtered by a concentric center-surround (CS) antagonistic receptive field of the silicon retina, is transferred to the orientation chip. The image transfer from the silicon retina to the orientation chip is carried out with analog signals. The orientation chip selectively aggregates multiple pixels of the silicon retina, mimicking the feedforward model proposed by Hubel and Wiesel. The chip provides the orientation-selective (OS) outputs which are tuned to 0 degrees, 60 degrees, and 120 degrees. The feed-forward aggregation reduces the fixed pattern noise that is due to the mismatch of the transistors in the orientation chip. The spatial properties of the orientation selective response were examined in terms of the adjustable parameters of the chip, i.e., the number of aggregated pixels and size of the receptive field of the silicon retina. The multichip aVLSI architecture used in the present study can be applied to implement higher order cells such as the complex cell of the primary visual cortex.

The neuromorphic silicon retina is a novel analog very large scale integrated (VLSI) circuit that emulates the structure and the function of the retinal neuronal circuit. We fabricated a neuromorphic silicon retina in which sample/hold circuits were embedded to generate fluctuation-suppressed outputs in the previous study. This silicon retina, however, has a trade-off problem between the resolution and the computational complexity as the neuromorphic silicon retina generally does, since the photo sensors and processing circuits are fabricated on a single-ship. To solve the problem, we have designed a multi-chip silicon retina in which the functional network circuits are divided into two chips, the photoreceptor network chip (P chip) and the horizontal cell network chip (H chip). The output images of the chips are transferred to the H chip with analog voltages through line-parallel transfer bus. The sample/hold circuits embedded in the H chip compensate for the pattern noise generated on the transfer pathway. The number of pixels and the fill factor increased three times and twice as much as the previously fabricated chip, respectively.

A silicon retina that emulates the sustained and the transient responses in the vertebrate retina was fabricated. The circuit of the chip consists of two layers of resistive network that have different length constants. The output emulating the sustained response possesses a Laplacian-Gaussian-like receptive field and, therefore, carries out a smoothing and contrast-enhancement on the input images. This receptive field was realized by subtracting voltages distributing over the two resistive networks. The output emulating the transient response was obtained by subtracting consecutive images that were smoothed out by the resistive network and is sensitive to moving objects. The outputs of these two channels can be obtained alternately from the silicon retina in real time, within time delays not exceeding a few tens of milliseconds, with indoor illumination. The outputs of the chip are offset-suppressed analog voltages since the uncontrollable mismatches of transistor characteristics are compensated for with the aid of sample/hold circuits embedded in each pixel circuit. The silicon retina fabricated in the present study can be readily used in current engineering applications, e.g., robot vision.

We have fabricated an analog VLSI circuit with 1D /spl nabla//sup 2/G-like receptive field. Into the chip are incorporated active pixel sensors and sample/hold circuits to achieve high gain and low noise. A real time edge detection was carried out under indoor fluorescence with the chip, indicating that the neuromorphic retina fabricated in the present study is usable for industrial applications.

A linear analog network model is proposed to characterize the function of the outer retinal circuit in terms of the standard regularization theory. Inspired by the function and the architecture of the model, a vision chip has been designed using analog CMOS Very Large Scale Integrated circuit technology. In the chip, sample/hold amplifier circuits are incorporated to compensate for statistic transistor mismatches. Accordingly, extremely low noise outputs were obtained from the chip. Using the chip and a zero-crossing detector, edges of given images were effectively extracted in indoor illumination.

A linear analog network model is proposed to describe the neural circuit of the outer retina. Using the model, we characterized the spatial filtering properties of the circuit in terms of the standard regularization theory in which the early vision problems are attributed to minimization of a cost function. Inspired by the algorithm/architecture of the circuit, a vision chip was designed using analog CMOS VLSI circuit technology. The chip fabricated is an one-dimensional 100 pixel line sensor. In the chip, sample/hold amplifier circuits are incorporated to compensate for the fixed pattern noise of photosensors as well as statistic transistor mismatch. The chip effectively extracts contour of given images by controlling the receptive field size, simulating a light-adaptive mechanism suggested in the vertebrate retinal circuit.