Smart vision sensors represent a niche in machine vision where high performance in terms of frame rates, processing speed, and flexibility is the most essential feature. With the integration of photosensitive devices with processing elements on one and the same substrate it is possible to reach high performance at low complexity and low cost. The MAPP2500 from IVP represents state of the art in this development of CMOS imaging and on-chip processing.

In the original near-sensor image processing (NSIP) concept, the intensity level for an individual sensor element is mapped onto the time axis so that the time to reach the threshold is inversely proportional to the intensity. In many cases this is favorable, in some cases there is a need for another mapping between time and intensity. We show that such mappings can be achieved by either inserting delays or by varying the threshold voltage. The result is that to achieve a linear mapping it is much more favorable to use a variable threshold since the processing power of the sensor is then better utilized. An interesting result, when it comes to adaptive mapping, is that the traditionally time consuming operation histogram equalization can be accomplished almost for free in the NSIP concept. We also show that other histogram-based, i.e., data-dependent, mapping can be easily implemented.

Near-sensor image processing (NSIP) is a new approach to low-level image processing in which the ability to perform global operations is crucial. We define a 2-D global logic unit (GLU), which consists of simple logical circuits arranged in a regular net, and describe how to perform powerful segmentation and feature-extracting operations using this net. Typical segmentation operations performed by the net are hole filling, smallest circumscribing rectangle, thresholding with hysteresis, and arbitrary propagation patterns. We describe how to obtain features like the position of an object in the image. Finally, we show that the NSIP concept, including the global functions, can be implemented in a single chip using today's VLSI technology.

Block-based motion compensation has been widely used for the removal of temporal redundancy existing in image sequences. However, the prediction image generated by this technique suffers from a perceptually annoying blocking-effect. This becomes especially serious in video transmission at very low bit rates. To overcome this problem, a transformed block-based motion compensation technique is proposed in this paper. Preliminary results show that both subjective and objective improvements can be obtained when a transformed block-based motion compensation scheme is employed instead of a direct block-based scheme.>

This paper addresses the issue of two-view facial motion estimation for model-based facial image coding. A new approach to estimate the motion of the head and the facial expressions is presented, which can be viewed as a two-step procedure as compared with our previous approach. In this new approach M-estimation technique is used to eliminate the effect of the facial expressions on the estimation of the head movement. In this way the global head motion can be more reliably recovered. Once the global motion is obtained, the facial expressions can be estimated. Some experimental results on synthesized and real image sequences demonstrate the effectiveness of the new algorithm.>

The paper introduces the concept of near-sensor image processing. By this, the authors mean techniques in which the physical properties of the image sensor itself is utilized to do part of the signal processing task. It is shown that the analog-temporal behavior of photodiodes combined with thresholding amplifiers can be used favorably to do certain low-level image processing tasks including median filtering and convolution. The given examples also show how adaptivity to different light levels can be achieved in a natural way. To extract features from the image, such as moments and shape factors, the authors introduce a simple measurement function.

This paper presents a review of promising techniques for very low bit-rate, below 64 kb/s, image sequence coding. Image sequence coding at such low rates will be a crucial technique in forthcoming visual services, e.g., visual information transmission and storage. A typical application is to transmit moving videophone scenes through the existing analog telephone lines or via a mobile channel. Two types of potential coding techniques are addressed: waveform-based image sequence coding and model-based image sequence coding.

An approach to estimating the motion of the head and facial expressions in model-based facial image coding is presented. An affine nonrigid motion model is set up. The specific knowledge about facial shape and facial expression is formulated in this model in the form of parameters. A direct method of estimating the two-view motion parameters that is based on the affine method is discussed. Based on the reasonable assumption that the 3-D motion of the face is almost smooth in the time domain, several approaches to predicting the motion of the next frame are proposed. Using a 3-D model, the approach is characterized by a feedback loop connecting computer vision and computer graphics. Embedding the synthesis techniques into the analysis phase greatly improves the performance of motion estimation. Simulations with long image sequences of real-world scenes indicate that the method not only greatly reduces computational complexity but also substantially improves estimation accuracy.>

The issue of analysis/synthesis of human facial movements in model-based facial image coding is addressed. An affine model which is especially suitable to such a type of nonrigid motion as facial expressions is set up. Based on the affine model, a direct two-view motion estimation approach is presented. This approach is much faster than other existing approaches. To remove the error-accumulation problem associated with long sequence motion estimation, a new scheme which is equipped with a feedback loop connecting computer vision and computer graphics is used. Experimental results show that the method not only greatly reduces the computational complexity but also substantially improves the estimation accuracy. The synthesized image sequence using the estimated motion parameters, a 3-D model of the face, and a frame of textured image looks very natural.>