High-performance dense stereo is a critical component of computer vision applications like 3D reconstruction, robot navigation, and augmented reality. In this paper, we present a low-power, high performance FPGA implementation of a stereo algorithm suitable for embedded real-time platforms. The design is scalable for higher resolution images and frame rates and supporting different cameras and application requirements. We achieve this by designing highly parallel computation cores with very efficient memory access to the image data. Using a prototype board, we demonstrate real-time stereo processing with 640×480 pixel GigE Vision cameras at 30 frames per second. We show that this FPGA design is 10 times lower power, more scalable and has lower latency, as compared to a GPU based implementation of the same stereo algorithm.

In scenes of significantly varying lighting conditions, under and over exposed regions can suffer from a loss of information. Similarly, the presence of spatial depth within a scene can cause some image regions to be out of focus. Several methods of addressing these issues exist, including tone mapping for true high dynamic range representation and exposure fusion for combining varied-exposure low dynamic range images, as solutions to the former, and image fusion and segmentation etc. to address the latter. This paper proposes an overhauled method of exposure fusion that solves the exposure and focus problems simultaneously, achieving a well-exposed, all-in-focus result. Smart, scene-based data acquisition techniques for reducing both required input data and computational resources are discussed. A platform for a realtime system implementation is also presented.

This paper describes a family of reconfigurable processing elements (RPEs) used to support video processing for the Sarnoff Vision Front End 200 (VFE-200) vision system. Within the VFE-200 RPEs have been used to estimate visual motion, compute 3D scene structure using stereo analysis, perform geometric transformations (warps) on imagery with interpolation, and to act as triple ported frame store memory units. The RPEs described in this paper incorporate complex DRAM memory control interfaces, high precision fixed- and floating-point arithmetic (including floating point division), and sophisticated hybrids of memory and computational functions. Within this paper, the architecture and implementation of the RPEs and the VFE-200 are described, and examples of how the RPEs are used to support specific computer vision functions at real-time video rates are presented.

We describe a real-time system designed to construct a stable view of a scene through aligning images of an incoming video stream and dynamically constructing an image mosaic. This system uses a video processing unit developed by the David Sarnoff Research Center called the Vision Front End (VFE-100) for the pyramid-based image processing tasks required to implement this process. This paper includes a description of the multiresolution coarse-to-fine image registration strategy, the techniques used for mosaic construction, the implementation of this process on the VFE-100 system, and experimental results showing image mosaics constructed with the VFE-100.>

A segmented pipeline architecture for multiresolution, focal, array processing is presented. A buffer is introduced at each point in a pipeline computation at which changes in sample density or analysis area may take place. These buffers divide the pipeline into segments, each with constant data load. When active, a segment runs at its full design rate. Efficiency is maintained by switching processing elements between segments as image data flows through the system. The segmented pipeline architecture is illustrated with an application to image motion analysis.>