Stereo matching, a key element towards extracting depth information from stereo images, is widely used in several embedded consumer electronic and multimedia systems. Such systems demand high processing performance and accurate depth perception, while their deployment in embedded and mobile environments implies that cost, energy and memory overheads need to be minimized. Hardware acceleration has been demonstrated in efficient embedded stereo vision systems. To this end, this paper presents the design and implementation of a hardware-based stereo matching system able to provide high accuracy and concurrently high performance for embedded vision devices, which are associated with limited hardware and power budget. We first implemented a compact and efficient design of the guided image filter, an edge-preserving filter, which reduces the hardware complexity of the implemented stereo algorithm, while at the same time maintains high-quality results. The guided filter design is used in two parts of the stereo matching pipeline, showing that it can simplify the hardware complexity of the Adaptive Support Weight aggregation step, and efficiently enable a powerful disparity refinement unit, which improves matching accuracy, even though cost aggregation is based on simple, fixed support strategies. We implemented several variants of our design on a Kintex-7 FPGA board, which was able to process HD video (1,280 × 720) in real-time (60 fps), using ~57.5k and ~71k of the FPGA's logic (CLB) and register resources, respectively. Additionally, the proposed stereo matching design delivers leading accuracy when compared to state-of-the-art hardware implementations based on the Middlebury evaluation metrics (at least 1.5 percent less bad matching pixels).

An embedded, real-time, and low power obstacle avoidance system is a critical component towards fully autonomous robots that can be used in safety missions, space exploration, and transportation systems among others. In this paper a complete prototyping platform for the evaluation of obstacle avoidance systems and autonomous robots is realized on reconfigurable hardware. An efficient stereo vision algorithm for producing the necessary 3D and an obstacle avoidance subsystem were both implemented on an ATLYS Spartan-6 FPGA board equipped with a VmodCam stereo camera module. A modified FDX Vantage 1/10 electric car platform was used for testing the proposed architecture in indoor and outdoor real-world scenes. The system receives stereo image data from the VmodCam module and a decision-making algorithm is applied on a specified Region of Interest (RoI) on the produced disparity map. The algorithm outputs the direction that the robot should move to in order to avoid any obstacles present. Experimental evaluation results indicate that the FPGA-based robotic platform can avoid obstacles in real-time (i.e. can process and identify obstacles within a 1/30th of a second that a stereo image takes to be processed) in both indoor and outdoor environments, with 91.7% accuracy, equivalent to software implementations. The overall power consumption of the proposed architecture, excluding the electronic car platform, is 6 W, making it ideal for use on mobile robots, without becoming a significant drain on its battery life.

In this paper, we propose an FPGA-based emulation framework that can provide dynamic vulnerability analysis for hardware-accelerated computer vision applications. The framework can be integrated alongside the targeted application, to allow for run-time, in-field, dynamically adjusted vulnerability analysis in real-world conditions, taking into consideration the non-deterministic parameters of the computer vision algorithm computations. We evaluate the proposed framework in real-time using an FPGA platform, for an obstacle avoidance (OA) computer vision application and its disparity estimation kernel to study the impact of Single-Event Upsets (SEUs).

Emerging embedded vision systems utilize disparity estimation as a means to perceive depth information to intelligently interact with their host environment and take appropriate actions. Such systems demand high processing performance and accurate depth perception while requiring low energy consumption, especially when dealing with mobile and embedded applications, such as robotics, navigation, and security. The majority of real-time dedicated hardware implementations of disparity estimation systems have adopted local algorithms relying on simple cost aggregation strategies with fixed and rectangular correlation windows. However, such algorithms generally suffer from significant ambiguity along depth borders and areas with low texture. To this end, this article presents the hardware architecture of a disparity estimation system that enables good performance in both accuracy and speed. The architecture implements an adaptive support weight stereo correspondence algorithm that integrates image segmentation information in an attempt to increase the robustness of the matching process. The article also presents hardware-oriented algorithmic modifications/optimization techniques that make the algorithm hardware-friendly and suitable for efficient dedicated hardware implementation. A comparison to the literature asserts that an FPGA implementation of the proposed architecture is among the fastest implementations in terms of million disparity estimations per second (MDE/s), and with an overall accuracy of 90.21%, it presents an effective processing speed/disparity map accuracy trade-off.

Stereo matching is a vital task in several emerging embedded vision applications requiring high-quality depth computation and real-time frame-rate. Although several stereo matching dedicated-hardware systems have been proposed in recent years, only few of them focus on balancing accuracy and speed. This paper proposes a hardware-based stereo matching architecture that aims to provide high accuracy and concurrently high performance in embedded vision applications. The proposed architecture integrates a compact and efficient design of the recently proposed guided image filter; an edge-preserving filter that reduces the hardware complexity of the implemented stereo algorithm, while at the same time maintains high-quality results. A prototype of the architecture has been implemented on a Kintex-7 FPGA board, achieving 60 fps for 720p resolution images. Moreover, the proposed design delivers leading accuracy when compared to state-of-the-art hardware implementations.

Retinal vessel tree extraction is a complex and computationally intensive task used in several medical and biometric applications. The emergence of portable biometric authentication applications, as well as on-site biomedical diagnostics, raises the need for hardware-accelerated, power-efficient architectures that can satisfy the performance and accuracy requirements of retinal vessel tree extraction. As such, this paper presents a VLSI implementation of a retina vessel segmentation system, in an attempt to illustrate the advantages and performance benefits that result from a dedicated VLSI solution. The proposed design implements an unsupervised, vessel segmentation algorithm, which utilizes match filtering with signed integers to enhance the difference between the blood vessels and the rest of the retina. The design simplifies the process of obtaining a binary map of the vessel tree by using parallel processing and efficient resource sharing, thus offering real-time performance. FPGA-based simulation results indicate significant performance improvements (up to 90x) when compared to existing hardware and software implementations.

Stereo Vision, a technique aimed at inferring depth information from stereo images, has been used in a wide range of computer vision applications, with real-time requirements in emerging embedded vision systems. Computation of the disparity map, a vital step in extracting depth information from stereo images, requires a significant amount of computational resources. As such, existing software implementations require high-end hardware platforms to achieve real-time frame rates, suggesting that dedicated hardware mechanisms might be more suitable for embedded applications. In this paper, we present a disparity map computation architecture targeting embedded stereo vision applications with hard real-time requirements. The architecture integrates a hardware edge detection mechanism that reduces the search space, improving the overall performance, and is configurable in terms of various application parameters, making it suitable for a number of application environments. The paper also presents a study on the impact of the various parameters in terms of the performance and hardware/power overheads. An experimental prototype of the architecture was implemented on the Xilinx ML505 FPGA Evaluation Platform, achieving 50 Frames Per Second (fps) for 1,280 × 1,024 image sizes. Moreover, the quality of the disparity maps generated by the proposed system is comparable to other existing hardware implementations featuring local stereo correspondence methods.

Emerging embedded 3D vision systems for robotics and security applications utilize object detection to perform video analysis in order to intelligently interact with their host environment and take appropriate actions. Such systems have high performance and high detection-accuracy demands, while requiring low energy consumption, especially when dealing with embedded mobile systems. However, there is a large image search space involved in object detection, primarily because of the different sizes in which an object may appear, which makes it difficult to meet these demands. Hence, it is possible to meet such constraints by reducing the search space involved in object detection. To this end, this article proposes a depth and edge accelerated search method and a dedicated hardware architecture that implements it to provide an efficient platform for generic real-time object detection. The hardware integration of depth and edge processing mechanisms, with a support vector machine classification core onto an FPGA platform, results in significant speed-ups and improved detection accuracy. The proposed architecture was evaluated using images of various sizes, with results indicating that the proposed architecture is capable of achieving real-time frame rates for a variety of image sizes (271 fps for 320 × 240, 42 fps for 640 × 480, and 23 fps for 800 × 600) compared to existing works, while reducing the false-positive rate by 52%.

Disparity estimation in stereoscopic vision is a vital step for the extraction of depth information from stereo images. This paper presents the hardware implementation of a disparity estimation system that enables good performance in both accuracy and speed. The architecture implements an adaptive support weight stereo correspondence algorithm, which integrates information obtained from image segmentation, in an attempt to increase the robustness of the matching process. The proposed system integrates optimization techniques that make the algorithm hardware-friendly and suitable for embedded vision systems. A prototype of the architecture was implemented on an FPGA, achieving 30 fps for 640×480 image sizes. The quality of the disparity maps generated by the proposed system is also better than other existing hardware implementations featuring fixed support local correspondence methods.

Object detection is a vital task in several emerging applications, requiring real-time detection frame-rate and low energy consumption for use in embedded and mobile devices. This paper proposes a hardware-based, depth-directed search method for reducing the search space involved in object detection, resulting in significant speed-ups and energy savings. The proposed architecture utilizes the disparity values computed from a stereoscopic camera setup, in an attempt to direct the detection classifier to regions that contain objects of interest. By eliminating large amounts of search data, the proposed system achieves both performance gains and reduced energy consumption. FPGA simulation results indicate performance speedups up to 4.7 times and high energy savings ranging from 41-48%, when compared to the traditional sliding window approach.

Object detection is a vital task in several existing as well as emerging applications, requiring real-time processing and low energy consumption, and often with limited available hardware budget in the case of embedded and mobile devices. This paper proposes an FPGA-based object detection system that utilizes edge information to reduce the search space involved in object detection. By eliminating large amounts of search data, the proposed system achieves both performance gains, and reduced energy consumption, while requiring minimal additional hardware, making it suitable for resource-constrained FPGAs. Implementation results on an FPGA indicate performance speedups up to 4.9 times, and high energy savings ranging from 73-78%, when compared to the traditional sliding window approach for FPGA implementations.

Stereoscopic 3D reconstruction is an important algorithm in the field of Computer Vision, with a variety of applications in embedded and real-time systems. Existing software-based implementations cannot satisfy the performance requirements for such constrained systems; hence an embedded hardware mechanism might be more suitable. In this paper, we present an architecture of a 3D reconstruction system for stereoscopic images, which we implement on Virtex2 Pro FPGA. The architecture uses a Sobel edge detector to achieve real-time (75 fps) performance, and is configurable in terms of various application parameters, making it suitable for a number of application environments. The paper also presents a design exploration on algorithmic parameters such as disparity range, correlation window size, and input image size, illustrating the impact on the performance for each parameter.

QAM is a widely used multi-level modulation technique, with a variety of applications in data radio communication systems. Most existing implementations of QAM-based systems use high levels of modulation in order to meet the high data rate constraint of emerging applications. This work presents the architecture of a highly-parallel MPSoC-based QAM modulator that offers multi-rate modulation. The proposed MPSoC architecture is modular and provides flexibility via dynamic reconfiguration of the QAM, offering high data rates (more than 1 Gbps), even at low modulation levels (16-QAM). Furthermore, the proposed QAM implementation integrates a hardware-based resource allocation algorithm for dynamic load balancing.