Numerous camera sensors have been recently proposed to estimate the depth of objects in the scene. However, many of these sensors either require multiple imaging elements or an active illuminant. In this paper, we propose a new depth camera, termed offset aperture (OA) camera, which provides depth of the scene with a single shot without an active illuminant. The camera is based on a 4-color sensor that senses the infrared (IR) signal along with the visible RGB signals. Separate OAs are used for IR and RGB images. A depth-dependent disparity is generated between the IR and RGB images. There is no disparity across the R, G, and B channel images, which allows the RGB image to be well-aligned. Numerous techniques are used in this work to observe good-quality depth maps and images. Leakage compensation has been proposed to remove crosstalk between the IR and RGB images. A comparison with another 4-color sensor called Dual Aperture camera shows that OA camera provides 88% improvement in depth sensitivity and 45% improvement in color quality, on average. The proposed OA camera can be used for numerous applications including occlusion handling in stereo, selective focus, and night vision. The advantages of the proposed OA camera include competitive image quality, capability of depth extraction, well-aligned RGB images, and small footprint of the camera, which make OA camera a suitable candidate for mobile applications.

Crowd anomaly detection is a key research area in vision-based surveillance. Most of the crowd anomaly detection algorithms are either too slow, bulky, or power-hungry to be applicable for battery-powered surveillance cameras. In this paper, we present a new crowd anomaly detection algorithm. The proposed algorithm creates a feature for every superpixel that includes the contribution from the neighboring superpixels only if their direction of motion conforms with the dominant direction of motion in the region. We also propose using univariate Gaussian discriminant analysis with the K-means algorithm for classification. Our method provides superior accuracy over numerous deep learning-based and handcrafted feature-based approaches. We also present a low-power FPGA implementation of the proposed method. The algorithm is developed such that features are extracted over non-overlapping pixels. This allows gating inputs to numerous modules resulting in higher power efficiency. The maximum energy required per pixel is 2.43 nJ in our implementation. 126.65 Mpixels can be processed per second by the proposed implementation. The speed, power, and accuracy performance of our method make it competitive for surveillance applications, especially battery-powered surveillance cameras.

This paper presents a Offset Aperture (OA) based single camera system and proposes a optimized vision processor, a new hardware architecture for fast, low-energy, and low-complexity depth extraction. The proposed design was fabricated in 110nm CMOS image sensor technology and supports 32-level depth resolution on 1920×1080 full HD image with 30fps, consuming 280.53mW from 1.5V supply and a mere 2.8% of bad classification. The low-complexity algorithms are employed to eliminate the DRAM access, thereby the proposed OA architecture can be directly embedded with the CMOS image sensor and commercial image processing chip.

Due to the increasing demand for 3D applications, development of novel depth-sensing cameras is being actively pursued. However, most of these cameras still face the challenge of high energy consumption and slow speed in the depth extraction process. This becomes a serious bottleneck in embedded implementations where real-time performance is required, constrained by power and area. This work proposes Offset Aperture (OA) camera, a new hardware architecture for fast, low-energy, and low-complexity depth extraction. Optimal implementations of pre-processing, cost-volume generation and cost-aggregation are presented. The whole depth-extraction pipeline has been implemented on a Field Programmable Gate Array (FPGA). Overall, a mere 2.8% of bad classification was achieved with the proposed system. Also, the proposed system can process 37 VGA frames per second while consuming 0.224 μJ/pixel. High accuracy, speed and low energy consumption of the proposed OA architecture make it suitable for embedded applications.