Brightness and contrast heavily influence image visual quality; therefore, modern digital camera image processing pipelines typically include a brightness and contrast enhancement (BCE) algorithm that enhances visual quality by applying tone mapping to the image. There are many BCE methods published in the literature that are variations of histogram equalization (HE) and contrast stretching (CS). When tested on large image databases, there are always certain images where these algorithms fail because image content is very diverse and a fixed method fails to adapt to this large variation. Our paper addresses this problem. We have developed an example-based BCE algorithm that can adapt its behavior to different scene types by using training examples that are hand-tuned by human observers for optimal visual quality. Our algorithm models the optimal enhancement function from these training images using Principal Component Analysis (PCA). Then, given a new image, the algorithm predicts the best amount of enhancement by extrapolating from closest training images. We have performed perceptual evaluations that conclude that our algorithm effectively enhances brightness and contrast judged by human observers.

In this paper, we describe an embedded programmable stereoscopic 3D camera system for mobile platforms. With the proliferation of stereoscopic televisions, computer monitors, and even auto-stereoscopic LCDs, as well as a growing inventory of stereoscopic games and movies, there is a growing demand for stereoscopic 3D content creation. We present each of the key components that enable the stereoscopic 3D video capture and playback on the embedded platform, including imaging, codec, graphics, and display subsystems. We describe several unique and new 3D image processing algorithms and explain how they are integrated into the stereo 3D system.

This paper presents an algorithm that stabilizes video and reduces rolling shutter distortions using a six-parameter affine model that explicitly contains parameters for translation, rotation, scaling, and skew to describe transformations between frames. Rolling shutter distortions, including wobble, skew and vertical scaling distortions, together with both translational and rotational jitter are corrected by estimating the parameters of the model and performing compensating transformations based on those estimates. The results show the benefits of the proposed algorithm quantified by the Interframe Transformation Fidelity (ITF) metric.

We propose a video stabilization algorithm that removes camera jitter in complex scenes with moving objects. Our algorithm is based on analyzing the content of the video according to the accumulated motion between the scene and the camera, which provides a simple and robust way of estimating jitter in the presence of camera panning and moving objects. We also describe a method of reducing the computational complexity of our algorithm by using a hierarchical clustering technique that optimizes the motion estimation resolution at different scene locations.

The active appearance model (AAM) is a powerful tool for modeling images of deformable objects and has been successfully used in a variety of alignment, tracking, and recognition applications. AAM uses subspace-based deformable models to represent the images of a certain object class. In general, fitting such complicated models to previously unseen images using standard optimization techniques is a computationally complex task because the gradient matrix has to be numerically computed at every iteration. The critical feature of AAM is a fast convergence scheme which assumes that the gradient matrix is fixed around the optimal coefficients for all images. Our work in this paper starts with the observation that such a fixed gradient matrix inevitably specializes to a certain region in the texture space, and the fixed gradient matrix is not a good estimate of the actual gradient as the target texture moves away from this region. Hence, we propose an adaptive AAM algorithm that linearly adapts the gradient matrix according to the composition of the target image's texture to obtain a better estimate for the actual gradient. We show that the adaptive AAM significantly outperforms the basic AAM, especially in images that are particularly challenging for the basic algorithm. In terms of speed and accuracy, the idea of a linearly adaptive gradient matrix presented in this paper provides an interesting compromise between a standard optimization technique that recomputes the gradient at every iteration and the fixed gradient matrix approach of the basic AAM.

The active appearance model (AAM) algorithm is a powerful tool for modeling images of deformable objects. AAM combines a subspace-based deformable model of an object's appearance with a fast and robust method of fitting this model to a previously unseen image. The speed of this algorithm comes from the assumption that the gradient matrix is fixed around the optimal coefficients for all images. In this paper, we propose a convergence scheme for AAM that adapts this gradient matrix to the target image's texture during convergence by adding linear modes of change that are based on the texture eigenvectors of AAM. We show that this adaptive strategy for the gradient matrix provides a significant increase in the performance of the AAM algorithm.

Face recognition is an important part of today's emerging biometrics and video surveillance markets. Recent years have witnessed an exploding interest in the development of face recognition algorithms and products. Currently, face recognition systems are usually implemented on general purpose processors. As face recognition algorithms move from research labs to the real world, power consumption and cost become critical issues. This motivates searching for implementations using a digital signal processor (DSP). Our goal is to explore the feasibility of implementing DSP-based face recognition systems. To achieve this goal, we implement a fully automatic face recognition system on Texas Instruments' TMS320C6416 DSP, profile performance, and analyze opportunities for optimization. The results of our experiments demonstrate that a generic C implementation with a modest C level optimization effort results in a face recognition software prototype that has low CPU and memory requirements. Hence, it appears that well-optimized face recognition implementations on DSPs can be an effective choice for embedded face recognition products.

Face images that are captured by surveillance cameras usually have a very low resolution, which significantly limits the performance of face recognition systems. In the past, super-resolution techniques have been proposed to increase the resolution by combining information from multiple images. These techniques use super-resolution as a preprocessing step to obtain a high-resolution image that is later passed to a face recognition system. Considering that most state-of-the-art face recognition systems use an initial dimensionality reduction method, we propose to transfer the super-resolution reconstruction from pixel domain to a lower dimensional face space. Such an approach has the advantage of a significant decrease in the computational complexity of the super-resolution reconstruction. The reconstruction algorithm no longer tries to obtain a visually improved high-quality image, but instead constructs the information required by the recognition system directly in the low dimensional domain without any unnecessary overhead. In addition, we show that face-space super-resolution is more robust to registration errors and noise than pixel-domain super-resolution because of the addition of model-based constraints.

Face images that are captured by surveillance cameras usually have a very low resolution, which significantly limits the performance of face recognition systems. In the past, super-resolution techniques have been proposed that attempt to increase the resolution by combining information from multiple images. These techniques use super-resolution as a preprocessing system to obtain a high resolution image that can later be passed to a face recognition system. Considering that most state-of-the-art face recognition systems use an initial dimensionality reduction method, we propose embedding the super-resolution algorithm into the face recognition system so that super-resolution is not performed in the pixel domain, but is instead performed in a reduced dimensional domain. The advantage of such an approach is a significant decrease in the computational complexity of the super-resolution algorithm because the algorithm no longer tries to construct a visually improved high quality image, but instead constructs the information required by the recognition algorithm directly in the lower dimensional domain without any unnecessary overhead.

In this paper, we present a segmented linear subspace model for face recognition that is robust under varying illumination conditions. The algorithm generalizes the 3D illumination subspace model by segmenting the image into regions that have surface normals whose directions are close to each other. This segmentation is performed using a K-means clustering algorithm and requires only a few training images under different illuminations. When the linear subspace model is applied to the segmented image, recognition is robust to attached and cast shadows, and the recognition rate is equal to that of computationally more complex systems that require constructing the 3D surface of the face.