The human visual system is not very sensitive to the absolute luminance of an image, but rather responds to local luminance changes, i.e., the gradient of an image. In this paper, we propose a constrained optimization approach for image gradient enhancement. The gradient strength of the enhanced image can be controlled directly using a target gradient strength parameter in the cost function. To suppress artifacts and ensure that contrast improves, a novel constraint is included in the optimization. Due to the number of variables in optimization-based image enhancement techniques being equal to the number of gray scales, we quantize the image using a k -means clustering-based histogram mergence (KCHM) method before enhancement. KCHM can significantly reduce the number of image gray scales while effectively preserving the subjective quality. This is useful considering the reduction of variables is good for solving optimization and reducing computation cost. Experimental results demonstrate that the proposed method can significantly improve the subjective image quality by enhancing both the contrast and the image gradient.

Most existing camera placement algorithms focus on coverage and/or visibility analysis, which ensures that the object of interest is visible in the camera's field of view (FOV). However, visibility, which is a fundamental requirement of object tracking, is insufficient for automated persistent surveillance. In such applications, a continuous consistently labeled trajectory of the same object should be maintained across different camera views. Therefore, a sufficient uniform overlap between the cameras' FOVs should be secured so that camera handoff can successfully and automatically be executed before the object of interest becomes untraceable or unidentifiable. In this paper, we propose sensor-planning methods that improve existing algorithms by adding handoff rate analysis. Observation measures are designed for various types of cameras so that the proposed sensor-planning algorithm is general and applicable to scenarios with different types of cameras. The proposed sensor-planning algorithm preserves necessary uniform overlapped FOVs between adjacent cameras for an optimal balance between coverage and handoff success rate. In addition, special considerations such as resolution and frontal-view requirements are addressed using two approaches: 1) direct constraint and 2) adaptive weights. The resulting camera placement is compared with a reference algorithm published by Erdem and Sclaroff. Significantly improved handoff success rates and frontal-view percentages are illustrated via experiments using indoor and outdoor floor plans of various scales.

In order to achieve improved recognition performance in comparison with conventional broadband images, this paper addresses a new method that automatically specifies the optimal spectral range for multispectral face images according to given illuminations. The novelty of our method lies in the introduction of a distribution separation measure and the selection of the optimal spectral range by ranking these separation values. The selected spectral ranges are consistent with the physics analysis of the multispectral imaging process. The fused images from these chosen spectral ranges are verified to outperform the conventional broadband images by 3%-20%, based on a variety of experiments with indoor and outdoor illuminations using two well-recognized face-recognition engines. Our discovery can be practically used for a new customized sensor design associated with given illuminations for improved face-recognition performance over the conventional broadband images.

Camera handoff is a crucial step to generate a continuously tracked and consistently labeled trajectory of the object of interest in multi-camera surveillance systems. Most existing camera handoff algorithms concentrate on data association, namely consistent labeling, where images of the same object are matched across different cameras. However, most real-time object tracking systems see a decrease in the system's frame rate as the number of tracked objects increases. To address this issue, we propose to incorporate an adaptive resource management mechanism into camera handoff. In so doing, cameras¿ resources can be dynamically allocated to multiple objects according to their priorities and hence the required minimum frame rate can be maintained. Experimental results illustrate that the proposed camera handoff algorithm is capable of maintaining a constant frame rate and of achieving a substantially improved handoff success rate by approximately 20% in comparison with the algorithm presented by Khan and Shah.

Most existing camera placement algorithms focus on coverage and/or visibility analysis, which ensures that the object of interest is visible in the camera—s field of view (FOV). However, visibility, a fundamental requirement of object tracking, is insufficient for persistent and automated tracking. In such applications, a continuous and consistently labeled trajectory of the same object should be maintained across different cameras’ views. Therefore, a sufficient overlap between the cameras’ FOVs should be secured so that camera handoff can be executed successfully and automatically before the object of interest becomes untraceable or unidentifiable. The proposed sensor planning method improves existing algorithms by adding handoff rate analysis, which preserves necessary overlapped FOVs for an optimal handoff success rate. In addition, special considerations such as resolution and frontal view requirements are addressed using two approaches: direct constraint and adaptive weight. The resulting camera placement is compared with a reference algorithm by Erdem and Sclaroff. Significantly improved handoff success rate and frontal view percentage are illustrated via experiments using typical office floor plans.

Multispectral imaging and fusion of multispectral images in visible spectrum have been employed and proved to enhance the recognition performance over conventional broad-band images, especially when the illumination changes. In this paper, we further our study toward the recognition performance of a smaller number of spectral band images. Given the illumination, we propose an automated method that specifies the optimal spectral ranges to achieve an improved recognition performance compared with conventional broad-band images. Our discovery can be practically used for a new customized sensor design associated with given illuminations for an improved face recognition performance over conventional broad-band images.

In this paper, we propose a series of techniques to enhance the computational performance of existing Belief Propagation (BP) based stereo matching that relies on automatic estimation of the Markov random field (MRF) parameters. First, we show how convergence in matching can be achieved faster than with the existing message comparison technique by skipping comparisons in early inferences. Second, assuming that a stereo pair is captured with identical cameras, we apply a hypothesis called noise equivalence to pre-estimate the likelihood parameters and thus, avoid costly nested inferences to reduce the computational time. The likelihood parameters and intensity information are used for accelerated message propagation in image regions lacking gradients. Third, the prior model parameters are estimated with a combination of maximum likelihood (ML) estimation and disparity gradient constraint to further reduce the computational time. Supporting experiments for the proposed algorithms show encouraging results on ground truth test images.

Most existing sensor planning algorithms find it difficult to tackle the discrepancy between a PTZ camera¿s limited instant field of view (FOV) and panoramic achievable FOV. In this paper, we introduce the probability of camera overload to resolve this discrepancy and present a sensor planning algorithm for PTZ cameras under the same framework as static cameras. The resulting camera placement achieves the optimal balance between coverage and handoff success rate. Furthermore, our algorithm is able to incorporate the target¿s dynamics into sensor planning. As a result, the system¿s handoff success rate can be maintained in environments with various target densities. Experimental results and comparisons with a reference algorithm proposed by Erdem and Sclaroff verify the effectiveness of our algorithm via a significantly improved handoff success rate.

Dual-camera systems have been widely used in surveillance because of the ability to explore the wide field of view (FOV) of the omnidirectional camera and the wide zoom range of the PTZ camera. Most existing algorithms require a priori knowledge of the omnidirectional camera's projection model to solve the nonlinear spatial correspondences between the two cameras. To overcome this limitation, two methods are proposed: 1) geometry and 2) homography calibration, where polynomials with automated model selection are used to approximate the camera's projection model and spatial mapping, respectively. The proposed methods not only improve the mapping accuracy by reducing its dependence on the knowledge of the projection model but also feature reduced computations and improved flexibility in adjusting to varying system configurations. Although the fusion of multiple cameras has attracted increasing attention, most existing algorithms assume comparable FOV and resolution levels among multiple cameras. Different FOV and resolution levels of the omnidirectional and PTZ cameras result in another critical issue in practical tracking applications. The omnidirectional camera is capable of multiple object tracking while the PTZ camera is able to track one individual target at one time to maintain the required resolution. It becomes necessary for the PTZ camera to distribute its observation time among multiple objects and visit them in sequence. Therefore, this paper addresses a novel scheme where an optimal visiting sequence of the PTZ camera is obtained so that in a given period of time the PTZ camera automatically visits multiple detected motions in a target-hopping manner. The effectiveness of the proposed algorithms is illustrated via extensive experiments using both synthetic and real tracking data and comparisons with two reference systems.

We present an optimized full scale automatic camera calibration approach which is both accurate and simple to implement. The method can be applied to a wide range of cameras equipped with normal, wide-angle, fish-eye, and telephoto lenses. The procedure does not require prior knowledge of any parameters. The method uses a simple planar calibration pattern which is observed from different positions. Closed-form estimates for the intrinsic and extrinsic parameters are computed followed by nonlinear optimization. A polynomial function describes the lens projection instead of the commonly used radial model. Statistical information criteria are used to automatically determine the complexity of the lens distortion model. The proposed complete calibration method is shown to obtain lower reprojection error than some commonly used methods while maintaining a low complexity distortion model.

Dual camera systems (omnidirectional and slaved PTZ cameras) are widely used in public area monitoring and target tracking. However, due to their low and non-uniform resolution, omnidirectional cameras are only able to provide moderate accuracy in both motion/target detection and tracking. To overcome this disadvantage, two methods are proposed: geometry mapping based on a polynomial imaging model and closed loop cooperative tracking. Using a unified polynomial approximation [1], the geometry relationship between the omnidirectional and PTZ cameras can be formulated in a general solution from camera calibration with a fully automated model selection. Distributed Kalman filters are developed to exchange estimated trajectories among cameras forming closed loop sub-systems for individual cameras. The effectiveness of the proposed algorithms is illustrated via experiments and comparisons with existing systems.

Contrast enhancement has an important role in image processing applications. Conventional contrast enhancement techniques either fail to produce satisfactory results for a broad variety of low-contrast images, or cannot be automatically applied to different images, because their parameters must be specified manually to produce a satisfactory result for a given image. This paper describes a new automatic method for contrast enhancement. The basic procedure is to first group the histogram components of a low-contrast image into the proper number of bins according to a selected criterion, then redistribute these bins uniformly over the grayscale, and finally ungroup the previously grouped gray-levels. Accordingly, this new technique is named gray-level grouping (GLG). GLG not only produces results superior to conventional contrast enhancement techniques, but is also fully automatic in most circumstances, and is applicable to a broad variety of images

In this effort, we propose a new image fusion technique, utilizing empirical mode decomposition (EMD), for improved face recognition. EMD is a non-parametric data-driven analysis tool that decomposes non-linear non-stationary signals into intrinsic mode functions (IMFs). In this method, we decompose images from different imaging modalities into their IMFs. Fusion is performed at the decomposition level and the fused IMFs are reconstructed to form the fused image. The effect of fusion on face recognition is measured by obtaining the cumulative match characteristics (CMCs) between galleries and probes. Apart from conducting face recognition tests on visible and infrared raw datasets, we use datasets fused by averaging, principal component (PCA) fusion, wavelet based fusion and our method, for comparison. The face recognition rate due to EMD fused images is higher than the face recognition rates due to raw visible, raw infrared and other fused images. Examples of the fused images and illustrative CMC comparison charts are shown.

In size preserving video tracking, the camera's focal length (zoom) is adjusted automatically to compensate for the changes in the target's image size caused by the relative motion between the camera and the target. The accurate estimation of these changes is paramount to the system performance. Structure from motion (SFM) based on the weak perspective projection model has been applied to real time target scale estimation (B. Tordoff, et al., 2004). In this paper we design target scale estimation algorithms with linear solutions based on more advanced projection models: paraperspective and perspective. The performances of the algorithms using three projection models are examined and compared. Experimental results show that the proposed algorithm based on the paraperspective projection model produces the best performance with respect to accuracy, complexity, and robustness.

Applications of digital imaging with extreme zoom are traditionally found in astronomy and wild life monitoring. More recently, the need for such capabilities has extended to wide area monitoring such as forest fires, harbors, and waterways. This paper presents a number of sensor arrangements for the acquisition of high magnification images. It also describes user interfaces developed for the system’s remote control and data acquisition and processing. Hardware and software considerations are addressed, focusing on optical setups and image de-blurring techniques. An image restoration algorithm tailored to such systems is developed, where an adaptive sharpness measure is employed as a cost function to guide the search for an optimal point spread function (PSF) for image de-blur. Experimental results demonstrate a considerably enhanced robustness to noise and ability to select the optimum PSF, producing much superior restored images.

Contrast enhancement has an important role in image processing applications. Conventional contrast enhancement techniques either often fail to produce satisfactory results for a broad variety of low-contrast images, or cannot be automatically applied to different images, because their parameters must be specified manually to produce a satisfactory result for a given image. This paper describes a new automatic method for contrast enhancement. The basic procedure is to first group the histogram components of a low-contrast image into a proper number of bins according to a selected criterion, then redistribute these bins uniformly over the grayscale, and finally ungroup the previously grouped gray-levels. Accordingly, this new technique is named gray-level grouping (GLG). GLG not only produces results superior to conventional contrast enhancement techniques, but is also fully automatic in most circumstances, and is applicable to a broad variety of images. An extension of GLG, selective GLG (SGLG), and its variations will be discussed in Part II of this paper. SGLG selectively groups and ungroups histogram components to achieve specific application purposes, such as eliminating background noise, enhancing a specific segment of the histogram, and so on. The extension of GLG to color images will also be discussed in Part II.

This is Part II of the paper, "Gray-Level Grouping (GLG): an Automatic Method for Optimized Image Contrast Enhancement". Part I of this paper introduced a new automatic contrast enhancement technique: gray-level grouping (GLG). GLG is a general and powerful technique, which can be conveniently applied to a broad variety of low-contrast images and outperforms conventional contrast enhancement techniques. However, the basic GLG method still has limitations and cannot enhance certain classes of low-contrast images well, e.g., images with a noisy background. The basic GLG also cannot fulfill certain special application purposes, e.g., enhancing only part of an image which corresponds to a certain segment of the image histogram. In order to break through these limitations, this paper introduces an extension of the basic GLG algorithm, selective gray-level grouping (SGLG), which groups the histogram components in different segments of the grayscale using different criteria and, hence, is able to enhance different parts of the histogram to various extents. This paper also introduces two new preprocessing methods to eliminate background noise in noisy low-contrast images so that such images can be properly enhanced by the (S)GLG technique. The extension of (S)GLG to color images is also discussed in this paper. SGLG and its variations extend the capability of the basic GLG to a larger variety of low-contrast images, and can fulfill special application requirements. SGLG and its variations not only produce results superior to conventional contrast enhancement techniques, but are also fully automatic under most circumstances, and are applicable to a broad variety of images.

This paper describes a series of linear and nonlinear pseudocoloring maps designed and applied to single energy X-ray luggage scans to assist airport screeners in identifying and detecting threat items, particularly hard to see low-density weapons in luggage. Considerations of the psychological and physiological processing involved in the human perception of color as well as the effects of using various color models, such as the RGB and HSI models, were explored. Original grayscale data, various enhanced images, and segmented scenes were used as input to the various color mapping schemes designed in this research. A highly interactive user-friendly graphical interface and a portable test were developed and used in a performance evaluation study involving a population of actual federal airport screeners. The study proved the advantages of using color over gray level data and also allowed the ranking of color maps and selection of the best performing color schemes. Rate improvements in weapon detection of up to 97% were achieved through the use of color

This paper proposes a fully digital auto-focusing algorithm for restoring the image with differently out-of-focused objects, which can restore background as well as all objects. In this paper, we assume that out-of-focus blur is isotropic such as circle of confusion (COC) or two-dimensional Gaussian blur. Therefore, the proposed algorithm can segment and estimate the point spread function (PSF) by using the size of ramp in the one-dimensional step response. The proposed algorithm can be developed by object-based image segmentation and restoration algorithm. Experimental results show that the proposed object-based image restoration algorithm can efficiently remove the space-variant out of focus blur from the image with multiple blurred objects.

We present a combined real-time face region tracking and highly accurate face recognition technique for an intelligent surveillance system. High-resolution face images are very important to achieving accurate identification of a human face. Conventional surveillance or security systems, however, usually provide poor image quality because they use only fixed cameras to record scenes passively. We have implemented a real-time surveillance system that tracks a moving face using four pan-tilt-zoom (PTZ) cameras. While tracking, the region-of-interest (ROI) can be obtained by using a low-pass filter and background subtraction with the PTZ. Color information in the ROI is updated to extract features for optimal tracking and zooming. FaceIt/sup /spl reg//, which is one of the most popular face recognition software packages, is evaluated and then used to recognize the faces from the video signal. Experimentation with real human faces showed highly acceptable results in the sense of both accuracy and computational efficiency.

In this paper, we describe an algorithm to measure the shape complexity for discrete approximations of planar curves in 2D images and manifold surfaces for 3D triangle meshes. We base our algorithm on shape curvature, and thus we compute shape information as the entropy of curvature. We present definitions to estimate curvature for both discrete curves and surfaces and then formulate our theory of shape information from these definitions. We demonstrate our algorithm with experimental results.

Tracking and recognizing non-rigid objects in video image sequences are complex tasks of increasing importance to many applications. In this paper, we present a hierarchical realization of an enhanced active shape model for color video tracking and we study the performance of hierarchical and non-hierarchical implementations in the RGB, YUV, and HSI color spaces.