The human visual system is not very sensitive to the absolute luminance of an image, but rather responds to local luminance changes, i.e., the gradient of an image. In this paper, we propose a constrained optimization approach for image gradient enhancement. The gradient strength of the enhanced image can be controlled directly using a target gradient strength parameter in the cost function. To suppress artifacts and ensure that contrast improves, a novel constraint is included in the optimization. Due to the number of variables in optimization-based image enhancement techniques being equal to the number of gray scales, we quantize the image using a k -means clustering-based histogram mergence (KCHM) method before enhancement. KCHM can significantly reduce the number of image gray scales while effectively preserving the subjective quality. This is useful considering the reduction of variables is good for solving optimization and reducing computation cost. Experimental results demonstrate that the proposed method can significantly improve the subjective image quality by enhancing both the contrast and the image gradient.

Most existing camera placement algorithms focus on coverage and/or visibility analysis, which ensures that the object of interest is visible in the camera's field of view (FOV). However, visibility is inadequate for continuous and automated tracking. In such applications, a sufficient overlap between FOVs should be secured so that camera handoff can be executed successfully and automatically before the object of interest becomes untraceable or unidentifiable. In addition, most of the existing algorithms employ discrete solution space, which suffers from limited solution accuracy and high computational complexity due to the high dimension of sampled solution space. In this paper, we propose to perform the optimisation of the camera parameters in a continuous solution space. In addition, to incorporate the optimisation of coverage and sufficient overlapped FOVs, a weighted sum approach is utilised to translate a multiple objective optimisation problem into a single objective optimisation process in our previous work. Significantly improved handoff success rates are illustrated via experiments using typical office floor plans in comparison with Erdem and Sclaroff's method. Improved accuracy, enhanced robustness, completeness of the solution set, and reduced computational complexity are accomplished in comparison with our previous algorithms.

In this paper, we propose a new cluster validity index (CVI) based on geometrical shape. Classic CVIs are based on a combination of separation and compactness measures and may include a measure of overlap between clusters. The proposed CVI combines measures of compactness and over-lap using n-sphere shape. We conducted experiments on several real data sets from the UCI repository and compared the performances of the proposed CVI with widely used CVIs. Results demonstrate that the proposed CVI performs better than the others even when used with complicated data sets.

In this paper, we propose a novel outdoor scene image segmentation algorithm based on background recognition and perceptual organization. We recognize the background objects such as the sky, the ground, and vegetation based on the color and texture information. For the structurally challenging objects, which usually consist of multiple constituent parts, we developed a perceptual organization model that can capture the nonaccidental structural relationships among the constituent parts of the structured objects and, hence, group them together accordingly without depending on a priori knowledge of the specific objects. Our experimental results show that our proposed method outperformed two state-of-the-art image segmentation approaches on two challenging outdoor databases (Gould data set and Berkeley segmentation data set) and achieved accurate segmentation quality on various outdoor natural scene environments.

Most existing performance evaluation methods concentrate on defining separate metrics over a wide range of conditions and generating standard benchmarking video sequences for examining the effectiveness of video tracking systems. In other words, these methods attempt to design a robustness margin or factor for the system. These methods are deterministic in which a robustness factor, for example, 2 or 3 times the expected number of subjects to track or the strength of illumination would be required in the design. This often results in over design, thus increasing costs, or under design causing failure by unanticipated factors. In order to overcome these limitations, we propose in this paper an alternative framework to analyze the physics of the failure process and, through the concept of reliability, determine the time to failure in automated video tracking systems. The benefit of our proposed framework is that we can provide a unified and statistical index to evaluate the performance of automated video tracking system for a task to be performed. At the same time, the uncertainty problem about a failure process, which may be caused by the system's complexity, imprecise measurements of the relevant physical constants and variables, or the indeterminate nature of future events, can be addressed accordingly based on our proposed framework.

In this paper, we present a new method to enhance depth images captured by a time-of-flight (TOF) depth sensor spatially and temporally. In practice, depth images obtained from TOF depth sensors have critical problems, such as optical noise existence, unmatched boundaries, and temporal inconsistency. In this work, we improve depth quality by performing a newly-designed joint bilateral filtering, color segmentation-based boundary refinement, and motion estimation-based temporal consistency. Experimental results show that the proposed method significantly minimizes the inherent problems of the depth images so that we can use them to generate a dynamic and realistic 3D scene.

Most existing camera placement algorithms focus on coverage and/or visibility analysis, which ensures that the object of interest is visible in the camera's field of view (FOV). However, visibility, which is a fundamental requirement of object tracking, is insufficient for automated persistent surveillance. In such applications, a continuous consistently labeled trajectory of the same object should be maintained across different camera views. Therefore, a sufficient uniform overlap between the cameras' FOVs should be secured so that camera handoff can successfully and automatically be executed before the object of interest becomes untraceable or unidentifiable. In this paper, we propose sensor-planning methods that improve existing algorithms by adding handoff rate analysis. Observation measures are designed for various types of cameras so that the proposed sensor-planning algorithm is general and applicable to scenarios with different types of cameras. The proposed sensor-planning algorithm preserves necessary uniform overlapped FOVs between adjacent cameras for an optimal balance between coverage and handoff success rate. In addition, special considerations such as resolution and frontal-view requirements are addressed using two approaches: 1) direct constraint and 2) adaptive weights. The resulting camera placement is compared with a reference algorithm published by Erdem and Sclaroff. Significantly improved handoff success rates and frontal-view percentages are illustrated via experiments using indoor and outdoor floor plans of various scales.

Due to the capacity of pan-tilt-zoom (PTZ) cameras to simultaneously cover a panoramic area and maintain high resolution imagery, researches in automated surveillance systems with multiple PTZ cameras have become increasingly important. Most existing algorithms require the prior knowledge of intrinsic parameters of the PTZ camera to infer the relative positioning and orientation among multiple PTZ cameras. To overcome this limitation, we propose a novel mapping algorithm that derives the relative positioning and orientation between two PTZ cameras based on a unified polynomial model. This reduces the dependence on the knowledge of intrinsic parameters of PTZ camera and relative positions. Experimental results demonstrate that our proposed algorithm presents substantially reduced computational complexity and improved flexibility at the cost of slightly decreased pixel accuracy, as compared with the work of Chen and Wang. This slightly decreased pixel accuracy can be compensated by consistent labeling approaches without added cost for the application of automated surveillance systems along with changing configurations and a larger number of PTZ cameras.

We propose a novel scene image segmentation algorithm based on Perceptual Organization. We develop a Perceptual Organization model by quantitatively incorporating a list of Gestalt laws. The Perceptual Organization model can capture the non-accidental structural relations among the constituent parts of an object. The experimental results show that our proposed method outperformed two competing image segmentation approaches and achieved good segmentation quality on various natural scene environments.

In this effort, a data-driven and application independent technique to combine focal information from different focal planes is presented. Input images, acquired by imaging systems with limited depth of field, are decomposed using a relatively new analysis tool called curvelets. The extracted curvelets are representative of polar `wedges' from the frequency domain. Fusion is performed on medial and peripheral curvelets by relevant fusion rules and the fused image combines information from different focal planes, while extending the depth of field of the scene. The main contribution in this effort is the direct use of curvelets in combining multifocal images. Several illustrative examples and objective comparisons are provided.

Supershape model is a recent primitive that represents numerous 3D shapes with several symmetry axes. The main interest of this model is its capability to reconstruct more complex shape than superquadric model with only one implicit equation. In this paper we propose a genetic algorithms to reconstruct a point cloud using those primitives. We used the pseudo-Euclidean distance to introduce a threshold to handle real data imperfection and speed up the process. Simulations using our proposed fitness functions and a fitness function based on inside-outside function show that our fitness function based on the pseudo-Euclidean distance performs better.

In order to achieve improved recognition performance in comparison with conventional broadband images, this paper addresses a new method that automatically specifies the optimal spectral range for multispectral face images according to given illuminations. The novelty of our method lies in the introduction of a distribution separation measure and the selection of the optimal spectral range by ranking these separation values. The selected spectral ranges are consistent with the physics analysis of the multispectral imaging process. The fused images from these chosen spectral ranges are verified to outperform the conventional broadband images by 3%-20%, based on a variety of experiments with indoor and outdoor illuminations using two well-recognized face-recognition engines. Our discovery can be practically used for a new customized sensor design associated with given illuminations for improved face-recognition performance over the conventional broadband images.

Camera handoff is a crucial step to generate a continuously tracked and consistently labeled trajectory of the object of interest in multi-camera surveillance systems. Most existing camera handoff algorithms concentrate on data association, namely consistent labeling, where images of the same object are matched across different cameras. However, most real-time object tracking systems see a decrease in the system's frame rate as the number of tracked objects increases. To address this issue, we propose to incorporate an adaptive resource management mechanism into camera handoff. In so doing, cameras¿ resources can be dynamically allocated to multiple objects according to their priorities and hence the required minimum frame rate can be maintained. Experimental results illustrate that the proposed camera handoff algorithm is capable of maintaining a constant frame rate and of achieving a substantially improved handoff success rate by approximately 20% in comparison with the algorithm presented by Khan and Shah.

The estimation of the fundamental matrix is the key step in feature-based camera ego-motion estimation for applications in scene modeling and vehicle navigation. In this paper, we present a new method of analyzing and further reducing the risk in the fundamental matrix due to the choice of a particular feature detector, the choice of the matching algorithm, the motion model, iterative hypothesis generation and verification paradigms. Our scheme makes use of model-selection theory to guide the switch to optimal methods for fundamental matrix estimation within the hypothesis-and-test architecture. We demonstrate our proposed method for vision-based robot localization in large-scale environments where the environment is constantly changing and navigation within the environment is unpredictable.

Most existing camera placement algorithms focus on coverage and/or visibility analysis, which ensures that the object of interest is visible in the camera—s field of view (FOV). However, visibility, a fundamental requirement of object tracking, is insufficient for persistent and automated tracking. In such applications, a continuous and consistently labeled trajectory of the same object should be maintained across different cameras’ views. Therefore, a sufficient overlap between the cameras’ FOVs should be secured so that camera handoff can be executed successfully and automatically before the object of interest becomes untraceable or unidentifiable. The proposed sensor planning method improves existing algorithms by adding handoff rate analysis, which preserves necessary overlapped FOVs for an optimal handoff success rate. In addition, special considerations such as resolution and frontal view requirements are addressed using two approaches: direct constraint and adaptive weight. The resulting camera placement is compared with a reference algorithm by Erdem and Sclaroff. Significantly improved handoff success rate and frontal view percentage are illustrated via experiments using typical office floor plans.

Multispectral imaging and fusion of multispectral images in visible spectrum have been employed and proved to enhance the recognition performance over conventional broad-band images, especially when the illumination changes. In this paper, we further our study toward the recognition performance of a smaller number of spectral band images. Given the illumination, we propose an automated method that specifies the optimal spectral ranges to achieve an improved recognition performance compared with conventional broad-band images. Our discovery can be practically used for a new customized sensor design associated with given illuminations for an improved face recognition performance over conventional broad-band images.

In this paper, we propose a series of techniques to enhance the computational performance of existing Belief Propagation (BP) based stereo matching that relies on automatic estimation of the Markov random field (MRF) parameters. First, we show how convergence in matching can be achieved faster than with the existing message comparison technique by skipping comparisons in early inferences. Second, assuming that a stereo pair is captured with identical cameras, we apply a hypothesis called noise equivalence to pre-estimate the likelihood parameters and thus, avoid costly nested inferences to reduce the computational time. The likelihood parameters and intensity information are used for accelerated message propagation in image regions lacking gradients. Third, the prior model parameters are estimated with a combination of maximum likelihood (ML) estimation and disparity gradient constraint to further reduce the computational time. Supporting experiments for the proposed algorithms show encouraging results on ground truth test images.

In this paper, we derive a data mining framework to analyze 3D features on human faces. The framework leverages kernel density estimators, genetic algorithm and an information complexity criterion to identify discriminant feature-clusters of lower dimensionality. We apply this framework on human face anthropometry data of 32 features collected from each of the 300 3D face mesh models. The feature-subsets that we infer as the output establishes domain knowledge for the challenging problem of 3D face recognition with dense 3D gallery models and sparse or low resolution probes.

Here, a versatile data-driven application independent method to extend the depth of field is presented. The principal contribution in this effort is the use of features extracted by empirical mode decomposition, namely Intrinsic Mode Images, for fusion. The input images are decomposed into intrinsic mode images and fusion is performed on the extracted oscillatory modes, by means of weighing schemes that allow emphasis of focused regions in each input image. The fused image unifies information from all focal planes, while maintaining the verisimilitude of the scene. In order to validate the fusion performance of our method, we have compared our results with those of region-based and multiscale decomposition based fusion techniques. Several illustrative examples and objective comparisons are provided.

Belief propagation methods are the state-of-the-art with multisensor state localization problems. However, when localization applications have to deal with multimodality sensors whose functionality depends on the environment of operation, we understand the need for an inference framework to identify confident and reliable sensors. Such a framework helps eliminate failed/non-functional sensors from the fusion process minimizing uncertainty while propagating belief. We derive a framework inspired from model selection theory and demonstrate results on real world multisensor robot state localization and multicamera target tracking applications.

Most existing sensor planning algorithms find it difficult to tackle the discrepancy between a PTZ camera¿s limited instant field of view (FOV) and panoramic achievable FOV. In this paper, we introduce the probability of camera overload to resolve this discrepancy and present a sensor planning algorithm for PTZ cameras under the same framework as static cameras. The resulting camera placement achieves the optimal balance between coverage and handoff success rate. Furthermore, our algorithm is able to incorporate the target¿s dynamics into sensor planning. As a result, the system¿s handoff success rate can be maintained in environments with various target densities. Experimental results and comparisons with a reference algorithm proposed by Erdem and Sclaroff verify the effectiveness of our algorithm via a significantly improved handoff success rate.

In this paper we present a novel place recognition method. Instead of directly using large numbers of SIFT features as visual landmarks, we first use a jigsaw puzzle image segmentation algorithm to segment the input scene image into regions that may correspond to objects or parts of objects. Based on these image regions, we further detect a set of salient objects to represent a place and only those SIFT descriptors that were contained in these salient objects were kept in the database. We also designed a range-tree data structure to organize these salient objects to increase the matching efficiency. Experiments show that place recognition can be achieved accurately and efficiently with these salient objects.

Dual-camera systems have been widely used in surveillance because of the ability to explore the wide field of view (FOV) of the omnidirectional camera and the wide zoom range of the PTZ camera. Most existing algorithms require a priori knowledge of the omnidirectional camera's projection model to solve the nonlinear spatial correspondences between the two cameras. To overcome this limitation, two methods are proposed: 1) geometry and 2) homography calibration, where polynomials with automated model selection are used to approximate the camera's projection model and spatial mapping, respectively. The proposed methods not only improve the mapping accuracy by reducing its dependence on the knowledge of the projection model but also feature reduced computations and improved flexibility in adjusting to varying system configurations. Although the fusion of multiple cameras has attracted increasing attention, most existing algorithms assume comparable FOV and resolution levels among multiple cameras. Different FOV and resolution levels of the omnidirectional and PTZ cameras result in another critical issue in practical tracking applications. The omnidirectional camera is capable of multiple object tracking while the PTZ camera is able to track one individual target at one time to maintain the required resolution. It becomes necessary for the PTZ camera to distribute its observation time among multiple objects and visit them in sequence. Therefore, this paper addresses a novel scheme where an optimal visiting sequence of the PTZ camera is obtained so that in a given period of time the PTZ camera automatically visits multiple detected motions in a target-hopping manner. The effectiveness of the proposed algorithms is illustrated via extensive experiments using both synthetic and real tracking data and comparisons with two reference systems.

Multifocus fusion is the process of fusing focal information from a set of input images into one all-in-focus image. Here, a versatile multifocus fusion algorithm is presented for application-independent fusion. A focally connected region is a region or a set of regions in an input image that falls under the depth of field of the imaging system. Such regions are segmented adaptively under the predicate of focal connectivity and fused by partition synthesis. The fused image has information from all focal planes, while maintaining the visual verisimilitude of the scene. In order to validate the fusion performance of our method, we have compared our results with those of tiling and multiscale fusion techniques. In addition to performing a seamless fusion of the focally connected regions, our method out performs the competing methods regarding overall sharpness in all our experiments. Several illustrative examples of multifocus fusion are shown and objective comparisons are provided.

Multifocus fusion is the process of unifying focal information from a set of input images acquired with limited depth of field. In this effort, we present a general purpose multifocus fusion algorithm, which can be applied to varied applications ranging from microscopic to long range scenes. The main contribution in this paper is the segmentation of the input images into partitions based on focal connectivity. Focal connectivity is established by isolating regions in an input image that fall on the same focal plane. Our method uses focal connectivity and does not rely on physical properties like edges directly for segmentation. Our method establishes sharpness maps to the input images, which are used to isolate and attribute image partitions to input images. The partitions are mosaiced seamlessly to form the fused image. Illustrative examples of multifocus fusion using our method are shown. Comparisons against existing methods are made and the results are discussed.

We present an optimized full scale automatic camera calibration approach which is both accurate and simple to implement. The method can be applied to a wide range of cameras equipped with normal, wide-angle, fish-eye, and telephoto lenses. The procedure does not require prior knowledge of any parameters. The method uses a simple planar calibration pattern which is observed from different positions. Closed-form estimates for the intrinsic and extrinsic parameters are computed followed by nonlinear optimization. A polynomial function describes the lens projection instead of the commonly used radial model. Statistical information criteria are used to automatically determine the complexity of the lens distortion model. The proposed complete calibration method is shown to obtain lower reprojection error than some commonly used methods while maintaining a low complexity distortion model.

Interest point detectors are the starting point in image analysis for depth estimation using epipolar geometry and camera ego-motion estimation. With several detectors defined in the literature, some of them outperforming others in a specific application context, we introduce multi-feature sample consensus (MuFeSaC) as an adaptive and automatic procedure to choose a reliable feature detector among competing ones. Our approach is derived based on model selection criteria that we demonstrate for mobile robot self-localization in outdoor environments consisting of both man-made structures and natural vegetation.

Our sensor selection algorithm targets the problem of global self-localization of multi-sensor mobile robots. The algorithm builds on the probabilistic reasoning using Bayes filters to estimate sensor measurement uncertainty and sensor validity in robot localization. For quantifying measurement uncertainty we score the Bayesian belief probability density using a model selection criterion, and for sensor validity, we evaluate belief on pose estimates from different sensors as a multi-sample clustering problem. The minimization of the combined uncertainty (measurement uncertainly score + sensor validity score) allows us to intelligently choose a subset of sensors that contribute to accurate localization of the mobile robot. We demonstrate the capability of our sensor selection algorithm in automatically switching pose recovery methods and ignoring non-functional sensors for localization on real-world mobile platforms equipped with laser scanners, vision cameras, and other hardware instrumentation for pose estimation.

Dual camera systems (omnidirectional and slaved PTZ cameras) are widely used in public area monitoring and target tracking. However, due to their low and non-uniform resolution, omnidirectional cameras are only able to provide moderate accuracy in both motion/target detection and tracking. To overcome this disadvantage, two methods are proposed: geometry mapping based on a polynomial imaging model and closed loop cooperative tracking. Using a unified polynomial approximation [1], the geometry relationship between the omnidirectional and PTZ cameras can be formulated in a general solution from camera calibration with a fully automated model selection. Distributed Kalman filters are developed to exchange estimated trajectories among cameras forming closed loop sub-systems for individual cameras. The effectiveness of the proposed algorithms is illustrated via experiments and comparisons with existing systems.

Contrast enhancement has an important role in image processing applications. Conventional contrast enhancement techniques either fail to produce satisfactory results for a broad variety of low-contrast images, or cannot be automatically applied to different images, because their parameters must be specified manually to produce a satisfactory result for a given image. This paper describes a new automatic method for contrast enhancement. The basic procedure is to first group the histogram components of a low-contrast image into the proper number of bins according to a selected criterion, then redistribute these bins uniformly over the grayscale, and finally ungroup the previously grouped gray-levels. Accordingly, this new technique is named gray-level grouping (GLG). GLG not only produces results superior to conventional contrast enhancement techniques, but is also fully automatic in most circumstances, and is applicable to a broad variety of images

We propose a simple nonparametric linear regression tool, known as kernel regression (KR), to estimate the illumination chromaticity. We design a Gaussian kernel whose bandwidth is selected empirically. Previously, nonlinear techniques like neural networks (NN) and support vector machines (SVM) are applied to estimate the illumination chromaticity. However, neither of the techniques was compared with linear regression tools. We show that the proposed method performs better chromaticity estimation compared to NN, SVM, and linear ridge regression (RR) approach on the same data set.

In this paper, we apply supershapes and R-functions to surface recovery from 3D data sets. Individual supershapes are separately recovered from a segmented mesh. R-functions are used to perform Boolean operations between the reconstructed parts to obtain a single implicit equation of the reconstructed object that is used to define a global error reconstruction function. We present surface recovery results ranging from single synthetic data to real complex objects involving the composition of several supershapes and holes.

In this effort, we propose a new image fusion technique, utilizing empirical mode decomposition (EMD), for improved face recognition. EMD is a non-parametric data-driven analysis tool that decomposes non-linear non-stationary signals into intrinsic mode functions (IMFs). In this method, we decompose images from different imaging modalities into their IMFs. Fusion is performed at the decomposition level and the fused IMFs are reconstructed to form the fused image. The effect of fusion on face recognition is measured by obtaining the cumulative match characteristics (CMCs) between galleries and probes. Apart from conducting face recognition tests on visible and infrared raw datasets, we use datasets fused by averaging, principal component (PCA) fusion, wavelet based fusion and our method, for comparison. The face recognition rate due to EMD fused images is higher than the face recognition rates due to raw visible, raw infrared and other fused images. Examples of the fused images and illustrative CMC comparison charts are shown.

In size preserving video tracking, the camera's focal length (zoom) is adjusted automatically to compensate for the changes in the target's image size caused by the relative motion between the camera and the target. The accurate estimation of these changes is paramount to the system performance. Structure from motion (SFM) based on the weak perspective projection model has been applied to real time target scale estimation (B. Tordoff, et al., 2004). In this paper we design target scale estimation algorithms with linear solutions based on more advanced projection models: paraperspective and perspective. The performances of the algorithms using three projection models are examined and compared. Experimental results show that the proposed algorithm based on the paraperspective projection model produces the best performance with respect to accuracy, complexity, and robustness.

This paper presents a tactical path planning algorithm for following ridges or valleys across a 3D terrain. The intent is to generate a path that enables an unmanned vehicle to surveil with maximum observability by traversing the ridges of a terrain or to operate with maximum covertness by navigating the valleys. The input to the algorithm is a 3D triangle mesh model for the terrain of interest. This mesh may be non-uniform and non-regular. Thus, the algorithm leverages research from computer graphics and computer vision to identify ridge-valley features on the terrain. These features serve as "obstacles" for an artificial potential field algorithm. The valleys are obstacles for a surveillance path, or the ridges are obstacles for a covert path. We incorporate geodesic-rather than Euclidean-distances into the potential field formulation to extend path planning to 3D surfaces. We present the theory of our proposed algorithm and provide experimental results

Applications of digital imaging with extreme zoom are traditionally found in astronomy and wild life monitoring. More recently, the need for such capabilities has extended to wide area monitoring such as forest fires, harbors, and waterways. This paper presents a number of sensor arrangements for the acquisition of high magnification images. It also describes user interfaces developed for the system’s remote control and data acquisition and processing. Hardware and software considerations are addressed, focusing on optical setups and image de-blurring techniques. An image restoration algorithm tailored to such systems is developed, where an adaptive sharpness measure is employed as a cost function to guide the search for an optimal point spread function (PSF) for image de-blur. Experimental results demonstrate a considerably enhanced robustness to noise and ability to select the optimum PSF, producing much superior restored images.

Contrast enhancement has an important role in image processing applications. Conventional contrast enhancement techniques either often fail to produce satisfactory results for a broad variety of low-contrast images, or cannot be automatically applied to different images, because their parameters must be specified manually to produce a satisfactory result for a given image. This paper describes a new automatic method for contrast enhancement. The basic procedure is to first group the histogram components of a low-contrast image into a proper number of bins according to a selected criterion, then redistribute these bins uniformly over the grayscale, and finally ungroup the previously grouped gray-levels. Accordingly, this new technique is named gray-level grouping (GLG). GLG not only produces results superior to conventional contrast enhancement techniques, but is also fully automatic in most circumstances, and is applicable to a broad variety of images. An extension of GLG, selective GLG (SGLG), and its variations will be discussed in Part II of this paper. SGLG selectively groups and ungroups histogram components to achieve specific application purposes, such as eliminating background noise, enhancing a specific segment of the histogram, and so on. The extension of GLG to color images will also be discussed in Part II.

This is Part II of the paper, "Gray-Level Grouping (GLG): an Automatic Method for Optimized Image Contrast Enhancement". Part I of this paper introduced a new automatic contrast enhancement technique: gray-level grouping (GLG). GLG is a general and powerful technique, which can be conveniently applied to a broad variety of low-contrast images and outperforms conventional contrast enhancement techniques. However, the basic GLG method still has limitations and cannot enhance certain classes of low-contrast images well, e.g., images with a noisy background. The basic GLG also cannot fulfill certain special application purposes, e.g., enhancing only part of an image which corresponds to a certain segment of the image histogram. In order to break through these limitations, this paper introduces an extension of the basic GLG algorithm, selective gray-level grouping (SGLG), which groups the histogram components in different segments of the grayscale using different criteria and, hence, is able to enhance different parts of the histogram to various extents. This paper also introduces two new preprocessing methods to eliminate background noise in noisy low-contrast images so that such images can be properly enhanced by the (S)GLG technique. The extension of (S)GLG to color images is also discussed in this paper. SGLG and its variations extend the capability of the basic GLG to a larger variety of low-contrast images, and can fulfill special application requirements. SGLG and its variations not only produce results superior to conventional contrast enhancement techniques, but are also fully automatic under most circumstances, and are applicable to a broad variety of images.

This paper describes a series of linear and nonlinear pseudocoloring maps designed and applied to single energy X-ray luggage scans to assist airport screeners in identifying and detecting threat items, particularly hard to see low-density weapons in luggage. Considerations of the psychological and physiological processing involved in the human perception of color as well as the effects of using various color models, such as the RGB and HSI models, were explored. Original grayscale data, various enhanced images, and segmented scenes were used as input to the various color mapping schemes designed in this research. A highly interactive user-friendly graphical interface and a portable test were developed and used in a performance evaluation study involving a population of actual federal airport screeners. The study proved the advantages of using color over gray level data and also allowed the ranking of color maps and selection of the best performing color schemes. Rate improvements in weapon detection of up to 97% were achieved through the use of color

In this paper we introduce a new 3D automatic registration method based on Gaussian fields and energy minimization. The method defines a simple C/sup /spl infin// energy function, which is convex in a large neighborhood of the alignment parameters; allowing for the use of powerful standard optimization techniques. We show that the size of the region of convergence can be significantly extended reducing the need for close initialization and overcoming local convergence problems of the standard iterative closest point (ICP) algorithms. Furthermore, the Gaussian criterion can be evaluated with linear computational complexity using fast Gauss transform methods, allowing for an efficient implementation of the registration algorithm. Experimental analysis of the technique using real world datasets shows the usefulness as well as the limits of the approach.

We present a new multiresolution analysis framework for irregular meshes with attributes based on the lifting scheme. We introduce a surface prediction operator to compute the detail coefficients for the geometry and the attributes of the model. Attribute analysis gives appearance information to complete the geometrical analysis of the model. A set of experimental results are given to show the efficiency of our framework. We present two applications to adaptive visualization and denoising.

We present a new and efficient algorithm to accurately polygonize an implicit surface generated by multiple Boolean operations with globally deformed primitives. Our algorithm is special in the sense that it can be applied to objects with both an implicit and a parametric representation, such as superquadrics, supershapes, and Dupin cyclides. The input is a Constructive Solid Geometry tree (CSG tree) that contains the Boolean operations, the parameters of the primitives, and the global deformations. At each node of the CSG tree, the implicit formulations of the subtrees are used to quickly determine the parts to be transmitted to the parent node, while the primitives' parametric definition are used to refine an intermediary mesh around the intersection curves. The output is both an implicit equation and a mesh representing its solution. For the resulting object, an implicit equation with guaranteed differential properties is obtained by simple combinations of the primitives' implicit equations using R-functions. Depending on the chosen R-function, this equation is continuous and can be differentiable everywhere. The primitives' parametric representations are used to directly polygonize the resulting surface by generating vertices that belong exactly to the zero-set of the resulting implicit equation. The proposed approach has many potential applications, ranging from mechanical engineering to shape recognition and data compression. Examples of complex objects are presented and commented on to show the potential of our approach for shape modeling.

This paper proposes a fully digital auto-focusing algorithm for restoring the image with differently out-of-focused objects, which can restore background as well as all objects. In this paper, we assume that out-of-focus blur is isotropic such as circle of confusion (COC) or two-dimensional Gaussian blur. Therefore, the proposed algorithm can segment and estimate the point spread function (PSF) by using the size of ramp in the one-dimensional step response. The proposed algorithm can be developed by object-based image segmentation and restoration algorithm. Experimental results show that the proposed object-based image restoration algorithm can efficiently remove the space-variant out of focus blur from the image with multiple blurred objects.

We present a combined real-time face region tracking and highly accurate face recognition technique for an intelligent surveillance system. High-resolution face images are very important to achieving accurate identification of a human face. Conventional surveillance or security systems, however, usually provide poor image quality because they use only fixed cameras to record scenes passively. We have implemented a real-time surveillance system that tracks a moving face using four pan-tilt-zoom (PTZ) cameras. While tracking, the region-of-interest (ROI) can be obtained by using a low-pass filter and background subtraction with the PTZ. Color information in the ROI is updated to extract features for optimal tracking and zooming. FaceIt/sup /spl reg//, which is one of the most popular face recognition software packages, is evaluated and then used to recognize the faces from the video signal. Experimentation with real human faces showed highly acceptable results in the sense of both accuracy and computational efficiency.

In this paper, we describe an algorithm called fast marching watersheds that segments a triangle mesh into visual parts. This computer vision algorithm leverages a human vision theory known as the minima rule. Our implementation computes the principal curvatures and principal directions at each vertex of a mesh, and then our hill-climbing watershed algorithm identifies regions bounded by contours of negative curvature minima. These regions fit the definition of visual parts according to the minima rule. We present evaluation analysis and experimental results for the proposed algorithm.

In this paper, we describe an algorithm to measure the shape complexity for discrete approximations of planar curves in 2D images and manifold surfaces for 3D triangle meshes. We base our algorithm on shape curvature, and thus we compute shape information as the entropy of curvature. We present definitions to estimate curvature for both discrete curves and surfaces and then formulate our theory of shape information from these definitions. We demonstrate our algorithm with experimental results.

This paper proposes a new, efficient surface representation method for surface matching. A feature carrier for a surface point, which is a set of two-dimensional (2-D) contours that are the projections of geodesic circles on the tangent plane, is generated. The carrier is named point fingerprint because its pattern is similar to human fingerprints and plays a role in discriminating surface points. Corresponding points on surfaces from different views are found by comparing their fingerprints. The point fingerprint is able to carry curvature, color, and other information which can improve matching accuracy, and the matching process is faster than 2-D image comparison. A novel candidate point selection method based on the fingerprint irregularity is introduced. Point fingerprint is successfully applied to pose estimation of real range data.

Tracking and recognizing non-rigid objects in video image sequences are complex tasks of increasing importance to many applications. In this paper, we present a hierarchical realization of an enhanced active shape model for color video tracking and we study the performance of hierarchical and non-hierarchical implementations in the RGB, YUV, and HSI color spaces.

We propose a novel algorithm to smooth and simplify simultaneously range images and also triangle meshes derived from those images. These data sets often suffer from noise and over-sampling. To overcome these issues, smoothing from image processing and simplification from computer graphics attempt to minimize noise and reduce complexity, respectively. Typically, these algorithms are separate and distinct steps, but we combine them into one algorithm. We employ surface normal voting to generate robust orientation estimates and then extend the quadric error metric framework to smooth noise while simplifying the surface. We demonstrate the capabilities of this algorithm with both synthetic and real data. The proposed algorithm provides significant noise smoothing improvement when compared to the standard Garland and Heckbert (1998) quadric simplification algorithm.

Triangle meshes are widely used in representing surfaces in computer vision and computer graphics. Although 2D image processing-based edge detection techniques have been popular in many application areas, they are not well developed for surfaces represented by triangle meshes. This paper proposes a robust edge detection algorithm for triangle meshes and its applications to surface segmentation and adaptive surface smoothing. The proposed edge detection technique is based on eigen analysis of the surface normal vector field in a geodesic window. To compute the edge strength of a certain vertex, the neighboring vertices in a specified geodesic distance are involved. Edge information are used further to segment the surfaces with the watershed algorithm and to achieve edge-preserved, adaptive surface smoothing. The proposed algorithm is novel in robustly detecting edges on triangle meshes against noise. The 3D watershed algorithm is an extension from previous work. Experimental results on surfaces reconstructed from multi-view real range images are presented.

This paper presents a simple and efficient algorithm for part decomposition of compound objects based on Gaussian curvature analysis. The proposed algorithm consists of three major steps, Gaussian curvature estimation, boundary detection, and region growing. Boundaries between two articulated parts are composed of points with highly negative curvature based on the transversality regularity. These boundaries are therefore detected by thresholding estimated Gaussian curvatures for each vertex. A component labeling operation is then performed to grow non-boundary vertices into parts. The original contributions of this paper include: (i) novel, curvature analysis-based decomposition of 3-D models represented by triangle meshes into functional parts instead of surfaces and (ii) large mesh (over 100,000 triangles) handling capability with low computational cost and easy implementation. Experiments were conducted on a large number of both synthetic and real 3-D models. Experimental results demonstrated the performance and efficiency of the proposed algorithm.

Effective image segmentation of a digitized scene into a set of recognizable objects requires the development of sophisticated scene analysis algorithms. Progress in this area has been made through the development of a statistical-based deformable model that improves upon existing point distribution models (PDMs) for boundary-based object segmentation. Existing PDM boundary finding techniques often suffer from the shortcoming that global shape and gray-level information are treated independently during boundary optimization. A deformable model algorithm is under development in which the objective function used during optimization of the boundary encompasses several important characteristics. Most importantly the objective function includes both shape and gray-level characteristics, so optimization occurs with respect to both pieces of information simultaneously. This algorithm has been applied to geometric test images and a simple industrial-type scene for which results are presented.

This paper describes the application of a new probabilistic shape and appearance model (PSAM) algorithm to the task of detecting polycystic kidney disease (PKD) in X-ray computed tomography images of laboratory mice. The genetically engineered PKD mouse is a valuable animal model that can be used to develop new treatments for kidney-related problems in humans. PSAM is a statistical-based deformable model that improves upon existing point distribution models for boundary-based object segmentation. This new deformable model algorithm finds the optimal boundary position using an objective function that has several unique characteristics. Most importantly, the objective function includes both global shape and local gray-level characteristics, so optimization occurs with respect to both pieces of information simultaneously. PSAM is employed to segment the mouse kidneys and then texture measurements are applied within kidney boundaries to detect PKD. The challenges associated with the segmentation non-rigid organs along with the availability of a priori information led to the choice of a trainable, deformable model for this application. In 103 kidney images that were analyzed as part of a preclinical animal study, the mouse kidneys and spine were segmented with an average error of 2.4 pixels per boundary point. In all 103 cases, the kidneys were successfully segmented at a level where PKD could be detected using mean-of-local-variance texture measurements within the located boundary.

In this paper, we describe a robust method for the estimation of curvature on a triangle mesh, where this mesh is a discrete approximation of a piecewise smooth surface. The proposed method avoids the computationally expensive process of surface fitting and instead employs normal voting to achieve robust results. This method detects crease discontinuities on the surface to improve estimates near those creases. Using a voting scheme, the algorithm estimates both principal curvatures and principal directions for smooth patches. The entire process requires one user parameter-the voting neighborhood size, which is a function of sampling density, feature size, and measurement noise. We present results for both synthetic and real data and compare these results to an existing algorithm developed by Taubin (1995).

This paper proposes a new efficient surface representation method for the application of surface matching. We generate a feature carrier for the surface point, which is a set of 2D contours that are the projection of geodesic circles onto the tangent plane. The carrier is named point's fingerprint because its pattern is similar to human fingerprint and discriminating for each point. Each point's fingerprint carries the information of the normal variation along geodesic circles. Corresponding points on surfaces from different views are found by comparing fingerprints of the points. This representation scheme includes more local geometry information than some previous works that only use one contour as the feature carrier. It is not histogram based so that it is able to carry more features to improve comparison accuracy. To speed up the matching, we use a novel candidate point selection method based on the shape irregularity of the projected local geodesic circle. The point's fingerprint is successfully used to register both synthetic and real 2 1/2 data.

A new method is presented for the localization and recognition of three-dimensional objects using color information. In the first processing step, we estimate depth information by either applying a chromatic block matching method to color stereo images or acquiring a range image from a laser scanner. Second, the computed depth maps are segmented to distinguish between the image background and the objects that should be recognized. Assuming that the segmented regions represent single objects in the three-dimensional scene, feature vectors are generated based on color histograms. The Euclidean distance is used as well as the scalar product to measure the similarity between the feature vectors computed from the color image and the feature vectors stored in a database.

We propose an adaptive regularization algorithm for smoothing dense range images using a novel, first order stabilizing function. The stabilizer we suggest is based upon minimizing the reconstructed surface area and is derived in the native, spherical coordinate system of the range scanner. This allows adjustments to be made along only the direction of measurement, thereby preventing the data overlapping problem that can arise in dense images. Adaptation is achieved by adjusting the regularization parameter according to the results of 2D edge analysis. Results indicate effective noise suppression along with well preserved edges and details in the reconstructed, 3D surfaces.

The focus of this paper is to design and implement a system capable of automatically reconstructing a prototype 3D model from a minimum number of range images of an object. Given an ideal 3D object model, the system iteratively renders range and intensity images of the model from a specified position, assimilates the range information into a prototype model, and determines the sensor pose (position and orientation) from which an optimal amount of previously unrecorded information may be acquired. Reconstruction is terminated when the model meets a given threshold of accuracy. Such a system has applications in the context of robot navigation, manufacturing, or hazardous materials handling. The system has been tested successfully on several synthetic data models, and each set of results was found to be reasonably consistent with an intuitive human search. The number of views necessary to reconstruct an adequate 3D prototype depends on the complexity of the object or scene and the initial data collected. The prototype models which the system recovers compare well with the ideal models.

Wide-Angle lenses are not often used for 3D reconstruction tasks, in spite of the potential advantages offered by their increased field-of-view, because (1) existing algorithms for high-distortion lens compensation perform poorly at image extremities and (2) procedures for the reconstruction of recti-linear images place a large burden on system resources. In this paper, a projection model based on quadric surfaces is presented which accurately characterizes the effect of wide-angle lenses across the entire image and allows for the use of novel feature matching strategies that do not require nonlinear distortion compensation.

This paper presents a novel fast superresolution algorithm using motion adaptive relaxation parameters. By adopting the fast iterative algorithm based on pre-conditioner, the proposed algorithm can rapidly restore high resolution details in the original high resolution image. Moreover, the proposed algorithm provides a better interpolated images by using the motion adaptive relaxation parameter which controls the convergence rate. Finally, it can be applied to the general image sequence with differently moving objects. As a result, we can obtain high resolution frames from image sequences in almost real-time.

Pose estimation is an important operation for many vision tasks. In this paper, the authors propose an algorithm for pose estimation based on the volume measurement of tetrahedra composed of feature-point triplets extracted from an arbitrary quadrangular target and the lens center of the vision system. The inputs to this algorithm are the six distances joining all feature pairs and the image coordinates of the quadrangular target. The outputs of this algorithm are the effective focal length of the vision system, the interior orientation parameters of the target, the exterior orientation parameters of the camera with respect to an arbitrary coordinate system if the target coordinates are known in this frame, and the final pose of the camera. The authors have also developed a shape restoration technique which is applied prior to pose recovery in order to reduce the effects of inaccuracies caused by image projection. An evaluation of the method has shown that this pose estimation technique is accurate and robust. Because it is based on a unique and closed form solution, its speed makes it a potential candidate for solving a variety of landmark-based tracking problems.>

An uncertainty and data fusion approach was developed and tested. This fusion algorithm is based on the interaction between two constraints: (1) the principle of knowledge source corroboration, which tends to maximize the final belief in a given proposition (often modeled by a probability density function or fuzzy membership distribution), if either of the knowledge sources supports the occurrence of this proposition, and (2) the principle of belief enhancement/withdrawal, which adjusts the belief of one knowledge source according to the belief of the second knowledge source by maximizing the similarity between the two source outputs. This method has been tested using various features from synthetic and real data of various types of many dimensionalities, resulting fused data which satisfy both principles.>

Parallel range segmentation tools are analyzed in terms of a general model for distributed-memory algorithms using n-cube topologies. The model explicitly relates the concepts of speedup, efficiency and load balancing and provides criteria for optimum assignment of nodes. Low-level computer vision functions are classified as either local or global. Experimental results are given showing that the efficiency of local operators is determined by load balancing, while the efficiency of global operators is more dependent on communication overhead.>

An algorithm is proposed for pose estimation based on the volume measurement of tetrahedra composed of feature-point triplets extracted from an arbitrary quadrangular target and the lens center of the vision system. Using a pinhole model (lens distortion is taken into account separately) and a quadrangular target, for which only the six distance measurements between all pairs of feature points are known, the complete pose is determined using an all-geometric closed-form solution for the six parameters of the pose (three translation components and three rotation components). This method has been tested using synthetic and real data and shown to be efficient, accurate, and robust. Its speed, in particular, makes it a potential candidate for real-time robotic tasks.>

The feasibility of realistic autonomous space manipulation tasks using multisensory information is shown through two experiments involving a fluid interchange system and a module interchange system. In both cases, autonomous location of the mating element, autonomous location of the guiding light target, mating, and demating of the system are performed. Specifically, vision-driven techniques were implemented that determine the arbitrary two-dimensional position and orientation of the mating elements as well as the arbitrary three-dimensional position and orientation of the light targets. The robotic system is also equipped with a force/torque sensor that continuously monitors the six components of force and torque exerted on the end effector. Using vision, force, torque, proximity, and touch sensors, the fluid interchange system and the module interchange system experiments were accomplished autonomously and successfully.>

A general approach is presented for the integration of vision, range, proximity, and touch sensory data to derive a better estimate of the position and orientation (pose) of an object appearing in the work space. Efficient and robust methods for analyzing vision and range data to derive an interpretation of input images are discussed. Vision information analysis includes a model-based object recognition module and an image-to-world coordinate transformation module to identify the three-dimensional (3-D) coordinates of the recognized objects. The range information processing includes modules for reprocessing, segmentation, and 3-D primitive extraction. The multisensory information integration approach represents sensory information in a sensor-independent form and formulates an optimization problem to find a minimum-error solution to the problem. The capabilities of a multisensor robotic system are demonstrated by performing a number of experiments using an industrial robot equipped with several sensors of differing types.>

The authors describe research efforts directed towards the development of sensor-driven robotic systems for performing various inspection and manipulation tasks. Emphasis is on modules designed for analyzing vision and range sensor data for object recognition and pose estimation. The object recognition system uses a model-based approach. Pose estimation by vision sensor includes derivation of an image-to-world coordinate transformation module. Range data analysis includes modules for preprocessing, segmentation, and 3-D primitive extraction. Experiments are presented to demonstrate the applicability of the analysis modules using a laboratory-based robotic testbed.>