In this paper, we propose a novel outdoor scene image segmentation algorithm based on background recognition and perceptual organization. We recognize the background objects such as the sky, the ground, and vegetation based on the color and texture information. For the structurally challenging objects, which usually consist of multiple constituent parts, we developed a perceptual organization model that can capture the nonaccidental structural relationships among the constituent parts of the structured objects and, hence, group them together accordingly without depending on a priori knowledge of the specific objects. Our experimental results show that our proposed method outperformed two state-of-the-art image segmentation approaches on two challenging outdoor databases (Gould data set and Berkeley segmentation data set) and achieved accurate segmentation quality on various outdoor natural scene environments.

Most existing camera placement algorithms focus on coverage and/or visibility analysis, which ensures that the object of interest is visible in the camera's field of view (FOV). However, visibility, which is a fundamental requirement of object tracking, is insufficient for automated persistent surveillance. In such applications, a continuous consistently labeled trajectory of the same object should be maintained across different camera views. Therefore, a sufficient uniform overlap between the cameras' FOVs should be secured so that camera handoff can successfully and automatically be executed before the object of interest becomes untraceable or unidentifiable. In this paper, we propose sensor-planning methods that improve existing algorithms by adding handoff rate analysis. Observation measures are designed for various types of cameras so that the proposed sensor-planning algorithm is general and applicable to scenarios with different types of cameras. The proposed sensor-planning algorithm preserves necessary uniform overlapped FOVs between adjacent cameras for an optimal balance between coverage and handoff success rate. In addition, special considerations such as resolution and frontal-view requirements are addressed using two approaches: 1) direct constraint and 2) adaptive weights. The resulting camera placement is compared with a reference algorithm published by Erdem and Sclaroff. Significantly improved handoff success rates and frontal-view percentages are illustrated via experiments using indoor and outdoor floor plans of various scales.

We propose a novel scene image segmentation algorithm based on Perceptual Organization. We develop a Perceptual Organization model by quantitatively incorporating a list of Gestalt laws. The Perceptual Organization model can capture the non-accidental structural relations among the constituent parts of an object. The experimental results show that our proposed method outperformed two competing image segmentation approaches and achieved good segmentation quality on various natural scene environments.

Camera handoff is a crucial step to generate a continuously tracked and consistently labeled trajectory of the object of interest in multi-camera surveillance systems. Most existing camera handoff algorithms concentrate on data association, namely consistent labeling, where images of the same object are matched across different cameras. However, most real-time object tracking systems see a decrease in the system's frame rate as the number of tracked objects increases. To address this issue, we propose to incorporate an adaptive resource management mechanism into camera handoff. In so doing, cameras¿ resources can be dynamically allocated to multiple objects according to their priorities and hence the required minimum frame rate can be maintained. Experimental results illustrate that the proposed camera handoff algorithm is capable of maintaining a constant frame rate and of achieving a substantially improved handoff success rate by approximately 20% in comparison with the algorithm presented by Khan and Shah.

The estimation of the fundamental matrix is the key step in feature-based camera ego-motion estimation for applications in scene modeling and vehicle navigation. In this paper, we present a new method of analyzing and further reducing the risk in the fundamental matrix due to the choice of a particular feature detector, the choice of the matching algorithm, the motion model, iterative hypothesis generation and verification paradigms. Our scheme makes use of model-selection theory to guide the switch to optimal methods for fundamental matrix estimation within the hypothesis-and-test architecture. We demonstrate our proposed method for vision-based robot localization in large-scale environments where the environment is constantly changing and navigation within the environment is unpredictable.

Most existing camera placement algorithms focus on coverage and/or visibility analysis, which ensures that the object of interest is visible in the camera—s field of view (FOV). However, visibility, a fundamental requirement of object tracking, is insufficient for persistent and automated tracking. In such applications, a continuous and consistently labeled trajectory of the same object should be maintained across different cameras’ views. Therefore, a sufficient overlap between the cameras’ FOVs should be secured so that camera handoff can be executed successfully and automatically before the object of interest becomes untraceable or unidentifiable. The proposed sensor planning method improves existing algorithms by adding handoff rate analysis, which preserves necessary overlapped FOVs for an optimal handoff success rate. In addition, special considerations such as resolution and frontal view requirements are addressed using two approaches: direct constraint and adaptive weight. The resulting camera placement is compared with a reference algorithm by Erdem and Sclaroff. Significantly improved handoff success rate and frontal view percentage are illustrated via experiments using typical office floor plans.

In this paper, we derive a data mining framework to analyze 3D features on human faces. The framework leverages kernel density estimators, genetic algorithm and an information complexity criterion to identify discriminant feature-clusters of lower dimensionality. We apply this framework on human face anthropometry data of 32 features collected from each of the 300 3D face mesh models. The feature-subsets that we infer as the output establishes domain knowledge for the challenging problem of 3D face recognition with dense 3D gallery models and sparse or low resolution probes.

Belief propagation methods are the state-of-the-art with multisensor state localization problems. However, when localization applications have to deal with multimodality sensors whose functionality depends on the environment of operation, we understand the need for an inference framework to identify confident and reliable sensors. Such a framework helps eliminate failed/non-functional sensors from the fusion process minimizing uncertainty while propagating belief. We derive a framework inspired from model selection theory and demonstrate results on real world multisensor robot state localization and multicamera target tracking applications.

Most existing sensor planning algorithms find it difficult to tackle the discrepancy between a PTZ camera¿s limited instant field of view (FOV) and panoramic achievable FOV. In this paper, we introduce the probability of camera overload to resolve this discrepancy and present a sensor planning algorithm for PTZ cameras under the same framework as static cameras. The resulting camera placement achieves the optimal balance between coverage and handoff success rate. Furthermore, our algorithm is able to incorporate the target¿s dynamics into sensor planning. As a result, the system¿s handoff success rate can be maintained in environments with various target densities. Experimental results and comparisons with a reference algorithm proposed by Erdem and Sclaroff verify the effectiveness of our algorithm via a significantly improved handoff success rate.

In this paper we present a novel place recognition method. Instead of directly using large numbers of SIFT features as visual landmarks, we first use a jigsaw puzzle image segmentation algorithm to segment the input scene image into regions that may correspond to objects or parts of objects. Based on these image regions, we further detect a set of salient objects to represent a place and only those SIFT descriptors that were contained in these salient objects were kept in the database. We also designed a range-tree data structure to organize these salient objects to increase the matching efficiency. Experiments show that place recognition can be achieved accurately and efficiently with these salient objects.

Dual-camera systems have been widely used in surveillance because of the ability to explore the wide field of view (FOV) of the omnidirectional camera and the wide zoom range of the PTZ camera. Most existing algorithms require a priori knowledge of the omnidirectional camera's projection model to solve the nonlinear spatial correspondences between the two cameras. To overcome this limitation, two methods are proposed: 1) geometry and 2) homography calibration, where polynomials with automated model selection are used to approximate the camera's projection model and spatial mapping, respectively. The proposed methods not only improve the mapping accuracy by reducing its dependence on the knowledge of the projection model but also feature reduced computations and improved flexibility in adjusting to varying system configurations. Although the fusion of multiple cameras has attracted increasing attention, most existing algorithms assume comparable FOV and resolution levels among multiple cameras. Different FOV and resolution levels of the omnidirectional and PTZ cameras result in another critical issue in practical tracking applications. The omnidirectional camera is capable of multiple object tracking while the PTZ camera is able to track one individual target at one time to maintain the required resolution. It becomes necessary for the PTZ camera to distribute its observation time among multiple objects and visit them in sequence. Therefore, this paper addresses a novel scheme where an optimal visiting sequence of the PTZ camera is obtained so that in a given period of time the PTZ camera automatically visits multiple detected motions in a target-hopping manner. The effectiveness of the proposed algorithms is illustrated via extensive experiments using both synthetic and real tracking data and comparisons with two reference systems.

Interest point detectors are the starting point in image analysis for depth estimation using epipolar geometry and camera ego-motion estimation. With several detectors defined in the literature, some of them outperforming others in a specific application context, we introduce multi-feature sample consensus (MuFeSaC) as an adaptive and automatic procedure to choose a reliable feature detector among competing ones. Our approach is derived based on model selection criteria that we demonstrate for mobile robot self-localization in outdoor environments consisting of both man-made structures and natural vegetation.

Our sensor selection algorithm targets the problem of global self-localization of multi-sensor mobile robots. The algorithm builds on the probabilistic reasoning using Bayes filters to estimate sensor measurement uncertainty and sensor validity in robot localization. For quantifying measurement uncertainty we score the Bayesian belief probability density using a model selection criterion, and for sensor validity, we evaluate belief on pose estimates from different sensors as a multi-sample clustering problem. The minimization of the combined uncertainty (measurement uncertainly score + sensor validity score) allows us to intelligently choose a subset of sensors that contribute to accurate localization of the mobile robot. We demonstrate the capability of our sensor selection algorithm in automatically switching pose recovery methods and ignoring non-functional sensors for localization on real-world mobile platforms equipped with laser scanners, vision cameras, and other hardware instrumentation for pose estimation.

Contrast enhancement has an important role in image processing applications. Conventional contrast enhancement techniques either fail to produce satisfactory results for a broad variety of low-contrast images, or cannot be automatically applied to different images, because their parameters must be specified manually to produce a satisfactory result for a given image. This paper describes a new automatic method for contrast enhancement. The basic procedure is to first group the histogram components of a low-contrast image into the proper number of bins according to a selected criterion, then redistribute these bins uniformly over the grayscale, and finally ungroup the previously grouped gray-levels. Accordingly, this new technique is named gray-level grouping (GLG). GLG not only produces results superior to conventional contrast enhancement techniques, but is also fully automatic in most circumstances, and is applicable to a broad variety of images

This paper presents a tactical path planning algorithm for following ridges or valleys across a 3D terrain. The intent is to generate a path that enables an unmanned vehicle to surveil with maximum observability by traversing the ridges of a terrain or to operate with maximum covertness by navigating the valleys. The input to the algorithm is a 3D triangle mesh model for the terrain of interest. This mesh may be non-uniform and non-regular. Thus, the algorithm leverages research from computer graphics and computer vision to identify ridge-valley features on the terrain. These features serve as "obstacles" for an artificial potential field algorithm. The valleys are obstacles for a surveillance path, or the ridges are obstacles for a covert path. We incorporate geodesic-rather than Euclidean-distances into the potential field formulation to extend path planning to 3D surfaces. We present the theory of our proposed algorithm and provide experimental results

Contrast enhancement has an important role in image processing applications. Conventional contrast enhancement techniques either often fail to produce satisfactory results for a broad variety of low-contrast images, or cannot be automatically applied to different images, because their parameters must be specified manually to produce a satisfactory result for a given image. This paper describes a new automatic method for contrast enhancement. The basic procedure is to first group the histogram components of a low-contrast image into a proper number of bins according to a selected criterion, then redistribute these bins uniformly over the grayscale, and finally ungroup the previously grouped gray-levels. Accordingly, this new technique is named gray-level grouping (GLG). GLG not only produces results superior to conventional contrast enhancement techniques, but is also fully automatic in most circumstances, and is applicable to a broad variety of images. An extension of GLG, selective GLG (SGLG), and its variations will be discussed in Part II of this paper. SGLG selectively groups and ungroups histogram components to achieve specific application purposes, such as eliminating background noise, enhancing a specific segment of the histogram, and so on. The extension of GLG to color images will also be discussed in Part II.

This is Part II of the paper, "Gray-Level Grouping (GLG): an Automatic Method for Optimized Image Contrast Enhancement". Part I of this paper introduced a new automatic contrast enhancement technique: gray-level grouping (GLG). GLG is a general and powerful technique, which can be conveniently applied to a broad variety of low-contrast images and outperforms conventional contrast enhancement techniques. However, the basic GLG method still has limitations and cannot enhance certain classes of low-contrast images well, e.g., images with a noisy background. The basic GLG also cannot fulfill certain special application purposes, e.g., enhancing only part of an image which corresponds to a certain segment of the image histogram. In order to break through these limitations, this paper introduces an extension of the basic GLG algorithm, selective gray-level grouping (SGLG), which groups the histogram components in different segments of the grayscale using different criteria and, hence, is able to enhance different parts of the histogram to various extents. This paper also introduces two new preprocessing methods to eliminate background noise in noisy low-contrast images so that such images can be properly enhanced by the (S)GLG technique. The extension of (S)GLG to color images is also discussed in this paper. SGLG and its variations extend the capability of the basic GLG to a larger variety of low-contrast images, and can fulfill special application requirements. SGLG and its variations not only produce results superior to conventional contrast enhancement techniques, but are also fully automatic under most circumstances, and are applicable to a broad variety of images.

In this paper, we describe an algorithm called fast marching watersheds that segments a triangle mesh into visual parts. This computer vision algorithm leverages a human vision theory known as the minima rule. Our implementation computes the principal curvatures and principal directions at each vertex of a mesh, and then our hill-climbing watershed algorithm identifies regions bounded by contours of negative curvature minima. These regions fit the definition of visual parts according to the minima rule. We present evaluation analysis and experimental results for the proposed algorithm.

In this paper, we describe an algorithm to measure the shape complexity for discrete approximations of planar curves in 2D images and manifold surfaces for 3D triangle meshes. We base our algorithm on shape curvature, and thus we compute shape information as the entropy of curvature. We present definitions to estimate curvature for both discrete curves and surfaces and then formulate our theory of shape information from these definitions. We demonstrate our algorithm with experimental results.

This paper proposes a new, efficient surface representation method for surface matching. A feature carrier for a surface point, which is a set of two-dimensional (2-D) contours that are the projections of geodesic circles on the tangent plane, is generated. The carrier is named point fingerprint because its pattern is similar to human fingerprints and plays a role in discriminating surface points. Corresponding points on surfaces from different views are found by comparing their fingerprints. The point fingerprint is able to carry curvature, color, and other information which can improve matching accuracy, and the matching process is faster than 2-D image comparison. A novel candidate point selection method based on the fingerprint irregularity is introduced. Point fingerprint is successfully applied to pose estimation of real range data.

We propose a novel algorithm to smooth and simplify simultaneously range images and also triangle meshes derived from those images. These data sets often suffer from noise and over-sampling. To overcome these issues, smoothing from image processing and simplification from computer graphics attempt to minimize noise and reduce complexity, respectively. Typically, these algorithms are separate and distinct steps, but we combine them into one algorithm. We employ surface normal voting to generate robust orientation estimates and then extend the quadric error metric framework to smooth noise while simplifying the surface. We demonstrate the capabilities of this algorithm with both synthetic and real data. The proposed algorithm provides significant noise smoothing improvement when compared to the standard Garland and Heckbert (1998) quadric simplification algorithm.

Triangle meshes are widely used in representing surfaces in computer vision and computer graphics. Although 2D image processing-based edge detection techniques have been popular in many application areas, they are not well developed for surfaces represented by triangle meshes. This paper proposes a robust edge detection algorithm for triangle meshes and its applications to surface segmentation and adaptive surface smoothing. The proposed edge detection technique is based on eigen analysis of the surface normal vector field in a geodesic window. To compute the edge strength of a certain vertex, the neighboring vertices in a specified geodesic distance are involved. Edge information are used further to segment the surfaces with the watershed algorithm and to achieve edge-preserved, adaptive surface smoothing. The proposed algorithm is novel in robustly detecting edges on triangle meshes against noise. The 3D watershed algorithm is an extension from previous work. Experimental results on surfaces reconstructed from multi-view real range images are presented.

In this paper, we describe a robust method for the estimation of curvature on a triangle mesh, where this mesh is a discrete approximation of a piecewise smooth surface. The proposed method avoids the computationally expensive process of surface fitting and instead employs normal voting to achieve robust results. This method detects crease discontinuities on the surface to improve estimates near those creases. Using a voting scheme, the algorithm estimates both principal curvatures and principal directions for smooth patches. The entire process requires one user parameter-the voting neighborhood size, which is a function of sampling density, feature size, and measurement noise. We present results for both synthetic and real data and compare these results to an existing algorithm developed by Taubin (1995).