The use of shape from silhouette for reconstruction tasks is plagued by two types of real-world errors: camera calibration error and silhouette segmentation error. When either error is present, we call the problem the Shape from Inconsistent Silhouette (SfIS) problem. In this paper, we show how small camera calibration error can be corrected when using a previously-published SfIS technique to generate a reconstruction, by using an Iterative Closest Point (ICP) approach. We give formulations under two scenarios: the first of which is only external camera calibration parameters rotation and translation need to be corrected for each camera and the second of which is that both internal and external parameters need to be corrected. We formulate the problem as a 2D-3D ICP problem and find approximate solutions using a nonlinear minimization algorithm, the Levenberg-Marquadt method. We demonstrate the ability of our algorithm to create more representative reconstructions of both synthetic and real datasets of thin objects as compared to uncorrected datasets.

We present a robot self-localization approach that is based on using a cascade of filters that increasingly refine a robot's guess regarding where it is in a hallway system. The location refinement carried out by each stage of the cascade compares a signature extracted from a stereo pair of camera images taken at the current location of the robot with a database of such signatures collected previously during a training phase. A central question in this approach to robot localization is what signatures to use for each stage of the cascade. An answer to this question must recognize the special importance of the first stage of the cascade — we refer to this as the prefiltering stage. The signature used for prefiltering must be significantly viewpoint invariant, while possessing sufficient locale uniqueness to yield a set of possible locations for the robot that includes the true location with a high probability. On the other hand, the signature(s) used for downstream filtering in the cascade must then prune away the inapplicable locales from the list yielded by the prefilter. What that implies is that the downstream filters must be increasingly viewpoint variant and locale specific. Although the framework we propose allows for an arbitrary number of filters to follow the prefiltering stage, the results we present in this paper are for a two-stage cascade consisting of a prefilter followed by one additional filter. The signatures we use in our experiments are based on 3D-JUDOCA features that can be extracted from stereo pairs of images. The proposed framework for choosing the best signatures for the prefiltering stage and the filtering stage that follows was tested in a large indoor hallway system with a total linear length of 1539 m. The validation results we show are based on a dataset of 6209 stereo images collected by a robot from the hallways during its training phase. The performance evaluation presented in this paper demonstrates that our framework can lead to high localization accuracy with good time performance by a robot.

We present a robust approach for tracking human subjects as their limbs and torso are engaged in large articulated movements while the entire body is executing a large translational motion with respect to the pointing angle of the camera. While the articulated movements can be handled by the recently proposed Component-Based Multiple Instance Learning (CMIL) tracker, the large translational motions by the target require that we also use a motion prediction framework to more accurately estimate the most probable positions of the target in the next frame of a video sequence. In the work we report here, this prediction is carried out with a particle filter. This coupling between CMIL based tracking and particle filtering yields a much more accurate estimate of candidate positions of the target in the next frame given the position of the target in the current frame. We validate this new approach by demonstrating results on videos of human subjects that are simultaneously executing large articulated movements with their limbs and torso while the subjects themselves are in some translational motions with respect to the pointing angle of the camera.

Energy efficiency dominates practically every aspect of the design of a wireless sensor network and duty cycling is an important tool for achieving high energy efficiencies. Duty cycling for wireless camera networks meant for tracking objects is made complex by the nodes having to anticipate the arrival of the objects in their field-of-view. The consequences of an object arriving in the view-region of a camera when it is sleeping need no elaboration. Our work presents a predictive framework to provide nodes with an ability to anticipate the arrival of objects in the field-of-view of their cameras. Our predictive framework differs from others in that the nodes whose duty cycles are increased are at least one step removed from the immediate neighborhood of the nodes where the objects are currently visible. By eliminating the need for the currently busiest nodes to also be in charge of informing their nonbusy immediate neighbors to get ready for object arrival, we end up with a more robust strategy for updating the duty cycle at the nodes where the objects are highly likely to appear soon. The proposed scheme works by using an existing MAC header bit that is already in the 802.15.4 protocol and, in that sense, our anticipatory approach for notifying the nodes about the current state of the object location entails no additional expenditure of energy. Our contribution includes evaluations based on large-scale simulations as well as real experiments with an Imote2-based wireless camera network.

Our work is about a new class of object trackers that are based on a boosted Multiple Instance Learning (MIL) algorithm to track an object in a video sequence. We show how the scope of such trackers can be expanded to the tracking of articulated movements by humans that frequently result in large frame-to-frame variations in the appearance of what needs to be tracked. To deal with the problems caused by such variations, our paper presents a component based version of the boosted MIL algorithm. Components are the output of an image segmentation algorithm applied to the pixels in the bounding box encapsulating the object to be tracked. The components give the boosted MIL the additional degrees of freedom that it needs in order to deal with the large frame-to-frame variations associated with articulated movements.

Estimating the head pose with RGBD data when the pose is allowed to vary over a large angle remains challenging. In this paper, we show that an appearance-based construction of a set of locally optimum subspaces provides a good (fast and accurate) solution to the problem. At training time, our algorithm partitions the set of all images obtained by applying pose transformations to the 3D point cloud for a frontal view into appearance based clusters and represents each cluster with a local PCA space. Given a test RGBD images, we first find the appearance cluster that it belongs to and, subsequently, we find its pose from the training image that is closest to the test image in that cluster. Our paper compares the appearance-based local-subspace method with the pose-based local-subspace approach and with a PCA-based global subspace method. This comparison establishes the superiority of the appearance-based local-subspace approach.

Our work provides a fast approach-path-independent framework for the problem of place recognition and robot localization in indoor environments. The approach-path independence is achieved by using highly viewpoint-invariant 3D junction features extracted from stereo pairs of images; these are based on stereo reconstructions of the JUDOCA junctions extracted from the individual images of a stereo pair. The speed in place-recognition and robot-localization is achieved by using a novel cylindrical data structure - we refer to it as the Feature Cylinder - for representing either all of the 3D junction features found in a hallway system during the learning phase of the robot or a set of locale signatures derived from the data. For the case when all data is placed on the Feature Cylinder, we can use the 3D-POLY polynomial-time in a hypothesize-and-verify approach to place recognition. On the other hand, in the locale signature based approach, we can use the same data structure for constant-time place recognition.

Our work presents solutions to two related vexing problems in feature-based localization of ground targets in Unmanned Aerial Vehicle (UAV) images: (i) A good initial guess at the pose estimate that would speed up the convergence to the final pose estimate for each image frame in a video sequence; and (ii)Time-bounded estimation of the position of the ground target. We address both these problems within the framework of the Iterative Closest Point (ICP) algorithm that now has a rich tradition of usage in computer vision and robotics applications. We solve the first of the two problems by frame-to-frame propagation of the computed pose estimates for the purpose of the initializations needed by ICP. The second problem is solved by terminating the iterative estimation process at the expiration of the available time for each image frame. We show that when frame-to-frame homography is factored into the iterative calculations, the accuracy of the position calculated at the time of bailing out of the iterations is nearly always sufficient for the goals of UAV vision.

Most tracking algorithms implicitly apply a coarse segmentation of each target object using a simple mask such as a rectangle or an ellipse. Although convenient, such coarse segmentation results in several problems in tracking - drift, switching of targets, poor target localization, to name a few - since it inherently includes extra non-target pixels if the mask is larger than the target or excludes some portion of target pixels if the mask is smaller than the target. In this paper, we propose a novel probabilistic framework for jointly solving segmentation and tracking. Starting from a joint Gaussian distribution over all the pixels, candidate target locations are evaluated by first computing a pixel-level segmentation and then explicitly including this segmentation in the probability model. The segmentation is also used to incrementally update the probability model based on a modified probabilistic principal component analysis (PPCA). Our experimental results show that the proposed method of explicitly considering pixel-level segmentation as a part of solving the tracking problem significantly improves the robustness and performance of tracking compared to other state-of-the-art trackers, particularly for tracking multiple overlapping targets.

In this paper, we present a distributed multicamera face tracking system suitable for large wired camera networks. Unlike previous multicamera face tracking systems, our system does not require a central server to coordinate the entire tracking effort. Instead, an efficient camera clustering protocol is used to dynamically form groups of cameras for in-network tracking of individual faces. The clustering protocol includes cluster propagation mechanisms that allow the computational load of face tracking to be transferred to different cameras as the target objects move. Furthermore, the dynamic election of cluster leaders provides robustness against system failures. Our experimental results show that our cluster-based distributed face tracker is capable of accurately tracking multiple faces in real-time. The overall performance of the distributed system is comparable to that of a centralized face tracker, while presenting the advantages of scalability and robustness.

We present a system for constructing 3D models of real-world objects with optically challenging surfaces. The system utilizes a new range imaging concept called multi-peak range imaging, which stores multiple candidates of range measurements for each point on the object surface. The multiple measurements include the erroneous range data caused by various surface properties that are not ideal for structured-light range sensing. False measurements generated by spurious reflections are eliminated by applying a series of constraint tests. The constraint tests based on local surface and local sensor visibility are applied first to individual range images. The constraint tests based on global consistency of coordinates and visibility are then applied to all range images acquired from different viewpoints. We show the effectiveness of our method by constructing 3D models of five different optically challenging objects. To evaluate the performance of the constraint tests and to examine the effects of the parameters used in the constraint tests, we acquired the ground truth data by painting those objects to suppress the surface-related properties that cause difficulties in range sensing. Experimental results indicate that our method significantly improves upon the traditional methods for constructing reliable 3D models of optically challenging objects.

By active stereo we mean a stereo vision system that allows for independent panning and tilting for each of the two cameras. One advantage of active stereo in relation to regular stereo is the former's wider effective field of view; if an object is too close to the camera baseline, the depth to the object can still be estimated accurately by panning the cameras appropriately. Another advantage of active stereo is that it can yield a larger number of depth measurements simultaneously for each position of the platform on which the camera system is mounted. Panning and tilting over a large angular range, while being the main reason for the advantages of active stereo, also make it more challenging to calibrate such systems. For a calibration procedure to be effective for active stereo, the estimated parameters must be valid over the entire range of the pan and tilt angles. This paper presents a new approach to the calibration of such vision systems. Our method is based on the rationale that an active stereo calibration procedure must explicitly estimate the locations and the orientations of the pan and tilt rotating axes for the cameras through a closed-form solution. When these estimates for the axes are combined with the homogeneous transform relationships that link the various coordinate frames, we end with a calibration that is valid over a large variation in the pan and tilt angles.

This paper seeks to increase the efficiency of background subtraction algorithms for motion detection. Our method uses a quadtree-base hierarchical framework that samples a small portion of the pixels in each image and yet produces motion detection results that are very similar compared to the conventional methods that raster scan entire images. The hierarchical data structure presented in this paper can be used with any background subtraction algorithm that employs background modeling and motion detection on a per-pixel basis. We have tested our method using two common background subtraction algorithms: running average and mixture of Gaussian. Our experimental results show that the application of the hierarchical data structure significantly increases the processing speed for accurate motion detection. For example, the mixture of Gaussian method with our hierarchical data structure is able to process 1600 by 1200 images at 11~12 frames per second compared to 2~3 frames per second without using the hierarchical data structure.