This paper presents an end-to-end system for moving object verification in airborne video sequences. Using a sample selection module, the system first selects frames from a short sequence and stores them in an exemplar database. To handle appearance change due to potentially large aspect angle variations, a homography-based view synthesis method is then used to generate a novel view of each image in the exemplar database at the same pose as the testing object in each frame of a testing video segment. A rotationally invariant color matcher and a spatial-feature matcher based on distance transforms are combined using a weighted average rule to compare the novel view and the testing object. After looping over all testing frames, the set of match scores is passed to a temporal analysis module to examine the behavior of the testing object, and calculate a final likelihood. Very good verification performance is achieved over thousands of trials for both color and infrared video sequences using the proposed system.

In this paper, we propose a complete framework for improved synthesis and understanding of the human pose from a limited number of silhouette images. It combines the active image-based visual hull (IBVH) algorithm and a contour-based body part segmentation technique. We derive a simple, approximate algorithm to decide the extrinsic parameters of a virtual camera, and synthesize the turntable image collection of the person using the IBVH algorithm by actively moving the virtual camera on a properly computed circular trajectory around the person. Using the turning function distance as the silhouette similarity measurement, this approach can be used to generate the desired pose-normalized images for recognition applications. In order to overcome the inability of the visual hull (VH) method to reconstruct concave regions, we propose a contour-based human body part localization algorithm to segment the silhouette images into convex body parts. The body parts observed from the virtual view are generated separately from the corresponding body parts observed from the input views and then assembled together for a more accurate VH reconstruction. Furthermore, the obtained turntable image collection helps to improve the body part segmentation and identification process. By using the inner distance shape context (IDSC) measurement, we are able to estimate the body part locations more accurately from a synthesized view where we can localize the body part more precisely. Experiments show that the proposed algorithm can greatly improve body part segmentation and hence shape reconstruction results.

This paper presents an end-to-end verification system for moving objects in airborne video. Lacking prior training data, the object information is collected on the fly from a short real-time learning sequence. Using a sample selection module, the system selects samples from the learning sequence and stores them in an exemplar database. To handle appearance change due to potentially large aspect angle variations, a homography-based view synthesis method is used to generate a novel view of each image in the exemplar database at the same pose as the query object in each frame of a query sequence. A spatial match score is obtained using a Distance Transform to compare the novel view and query object. After looping over all query frames, the set of match scores is passed to a temporal analysis module to examine the behavior of the query object, and calculate a final likelihood. Very good verification performance is achieved over thousands of trials for both color and infrared video sequences using the proposed system.

This paper presents a method for synthesizing novel views of moving objects in airborne video. The object of interest is tracked using an appearance based visual tracking method. The on-object point correspondence is then built and used to estimate the homography induced by the ground plane. With known camera focal length, the surface normal to the ground plane and the camera motion between two views are factored out from the homography. In order to assure robustness of surface normal estimation, a rank one constraint is applied to decompose a matrix which contains the homographies from multiple frame pairs. Given a desired viewing direction, the novel image of the object is generated by warping the reference frame using the new homography between the desired viewpoint and the reference frame. Experimental results show that the method is robust and errors due to small depth variations of the object is negligible. 1

We introduce an active view synthesis approach from silhouettes. With the virtual camera moving on a properly selected circular trajectory around an object of interest, we achieve a collection of virtual views of the object, which is equivalent to the case that the object is on a rotating turntable and captured by a static camera whose optical axis is parallel to the turntable. We show how to derive the virtual camera's extrinsic parameters at each position on the trajectory. Using the turning function distance as the silhouette similarity measurement, this approach can be used to generate the desired pose-normalized images for recognition applications.

The paper introduces a two view tracking method which uses the homography relation between the two views to handle occlusions. An adaptive appearance-based model is incorporated in a particle filter to realize robust visual tracking. Occlusion is detected using robust statistics. When there is occlusion in one view, the homography from this view to other views is estimated from previous tracking results and used to infer the correct transformation for the occluded view. Experimental results show the robustness of the two view tracker.

In this paper, we present a method, which combines image-based visual hull and human body part segmentation for overcoming the inability of the visual hull method to reconstruct concave regions. The virtual silhouette image corresponding to the given viewing direction is first produced with image-based visual hull. Human body part localization technique is used to segment the input images and the rendered virtual silhouette image into convex body parts. The body parts in the virtual view are generated separately from the corresponding body parts in the input views and then assembled together. The previously rendered silhouette image is used to locate the corresponding body parts in input views and avoid the unconnected or squeezed regions in the assembled final view. Experiments show that this method can improve the reconstruction of concave regions for human postures and texture mapping.