Our goal is to develop tools that go beyond frame-constrained manipulation such as resizing, color correction, and simple transitions, and provide object-level operations within frames. Some of our targeted video editing tasks includes transferring a motion picture to a new still picture, importing a moving object into a new background, and compositing two video sequences. The challenges behind this kind of complex video editing tasks lie in two constraints: 1) Spatial consistency: imported objects should blend with the background seamlessly. Hence pixel replacement, which creates noticeable seams, is problematic. 2) Temporal coherency: successive frames should display smooth transitions. Hence frame-by-frame editing, which results in visual flicker, is inappropriate. Our work is aimed at providing an easy-to-use video editing tool that maximally satisfies the spatial and temporal constraints mentioned above and requires minimum user interaction. We propose a new framework for video editing in gradient domain. The spatio-temporal gradient fields of target videos are modified and/or mixed to generate a new gradient field which is usually not integrable. We propose a 3D video integration algorithm, which uses the variational method, to find the potential function whose gradient field is closest to the mixed gradient field in the sense of least squares. The video is reconstructed by solving a 3D Poisson equation. We derive an extension of current 2D gradient technique to 3D space, yielding in a novel video editing framework, which is very different from all current video editing software.

In this paper we present a graph cuts based active contours (GCBAC) approach to object segmentation problems. Our method is a combination of active contours and the optimization tool of graph cuts and differs fundamentally from traditional active contours in that it uses graph cuts to iteratively deform the contour. Consequently, it has the following advantages. (1) It has the ability to jump over local minima and provide a more global result. (2) Graph cuts guarantee continuity and lead to smooth contours free of self-crossing and uneven spacing problems. Therefore, the internal force, which is commonly used in traditional energy functions to control the smoothness, is no longer needed, and hence the number of parameters is greatly reduced. (3). Our approach easily extends to the segmentation of three and higher dimensional objects. In addition, the algorithm is suitable for interactive correction and is shown to always converge. Experimental results and analyses are provided.

In this paper, we present an object contour tracking approach using graph cuts based active contours (GCBAC). Our proposed algorithm does not need any a priori global shape model, which makes it useful for tracking objects with deformable shapes and appearances. GCBAC are not sensitive to initial conditions and always converge to the optimal contour within the dilated neighborhood of itself. Given an initial boundary near the object in the first frame, GCBAC can iteratively converge to an optimal object boundary. In each frame thereafter, the resulting contour in the previous frame is taken as initialization and the algorithm consists of two steps. In the first step, GCBAC are applied to the difference between this frame and its previous one. The resulting contour is taken as initialization of the second step, which applies GCBAC to current frame directly. To evaluate the tracking performance, we apply the algorithm to several real world video sequences. Experimental results are provided.

We present an iterative algorithm to reconstruct a 3D object surface from a sparse set of matched feature points on the input stereo images of the object. The initial matches are sparse and do not have to be accurate. The reconstructed 3D surface is represented in terms of triangular polygons whose vertices are initially the 3D points corresponding to these matched feature points. In order to render photorealistic images of the surface, these feature points are iteratively updated. New feature points are added into the feature point set as well as the depth estimates of the feature points are refined. Experimental results showing the updated correspondences, reconstructed surfaces and virtual views rendered from new directions are presented.