“Seeing through” occluders is one of the most important effects that can be achieved with synthetic aperture imaging. As well, the occlusion problem, a challenging task for many computer vision applications, can be easily handled. Synthetic aperture imaging takes advantage of the property that only objects on the focal plane are sharp. The resulting image that is obtained by averaging images from different views consists of blurry objects away from the focal plane and sharp objects on the focal plane. Removing the blurriness caused by defocusing in synthetic aperture images to achieve an all-in-focus “seeing through” image is a challenging research problem. In this paper, we propose a novel method to improve the image quality of synthetic aperture imaging using image matting via energy minimization by estimating the foreground and the background. In particular, we first estimate the out-of-focus region by focusing on the background objects in each camera view using energy minimization. Next, we utilize a labeling method to create a sharp “see through” synthetic aperture image of the hidden objects. Then, image matting is used to extract the alpha matte of the hidden objects. Finally, by compositing the hidden objects with the estimated background regions, a sharp “see through” synthetic aperture image is created. The experimental results show that the proposed method outperforms the traditional synthetic aperture imaging method [1] as well as its improved versions [2]-[4], which simply dim and blur the area in the image that is out of focus, and a recent all-in-focus method [5]. We show that both the occluded objects and the background can be combined using our method to create a sharp synthetic aperture image.

In this paper, we present a new method to estimate the housing parameters of an underwater camera by making full use of triple wavelength dispersion. Our method is based on an important finding that there is a closed-form solution to the distance from the camera center to the refractive interface once the refractive normal is known. The correctness of this finding is mathematically proved in this paper. To the best of our knowledge, such a finding has not been studied or reported, and hence is never proved theoretically. As well, the refractive normal can be estimated by solving a set of linear equations using wavelength dispersion. Our method does not require any calibration target, such as a checkerboard pattern, which may be difficult to manipulate when the camera is deployed deep undersea. Extensive experiments have been carried out which include simulations to verify the correctness and robustness to noise of our method and real experiments. The results of real experiments show that our method works as expected. The accuracy of our results is evaluated against the ground truth in both simulated and real experiments. Finally, we also show how we can apply dispersion to compute the 3D shape of an object using one single camera.

In this paper, we present a novel refractive calibration method for an underwater stereo camera system where both cameras are looking through multiple parallel flat refractive interfaces. At the heart of our method is an important finding that the thickness of the interface can be estimated from a set of pixel correspondences in the stereo images when the refractive axis is given. To our best knowledge, such a finding has not been studied or reported. Moreover, by exploring the search space for the refractive axis and using reprojection error as a measure, both the refractive axis and the thickness of the interface can be recovered simultaneously. Our method does not require any calibration target such as a checkerboard pattern which may be difficult to manipulate when the cameras are deployed deep undersea. The implementation of our method is simple. In particular, it only requires solving a set of linear equations of the form Ax = b and applies sparse bundle adjustment to refine the initial estimated results. Extensive experiments have been carried out which include simulations with and without outliers to verify the correctness of our method as well as to test its robustness to noise and outliers. The results of real experiments are also provided. The accuracy of our results is comparable to that of a state-of-the-art method that requires known 3D geometry of a scene.

3D reconstruction of transparent and specular objects is a very challenging topic in computer vision. For transparent and specular objects, which have complex interior and exterior structures that can reflect and refract light in a complex fashion, it is difficult, if not impossible, to use either passive stereo or the traditional structured light methods to do the reconstruction. We propose a frequency-based 3D reconstruction method, which incorporates the frequency-based matting method. Similar to the structured light methods, a set of frequency-based patterns are projected onto the object, and a camera captures the scene. Each pixel of the captured image is analyzed along the time axis and the corresponding signal is transformed to the frequency-domain using the Discrete Fourier Transform. Since the frequency is only determined by the source that creates it, the frequency of the signal can uniquely identify the location of the pixel in the patterns. In this way, the correspondences between the pixels in the captured images and the points in the patterns can be acquired. Using a new labelling procedure, the surface of transparent and specular objects can be reconstructed with very encouraging results.

With the exponential growth of web image data, image tagging is becoming crucial in many applications such as e-commerce. However, despite the great progress achieved in various image tagging technologies, none of them are able to incorporate browsing and discovery activities on web viewers in such a way that a user can easily query an image and ask the question "what is that in the image?". We have developed a comprehensive online image tagging system based on a Tagging-Tracking-Learning (TTL) framework to solve this problem. Tagging images using this system is able to turn common static web images into non-intrusive interactive images. The system tracks all browsing and interaction activities of users over time to filter out low quality tags and in turn helps the tagging process by alleviating manual operations. In this paper, we describe the implementation of the TTL framework and the novel algorithms developed. Usability studies of the system indicate that the TTL framework provides a better user experiences and simplifies the process of obtaining large tagged image collections over state-of-the-art approaches.

In this paper, we present a new approach for recovering spacetime-consistent depth maps from multiple video sequences captured by stationary, synchronized and calibrated cameras for depth based free viewpoint video rendering. Our two-pass approach is generalized from the recently proposed region-tree based binocular stereo matching method. In each pass, to enforce temporal consistency between successive depth maps, the traditional region-tree is extended into a temporal one by including connections to “temporal neighbor regions” in previous video frames, which are identified using estimated optical flow information. For enforcing spatial consistency, multi-view geometric constraints are used to identify inconsistencies between depth maps among different views which are captured in an inconsistency map for each view. Iterative optimizations are performed to progressively correct inconsistencies through inconsistency maps based depth hypotheses pruning and visibility reasoning. Furthermore, the background depth and color information is generated from the results of the first pass and is used in the second pass to enforce sequence-wise temporal consistency and to aid in identifying and correcting spatial inconsistencies. The extensive experimental evaluations have shown that our proposed approach is very effective in producing spatially and temporally consistent depth maps.