We introduce an efficient measure for estimating the degree of in-focus within an image region. This measure, harmonic mean of variances, is computed from the statistical properties of the image in its bandpass filtered versions. We incorporate the harmonic variance into a graph Laplacian spectrum based segmentation framework in order to accurately align the in-focus measure responses on the underlying image structure. Our results demonstrate the effectiveness of this novel measure for determining and segmenting in-focus regions in low depth-of-field images.

This paper investigates the role that nonlinear camera response functions (CRFs) have on image deblurring. We present a comprehensive study to analyze the effects of CRFs on motion deblurring. In particular, we show how nonlinear CRFs can cause a spatially invariant blur to behave as a spatially varying blur. We prove that such nonlinearity can cause large errors around edges when directly applying deconvolution to a motion blurred image without CRF correction. These errors are inevitable even with a known point spread function (PSF) and with state-of-the-art regularization-based deconvolution algorithms. In addition, we show how CRFs can adversely affect PSF estimation algorithms in the case of blind deconvolution. To help counter these effects, we introduce two methods to estimate the CRF directly from one or more blurred images when the PSF is known or unknown. Our experimental results on synthetic and real images validate our analysis and demonstrate the robustness and accuracy of our approaches.

This paper presents a novel coprime blurred pair (CBP) model to improve data security in camera surveillance. While most previous approaches have focused on completely encrypting the video stream, we introduce a spatial encryption scheme by strategically blurring the image/video contents. Specifically, we form a public stream and a private stream by blurring the original video data using two different kernels. Each blurred stream will provide the user who has lower clearance less access to personally identifiable details while still allowing behavior to be monitored. If the behavior is recognized as suspicious, a supervisor can use both streams to deblur the contents. Our approach is based on a new CBP theory where the two kernels are coprime when mapped to bivariate polynomials in the $(z)$ domain. We show that coprimality can be derived in terms of the rank of Bézout matrix formed by sampled polynomials, and we present an efficient algorithm to factor the Bézout matrix for recovering the latent image. To make our solution practical, we implement our decryption scheme on a graphics processing unit (GPU) to achieve real-time performance. Extensive experiments demonstrate that our new scheme can effectively protect sensitive identity information in surveillance videos and faithfully reconstruct the unblurred video stream when both CBP sequences are available.

We present a novel and simple computational imaging solution to robustly and accurately recover 3D dynamic fluid surfaces. Traditional specular surface reconstruction schemes place special patterns (checkerboard or color patterns) beneath the fluid surface to establish point-pixel correspondences. However, point-pixel correspondences alone are insufficient to recover surface normal or height and they rely on additional constraints to resolve the ambiguity. In this paper, we exploit using Bokode - a computational optical device that emulates a pinhole projector - for capturing ray-ray correspondences which can then be used to directly recover the surface normals. We further develop a robust feature matching algorithm based on the Active-Appearance Model to robustly establishing ray-ray correspondences. Our solution results in an angularly sampled normal field and we derive a new angular-domain surface integration scheme to recover the surface from the normal fields. Specifically, we reformulate the problem as an over-constrained linear system under spherical coordinate and solve it using Singular Value Decomposition. Experiments results on real and synthetic surfaces demonstrate that our approach is robust and accurate, and is easier to implement than state-of-the-art multi-camera based approaches.

Acquiring dynamic 3D fluid surfaces is a challenging problem in computer vision. Single or stereo camera based solutions are sensitive to refraction distortions, fast fluid motions, and calibration errors. In this paper, we present a multi-view based solution for robustly capturing fast evolving fluid wavefronts. We first construct a portable, 3×3 camera array system as the main acquisition device. We elaborately design the system to allow high-resolution and high-speed capture. To recover fluid surfaces, we place a known pattern beneath the surface and position the camera array on top to observe the pattern. By tracking the distorted feature points over time and across cameras, we obtain spatial-temporal correspondence maps and we use them for specular carving to reconstruct the time-varying surface. In case one of the cameras loses track due to distortions or blurs, we use the rest of the cameras to construct the surface and then apply multi-perspective warping to locate the lost-track feature points so that we can continue using the camera in later frames. Our experiments on synthetic and real data demonstrate that our multi-view framework is robust and reliable.

We present a new Coprime Blurred Pair (CBP) theory that may benefit a number of computer vision applications. A CBP is constructed by blurring the same latent image with two unknown kernels, where the two kernels are co-prime when mapped to bivariate polynomials under the z-transform. We first show that the blurred contents in a CBP are difficult to restore using conventional blind deconvolution methods based on sparsity priors. We therefore introduce a new coprime prior for recovering the latent image in a CBP. Our solution maps the CBP to bivariate polynomials and sample them on the unit circle in both dimension. We show that coprimality can be derived in terms of the rank of the Bézout Matrix [2] formed by the sampled polynomials and we present an efficient algorithm to factor the Bézout Matrix for recovering the latent image. Finally, we discuss applications of the CBP theory in privacy-preserving surveillance and motion deblurring, as well as physical implementations of CBPs using flutter shutter cameras.

Visual tracking plays an important role in many computer vision tasks. A common assumption in previous methods is that the video frames are blur free. In reality, motion blurs are pervasive in the real videos. In this paper we present a novel BLUr-driven Tracker (BLUT) framework for tracking motion-blurred targets. BLUT actively uses the information from blurs without performing debluring. Specifically, we integrate the tracking problem with the motion-from-blur problem under a unified sparse approximation framework. We further use the motion information inferred by blurs to guide the sampling process in the particle filter based tracking. To evaluate our method, we have collected a large number of video sequences with significant motion blurs and compared BLUT with state-of-the-art trackers. Experimental results show that, while many previous methods are sensitive to motion blurs, BLUT can robustly and reliably track severely blurred targets.

The classical optical flow assumes that a feature point maintains constant brightness across the frames. For fluid-type motions such as smoke or clouds, the constant brightness assumption does not hold, and accurately estimating the motion flow from their images is difficult. In this paper, we introduce a simple but effective Navier-Stokes (NS) potential flow model for recovering fluid-type motions. Our method treats the image as a wavefront surface and models the 3D potential flow beneath the surface. The gradient of the velocity potential describes the motion flow at every voxel. We first derive a general brightness constraint that explicitly models wavefront (brightness) variations in terms of the velocity potential. We then use a series of partial differential equations to separately model the dynamics of the potential flow. To solve for the potential flow, we use the Dirichlet-Neumann Operator (DNO) to simplify the 3D volumetric velocity potential to 2D surface velocity potential. We approximate the DNO via Taylor expansions and develop a Fourier domain method to efficiently estimate the Taylor coefficients. Finally we show how to use the DNO to recover the velocity potential from images as well as to propagate the wavefront (image) over time. Experimental results on both synthetic and real images show that our technique is robust and reliable.

We present a hybrid camera that combines the advantages of a high resolution camera and a high speed camera. Our hybrid camera consists of a pair of low-resolution high-speed (LRHS) cameras and a single high-resolution low-speed (HRLS) camera. The LRHS cameras are able to capture fast-motion with little motion blur. They also form a stereo pair and provide a low-resolution depth map. The HRLS camera provides a high spatial resolution but also introduces severe motion blur when capturing fast moving objects. We develop efficient algorithms to simultaneously motion-deblur the HRLS image and reconstruct a high resolution depth map. Our method estimates the motion flow in the LRHS pair and then warps the flow field to the HRLS camera to estimate the point spread function (PSF).We then deblur the HRLS image and use the resulting image to enhance the low-resolution depth map using joint bilateral filters. We demonstrate the hybrid camera in depth map super-resolution and motion deblurring with spatially varying kernels. Experiments show that our framework is robust and highly effective.

Caustics are important visual phenomena, as well as challenging global illumination effects in computer graphics. Physically caustics can be interpreted from one of two perspectives: in terms of photons gathered on scene geometry, or in terms of a pair of caustic surfaces. These caustic surfaces are swept by the foci of light rays. In this paper, we develop a novel algorithm to approximate caustic surfaces of sampled rays. Our approach locally parameterizes rays by their intersections with a pair of parallel planes. We show neighboring ray triplets are constrained to pass simultaneously through two slits, which rule the caustic surfaces. We derive a ray characteristic equation to compute the two slits, and hence, the caustic surfaces. Using the characteristic equation, we develop a GPU-based algorithm to render the caustics. Our approach produces sharp and clear caustics using much fewer ray samples than the photon mapping method and it also maintains high spatial and temporal coherency. Finally, we present a normal-ray surface representation that locally parameterizes the normals about a surface point as rays. Computing the normal ray caustic surfaces leads to a novel real-time discrete shape operator.