We address the problem of optical flow estimation between a stereo pair of images suitable for view interpolation applications. In recent years, deep learning based approaches are found to be capable of producing superior optical flow estimation results than that of traditional approaches. However, deep learning approaches generally suffer from large GPU memory constraints and hence their application is limited to low resolution images. Here, we present an approach to accurate optical flow estimation via the state-of-the-art deep learning based approaches that introduces fewer artifacts in view interpolation applications. We confirm the suitability of our approach through both qualitative and quantitative results.

This paper addresses the problem of finding semi-dense feature correspondences with better than quarter-pixel accuracy across stereo images in an efficient manner. Having a large number of correspondences is critical for many computer vision applications as more observations allow to better model application parameters. Also, performance of many applications can greatly improve with correspondences of at least quarter-pixel accuracy. However, defining a large number of quarter-pixel accurate feature correspondences effectively is still a challenging task. Specifically, dense matching attempts to find dense correspondences; while it is attractive to have a feature correspondence per pixel in a given image, such approaches are typically computationally heavy with at most half-pixel accuracy. In contrast, sparse feature matching approaches are computationally less expensive and generally benefit from better than quarter-pixel accuracy for the defined correspondences, but, they define correspondences for only a limited number of pixels in an image. In addition, they tend to have a higher rate of mismatches as they do not consider spatial smoothness at their core. This paper introduces a new framework to address the above shortcomings, aiming to significantly increase the number of correct matches with at least quarter-pixel accuracy, while keeping the computational cost smaller than that of dense matching alone.

This paper addresses the problem of robust feature detection and matching across different views of the same scene taken using two (or more) wide angle cameras, assuming that their intrinsic calibration parameters are approximately known. We introduce an end-to-end framework with an aim to enhance both precision and recall in the matching process, i.e. increase the number of correct matches across the fisheye images while reducing the number of false positive matches. Experimental results demonstrate the effectiveness of the proposed work.

While High Dynamic Range (HDR) image generation using computational approaches has seen widespread adoption in consumer photography, computational approaches for HDR video creation are only just beginning to emerge. In this paper, we present an algorithm for HDR video generation using short baseline camera arrays (cameras placed close together) with varying exposures. Our main contributions are an algorithm for disparity estimation that specifically targets short baseline camera arrays and an alignment algorithm that minimizes visual artifacts even in the presence of disparity errors. Finally, we present results using our algorithm on real world videos captured using a camera array to demonstrate the success of this application.

Computational photography techniques overcome limitations of traditional image sensors such as dynamic range and noise. Many computational imaging techniques have been proposed that process image stacks acquired using different exposure, aperture or gain settings, but far less attention has been paid to determining the parameters of the stack automatically. In this paper, we propose a novel computational imaging system that automatically and efficiently computes the optimal number of shots and corresponding exposure times and gains, taking into account characteristics of the scene and sensor. Our technique seamlessly integrates the use of multiple capture for both High Dynamic Range (HDR) imaging and denoising. The acquired images are then aligned, warped and merged in the raw Bayer domain according to a statistical noise model of the sensor to produce an optimal, potentially HDR and denoised image. The result is a fully automatic camera that constantly monitors the scene in front of it and decides how many images are required to capture it, without requiring the user to explicitly switch between different capture modalities.

Estimating the 3D orientation of the camera in a video sequence within a global frame of reference is useful for video stabilization when displaying the video in a virtual 3D environment, as well as for accurate navigation and other applications. This task requires the input of orientation sensors attached to the camera to provide absolute 3D orientation in a geographical frame of reference. However, high-frequency noise in the sensor readings makes it impossible to achieve accurate orientation estimates required for visually stable presentation of video sequences that were acquired with a camera subject to jitter, such as a handheld camera or a vehicle mounted camera. On the other hand, image alignment has proven successful for image stabilization, providing accurate frame-to-frame orientation estimates but drifting over time due to error and bias accumulation and lacking absolute orientation. In this paper we propose a practical method for generating high accuracy estimates of the 3D orientation of the camera within a global frame of reference by fusing orientation estimates from an efficient image-based alignment method, and the estimates from an orientation sensor, overcoming the limitations of the component methods.

Over-determined linear systems with noise in all measurements are common in computer vision, and particularly in motion estimation. Maximum likelihood estimators have been proposed to solve such problems, but except for simple cases, the corresponding likelihood functions are extremely complex, and accurate confidence measures do not exist. This paper derives the form of simple likelihood functions for such linear systems in the general case of heteroscedastic noise. We also derive a new algorithm for computing maximum likelihood solutions based on a modified Newton method. The new algorithm is more accurate, and exhibits more reliable convergence behavior than existing methods. We present an application to affine motion estimation, a simple heteroscedastic estimation problem.

We describe a probabilistic framework for detecting and tracking motion boundaries. It builds on previous work (M.J. Black and D.J. Fleet, 2000) that used a particle filter to compute a posterior distribution over multiple, local motion models, one of which was specific for motion boundaries. We extend that framework in two ways: 1) with an enhanced likelihood that combines motion and edge support, 2) with a spatiotemporal model that propagates beliefs between adjoining image neighborhoods to encourage boundary continuity and provide better temporal predictions for motion boundaries. Approximate inference is achieved with a combination of tools: sampled representations allow us to represent multimodal non-Gaussian distributions and to apply nonlinear dynamics, while mixture models are used to simplify the computation of joint prediction distributions.

This paper addresses the derivation of likelihood functions and confidence bounds for problems involving over-determined linear systems with noise in all measurements, often referred to as total-least-squares (TLS). It has been shown previously that TLS provides maximum likelihood estimates. But rather than being a function solely of the variables of interest, the associated likelihood functions increase in dimensionality with the number of equations. This has made it difficult to derive suitable confidence bounds, and impractical to use these probability functions with Bayesian belief propagation or Bayesian tracking. This paper derives likelihood functions that are defined only on the parameters of interest. This has two main advantages. First, the likelihood functions are much easier to use within a Bayesian framework; and second it is straightforward to obtain a reliable confidence bound on the estimates. We demonstrate the accuracy of our confidence bound in relation to others that have been proposed. Also, we use our theoretical results to obtain likelihood functions for estimating the direction of 3D camera translation.