In this work we propose a novel approach to the problem of multi-view stereo reconstruction. Building upon the previously proposed PatchMatch stereo and PM-Huber algorithm we introduce an extension to the multi-view scenario that employs an iterative refinement scheme. Our proposed approach uses an extended and robustified volumetric truncated signed distance function representation, which is advantageous for the fusion of refined depth maps and also for raycasting the current reconstruction estimation together with estimated depth normals into arbitrary camera views. We formulate the combined multi-view stereo reconstruction and refinement as a variational optimization problem. The newly introduced plane based smoothing term in the energy formulation is guided by the current reconstruction confidence and the image contents. Further we propose an extension of the PatchMatch scheme with an additional KLT step to avoid unnecessary sampling iterations. Improper camera poses are corrected by a direct image aligment step that performs robust outlier compensation by means of a recently proposed kernel lifting framework. To speed up the optimization of the variational formulation an adapted scheme is used for faster convergence.

The stereo correspondence problem is still a highly active topic of research with many applications in the robotic domain. Still many state of the art algorithms proposed to date are unable to reasonably handle high resolution images due to their run time complexities or memory requirements. In this work we propose a novel stereo correspondence estimation algorithm that employs binary locality sensitive hashing and is well suited to implementation on the GPU. Our proposed method is capable of processing very high-resolution stereo images at near real-time rates. An evaluation on the new Middlebury and Disney high-resolution stereo benchmarks demonstrates that our proposed method performs well compared to existing state of the art algorithms.

Most stereo correspondence algorithms match support windows at integer-valued disparities and assume a constant disparity value within the support window. The recently proposed Patch Match stereo algorithm by Bleyer et al. overcomes this limitation of previous algorithms by directly estimating planes. This work presents a method that integrates the Patch Match stereo algorithm into a variational smoothing formulation using quadratic relaxation. The resulting algorithm allows the explicit regularization of the disparity and normal gradients using the estimated plane parameters. Evaluation of our method in the Middlebury benchmark shows that our method outperforms the traditional integer-valued disparity strategy as well as the original algorithm and its variants in sub-pixel accurate disparity estimation.

This paper introduces a new method for remote surgical cutting by providing haptic guidance along a trajectory that is measured in-situ through a miniature, tool-tip mounted endoscopic camera. By directly augmenting the instrument with a camera, instead of relying on an additional conventional endoscope, the approach becomes independent from inherent calibration uncertainties of the telemanipulation system, other than the micro camera itself, and from registration of pre- and intra-operative surgical data. We calculate a smooth cut path with corresponding scalpel orientation to guide the user towards the optimal trajectory. Experiments are conducted with passive and active haptic virtual fixtures. During active fixturing the optimal forward velocity is calculated to follow the path. The results indicate an improvement in terms of position accuracy, blade orientation, and forward velocity.