Panretinal Laser Photocoagulation (PRP) using the slit-lamp is a widely used treatment for sight-threatening retina diseases. It is a challenging procedure, therefore a number of mapping techniques have been proposed to provide surgeons with reliable intra-operative navigation assistance and mapping. However, available methods are limited by tracking errors, caused by lens distortions, illumination variations and abrupt patient movement. To handle these issues, we propose a strategy for non-rigid fine adjustment of retina maps acquired using the slit-lamp. We describe a novel direct framework for alignment of multiple overlapping images via a Thin-Plate Spline model. Our method is able to cope with illumination variations between overlapping images and does not require a priori geometrical information. Experiments conducted on maps acquired from several human patients attest the method's ability to correct ghosts and blurring, improving the consistency and accuracy of the maps.

The slit lamp device is a very popular equipment for inspecting the human retina due to its magnification, stereoscopy and easy control. In this context, computer assistance can provide information overlay to surgeons, improving navigation, accuracy and safety in procedures where the slit-lamp is used. Toward this goal, we describe a robust method for tracking and mapping the human retina using slit-lamp images. Inspired in [1], the proposed method is essentially a direct visual tracking method using local illumination compensation, as well as an efficient pixel selection scheme for achieving fast tracking speeds. In this paper, we describe a novel forward-backward strategy for achieving superior tracking resilience. Experiments conducted on several human patients confirm the practical value of the system.

Gradient-based optimization is a very efficient strategy to solve the direct visual tracking (DVT) problem using transformation models with many degrees of freedom (DOF). Even though popular DVT methods use the sum of squared differences as similarity function, this approach is not robust to illumination variations often verified in practice. One technique to compensate illumination variations is through an illumination model, which, in turn, increases the total number of parameters to be computed. High quality augmented reality and robotic systems demand fast tracking speeds, which can be impaired by the computational complexity added by the illumination model. In this paper, we propose a robust DVT method capable of tracking in extreme illumination conditions. Building upon the sum of conditional variance as similarity function, we propose a novel tracking method that significantly reduces the computational effort compared to similar methods proposed in the literature. We provide extensive experiments and quantitative analysis using challenging videos to attest the advantages of the proposed method.

Laser photocoagulation is currently the standard treatment for sight-threatening diseases worldwide, namely diabetic retinopathy and retinal vein occlusions. The slit lamp biomicroscope is the most commonly used device for this procedure, specially for the treatment of the eye periphery. However, only a small portion of the retina can be visualized through the biomicroscope, complicating the task of localizing and identifying surgical targets, increasing treatment duration and patient discomfort. In order to assist surgeons, we propose a method for creating intraoperative retina maps for view expansion using a slit-lamp device. Based on the mosaicking method described by Richa et al, 2012, the proposed method is a combination of direct and feature-based methods, suitable for the textured nature of the human retina. In this paper, we describe three major enhancements to the original formulation. The first is a visual tracking method using local illumination compensation to cope with the challenging visualization conditions. The second is an efficient pixel selection scheme for increased computational efficiency. The third is an entropy-based mosaic update method to dynamically improve the retina map during exploration. To evaluate the performance of the proposed method, we conducted several experiments on human subjects with a computer-assisted slit-lamp prototype. We also demonstrate the practical value of the system for photo documentation, diagnosis and intraoperative navigation.

Methods for tracking an object have generally fallen into two groups: tracking by detection and tracking through local optimization. The advantage of detection-based tracking is its ability to deal with target appearance and disappearance, but it does not naturally take advantage of target motion continuity during detection. The advantage of local optimization is efficiency and accuracy, but it requires additional algorithms to initialize tracking when the target is lost. To bridge these two approaches, we propose a framework for unified detection and tracking as a time-series Bayesian estimation problem. The basis of our approach is to treat both detection and tracking as a sequential entropy minimization problem, where the goal is to determine the parameters describing a target in each frame. To do this we integrate the Active Testing (AT) paradigm with Bayesian filtering, and this results in a framework capable of both detecting and tracking robustly in situations where the target object enters and leaves the field of view regularly. We demonstrate our approach on a retinal tool tracking problem and show through extensive experiments that our method provides an efficient and robust tracking solution.

The goal of this paper is to introduce a direct visual tracking method based on an image similarity measure called the sum of conditional variance (SCV). The SCV was originally proposed in the medical imaging domain for registering multi-modal images. In the context of visual tracking, the SCV is invariant to non-linear illumination variations, multi-modal and computationally inexpensive. Compared to information theoretic tracking methods, it requires less iterations to converge and has a significantly larger convergence radius. The novelty in this paper is a generalization of the efficient second-order minimization formulation for tracking using the SCV, allowing us to combine the efficient second-order approximation of the Hessian with a similarity metric invariant to non-linear illumination variations. The result is a visual tracking method that copes with non-linear illumination variations without requiring the estimation of photometric correction parameters at every iteration. We demonstrate the superior performance of the proposed method through comparative studies and tracking experiments under challenging illumination conditions and rapid motions.

In the context of minimally invasive cardiac surgery, robotic assistance has significantly helped surgeons to overcome difficulties related to the minimally invasive procedure. Recently, techniques have been proposed for active canceling the beating heart motion for improving the accuracy of the surgical gestures. In this scenario, computer vision techniques can be applied for estimating the heart motion based solely on natural structures on the heart surface. However, visual tracking is complicated by the particular lighting conditions and clutter (smoke, liquids, etc) during surgery. Another challenging problem are the occasional occlusions by surgical instruments. In order to overcome these problems, we exploit the quasi-periodicity of the beating heart motion for increasing the robustness of the visual tracking task. In this paper, a novel time-varying dual Fourier series for modeling the quasi-periodic beating heart motion is proposed. For estimating the series parameters, an Extended Kalman Filter (EKF) is used. The proposed method is applied in a visual tracking task for bridging tracking disturbances and automatically reestablish tracking in cases of occlusions. The efficiency of the prediction method and the sensible improvements in the visual tracking task are demonstrated through in vivo experiments.

With the advent of new applications in cardiac robotic-assisted minimally invasive surgery (MIS), a demand for the design of efficient motion compensation systems was created. In this context, vision-based techniques seem to be a practical way to retrieve the motion of the beating heart since they do not require the introduction of additional sensors in the limited workspace. In this paper, we propose an efficient method for tracking the heart surface which incorporates two novelties. The first is a thin-plate splines (TPS) parametric model for the heart surface deformation that allows us to better track regions of the heart surface with little texture information which undergo large non-rigid deformations. The second novelty is the incorporation of a performing illumination compensation algorithm to cope with arbitrary illumination changes and increase tracking robustness.We also extend this framework for 3D tracking, to enable full compensation of the heart motion. Extensive experiments conducted onin-vivoandex-vivoheart images attest the notable performance of the algorithm.