Accurate instrument pose estimation is a crucial step towards the future of robotic surgery, enabling applications such as autonomous surgical task execution. Vision-based methods for surgical instrument pose estimation provide a practical approach to tool tracking, but they often require markers to be attached to the instruments. Recently, more research has focused on the development of markerless methods based on deep learning. However, acquiring realistic surgical data, with ground truth (GT) instrument poses, required for deep learning training, is challenging. To address the issues in surgical instrument pose estimation, we introduce the Surgical Robot Instrument Pose Estimation (SurgRIPE) challenge, hosted at the 26th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) in 2023. The objectives of this challenge are: (1) to provide the surgical vision community with realistic surgical video data paired with ground truth instrument poses, and (2) to establish a benchmark for evaluating markerless pose estimation methods. The challenge led to the development of several novel algorithms that showcased improved accuracy and robustness over existing methods. The performance evaluation study on the SurgRIPE dataset highlights the potential of these advanced algorithms to be integrated into robotic surgery systems, paving the way for more precise and autonomous surgical procedures. The SurgRIPE challenge has successfully established a new benchmark for the field, encouraging further research and development in surgical robot instrument pose estimation.

Unlike in natural images, in endoscopy there is no clear notion of an up-right camera orientation. Endoscopic videos therefore often contain large rotational motions, which require keypoint detection and description algorithms to be robust to these conditions. While most classical methods achieve rotation-equivariant detection and invariant description by design, many learning-based approaches learn to be robust only up to a certain degree. At the same time learning-based methods under moderate rotations often outperform classical approaches. In order to address this shortcoming, in this paper we propose RIDE, a learning-based method for rotation-equivariant detection and invariant description. Following recent advancements in group-equivariant learning, RIDE models rotation-equivariance implicitly within its architecture. Trained in a self-supervised manner on a large curation of endoscopic images, RIDE requires no manual labeling of training data. We test RIDE in the context of surgical tissue tracking on the SuPeR dataset as well as in the context of relative pose estimation on a repurposed version of the SCARED dataset. In addition we perform explicit studies showing its robustness to large rotations. Our comparison against recent learning-based and classical approaches shows that RIDE sets a new state-ofthe-art performance on matching and relative pose estimation tasks and scores competitively on surgical tissue tracking.

We propose a novel algorithm for incrementally reconstructing the visual hull in dynamic scenes by exploiting temporal consistency. Using the difference in the silhouette images between two frames we can efficiently locate the scene parts that have to be updated using ray casting. By only concentrating on the parts of the scene that have to be checked for changes, we achieve a significant speed up over traditional methods, which reconstruct each frame independently. Our method does not use any kind of scene model and works regardless of the number and shape of the objects in the scene. We test our algorithm on sequences taken with a multi-camera system and perform a detailed performance analysis showing that our method outperforms other existing methods.

This paper presents a new algorithm for reducing the minimal surface bias associated with volumetric graph cuts for 3D reconstruction from multiple calibrated images. The algorithm is based on an iterative graph-cut over narrow bands combined with an accurate surface normal estimation. At each iteration, we first optimize the normal to each surface patch in order to obtain a precise value for the photometric consistency measure. This helps in preserving narrow protrusions with high curvature which are very sensitive to the choice of normal. We then apply a volumetric graph-cut on a narrow band around the current surface estimate to determine the optimal surface inside this band. Using graph cuts on a narrow band allows us to avoid local minima inside the band while at the same time reducing the danger of taking ”shortcuts ” and converging to a wrong ”global ” minimum when using a wide band. Reconstruction results obtained on standard data sets clearly show the merits of the proposed algorithm. 1

We propose a method that dramatically improves the performance of template-based matching in terms of size of convergence region and computation time. This is done by selecting a subset of the template that verifies the assumption (made during optimization) of linearity or quadraticity with respect to the motion parameters. We call these subsets linear or quadratic subsets. While subset selection approaches have already been proposed, they generally do not attempt to provide linear or quadratic subsets and rely on heuristics such as textured-ness. Because a naive search for the optimal subset would result in a combinatorial explosion for large templates, we propose a simple algorithm that does not aim for the optimal subset but provides a very good linear or quadratic subset at low cost, even for large templates. Simulation results and experiments with real sequences show the superiority of the proposed method compared to existing subset selection approaches.

In this paper we present an efficient and robust real-time system for object contour tracking in image sequences. The developed application partly relies on an optimized implementation of a state-of-the-art curve fitting algorithm, and integrates important additional features in order to achieve robustness while keeping the speed of the main estimation algorithm. An application program has been developed, which requires only a few standard libraries available on most platforms, and runs at video frame rate on a common PC with standard hardware equipment.