Accurate counting of surgical instruments in Operating Rooms (OR) is a critical prerequisite for ensuring patient safety during surgery. Despite recent progress of large visual-language models and agentic AI, accurately counting such instruments remains highly challenging, particularly in dense scenarios where instruments are tightly clustered. To address this problem, we introduce Chain-of-Look, a novel visual reasoning framework that mimics the sequential human counting process by enforcing a structured visual chain, rather than relying on classic object detection which is unordered. This visual chain guides the model to count along a coherent spatial trajectory, improving accuracy in complex scenes. To further enforce the physical plausibility of the visual chain, we introduce the neighboring loss function, which explicitly models the spatial constraints inherent to densely packed surgical instruments. We also present SurgCount-HD, a new dataset comprising 1,464 high-density surgical instrument images. Extensive experiments demonstrate that our method outperforms state-of-the-art approaches for counting (e.g., CountGD, REC) as well as Multimodality Large Language Models (e.g., Qwen, ChatGPT) in the challenging task of dense surgical instrument counting. The code and dataset is available at https://github.com/rishi1134/CoLSR.git

We present RASO, a foundation model designed to Recognize Any Surgical Object, offering robust open-set recognition capabilities across a broad range of surgical procedures and object classes, in both surgical images and videos. RASO leverages a novel weakly-supervised learning framework that generates tag-image-text pairs automatically from large-scale unannotated surgical lecture videos, significantly reducing the need for manual annotations. Our scalable data generation pipeline gathers 2,200 surgical procedures and produces 3.6 million tag annotations across 2,066 unique surgical tags. Our experiments show that RASO achieves improvements of 2.9 mAP, 4.5 mAP, 10.6 mAP, and 7.2 mAP on four standard surgical benchmarks respectively in zero-shot settings, and surpasses state-of-the-art models in supervised surgical action recognition tasks. We will open-source our code, model, and dataset to facilitate further research.

Three-dimensional sensing of changing surgical scenes would improve the function of surgical robots. This paper explores the requirements and utility of a new type of depth sensor, the plenoptic camera, for surgical robots. We present a metric calibration procedure for the plenoptic camera and the registration of its coordinate frame to the robot (hand-eye calibration). We also demonstrate the utility in robotic needle insertion and application of sutures in phantoms. The metric calibration accuracy is reported as 1.14 ± 0.80 mm for the plenoptic camera and 1.57 ± 0.90 mm for hand-eye calibration. The accuracy of needle insertion task is 1.79 ± 0.35 mm for the entire robotic system. Additionally, the accuracy of suture placement with the presented system is reported at 1.80 ± 0.43 mm. Finally, we report consistent suture spacing with only 0.11 mm standard deviation between inter-suture distances. The measured accuracy of less than 2 mm with consistent suture spacing is a promising result to provide repeatable leak-free suturing with a robotic tool and a plenoptic depth imager.

This paper reports an evaluation of contact-less hand tracking sensors for the use of tele-operation, in particular for surgical robotics applications. Two hand tracking systems are investigated: 3Gear Systems interface with the Microsoft KinectTMsensor, and the Leap Motion sensor system. This paper reports an experimental evaluation and comparison of the two systems range, static positioning error, trajectory accuracy of single finger and hand motions, and latency. Latency and trajectory accuracy were found superior using the Leap system. KinectTM/3Gear was found superior when larger range and gesture control are necessary. 3Gear was used in a simulated surgical positioning task and demonstrated an average translational accuracy of 6.2mm. Given the data we have collected, we conclude that neither system, at present, possesses the high level of accuracy and robustness over the required range that would be a prerequisite for use as a medical robotics master.

This paper specifies and evaluates the accuracy of the Smart Tissue Anastomosis Robot (STAR). The STAR is a proof of concept vision-guided robotic system equipped with an actuated laparoscopic suturing tool and a multispectral vision system. The STAR supports image-based suturing commands and is capable of detecting near-infrared fluorescent (NIRF) markers that provide reliable visual segmentation and tracking. The paper reports the best case scenario accuracy specifications of the STAR as derived from its configuration and calibration parameters. We also evaluate experimentally the effects of overlaying NIRF markers on the accuracy of the STAR when these markers are used as the source of image-based commands and we compare these results to the accuracy of the STAR with image-based commands generated from plain color images. Our results demonstrate that the STAR is able to place sutures on a planar phantom with an average accuracy of 0.5 mm with a standard deviation of 0.2 mm and that NIRF markers have no statistically significant adverse effect on the accuracy.