Knot tying is a fundamental dexterous surgical subtask that is a key step in suturing. One challenge to robot augmentation is limited depth perception due to the small baseline of surgical endoscopic cameras. In this work, we present Surgical D-Knot: an augmented dexterity pipeline combining learned perception with model-based methods to perform surgical double knots using only one monocular RGB camera. This pipeline includes 2D grasp point identification, 3D suture thread grasping using local feature servoing, suture thread wrapping using relative motion and 2D re-grasp point identification. Human dexterity is required for initial thread setup and thread cutting after each double knot. Physical experiments with 120 double knot trials result in a success rate of 80.83% for the initial knot and 55.83% for the second knot. Translation of surgical knot tying to chicken skin results in success rates of 73.75% for the initial knot and 40% for the second knot. Each double knot requires on average 70 seconds. This is the first work to our knowledge that augments human dexterity for double knot tying. https://sites.google.com/view/surgicaldknot/

Vascular shunt insertion is a fundamental surgical procedure used to temporarily restore blood flow to tissues. It is often performed in the field after major trauma. We formulate a problem of automated vascular shunt insertion and propose a pipeline to perform Automated Vascular Shunt Insertion (AVSI) using a da Vinci Research Kit. The pipeline uses a learned visual model to estimate the locus of the vessel rim, plans a grasp on the rim, and moves to grasp at that point. The first robot gripper then pulls the rim to stretch open the vessel with a dilation motion. The second robot gripper then proceeds to insert a shunt into the vessel phantom (a model of the blood vessel) with a chamfer tilt followed by a screw motion. Results suggest that AVSI achieves a high success rate even with tight tolerances and varying vessel orientations up to 30°. Supplementary material, dataset, videos, and visualizations can be found at https://sites.google.com/berkeley.edu/autolab-avsi.

Peg transfer is a well-known surgical training task in the Fundamentals of Laparoscopic Surgery (FLS). While human surgeons teleoperate robots such as the da Vinci to perform this task with high speed and accuracy, it is challenging to automate. This paper presents a novel system and control method using a da Vinci Research Kit (dVRK) surgical robot and a Zivid depth sensor, and a human subjects study comparing performance on three variants of the peg-transfer task: unilateral, bilateral without handovers, and bilateral with handovers. The system combines 3D printing, depth sensing, and deep learning for calibration with a new analytic inverse kinematics model and time-minimized motion controller. In a controlled study of 3384 peg transfer trials performed by the system, an expert surgical resident, and 9 volunteers, results suggest that the system achieves accuracy on par with the experienced surgical resident and is significantly faster and more consistent than the surgical resident and volunteers. The system also exhibits the highest consistency and lowest collision rate. To our knowledge, this is the first autonomous system to achieve “superhuman” performance on a standardized surgical task. All data is available athttps://sites.google.com/view/surgicalpegtransferNote to Practitioners—This paper presents a new approach to calibrating cable-driven robots based on a combination of 3D printing, depth sensing, inverse kinematics, convex optimization, and deep learning. The approach is applied to calibrating the da Vinci, commercial surgical-assist robot, to automate a standard “pick and place” task. Experiments suggest that the resulting system matches human surgical expert performance in speed and accuracy and significantly outperforms humans in terms of consistency. All details on the system including CAD models, code, and user study data are available online.

Robotic Surgical Assistants (RSAs) are commonly used to perform minimally invasive surgeries by expert surgeons. However, long procedures filled with tedious and repetitive tasks such as suturing can lead to surgeon fatigue, motivating the automation of suturing. As visual tracking of a thin reflective needle is extremely challenging, prior work has modified the needle with nonreflective contrasting paint. As a step towards automation of a suturing subtask without modifying the needle, we propose HOUSTON: Handover of Unmodified, Surgical, Tool-Obstructed Needles, a problem and algorithm that uses a learned active sensing policy with a stereo camera to iteratively localize and align the needle into a visible and accessible pose for the other gripper. To compensate for robot positioning and needle perception errors, the algorithm then executes a high-precision grasping motion that uses multiple cameras. Physical experiments with the da Vinci Research Kit (dVRK) suggest a success rate of 96.7% on needles used in training, and 75 - 92.9% on needles unseen in training. On sequential handovers, HOUSTON successfully executes 32.4 handovers on average before failure. To our knowledge, this work is the first to study handover of unmodified surgical needles. See https: / /tinyurl. com/houston-surgery for additional materials including details about offline datasets and model architectures.

Assisting surgeons with automation of surgical subtasks is challenging due to backlash, hysteresis, and variable tensioning in cable-driven robots. These issues are exacerbated as surgical instruments are changed during an operation. In this work, we propose a framework for automation of high- precision surgical subtasks by learning local, sample-efficient, accurate, closed-loop policies that use visual feedback instead of robot encoder estimates. This framework, which we call deep Intermittent Visual Servoing (IVS), switches to a learned visual servo policy for high-precision segments of repetitive surgical tasks while relying on a coarse open-loop policy for the segments where precision is not necessary. We train the policy using only 180 human demonstrations that are roughly 2 seconds each. Results on a da Vinci Research Kit suggest that combining the coarse policy with half a second of corrections from the learned policy during each high-precision segment improves the success rate on the Fundamentals of Laparoscopic Surgery peg transfer task from 72.9% to 99.2%, 31.3% to 99.2%, and 47.2% to 100.0% for 3 instruments with differing cable properties. In the contexts we studied, IVS attains the highest published success rates for automated surgical peg transfer and is significantly more reliable than previous techniques when instruments are changed. Supplementary material is available at https://tinyurl.com/ivs-icra.