Optical coherence tomography (OCT) has been widely used for high-fidelity biological tissue scanning but is traditionally limited to small lateral fields of view that preclude large-area scanning. To overcome this problem, we propose an integration of an OCT sensor to a 6-DOF robot arm end-effector combined with a geometry-aware stitching model for surface and volumetric data stitching. We firstly develop a simple but efficient Robot-OCT calibration method by using a three-marker calibration pattern and implement an optimization solver. Given a pre-defined trajectory, a local planner is developed to update the sensor pose by using the OCT point cloud information in order to maintain the effective imaging depth based on the distance and orientation constraints. The system calibration method is verified through repeated experiments with the three-marker targets and the result shows an average testing error of$0.132 \pm 0.071 ~\text{mm}$. The geometry-aware OCT stitching framework is demonstrated based on the experiments of different scanning trajectories and 3D-printed phantoms for large-area scanning. The OCT stitched point cloud is compared with the ground truth from the phantom CAD model and the result show an average surface alignment error of$0.441 \pm 0.241 ~\text{mm}$for the path following tasks.

In microsurgery, surgeons frequently encounter challenges due to the need for exceptional precision and dexterity, the lack of depth perception for micro-scale surgical maneuvers, and the inevitable effects of fatigue and hand tremor. In surgical robotics, conventional intraoperative perception systems normally provide real-time image feedback, but depth and volumetric information is typically lacking. To overcome these challenges, we propose a teleoperated robotic system with two arms to provide high-fidelity intraoperative volumetric imaging during micro-scale tissue manipulation. This system incorporates an optical coherence tomography sensor for real-time 3D visualization and a dual-arm teleoperated robot system controlled by haptic input devices for accurate and precise manipulation. We characterize the system's performance through a precision positioning task and a vessel following task in a retinal model, which shows average positioning errors of approximately 232μm and 83 μm, respectively. We demonstrate the fully integrated system through the completion of an eggshell membrane peeling task that simulates retinal membrane peeling.

In robotic laser surgery, shape prediction of an one-shot ablation crater is an important problem for minimizing errant overcutting of healthy tissue during the course of pathological tissue resection and precise tumor removal. Since it is difficult to physically model the laser-tissue interaction due to the variety of optical tissue properties, complicated process of heat transfer, and uncertainty about the chemical reaction, we propose a 3D crater prediction model based on an entirely data-driven method without any assumptions of laser settings and tissue properties. Based on the crater prediction model, we formulate a novel robotic laser planning problem to determine the optimal laser incident configuration, which aims to create a crater that aligns with the surface target (e.g. tumor, pathological tissue). To solve the one-shot ablation crater prediction problem, we model the 3D geometric relation between the tissue surface and the laser energy profile as a non-linear regression problem that can be represented by a single-layer perceptron (SLP) network. The SLP network is encoded in a novel kinematic model to predict the shape of the post-ablation crater with an arbitrary laser input. To estimate the SLP network parameters, we formulate a dataset of one-shot laser-phantom craters reconstructed by the optical coherence tomography (OCT) B-scan images. To verify the method. The learned crater prediction model is applied to solve a simplified robotic laser planning problem modelled as a surface alignment error minimization problem. The initial results report about$(91.2\pm 3.0)\%$3D-crater-Intersection-over-Union (3D-crater-IoU) for the 3D crater prediction and an average of about 98.0% success rate for the simulated surface alignment experiments.

In this paper, we present an autonomous RGB-D and 2D ultrasound-guided robotic system for collecting 3D localized volumes of peripheral vessels. This compact design, with available commercial components, lends itself to platform utility throughout the human body. The fully integrated system works with force limits for future safety in human use. We propose a PID force controller for smooth and safe robot scanning following a priori 3D trajectory generated from a surface point cloud. System calibration is implemented to determine transformations among sensors, end-effector and robot base. A vascular localization pipeline that consists of detection and tracking is proposed to find the 3D vessel positions in real-time. Precision tests are performed with both predesignated and autonomously selected areas in an arm phantom. The average variance of the autonomously collected ultrasound images (to construct 3D volumes) between repeated tests is shown to be around 0.3 mm, similar to the theoretical spatial resolution a clinical ultrasound system. This fully integrated system demonstrates the capability of autonomous collection of peripheral vessels with built-in safety measures for future human testing.

This paper proposes a stereovision-guided robotic laser system that can conduct laser ablation on targets selected by human operators in the color image, referred as StereoCNC. Two digital cameras are integrated into a previously developed robotic laser system to add a color sensing modality and formulate the stereovision. A calibration method is implemented to register the coordinate frames between stereo cameras and the laser system, modelled as a 3D-to-3D Least-squares problem. This problem is solved by a RANSAC-based 3D rigid transformation method and the calibration reprojection errors are used to characterize a 3D error field by Gaussian process regression. This regression error model is used to predict an error value for each data point of a stereo-reconstructed point cloud and an optimization problem is formulated to adjust the surgical site to a new position with minimum reprojection errors. Based on the calibrated system and the error model, a stereovision-guided laser-tissue removal pipeline is proposed to precisely locate, target, and ablate a surface region. The pipeline is validated by the experiments on phantoms with color texture and various geometric shapes. The overall targeting accuracy of the system achieves an average RMSE of 0.13±0.02 mm and maximum error of 0.34±0.06 mm, as measured by pre- and post-laser ablation images. The results show potential applications of using the developed stereovision-guided robotic system for superficial laser surgery, including dermatologic applications or removal of exposed tumorous tissue in neurosurgery.

Laser scalpels are utilized across a variety of surgical and dermatological procedures due to their precision and non-contact nature. This paper presents a novel laser scalpel system for superficial laser therapy applications. The system integrates a RGB-D camera, a 3D triangulation sensor and a carbon dioxide (CO2) laser scalpel for computer-assisted laser surgery. To accurately ablate targets chosen from the color image, a 3D extrinsic calibration method between the RGB-D camera frame and the laser coordinate system is implemented. The accuracy of the calibration method is tested on phantoms with planar and cylindrical surfaces. Positive error and negative error, as defined as undershooting and overshooting over the target area, are reported for each test. For 60 total test cases, the root-mean-square of the positive and negative error in both planar and cylindrical phantoms is less than 1.0 mm, with a maximum absolute error less than 2.0 mm. This work demonstrates the feasibility of automated laser therapy with surgeon oversight via our sensor system.