Endoscopic reconstruction plays a crucial role in surgical robotics. The dynamic lighting conditions and integrated camera-light source in endoscopic scenes create a distinct reconstruction challenge: shape ambiguity. To mitigate this, we propose a Gaussian Splatting (GS) based framework for endoscopic scene reconstruction, enhanced with reflectance regularization. We embed every 3D Gaussian point with physical reflective attributes and combine this representation with a physically based inverse rendering framework. By jointly training 3DGS for view synthesis with this reflectance regularization, we are able to attain high-quality geometry without changing the volume rendering pipeline. Our experiments demonstrate the superiority in both geometry representation and rendering performance compared to existing GS approaches, making it a practical solution for endoscopic applications. Project is available at: https://med-air.github.io/GSR2.

Nasotracheal intubation (NTI) is critical for establishing artificial airways in clinical anesthesia and critical care. Current manual methods face significant challenges, including cross-infection, especially during respiratory infection care, and insufficient control of endoluminal contact forces, increasing the risk of mucosal injuries. While existing studies have focused on automated endoscopic insertion, the automation of NTI remains unexplored despite its unique challenges: Nasotracheal tubes exhibit greater diameter and rigidity than standard endoscopes, substantially increasing insertion complexity and patient risks. We propose a novel autonomous NTI system with two key components to address these challenges. First, an autonomous NTI system is developed, incorporating a prosthesis embedded with force sensors, allowing for safety assessment and data filtering. Then, the Recurrent Action-Confidence Chunking with Transformer (RACCT) model is developed to handle complex tube-tissue interactions and partial visual observations. Experimental results demonstrate that the RACCT model outperforms the ACT model in all aspects and achieves a 66% reduction in average peak insertion force compared to manual operations while maintaining equivalent success rates. This validates the system’s potential for reducing infection risks and improving procedural safety.

To steer a soft robot precisely in an unconstructed environment with minimal collision remains an open challenge for soft robots. When the environments are unknown, prior motion planning for navigation may not always be available. This paper presents a novel Sim-to-Real method to guide a cable-driven soft robot in a static environment under the Simulation Open Framework Architecture (SOFA). The scenario aims to resemble one of the steps during a simplified transoral tracheal intubation process where a robotic endotracheal tube is guided to the upper trachea-larynx location by a flexible video-assisted endoscope/stylet. In SOFA, we employ the quadratic programming inverse solver to obtain collision-free motion strategies for the endoscope/stylet manipulation based on the robot model and encode the virtual eye-in-hand vision. Then, we associate the anatomical features recognized by the virtual vision and the joint space motion using a closed-loop nonlinear autoregressive exogenous model (NARX) network. Afterward, we transfer the learned knowledge to the robot prototype, expecting it to navigate to the desired spot in a new phantom environment automatically based on its eye-in-hand vision only. Experiment results indicate that our soft robot can efficaciously navigate through the unstructured phantom to the desired spot with minimal collision motion according to what it has learned from the virtual environment. The results show that the average R-squared coefficient between the closed-loop NARX-forecasted and SOFA-referenced robot's cable and prismatic joint space motion are 0.963 and 0.997, respectively. The eye-in-hand visions also demonstrate good alignment between the robot tip and the glottis.

The rigid and straight nature of conventional surgical drills and screwdrivers makes it difficult to access the posterior mandible for fracture reduction without the creation of facial incisions. To assist transoral mandibular fracture reduction in hard-to-reach areas, we propose a shared-control dexterous robotic system. The end effector of this system is an articulated drilling/screwing tool to provide distal dexterity. This system uses an admittance-control-based approach to provide precision and stability during shared-control hole-drilling processes. A cadaver study showed the efficacy of the proposed system to assist plate fixation in the reduction of mandibular fractures. The proposed articulated surgical tool was capable of drilling holes in and driving screws into the mandible of a cadaver head. In addition, the shared-control robotic system ensured that the drill moved along its axial direction, leading to stable and precise hole drilling.

Soft robots have great potential in surgical applications due to their compliance and adaptability to their environment. However, their flexibility and nonlinearity bring challenges for precise modeling, sensing, and control, especially in constrained cavities. In this article, a simple, compact two-segment soft robot for flexible laser ablation is proposed. The proximal hydraulic-driven segment can offer omnidirectional bending so as to navigate toward lesions. The distal segment driven by tendons enables precise, fast steering of laser collimator for laser sweeping on lesion targets. The dynamics of such mechanical steering motion can be enhanced with a metal spring backbone integrated along the collimator, thus facilitating the control with certain linearity and responsiveness. A soft robot modeling and control scheme based on Koopman operators is proposed. We also design a disturbance observer so as to incorporate the controller feedback with real-time fiber optic shape sensing. Experimental validation is conducted on simulated orex-vivolaser ablation tasks, thus evaluating our control strategies in laser path following across various contours/patterns. As a result, such a simple compact laser manipulation can perform up to 6 Hz sweeping with precision of path following errors below 1 mm. Such modeling and control scheme could also be used on an endoscopic laser ablation robot with unsymmetric mechanism driven by two tendons.