Accurate visualization of subtle vascular dynamics is a knowledge-intensive challenge in minimally invasive surgery. Conventional imaging systems struggle to reveal these imperceptible motions amidst the dynamic complexity of surgical scenes, limiting decision-making reliability. We introduce EndoControlMag , a Lagrangian framework that employs mask-conditioned magnification to selectively enhance vascular motion while preserving the structural integrity of surrounding tissues in endoscopic videos. Our approach integrates two key designs: Periodic Reference Resetting (PRR) , which divides videos into short overlapping clips with dynamically updated reference frames to alleviate error accumulation while maintaining temporal coherence, and Hierarchical Tissue-aware Magnification (HTM) , which combines pretrained visual tracking for accurate vessel localization with dual-mode adaptive softening strategies. HTM employs either motion-based softening that modulates magnification strength proportional to observed tissue displacement, or distance-based exponential decay that simulates biomechanical force attenuation. This strategy enables robust performance across diverse surgical scenarios where motion-based softening excels with complex tissue deformations and distance-based softening provides stability under unreliable optical flow conditions. To validate generality and scalability, we construct EndoVMM24, a benchmark dataset spanning four surgical specialties and diverse intraoperative scenarios. Extensive quantitative metrics, qualitative assessments, and expert surgeon evaluations demonstrate that EndoControlMag significantly outperforms existing methods in magnification accuracy, image quality, and robustness. This work advances engineering informatics for surgical vision by providing a reproducible, context-aware framework that supports reliable decision-making in minimally invasive procedures. The code, dataset, and video results are available at https://cho-haz.github.io/EndoControlMag/ .

Open tracheostomy (OT) is considered the traditional way and golden standard for treating airway obstruction patients. However, OT has many unavoidable drawbacks, including strict performing scenarios, significant scarring, and the risk of surgeon infection. Percutaneous dilation tracheostomy (PDT) emerges, with advantages including a lower cost, smaller scarring, and better protection of surgeons from inflecting by aerosol. However, the outside-in puncture manner of PDT has a risk of piercing the post-tracheal wall and the esophagus with uncontrolled force. Additionally, locating tracheal rings and determining the puncture site externally can be challenging for certain patients, such as those who are obese or have undergone neck surgery, while this procedure typically relies on palpation and the surgeon's expertise. Hence, to improve the safety and simplicity of tracheostomy, a minimally-invasive endotracheal inside-out flexible needle-driving system towards microendoscope-guided robotic tracheostomy (MERT) has been proposed in this paper. Guided by an optical coherence tomography (OCT) probe and a microendoscope, the robot inserts into the trachea and performs an inside-out puncture using a flexible needle. The robot can work through a standard endotracheal tube (ETT), and the puncture direction of the flexible needle is variable. Kinematics and statics models of the flexible needle have been derived, and the minimum position errors generated in the kinematics and statics validation experiments are$0.57 \pm 0.21 \mathbf{~ m m}$and$0.27 \pm 0.21 \mathbf{~ m m}$. Finally, a porcine trachea puncture experiment is carried out, and the feasibility of the proposed system is verified.

Tracheotomy is commonly performed for patients needing prolonged intubation, airway obstruction, and neck injuries. Accurate placement of the incision and the tracheal window is paramount in order to avoid complications. Current surgical technique heavily relies on palpating cartilage landmarks on the neck to place the incision. In order to achieve the accelerated goals of the robot-assisted subtask in a tracheotomy, this paper proposes a novel autonomous palpation-based acquisition strategy - RASEC in the tracheal region, which can interactively predict the next acquisition point to maximize the expected information and minimize the costs of palpation procedure. We employ a Gaussian Process (GP) to model the distribution of hardness and utilize anatomical information as a priori input to guide the point of palpation for medical robots. The dynamic tactile sensor based on the resonant frequency is introduced to measure tissue hardness in the tracheal region by millimeter-scale contact to secure the interaction. We investigate the kernel fusion method to blend the Squared Exponential (SE) kernel with the Ornstein-Uhlenbeck (OU) kernel and optimize the Bayesian optimization search by leveraging the anatomical information of the larynx as a priori knowledge. Moreover, we further regularize the exploration and greed factors. The tactile sensor’s moving distance and the robotic base link’s rotation angle during the incision localization process are considered new factors in the acquisition strategy. Simulation and physical phantom experiments are conducted for comparison with state-of-the-art GP-based exploration approaches. The results show that the sensor’s moving distance was reduced by 53.1% and the rotation angle of the base was reduced by 75.2% of the previous values without sacrificing overall performance capabilities. The satisfying algorithmic index (average precision 0.932, average recall 0.973, average F1 score 0.952) with fewer central estimation distance errors (0.423 mm) and high resolution (1 mm) indicates the performance of the proposed RASEC in terms of exploration efficiency, cost awareness, and localization accuracy for incision localization and recommendation in real robot-assisted subtask in the tracheotomy procedure.Note to Practitioners—This work is well motivated to introduce the Level of Autonomy (LoA) 2 - task-level autonomy, specifically in the context of tracheotomy procedures. The incorporation of robotic palpation techniques aims to provide surgeons with enhanced capabilities for incision recommendations, which directly benefit surgeons to visualize hands-on information and localize the trachea regions more efficiently and further reduce cognitive load. To detect the trachea region for intubation incision without costly ergodic acquisition, this article suggests a highly efficient acquisition strategy utilizing the fusion kernel function and regularized impact factors, eliminating the time consumption for such localization task. The actual clinical value is that our proposed strategy can earn more time for further increasing the probability of patient resuscitation, to facilitate supervised autonomy in the real clinic scene.

Needle insertion is a critical component of many remote surgical procedures, including biopsies, injections, neurosurgery, and brachytherapy cancer treatments. However, precise visualization of the biopsy needle trajectory remains challenging due to specular reflection, speckle noise, and needle-like anatomical features. This paper proposes a visual feedback prediction framework for ultrasound-assisted percutaneous kidney biopsy in 5G remote surgery, aiming to enhance operator confidence, reduce procedure time, and minimize the risk of unintended bleeding. Building upon this framework, we design a Lightweight-Accuracy Needle Trajectory Prediction (LA-NTP) model by minimizing the backbone and optimizing the multi-module prediction process, incorporating innovative training strategies (i.e., angle-aware geometric and trajectory augmentation losses). The experimental results demonstrate that it achieves competitive performance with only 20.3% of the model size of the previous best real-time method and a 3.7-fold increase in inference speed. Even in challenging scenarios involving large insertion depths and steep angles, our method provides stable and precise navigation.

Generating a surgical report in robot-assisted surgery, in the form of natural language expression of surgical scene understanding, can play a significant role in document entry tasks, surgical training, and post-operative analysis. Despite the state-of-the-art accuracy of the deep learning algorithm, the deployment performance often drops when applied to the Target Domain (TD) data. For this purpose, we develop a multi-layer transformer-based model with the gradient reversal adversarial learning to generate a caption for the multi-domain surgical images that can describe the semantic relationship between instruments and surgical Region of Interest (ROI). In the gradient reversal adversarial learning scheme, the gradient multiplies with a negative constant and updates adversarially in backward propagation, discriminating between the source and target domains and emerging domain-invariant features. We also investigate model calibration with label smoothing technique and the effect of a well-calibrated model for the penultimate layer’s feature representation and Domain Adaptation (DA). We annotate two robotic surgery datasets of MICCAI robotic scene segmentation and Transoral Robotic Surgery (TORS) with the captions of procedures and empirically show that our proposed method improves the performance in both source and target domain surgical reports generation in the manners of unsupervised, zero-shot, one-shot, and few-shot learning.

This paper presents a sensor fusion-based surgical instrument tracking system which uses multiple monocular modules. The system is an optical tracking system, which has been widely utilized in the image-guide surgery because of its high accuracy and precision. However, the line-of-sight occlusion problem which remains unresolved in current systems frustrates surgeons during the operation. To address this challenge, we propose a surgical instrument tracking system based on multiple monocular modules. The rationale is to enable the system to track the surgical instruments inside the surgical site from different views. Three sensor fusion algorithms are proposed to integrate all sensor data from the multimodule system. In order to show the feasibility of the tracking system, simulations and comparison experiments have been carried out. The intensive investigation results give a practical instruction to the real implementation of the proposed system in image-guided interventions. Moreover, an image-guided surgical trial by using a cadaver head has been carried out to validate the feasibility of the proposed system and the tracking algorithms. The results from both the simulation and the cadaver trial have shown the effectiveness of the proposed robust fusion algorithm.

Robot-assisted single port access surgery (SPAS) brings significant benefits to the patients. However, it is challenging due to the limited incision size and requirements in precision, load capacity, and dexterity. To address these challenges, this paper presents a multichannel SPAS robotic system consisting of two channels of 5-degree-of-freedom (DOF) flexible manipulators and an endoscope channel. Each channel of the manipulators is designed based on a 3-DOF parallel mechanism with three sets of super-elastic Ni-Ti rods and universal joints. This design optimizes the structure for flexibility and safety considerations with a simplified structure compared with conventional parallel mechanisms. Flexible shafts are used for torque transmission from the actuation unit, allowing the motors to be farther away from the patient-side robotic system. The kinematics of the manipulator is derived, and then the reachable workspace and dexterity are analyzed. Furthermore, a prototype of the proposed robotic system is presented and evaluated through adequate experiments. The flexibility of the manipulator is verified via stiffness characterization test. The results of the accuracy tests confirmed that the robotic system can implement manipulation arm with acceptable accuracy. The feasibility and effectiveness of applying the robotic system to the practical operation are also demonstrated through experimentations.

Mandibular mobility plays a significant role in human daily lives, enabling food intake, respiration, speaking, and other oral activities. However, as for the patients suffering from mandibular mobility disorders, their mandible functions are deteriorated, which severely affects their quality of life. In this paper, we present a new solution to recover mandibular mobility: a soft oral rehabilitation robot (SORR), which is actuated by a novel type of soft pneumatic actuator (SPA). After identifying the biometrics of the human mandible, we illustrate the application-oriented design of the robot and the SPA. A static model of the SPA was established to predict its behavior and eligibility for the application. In the experimental characterization, we measured the elongation and force output of the SPA, and the detailed results are presented. The comparison and analysis of the results provide physical insight into the mechanisms of the SPA. As the first application of soft robot inside human body, this work enlightens profound application potentials of the silicone-based SPAs.