Magnetic compression anastomosis (MCA) offers a promising solution for minimally invasive anastomosis surgery. However, current MCA schemes lack safe, real-time localization and guidance for compression magnets, hindering surgeons' ability to control the compression magnets effectively in complex, unstructured endoluminal environments. To address these limitations, this paper introduces the MagsL-HUD endoscopic system, a novel solution that enables multi-magnetic six-degree-of-freedom (six-DoF) localization and head-up display (HUD) guidance within the endoscopic view (EV). Specifically, the system integrates a developed Endo-MagCap device with an orthogonal magnet configuration, along with a magnetic sensor array, to achieve real-time full-pose localization. An endoscopic camera model is incorporated for HUD visualization, enhancing intuitive interaction for surgeons' better-informed decisions. Eventually, the effectiveness of the MagsL-HUD endoscopic system is validated through laboratory experiments andex vivoanimal trials. The system demonstrates six-DoF tracking accuracy with average errors of 0.0070 m and 0.1437 rad, and 0.0071 m and 0.1721 rad in the designed trajectory cases for two compression magnets, respectively. Additionally,ex vivoporcine tests confirm the system's feasibility and applicability, successfully performing a stomach-colon MCA surgery with a final compression gap of approximately 0.00247 m. Further comparative studies demonstrate that the MagsL-HUD method has a compression success rate of 71.4$\%$versus 42.9$\%$of the non-HUD approaches in the designed tests. This work represents a significant step toward the clinical adoption of magnetic-assisted endoscopy for minimally invasive anastomosis surgeries, holding substantial practical significance for improving the safety and efficacy of MCA procedures in complex, unstructured endoluminal environments.

Endoscopic submucosal dissection (ESD) is a procedure targeted for early gastrointestinal cancer. Traction plays a crucial role in enhancing the efficiency of cutting lesions, thereby reducing procedural complexity and duration. From the perspective of traction devices, current non-magnetic ones hold shortcomings in complicating the workspace in directional tissue manipulation; Current magnetic traction devices cannot be prepared before the procedure, and require the withdrawal of endoscope in the midway to re-introduce the magnetic retractor to the lesion site. Towards these plights, this paper introduces a robotic flexible magnetic retractor designed for tissue manipulation during ESD. Precisely, the flexible prototype can be seamlessly inserted through the instrument channel of an endoscope to the lesion site without the need for endoscope withdrawal. Moreover, the introduction of robotic magnetic actuation enhances the agile control of magnetic retractors while alleviating the surgeon's workload in magnetic-retractor-assisted ESD. The experimental results validate the functionality and efficacy of the prototype magnetic retractor in magnetic traction-assisted ESD procedures. The retractor demonstrated its ability to provide adequate traction and accomplish clinical tasks. This innovative approach holds promise for enhancing the efficiency and outcomes of ESD procedures, offering a compelling alternative to traditional traction methods.

In robotic teleoperation systems, the effectiveness of teleoperation relies heavily on the operator performing control actions, underscoring the importance of master device design. Breaking away from the mechanical framework that relies on complex structures and restricts workspace, this paper adopts a new magnetic method and introduces a wireless magnetic master device (MMD) with an expandable, singularity-free workspace, designed to enhance maneuverability and operational reach. Specifically, the concept and development of the MMD for robotic teleoperation are presented, followed by an exploration of key technical aspects, including a six-degree-of-freedom (DoF) tracking approach using dual magnetic localization and controller design for the robotic arm using the MMD. Finally, the proposed method’s feasibility and accuracy are validated through magnetic localization and teleoperation control tests. Results indicate that the MMD achieves six-DoF localization with an average Euclidean localization error ofet= 0.0018 m and an average Geodesic distance error ofeR= 0.0604 rad within the central region of the workspace, and six-DoF manipulation (velocityet= 0.0062 m andeR=0.2091 rad in tests. Furthermore, the overall teleoperation control framework achieves tracking accuracy withet= 0.0085 m andeR= 0.1475 rad in robotic calligraphy task; achieves a broad scanning range of 0.2746 m along they-axis in robotic ultrasound scanning task; achieves a large-angle rotation of 9.15 rad in robotic clamp rotating task. Overall, this research advances master device design, aiming to inspire further innovation in master-follower teleoperation control for human-machine systems.

Uncertainty estimation plays an important role for future reliable deployment of deep segmentation models in safety-critical scenarios such as medical applications. However, existing methods for uncertainty estimation have been limited by the lack of explicit guidance for calibrating the prediction risk and model confidence. In this work, we propose a novel fine-grained reward maximization (FGRM) framework, to address uncertainty estimation by directly utilizing an uncertainty metric related reward function with a reinforcement learning based model tuning algorithm. This would benefit the model uncertainty estimation with direct optimization guidance for model calibration. Specifically, our method designs a new uncertainty estimation reward function using the calibration metric, which is maximized to fine-tune an evidential learning pre-trained segmentation model for calibrating prediction risk. Importantly, we innovate an effective fine-grained parameter update scheme, which imposes fine-grained reward-weighting of each network parameter according to the parameter importance quantified by the fisher information matrix. To the best of our knowledge, this is the first work exploring reward optimization for model uncertainty estimation in safety-critical vision tasks. The effectiveness of our method is demonstrated on two large safety-critical surgical scene segmentation datasets under two different uncertainty estimation settings. With real-time one forward pass at inference, our method outperforms state-of-the-art methods by a clear margin on all the calibration metrics of uncertainty estimation, while maintaining a high task accuracy for the segmentation results. Code is available at https://github.com/med-air/FGRM.