Magnetic compression anastomosis (MCA) offers a promising solution for minimally invasive anastomosis surgery. However, current MCA schemes lack safe, real-time localization and guidance for compression magnets, hindering surgeons' ability to control the compression magnets effectively in complex, unstructured endoluminal environments. To address these limitations, this paper introduces the MagsL-HUD endoscopic system, a novel solution that enables multi-magnetic six-degree-of-freedom (six-DoF) localization and head-up display (HUD) guidance within the endoscopic view (EV). Specifically, the system integrates a developed Endo-MagCap device with an orthogonal magnet configuration, along with a magnetic sensor array, to achieve real-time full-pose localization. An endoscopic camera model is incorporated for HUD visualization, enhancing intuitive interaction for surgeons' better-informed decisions. Eventually, the effectiveness of the MagsL-HUD endoscopic system is validated through laboratory experiments andex vivoanimal trials. The system demonstrates six-DoF tracking accuracy with average errors of 0.0070 m and 0.1437 rad, and 0.0071 m and 0.1721 rad in the designed trajectory cases for two compression magnets, respectively. Additionally,ex vivoporcine tests confirm the system's feasibility and applicability, successfully performing a stomach-colon MCA surgery with a final compression gap of approximately 0.00247 m. Further comparative studies demonstrate that the MagsL-HUD method has a compression success rate of 71.4$\%$versus 42.9$\%$of the non-HUD approaches in the designed tests. This work represents a significant step toward the clinical adoption of magnetic-assisted endoscopy for minimally invasive anastomosis surgeries, holding substantial practical significance for improving the safety and efficacy of MCA procedures in complex, unstructured endoluminal environments.

The remote center of motion (RCM) constraint is a vital requirement in the design of robotic systems for transrectal ultrasound (TRUS) probe-scanning. This paper presents the design and development of a novel RCM-constrained manipulator specifically tailored for TRUS probe-scanning applications. The proposed system features a six-degree-of-freedom (6-DoF) parallel-serial hybrid mechanism that enables the TRUS probe to perform pivot and spin rotations while maintaining the RCM constraint. Subsequently, the kinematic model incorporating the RCM constraint is derived. Additionally, a geometry-aware path planning method is then introduced, considering variations in the desired rotation targets. This method parameterizes distance metrics on SO(3) (a Lie group) using coordinate-free Riemannian geometry, enabling the dynamic optimization of rotation orders to minimize the calculated Riemannian metrics. Furthermore, a smooth rotational trajectory generation method is proposed, constructing rotation curves between the ordered matrices on SO(3) while minimizing angular acceleration. Both simulations and experimental results validate the effectiveness and practicality of the proposed manipulator and its path planning method.

Despite advantages from neurosurgical systems, achieving intuitive and safe collaboration with robot during the coarse positioning of instrument insertion end effector (IIEE) remains a critical issue. In this paper, we propose a novel non-contact hand-guided method for such advancement based on magnetic sensing. First, a wearable magnet band and a magnetic sensor are designed, based on which the magnetic localization is achieved for surgeon’s hand location detection. Second, a quadratic programming-based control is implemented, to guarantee the pose-based servo performance, higher rotational manipulability for IIEE fine alignment, and joint position&velocity limits avoidance. For evaluation, two experiments are designated and conducted. Results show that the magnetic localization algorithm can achieve < 4.7 mm and 2.6° errors in a dynamic path tracking test, which can provide an accurate magnet location for hand guidance. Moveover, workflow of the proposed solution in a brain biopsy scenario demonstrates its enhancement of IIEE rotational manipulability (11.6% increase at final configuration), and safety improvement of collision avoidance when other surgeon approaches for cannula delivery. This research contributes to enhanced intuitiveness and safety for surgeon-robot collaborative coarse positioning of IIEE in neurosurgery.

In robotic teleoperation systems, the effectiveness of teleoperation relies heavily on the operator performing control actions, underscoring the importance of master device design. Breaking away from the mechanical framework that relies on complex structures and restricts workspace, this paper adopts a new magnetic method and introduces a wireless magnetic master device (MMD) with an expandable, singularity-free workspace, designed to enhance maneuverability and operational reach. Specifically, the concept and development of the MMD for robotic teleoperation are presented, followed by an exploration of key technical aspects, including a six-degree-of-freedom (DoF) tracking approach using dual magnetic localization and controller design for the robotic arm using the MMD. Finally, the proposed method’s feasibility and accuracy are validated through magnetic localization and teleoperation control tests. Results indicate that the MMD achieves six-DoF localization with an average Euclidean localization error ofet= 0.0018 m and an average Geodesic distance error ofeR= 0.0604 rad within the central region of the workspace, and six-DoF manipulation (velocityet= 0.0062 m andeR=0.2091 rad in tests. Furthermore, the overall teleoperation control framework achieves tracking accuracy withet= 0.0085 m andeR= 0.1475 rad in robotic calligraphy task; achieves a broad scanning range of 0.2746 m along they-axis in robotic ultrasound scanning task; achieves a large-angle rotation of 9.15 rad in robotic clamp rotating task. Overall, this research advances master device design, aiming to inspire further innovation in master-follower teleoperation control for human-machine systems.

Kinematic calibration is crucial to improve the positioning accuracy of serial robots. This paper proposes a novel algorithm for robotic kinematic calibration based on an augmented product of exponentials (POE)-based kinematic model using Gaussian mixture models (GMMs) with only position data. In this algorithm, non-geometric errors that cannot be fitted by varying the parameters within the traditional robot model are also considered and compensated. This approach involving a three-stage calibration process which is used to identify the kinematic model parameters and to train the GMMs will be presented in this paper. Finally, this algorithm will be applied to two serial robots for simulation and experimental validation. The effectiveness of the proposed algorithm is verified from both results and significant improvement on error reduction from 26 % to 96% can be observed through the comparison with other existing approaches.

This paper is concerned with the issue of the kinematic model-based bending control for the magnetically actuated robotic endoscope and its application for automatic retroflexion. By the utilization of the Cosserat rod theory and the transformation in the magnetic tip of the endoscope, the comprehensive kinematic model of the magnetically-actuated robotic endoscope is established. Afterward, a magnetic control scheme for the bending motion is proposed by co-developing an error feedback PID control strategy and the model-based feedback approach. Moreover, as one unique kind of bending motion, retroflexion is taken into account, and the strategy aimed at the bid of compact space retroflexion is presented by virtue of the introduction of serial waypoints pursuing the position of the magnetic tip being close to the midline as possible. Eventually, the developed modeling and bending control scheme and the compact space retroflexion strategy are examined in a magnetically actuated robotic endoscope system to manifest the effectiveness and applicability of the theoretical approach. The experimental results indicate that the designed controller can drive the endoscope to bend to the desired pose and show a reduction of about 47.01% in the sweeping area and 79.25% in the last distance to midline achieved by conducting compact space retroflexion in comparison to “U” type one.

Colorectal cancer incidence has been steadily rising worldwide. Magnetic colonoscopes provide new approaches to conduct colon inspection and treatment. This paper presents a novel electromagnetically actuated soft-tethered colonoscope to achieve precise and stable orientation control. An inflated balloon is designed to eliminate the unpredictable disturbance of the floating tether. A 2OR Pseudo-Rigid-Body (PRB) model of the soft tether is developed to analyze the relationship between the tether deflection and applied force and torque. A closed-loop control framework is constructed with visual position feedback. Experiments are first conducted to validate the assumption of the PRB model and the efficacy of the magnetic field model. Then, trajectory tracking tasks and disturbance rejection tests are performed to validate the feasibility of the proposed solution and closed-loop control. Results show that the colonoscope can stably and accurately orient to the desired orientation with an absolute mean position error of less than 0.5 mm and an average velocity of 3.5 mm/s. The distal tip can quickly re-stabilize to the desired orientation even when a large disturbance exists.

In this paper, we proposed an autonomous robotic flexible endoscope system for the laparoscopic bariatric surgery (LBS). This system comprises a UR5 robot and a flexible endoscope equipped with a novel continuum joint, named reinforced double helix continuum mechanism. Compared with the simple helix structure, the compressional and torsional stiffness of the proposed joint are improved significantly. To automate the robotic flexible endoscope, image-based visual servoing technique is employed. A deep learning algorithm named TernausNet-16 is improved and incorporated into the control framework to detect surgical instruments inside the camera view. The experimental studies verified the effectiveness and feasibility of the robotic flexible endoscope system for the visual serviong control scheme assisted by deep learning methods.