Robotic ultrasound (US) systems represent an emerging frontier in medical imaging. A fundamental component of these systems is the rigid body transformation between the robot flange and the attached US probe, which enables mapping of visual data from image space to the robot's reference frame. Traditionally, calibrating this transformation has been a tedious process, complicated by equipment demands and operational constraints arising from the probe's narrow field of view. This work presents a novel calibration strategy based on conic sections geometry, which offers several key simplifications over existing approaches: 1). It requires no external equipment beyond a single cone phantom; 2). It operates with a small input size and imposes no strict alignment or motion constraints on the US scan plane during calibration; and 3). It employs a straightforward pattern analysis pipeline to process images acquired from phantom scans. Experimental validation results show that the proposed method achieves accuracy comparable to existing state-of-the-art approaches while delivering superior precision, thereby demonstrating enhanced calibration reproducibility enabled by its streamlined workflow. These advantages make this method particularly suitable for application in clinical scenarios that require frequent and efficient calibration.

Magnetic compression anastomosis (MCA) offers a promising solution for minimally invasive anastomosis surgery. However, current MCA schemes lack safe, real-time localization and guidance for compression magnets, hindering surgeons' ability to control the compression magnets effectively in complex, unstructured endoluminal environments. To address these limitations, this paper introduces the MagsL-HUD endoscopic system, a novel solution that enables multi-magnetic six-degree-of-freedom (six-DoF) localization and head-up display (HUD) guidance within the endoscopic view (EV). Specifically, the system integrates a developed Endo-MagCap device with an orthogonal magnet configuration, along with a magnetic sensor array, to achieve real-time full-pose localization. An endoscopic camera model is incorporated for HUD visualization, enhancing intuitive interaction for surgeons' better-informed decisions. Eventually, the effectiveness of the MagsL-HUD endoscopic system is validated through laboratory experiments andex vivoanimal trials. The system demonstrates six-DoF tracking accuracy with average errors of 0.0070 m and 0.1437 rad, and 0.0071 m and 0.1721 rad in the designed trajectory cases for two compression magnets, respectively. Additionally,ex vivoporcine tests confirm the system's feasibility and applicability, successfully performing a stomach-colon MCA surgery with a final compression gap of approximately 0.00247 m. Further comparative studies demonstrate that the MagsL-HUD method has a compression success rate of 71.4$\%$versus 42.9$\%$of the non-HUD approaches in the designed tests. This work represents a significant step toward the clinical adoption of magnetic-assisted endoscopy for minimally invasive anastomosis surgeries, holding substantial practical significance for improving the safety and efficacy of MCA procedures in complex, unstructured endoluminal environments.

Endoscopic submucosal dissection (ESD) is a procedure targeted for early gastrointestinal cancer. Traction plays a crucial role in enhancing the efficiency of cutting lesions, thereby reducing procedural complexity and duration. From the perspective of traction devices, current non-magnetic ones hold shortcomings in complicating the workspace in directional tissue manipulation; Current magnetic traction devices cannot be prepared before the procedure, and require the withdrawal of endoscope in the midway to re-introduce the magnetic retractor to the lesion site. Towards these plights, this paper introduces a robotic flexible magnetic retractor designed for tissue manipulation during ESD. Precisely, the flexible prototype can be seamlessly inserted through the instrument channel of an endoscope to the lesion site without the need for endoscope withdrawal. Moreover, the introduction of robotic magnetic actuation enhances the agile control of magnetic retractors while alleviating the surgeon's workload in magnetic-retractor-assisted ESD. The experimental results validate the functionality and efficacy of the prototype magnetic retractor in magnetic traction-assisted ESD procedures. The retractor demonstrated its ability to provide adequate traction and accomplish clinical tasks. This innovative approach holds promise for enhancing the efficiency and outcomes of ESD procedures, offering a compelling alternative to traditional traction methods.

Magnetically actuated capsule endoscopic robots (MACERs) are becoming increasingly popular because they can reach deep diseased regions inside the body that are difficult or inaccessible to traditional endoscopes without the restriction of mechanical transmission medium. However, MACERs are highly nonlinear, hence achieving obstacle avoidance, safe, and stable target tracking control of MACERs remains a challenging research topic. Therefore, to satisfy the diagnosis and treatment needs of the deep diseased regions inside the body, this paper designs a MACER target tracking neural network control method with obstacle avoidance and noise-resistant capabilities. Firstly, the kinematics and obstacle avoidance model of the MACER are established, and then a moving target tracking control scheme of robot with joint motion constraints and obstacle avoidance capabilities is designed. Next, a noise-resistant neural network is designed to quickly solve the MACER’s control scheme, thereby achieving safe, obstacle avoidance, and stable target tracking control of the MACER. Finally, the effectiveness and practicability of the proposed method are checked by simulation analysis and experiment on MACER, and compared with the existing methods. The experimental results indicate that the neural network method proposed can effectively control the MACER to track the target motion along the gastric wall curve. Compared with existing methods, the designed method has stronger anti-noise interference ability, the convergence accuracy of the proposed method is improved by 1.3 times, and the computational burden is reduced by 26.7 times.

In modern healthcare, the demand for autonomous robotic assistants has grown significantly, particularly in the operating room, where surgical tasks require precision and reliability. Robotic scrub nurses have emerged as a promising solution to improve efficiency and reduce human error during surgery. However, challenges remain in terms of accurately grasping and handing over surgical instruments, especially when dealing with complex objects in dynamic environments. In this work, we introduce RoboNurse-VLA, a novel robotic scrub nurse system based on a Vision-Language-Action (VLA) model. RoboNurse-VLA integrates Segment Anything Model 2 (SAM 2) and Llama 2, leveraging an LLM head to enhance reasoning capabilities. By combining SAM 2’s mask generation with Llama 2’s advanced reasoning, RoboNurse-VLA can accurately interpret task requirements, identify optimal grasping points, and determine appropriate handover poses. Designed for real-time operation, RoboNurse-VLA enables precise grasping and seamless handover of surgical instruments based on voice commands from the surgeon. Utilizing state-of-the-art vision and language models, it effectively addresses challenges related to object detection, pose optimization, and handling difficult-to-grasp instruments. Extensive evaluations demonstrate that RoboNurse-VLA outperforms existing models, achieving high success rates in surgical instrument handovers, even for previously unseen tools and complex objects. This work represents a significant advancement in autonomous surgical assistance, highlighting the potential of VLA models for real-world medical applications. More details can be found at https:// robonurse-vla.github.io.

The remote center of motion (RCM) constraint is a vital requirement in the design of robotic systems for transrectal ultrasound (TRUS) probe-scanning. This paper presents the design and development of a novel RCM-constrained manipulator specifically tailored for TRUS probe-scanning applications. The proposed system features a six-degree-of-freedom (6-DoF) parallel-serial hybrid mechanism that enables the TRUS probe to perform pivot and spin rotations while maintaining the RCM constraint. Subsequently, the kinematic model incorporating the RCM constraint is derived. Additionally, a geometry-aware path planning method is then introduced, considering variations in the desired rotation targets. This method parameterizes distance metrics on SO(3) (a Lie group) using coordinate-free Riemannian geometry, enabling the dynamic optimization of rotation orders to minimize the calculated Riemannian metrics. Furthermore, a smooth rotational trajectory generation method is proposed, constructing rotation curves between the ordered matrices on SO(3) while minimizing angular acceleration. Both simulations and experimental results validate the effectiveness and practicality of the proposed manipulator and its path planning method.

Despite advantages from neurosurgical systems, achieving intuitive and safe collaboration with robot during the coarse positioning of instrument insertion end effector (IIEE) remains a critical issue. In this paper, we propose a novel non-contact hand-guided method for such advancement based on magnetic sensing. First, a wearable magnet band and a magnetic sensor are designed, based on which the magnetic localization is achieved for surgeon’s hand location detection. Second, a quadratic programming-based control is implemented, to guarantee the pose-based servo performance, higher rotational manipulability for IIEE fine alignment, and joint position&velocity limits avoidance. For evaluation, two experiments are designated and conducted. Results show that the magnetic localization algorithm can achieve < 4.7 mm and 2.6° errors in a dynamic path tracking test, which can provide an accurate magnet location for hand guidance. Moveover, workflow of the proposed solution in a brain biopsy scenario demonstrates its enhancement of IIEE rotational manipulability (11.6% increase at final configuration), and safety improvement of collision avoidance when other surgeon approaches for cannula delivery. This research contributes to enhanced intuitiveness and safety for surgeon-robot collaborative coarse positioning of IIEE in neurosurgery.

In robotic teleoperation systems, the effectiveness of teleoperation relies heavily on the operator performing control actions, underscoring the importance of master device design. Breaking away from the mechanical framework that relies on complex structures and restricts workspace, this paper adopts a new magnetic method and introduces a wireless magnetic master device (MMD) with an expandable, singularity-free workspace, designed to enhance maneuverability and operational reach. Specifically, the concept and development of the MMD for robotic teleoperation are presented, followed by an exploration of key technical aspects, including a six-degree-of-freedom (DoF) tracking approach using dual magnetic localization and controller design for the robotic arm using the MMD. Finally, the proposed method’s feasibility and accuracy are validated through magnetic localization and teleoperation control tests. Results indicate that the MMD achieves six-DoF localization with an average Euclidean localization error ofet= 0.0018 m and an average Geodesic distance error ofeR= 0.0604 rad within the central region of the workspace, and six-DoF manipulation (velocityet= 0.0062 m andeR=0.2091 rad in tests. Furthermore, the overall teleoperation control framework achieves tracking accuracy withet= 0.0085 m andeR= 0.1475 rad in robotic calligraphy task; achieves a broad scanning range of 0.2746 m along they-axis in robotic ultrasound scanning task; achieves a large-angle rotation of 9.15 rad in robotic clamp rotating task. Overall, this research advances master device design, aiming to inspire further innovation in master-follower teleoperation control for human-machine systems.

In minimally invasive surgery (MIS), the field of view (FOV) control is crucial. Autonomous endoscope robots have been developed to facilitate MIS procedures by enabling autonomous surgical target tracking, thus reducing the workload on surgeons. However, existing visual servoing-based target tracking methods for autonomous endoscopes often overlook the insecurity stemming from restricted workspace conditions. Instances, such as collisions between the endoscope robot’s tip and the patient’s chest or abdominal wall pose risks to patient tissue, while extensive motion of the endoscope shaft may damage incision port tissue. Addressing these security concerns, this article proposes a novel approach called virtual fixture-based restricted workspace constraint (RWSC) to reconstruct the endoscope robot’s movement range. A quadratic programming (QP) optimization framework is employed to govern the robot’s motion, ensuring autonomous target tracking while adhering to RWSCs. To solve the QP problem, we propose an adaptive zeroing neural network (ZNN) featuring a newly designed activation function (AF). This AF enhances the ZNN with predefined-time convergence and noise rejection capabilities, making it especially suitable for time-sensitive and noise-prone surgical applications. Theoretical analysis and experimental results demonstrate that our adaptive ZNN achieves shorter convergence times than existing neural dynamic-based QP solvers. Physical validations show the efficacy of the proposed RWSCs in limiting the workspace of the endoscope robot, while the FOV control strategy enables autonomous target tracking of flexible endoscopes under diverse constraints and objectives.

This article proposes an adaptive intermittent control and an adaptive optimal control for quarter-vehicle active suspension systems. This article leverages integral-type barrier Lyapunov functions (BLFs) to ensure that the vertical displacement and vertical displacement velocity always satisfy state-related constraints. In order to stabilize the vehicle’s attitude and improve passenger comfort, an adaptive intermittent control method is designed to seek the dwell-time condition, achieving a balance between controllable and uncontrollable subsystems. In addition, to reduce the power consumption of the control input, an adaptive optimal control method is designed by designing optimal cost functions and employing the backstepping algorithm under the framework of actor–critic neural networks (critic NNs). The stability of the quarter-vehicle active suspension system is analyzed based on the Lyapunov theory. Finally, simulation results demonstrate significant effects on passenger comfort and reduced control input power for both control methods.

Hand extension is crucial for stroke survivors with spasticity, where their fingers become rigid and their thumb remains curled within the palm. Due to the underactuated nature of the hand, the dominance of flexor muscles over extensors, and the limited surface area available, developing an extension glove with thumb assistance poses a challenge for researchers. This paper introduces a fully wearable soft hand extension glove based on the X-pouch and strap system, addressing the above challenges. The glove enables adequate finger extension, thumb abduction, and extension for high MAS score patients. Modelling and testing revealed extension torques of up to 2.7 Nm at the MCP joint and 0.67 Nm at the PIP and DIP joints. Performance evaluation, including comparison with existing methods, demonstrated the glove’s superior extension capabilities using a model hand with realistic stiffness. Furthermore, the glove’s effectiveness was confirmed through testing on a stroke patient with MAS = 2, validating its on-body functionality.

In response to torso-oriented magnetic localization tasks that require the system to have interactivity and flexibility with guaranteed accuracy, a novel bio-inspired magnetic sensor array configuration is proposed in this paper. Precisely, the ideas of the natural characteristics of octopus flexible tentacles and the "wrap" morphology are integrated into the design of the magnetic localization system based on the sensor array method. It is worth mentioning that such a design enhances the interactivity and flexibility of the localization system compared to the general planar sensor array strategy. Apart from the concept introduction, the geometry analysis of the proposed configuration is presented based on the constant curvature model. Besides, the magnetic localization algorithm for the system is presented by constructing a magnetic tracking optimization function. Eventually, the proposed concept and developed algorithm are examined in the sensor-array-simulation environment to manifest their effectiveness and applicability. The experimental results indicate that the octopus-inspired-configuration sensor array achieves a mean accuracy at a centimeter-level in our cases, and has better accuracy with a mean value of ē as 0.0178 m and ${\overline {SQR} _{ave{\text{ }}}}$ as 0.0883 for the center interest space compared to general planar configuration one. Moreover, the effect of the configuration error is analyzed. These results verify the feasibility and superiority of the proposed concept and hold significant practical significance in addressing the challenge associated with magnetic localization tasks toward the clinical application scenarios.

In this paper, the near optimal control problem of discrete-time switched nonlinear system with hysteresis is studied, that is, an optimal control scheme is designed for the approximate model of discrete-time switched nonlinear system. Firstly, through system transformation, the original system is replaced with a prediction model that can avoid non causal problems caused by the use of future information in the current step. By using action-critic network, an adaptive control method is put forward to offset the influence of hysteresis. Meanwhile, the designed controller ensures the stability of the closed-loop system under arbitrary switching signal and minimizes the performance index of the subsystem corresponding to the working interval. Finally, the simulation verifies the significance of the proposed method.

Manual rigid endoscopes have defects such as a low efficiency, difficult operation, and safety risks, and the antinoise interference ability, convergence speed, and control accuracy of the neural network control technology for the existing autonomous endoscopes are often ignored. Solving these problems is important for the stable operation of endoscopes. Therefore, a new adaptive fast convergent antinoise dual neural network (AFA-DNN) controller for the visual servo control of ten-degree of freedom flexible endoscope robots (FERs) with physical constraints is proposed in this work. First, the control scheme of the FERs is formulated as a quadratic programming problem, and then, an AFA-DNN visual servo controller is designed for the FERs. The adaptive gains of the controller can accelerate the convergence, improve the antinoise ability, and increase the convergence accuracy of the controller. Then, according to the Lyapunov theory, the fast convergence of the AFA-DNN in finite time is proven for both noise-free and noisy conditions. The experimental results indicate that the FER controlled by the proposed AFA-DNN can accurately track various trajectories and that the AFA-DNN has a better antinoise interference ability, higher convergence accuracy, and faster convergence speed than conventional methods. The convergence speed of the AFA-DNN is increased by a factor of 4.22 by using the adaptive gains. Experiments also indicate that the AFA-DNN remains well functioning under various noise disturbances (such as constant, periodic, linear, and Gaussian noise).

This article studies the adaptive control about the geodetic fixed positions and heading of three-degree-of-freedom dual-propeller vessel. During the navigation of a vessel at sea, due to the unpredictable sea, on the one hand, it is important to ensure that the vessel can smoothly follow the desired geodesic fixed position and heading; on the other hand, when the sailing environment is harsh, it is even more important that the vessel can adapt to the desired geodesic fixed position and heading that change at any time for safe driving. Therefore, this article selects the time-varying function related to the desired geodesic fixed position and heading as the constraint condition, and the constraint condition will change in real time as the expected position and heading change. The design of the control strategy is difficult, and the designed control strategy will be more suitable for complex maritime navigation conditions. First, the article constructs a log-type barrier Lyapunov function. Second, by introducing an unknown external disturbance observer, the external disturbances caused by the environment that may be encountered during the vessel's voyage can be observed. Then, combined with the backstepping algorithm, a neural network (NN) control strategy and adaptive law are designed. Among them, for the uncertain function in the process of designing the control strategy, the NN is used to approximate it. Furthermore, through the Lyapunov stability analysis, it is shown that applying the designed control strategy to the vessel system in this article can ensure that the system is closed-loop stable. The final simulation experiment shows the effectiveness of the designed control strategy.

In this article, the problem of full state constraints for a class of uncertain nonlinear switched systems with time-varying delays under average dwell time is studied, and an adaptive event-triggered mechanism is proposed. The Lyapunov–Krasovskii function (LKF) is employed to solve the trouble caused by time-varying delays, neural networks are selected to approximate the uncertain terms in the system, and the state constraint problem is solved by constructing tan barrier Lyapunov function (Tan-BLF). What’s more, the constraint boundaries considered in this article can be expressed as functions that rely on time and historical information of the system. In addition, the mismatch behavior between subsystem and its controller is also considered. Finally, numerical simulation results verify the availability of the control strategy.

Kinematic calibration is crucial to improve the positioning accuracy of serial robots. This paper proposes a novel algorithm for robotic kinematic calibration based on an augmented product of exponentials (POE)-based kinematic model using Gaussian mixture models (GMMs) with only position data. In this algorithm, non-geometric errors that cannot be fitted by varying the parameters within the traditional robot model are also considered and compensated. This approach involving a three-stage calibration process which is used to identify the kinematic model parameters and to train the GMMs will be presented in this paper. Finally, this algorithm will be applied to two serial robots for simulation and experimental validation. The effectiveness of the proposed algorithm is verified from both results and significant improvement on error reduction from 26 % to 96% can be observed through the comparison with other existing approaches.

This paper is concerned with the issue of the kinematic model-based bending control for the magnetically actuated robotic endoscope and its application for automatic retroflexion. By the utilization of the Cosserat rod theory and the transformation in the magnetic tip of the endoscope, the comprehensive kinematic model of the magnetically-actuated robotic endoscope is established. Afterward, a magnetic control scheme for the bending motion is proposed by co-developing an error feedback PID control strategy and the model-based feedback approach. Moreover, as one unique kind of bending motion, retroflexion is taken into account, and the strategy aimed at the bid of compact space retroflexion is presented by virtue of the introduction of serial waypoints pursuing the position of the magnetic tip being close to the midline as possible. Eventually, the developed modeling and bending control scheme and the compact space retroflexion strategy are examined in a magnetically actuated robotic endoscope system to manifest the effectiveness and applicability of the theoretical approach. The experimental results indicate that the designed controller can drive the endoscope to bend to the desired pose and show a reduction of about 47.01% in the sweeping area and 79.25% in the last distance to midline achieved by conducting compact space retroflexion in comparison to “U” type one.

Characteristics of wireless manipulation based on permanent magnet coupling, including anchoring distance, panning torque, and translational force, are assessed in this paper. The study focuses on a typical scenario where a slave robot embedded with a small permanent magnet can be remotely controlled within a constrained area by a master large permanent magnet placed outside the environment at a certain distance from it. The key parameters (force and torque) acting on the slave robot are quantified and evaluated. In this article, several combinations of permanent magnets with various dimensions and configurations are studied using finite element methods. Based on the obtained results, we create a lookup table for each parameter, serving as a guideline to help interested researchers choose suitable magnetic combinations for their applications.

Generally, the infinity-norm joint-velocity minimization (INVM) of physically constrained kinematically redundant robots can be formulated as time-variant linear programming (TVLP) with equality and inequality constraints. Zeroing neural network (ZNN) is an effective neural method for solving equality-constrained TVLP. For inequality-constrained TVLP, however, existing ZNNs become incompetent due to the lack of relevant derivative information and the inability to handle inequality constraints. Currently, there is no capable ZNN in the literature that has achieved the INVM of redundant robots under joint limits. To fill this gap, a classical INVM scheme is first introduced in this article. Then, a new joint-limit handling technique is proposed and employed to convert the INVM scheme into a unified TVLP with full derivative information. By using a perturbed Fisher-Burmeister function, the TVLP is further converted into a nonlinear equation. These conversion techniques lay a foundation for the success of designing a capable ZNN. To solve the nonlinear equation and the TVLP, a novel continuous-time ZNN (CTZNN) is designed and its corresponding discrete-time ZNN (DTZNN) is established using an extrapolated backward differentiation formula. Theoretical analysis is rigorously conducted to prove the convergence of the neural approach. Numerical studies are performed by comparing the DTZNN solver and the state-of-the-art (SOTA) linear programming (LP) solvers. Comparative results show that the DTZNN consumes the least computing time and can be a powerful alternative to the SOTA solvers. The DTZNN and the INVM scheme are finally applied to control two kinematically redundant robots. Both simulative and experimental results show that the robots successfully accomplish user-specified path-tracking tasks, verifying the effectiveness and practicability of the proposed neural approach and the INVM scheme equipped with the new joint-limit handling technique.

To solve the problems of discomfort and potential colon perforations of patients that arise when standard colonoscopes are used for colonoscopy, an electromagnetically actuated soft-tethered colonoscope robot (EASCR) is here introduced. Owing to EASCRs’ highly nonlinear and complex application environments, the hybrid vision/magnetic-force tracking control for these types of robots remains a challenging research issue, and the lack of current constraints may also give rise to safety concerns. Therefore, a hybrid vision/magnetic-force fast convergent dual neural network (DNN) tracking controller for an EASCR with current constraints is developed to alleviate patient discomfort and ensure the safe and smooth progression of colonoscopy. First, EASCR motion/vision and electromagnetically actuated force nonlinear coupling models are established, and a quadratic programming visual servo-tracking control scheme with current constraints is designed. Second, a novel DNN solver for the nonlinear control scheme is developed, and its convergence in finite time is strictly proved. The results of simulations and experiments indicate that the designed control method can well control EASCRs with current constraints to achieve tracking tasks, and it has a stronger anti-disturbance ability, faster convergence, and higher convergence accuracy than existing methods.

Posture sensing of soft actuators is critical for performing closed-loop control of soft robots. This paper presents a novel end-to-end posture perception method for soft actuators by developing long short-term memory (LSTM) neural networks. A novel flexible bending sensor developed from off-the-shelf conductive silicon material was proposed and used for posture sensing. In the proposed method, the hysteresis of the soft robot and non-linear sensing signals from the flexible bending sensors have also been considered. With one-step calibration from the sensor output, the posture of the soft actuator could be captured by the LSTM network. The method was validated on a finger-size one DOF pneumatic fiber-reinforced bending actuator. Four kirigami-inspired flexible piezoresistive transducers were placed on the top surface of the actuator. Results show that the transducers could sense the posture of the actuator with acceptable accuracy. We believe our work could benefit soft robot dynamic posture perception and closed-loop control.

This paper presents a long-range magnetic actuated and guided endoscope for uniport video-assisted thoracic surgery (VATS). In VATS, the incision is quite narrow and part of the chest wall may be very thick. So, the magnetic endoscope system is required to produce sufficient attractive force at a considerable distance with a compact dimension. In this paper, a magnetic endoscope system is developed to meet the aforementioned clinical demands. In the system, both the internal and external units consist of two cylindrical magnets at both ends and a semi-cylindrical magnet in the middle. Coupled with the magnetic field from the external unit, the internal endoscope can achieve anchoring, tilting, panning, and translating to provide the desired view for the surgeon. The rotation of the endoscope is dynamically modeled by combining magnetic theory and coordinate transformation. The prototype is made with a boundary box of 10×14×56 mm, which can be inserted through the narrow incision in VATS. In the experiment, the developed models of anchoring, tilting, and panning were verified. The magnet configuration in the system can achieve a static anchoring distance of 95 mm and exhibits enhancement in attractive force compared with other designs.

In this article, a fuzzy adaptive output feedback control strategy is designed for a class of uncertain nonlinear switched system with full state constraints under arbitrary switching signal. The states of the system studied in this article are unmeasurable, so a fuzzy observer is designed to estimate the unmeasurable states. At the same time, in order to ensure that the states of the system do not violate the constraints related to the desired output and states, the log-type barrier Lyapunov function method is selected to solve this constraint problem. Finally, through Lyapunov stability theory analysis, it is found that the designed control strategy can ensure that all signals in the closed-loop system are bounded, and the states of the system do not violate their corresponding constraints. In addition, a numerical simulation verifies the effectiveness of the control strategy.

Superior dexterity and extreme flexibility are typical advantages for concentric cable-driven manipulators working in confined spaces. However, its inverse kinematics and configuration planning are very complicated. In this article, we propose a spatial biarc method for the above problem. The distinguishing feature of this method is that input parameters are two positions and two direction vectors in three-dimensional (3-D) space, and the output is a reasonable spatial biarc for controlling a concentric cable-driven manipulator in 3-D space. This method has the following three advantages. First, the positions and direction vectors of the base and inner distal tip are considered simultaneously. In addition, the length and ratio of the overlapped section and separated section can be adjusted by changing the length of the direction vectors. Furthermore, by judging the angular value of the direction vectors, one can predetermine whether the spatial configuration of the entire arm is C- or S-shaped. The proposed method realizes the parameterization of a concentric cable-driven manipulator, which makes it convenient to intuitively control the manipulator to achieve interference-free motion trajectory planning in confined spaces. Finally, trajectory tracking inspections are simulated and experimentally executed. It can be seen from results that the proposed spatial biarc method can provide reasonable solutions for concentric cable-driven manipulators. The method is especially favorable in terms of 3-D-pose-determination problem and trajectory-planning problem. It can also be applied to other manipulators with similar configurations. Without loss of generality, when the given points and direction vectors are coplanar, the proposed spatial biarc method can be transformed to a planar biarc method.

Recently, flexible robotic systems are developed to enhance minimally invasive interventions on internal organs located in confined areas of human body. These surgical devices are designed to navigate anatomical pathways via single-port access, such as natural orifices or minimal incisions and intraluminal interventions. With improved precision, spatial flexibility and dexterity, the robotic technology can enhance surgery such that minimally invasive flexible access would become a faster, safer, and more convenient method for intra-body interventions without multiple or wide incisions. However, a lot of works are still required for global acceptance of existing flexible robotic surgical platforms. This review provides extended insights on the design details of two types of flexible robotic systems used for endoscopic and endovascular procedures. As of today, several prototypes of both platforms have been proposed; however, their global acceptability and applicability remains very low. To address these, we present an extensive review on design constraints and control methods which are vital for safer, faster, and better operation of the flexible robotic systems in minimally invasive surgery (MIS). Finally, research trends of flexible robotic systems and their clinical application status in MIS are discussed along with some of the technical and technological challenges hindering their prominence.

Colorectal cancer incidence has been steadily rising worldwide. Magnetic colonoscopes provide new approaches to conduct colon inspection and treatment. This paper presents a novel electromagnetically actuated soft-tethered colonoscope to achieve precise and stable orientation control. An inflated balloon is designed to eliminate the unpredictable disturbance of the floating tether. A 2OR Pseudo-Rigid-Body (PRB) model of the soft tether is developed to analyze the relationship between the tether deflection and applied force and torque. A closed-loop control framework is constructed with visual position feedback. Experiments are first conducted to validate the assumption of the PRB model and the efficacy of the magnetic field model. Then, trajectory tracking tasks and disturbance rejection tests are performed to validate the feasibility of the proposed solution and closed-loop control. Results show that the colonoscope can stably and accurately orient to the desired orientation with an absolute mean position error of less than 0.5 mm and an average velocity of 3.5 mm/s. The distal tip can quickly re-stabilize to the desired orientation even when a large disturbance exists.

Crawling through various terrains has been a long research interest. In recent years, quite a number of soft crawling robots have been developed. However, locomoting in an elastic, humid, and slippery environment remains a challenge. In nature, gastropods, such as snails, live in humid environment and could crawl through all kinds of surface conditions by using wet adhesion. In the wet adhesive locomotion, the mucus is crucial in adhering the gastropod while allowing forward motion. Previously, we presented one snail-like soft robot that mimics the gastropods. In this work, we propose a second version and present a theoretical model of the mucus simulant. In addition, the dynamic model of the soft robot’s wet adhesive locomotion is developed for the first time. Results show that the speed of the current version is 5 times than that of the previous one through the optimization of design. Also shown by the results that the mucus helps to speed up the robot by at least 2.7 times.

In this paper, we proposed an autonomous robotic flexible endoscope system for the laparoscopic bariatric surgery (LBS). This system comprises a UR5 robot and a flexible endoscope equipped with a novel continuum joint, named reinforced double helix continuum mechanism. Compared with the simple helix structure, the compressional and torsional stiffness of the proposed joint are improved significantly. To automate the robotic flexible endoscope, image-based visual servoing technique is employed. A deep learning algorithm named TernausNet-16 is improved and incorporated into the control framework to detect surgical instruments inside the camera view. The experimental studies verified the effectiveness and feasibility of the robotic flexible endoscope system for the visual serviong control scheme assisted by deep learning methods.

Driven by the SARS-CoV-2 pandemic, demand for oropharyngeal swab sampling (OP-swabs) is surging. However, medical staff can easily become infected by the virus during the sampling process. In an effort to combat this, we developed a novel, intrinsically safe rigid- flexible coupling (RFC) manipulator to improve the safety and reliability of OP-swab sampling to test for COVID-19, which is presented herein. Suitable sampling areas and the necessary contact force for OP-swab sampling tasks are carefully investigated, and three typical sampling paths outlined that could be performed by a robotic system. This is followed by a detailed description of an intrinsically safe bionic micro-pneumatic actuator (MPA) that was designed and fabricated as the main component of the RFC manipulator. The developed RFC manipulator’s kinematic modeling, motion planning, and force control capacities were designed for OP-swab sampling scenarios. The system was then validated using both an oral cavity phantom and human volunteers, with comparative experiments on the swab quality of the OP-swab sampling approach conducted in both robotic and manual modes. The results indicate that fully-automated sampling based on this design would be feasible.

A flexible endoscope introduces more dexterity to the image capturing in endoscopic surgery. However, manual control or automatic control based on instrument tracking does not handle the misorientation between the endoscopic video and the surgeon. We propose an automatic flexible endoscope control method that tracks the surgeon's head with respect to the object in the surgical scene. The robotic flexible endoscope is actuated so that it captures the surgical scene from the same perspective as the surgeon. The surgeon wears a head-mounted display to observe the endoscopic video. The frustum of the flexible endoscope is rendered as an augmented reality overlay to provide surgical guidance. We developed the prototype, FlexiVision, integrating a 6-DOF robotic flexible endoscope based on the da Vinci Research Kit and Microsoft HoloLens. We evaluated the proposed automatic control method via a lesion observation task, and evaluated the AR surgical guidance in a lesion targeting task. The multi-user study results demonstrated that, for both tasks, FlexiVision significantly reduced the completion time (by 59% and 58%), number of errors (by 75% and 95%) and subjective task load level. With FlexiVision, the flexible endoscope could act as the surgeon's eyes teleported into the abdominal cavity of the patient.

This article designs and analyzes a recurrent neural network (RNN) for the visual servoing of a flexible surgical endoscope. The flexible surgical endoscope is based on a commercially available UR5 robot with a flexible endoscope attached as an end-effector. Most of the existing visual servo control frameworks of the robotic endoscopes or robot arms have not considered either the physical limits of the robot or the remote center of motion (RCM) constraints (i.e., the fulcrum effect). To tackle this issue, this article first conducts the kinematic modeling of the flexible robotic endoscope to achieve automation by visual servo control. The kinematic modeling results in a quadratic programming (QP) framework with physical limits and RCM constraints involved, making the UR5 robot applicable to surgical field. To solve the QP problem and accomplish the visual task, an RNN activated by a sign-bi-power activation function (AF) is proposed. The motivation of using the sign-bi-power AF is to enable the RNN to exhibit an accelerated finite-time convergence, which is more preferred in time-critical applications. Theoretically, the finite-time convergence of the RNN is rigorously proved using the Lyapunov theory. Compared with the previous AFs applied to the RNN, theoretical analysis shows that the RNN activated by the sign-bi-power AF delivers an accelerated convergence speed. Comparative validations are performed, showing that the proposed finite-time convergent neural network is effective to achieve visual servoing of the flexible endoscope with physical limits and RCM constraints handled simultaneously.

Soft robots attract research interests worldwide. However, its control remains challenging due to the difficulty in sensing and accurate modeling. In this paper, we propose a novel iterative learning model predictive control (ILMPC) method for soft bending actuators. The uniqueness of our approach is the ability to improve model accuracy gradually. In this method, a pseudo-rigid-body model is used to take an initial guess of the bending behavior of the actuator and the model accuracy is improved with iterative learning. Compared with conventional model free iterative learning control (ILC), the proposed method significantly reduces the learning curve. Compared with the model predictive control (MPC), the proposed method does not rely on an accurate model and it will output a satisfactory model after the learning process. A soft-elastic composite actuator (SECA) is used to validate the proposed method. Both simulation and experimental results show that the proposed method outperforms the conventional MPC and ILC.

This paper presents a novel magnetic anchored and steered camera robot intended for minimally invasive surgery (MIS), particularly for single port access (SPA) surgery. The design aims to achieve both compactness and a planar pan/tilt workspace (instead of hemispheric) to lower robot footprint in vertical space. Robot comprises two 6mm×6mm diametrically magnetized internal permanent magnets (IPMs) fixed at either ends of a small cylindrical capsule, with camera module and a 45°mirror capped inside capsule. As such, camera view orientation can be steered in 2-DOF across range of 180° tilt and 360° panning, all within a planar workspace close to surface of anchor. Using only two small IPMs for all necessary functions (anchoring, translation along intra-abdominal surface, and steering) reduces bulk and length of robot. The robot is investigated first by finite element methods. Theoretical models for both tilting and panning were then built based on FEM results. The models are evaluated and verified by checking its predictions in benchtop experiments. Ex vivo evaluations was also utilized to prove feasibility of device in environment similar to human anatomy. Overall, the camera robot prototype is compact (4cm length; 7mm diameter), lightweight (3.6g), motor-free, and allow view orientation control (tilting and panning) in a planar workspace. Minimal footprint in vertical space is ideal for many MIS applications, where vertical space is extremely limited.

Navigation through tubular environment is fundamental in tasks such as pipeline inspection, gastrointestinal tract inspection, etc. Conventional pipeline inspection robots are mostly made by rigid materials and could not well adapt to the large size variation of the environment. Soft robots provide an additional solution for inspection of pipelines, especially with significant size variation. In this work, we present a soft robot for pipeline inspection, which consists of an earthworm-like soft robot and a Central Pattern Generator (CPG)-based control system. An analytical model is developed to predict the maximum pipe diameter that the robot could adapt to. For the current prototype, the robot could adapt to size change of three times. Experimental results show that this robot could navigate through pipelines with sharp turnings and with large diameter change.

Minimally invasive surgery (MIS) has seen substantial development in recent decades. In MIS, particularly in laparoendoscopic single site (LESS) surgery, the surgical site is always crowded. One potential solution is the magnetic anchored and guidance system (MAGS), which liberates instruments from constraint of the accessing port. Current MAGS are all made by rigid components, which relies on articulated joints to maneuver inside the surgical cavity. Moreover, fabricating and assembling the MAGS are difficult due to the small size of the components. In this work, we present the design, modeling, fabrication and testing of a novel soft MAGS endoscope system. The system includes the external controller and the internal unit, which contains a soft body, two magnets and a replaceable wireless camera. It is compact, safe, lightweight and easy to make. Experimental results show that the system could offer stable maneuvering both in lab settings and in porcine model.

In this paper, a concentric wire-driven mechanism (CWM) is presented. The CWM comprises of two nested wire-driven mechanisms (WDM). Each WDM contains a flexible backbone and a set of wires. The backbone bending is controlled by pulling the wires. The stiffness of the outer WDM is controllable and dominants that of the overlapped section. Therefore, in the overlapped section the inner WDM conforms to the shape of the outer WDM and the tip of the outer WDM serves as the base of the distal separate section of the inner WDM. Compared with conventional flexible mechanisms, i.e., tendon/wire/cable-driven mechanism and concentric tube mechanism, the CWM has a much wider workspace. This advantage is demonstrated by simulations based on kinematic modeling. A prototype is developed and the preliminary experimental results validate the design concept.

Flexible serpentine manipulators are widely used in surgical robots as it can be operated inside the patient's body cavity by backbone bending. However, during the bending the manipulator sweeps over a region, where sensitive organs may locate. This raises the safety concern. In this paper, a motion planning algorithm focusing on minimize the sweeping area for flexible serpentine manipulators is presented. Particularly, a three dimensional backward average neural dynamic model (BANDM) is proposed to build minimum sweeping area planning field in the configuration space of the serpentine manipulator. Given a target position, the motion sequence is generated automatically based on the established planning field. The simulations and experimental results validate the effectiveness and superiority of the proposed planning approach over conventional planning algorithms in terms of sweeping area with keeping target reach and obstacle avoidance.

In this paper, a novel constrained tendon-driven serpentine manipulator (CTSM) suited for minimally invasive surgery is presented. It comprises of a flexible backbone, a set of controlling tendons and a constraint. In the CTSM not only the curvature of the bending section can be controlled but also the length. Specifically, the curvature is controlled by the tendons, and the length is controlled by a constraint tube, which is translational and is concentric with the flexible backbone. The kinematic model of the CTSM is developed based on the piecewise constant curvature assumption. Analysis shows that by introducing the translational constraint both the workspace and dexterity of the manipulator are improved. The stiffer the constraint the larger the workspace expansion and the smaller the dexterity enhancement. A prototype is developed and the experimental results validate the design idea and analysis.

This paper presents a novel double-hull boat with two wire-driven flapping propulsors. The boat comprises two identical hulls and a control center. Each hull is propelled by a biomimetic wire-driven flapping propulsor, which has seven joints, and is driven by one servo motor. Propulsion model of the boat is developed by integrating the reaction force during the flapping cycle. Motion of the boat is studied, including cruising and turning around. Two cruising modes and three turning modes are presented. Results show that the boat cruises forward steadily. In the tests, the maximum speed of the boat is 0.323 BL/s; the minimum turning radius of the boat is 0.517 BL; the maximum turning speed is 13.9°/s.

This paper presents the idea of employing constraints to expand the workspace of Underactuated Flexible Manipulators (UFM). The constraints are divided into two types: bilateral constraint (BC) and unilateral constraint (UC). Under the constraints the UFM is segmented into two sections: the anterior constrained section and the distal free section; moreover, the motion of the UFM is altered, i.e., the end effector of the UFM can get access to positions where unreachable previously, or, the workspace is expanded. In the paper, constrained kinematics model is developed at first. The workspace is derived thereafter. To validate the idea, an underactuated wire-driven serpentine manipulator is built. In the experiments, the two types of constraints are applied. Experiment results agree well with the simulation results.

This paper presents a novel robot fish with vector propulsion. It can swim like a shark and/or a dolphin. The propulsor (tail) of the robot has an underactuated serpentine backbone and the actuation is done by two sets of orthogonally distributed wires. The backbone is composed of seven vertebras and an elastic rod. The vertebras are articulated by the rod and spherical joints. The horizontal flapping and vertical flapping are independently actuated by two motors. This enables the propulsor providing thrust in all directions. Propulsion model of the propulsor is developed by integrating the kinematic model and Lighthill's elongated body theory. A prototype is built. Tests show that the robot fish could flap its tail like the shark or the dolphin effectively. In the swimming tests, the maximum swimming speed of the robot is 0.35 BL/s.