Hysteresis in the tendons driving continuum robots is frequently regarded as a nuisance and a problem that is best avoided. Some prior work seeks to ameliorate the effects of hysteresis through the selection of materials. Others propose models of hysteresis to compensate for their effects. In this work, we present an empirically validated model of hysteresis in tendon-driven continuum robots. We demonstrate that hysteresis contributes to the stability of these robots by mitigating undesirable tensions in robot's backbone. As a result, a model-based approach to hysteresis can be used not just for compensation of a nuisance, but to enhance the utility of continuum robots in safety critical applications such as medical robots.

In this paper, we present a novel synergistic framework for learning shape estimation and a shape-aware whole-body control policy for tendon driven continuum robots. Our approach leverages the interaction between two Augmented Neural Ordinary Differential Equations (ANODEs) — the Shape-NODE and Control-NODE — to achieve continuous shape estimation and shape-aware control. The Shape-NODE integrates prior knowledge from Cosserat rod theory, allowing it to adapt and account for model mismatches, while the Control-NODE uses this shape information to optimize a whole-body control policy, trained in a Model Predictive Control (MPC) fashion. This unified framework effectively overcomes limitations of existing data-driven methods, such as poor shape awareness and challenges in capturing complex nonlinear dynamics. Extensive evaluations in both simulation and real-world environments demonstrate the framework's robust performance in shape estimation, trajectory tracking, and obstacle avoidance. The proposed method consistently outperforms state-of-the-art end-to-end, Neural-ODE, and Recurrent Neural Network (RNN) models, particularly in terms of tracking accuracy and generalization capabilities. The code and pretrained models are available at https://github.com/SIRGLab/WholeBodyControl_CTR.

Aiming at developing a safe, intuitive, and collaborative steerable drilling robotic system for pedicle screw fixation procedures, in this paper, we leverage our recently developed steerable drilling robotic framework, and developed a collaborative drilling mode to control this system. In this control mode, first a user positions a concentric tube steerable drilling robot (CT-SDR) in the workspace and aligns it based on a preplanned trajectory. Next, the CT-SDR is directly controlled by the user through an admittance mode to perform a drilling procedure and creating a J-shape tunnel. To evaluate the user comfort and intuitiveness of the drilling procedure using this system and the proposed control interface, we performed a user study with 11 subjects, who had no prior experience in using this system. The results of this study were analyzed using various qualitative and quantitative metrics.

Current pelvic fixation techniques rely on rigid drilling tools, which inherently constrain the placement of rigid medical screws in the complex anatomy of pelvis. These constraints prevent medical screws from following anatomically optimal pathways and force clinicians to fixate screws in linear trajectories. This suboptimal approach, combined with the unnatural placement of the excessively long screws, lead to complications such as screw misplacement, extended surgery times, and increased radiation exposure due to repeated X-ray images taken ensure to safety of procedure. To address these challenges, in this paper, we present the design and development of a unique 4-degree-of-freedom (DoF) pelvic concentric tube steerable drilling robot (pelvic CT-SDR). The pelvic CT-SDR is capable of creating long S-shaped drilling trajectories that follow the natural curvatures of the pelvic anatomy. The performance of the pelvic CT-SDR was thoroughly evaluated through several S-shape drilling experiments in simulated bone phantoms.

To address the screw loosening and pullout limitations of rigid pedicle screws in spinal fixation procedures, and to leverage our recently developed Concentric Tube Steerable Drilling Robot (CT-SDR) and Flexible Pedicle Screw (FPS), in this paper, we introduce the concept of Augmented Bridge Spinal Fixation (AB-SF). In this concept, two connecting J-shape tunnels are first drilled through pedicles of vertebra using the CT-SDR. Next, two FPSs are passed through this tunnel and bone cement is then injected through the cannulated region of the FPS to form an augmented bridge between two pedicles and reinforce strength of the fixated spine. To experimentally analyze and study the feasibility of AB-SF technique, we first used our robotic system (i.e., a CT-SDR integrated with a robotic arm) to create two different fixation scenarios in which two J-shape tunnels, forming a bridge, were drilled at different depth of a vertebral phantom. Next, we implanted two FPSs within the drilled tunnels and then successfully simulated the bone cement augmentation process.

In this paper, we introduce S3D: A Spatial Steerable Surgical Drilling Framework for Robotic Spinal Fixation Procedures. S3D is designed to enable realistic steerable drilling while accounting for the anatomical constraints associated with vertebral access in spinal fixation (SF) procedures. To achieve this, we first enhanced our previously designed concentric tube Steerable Drilling Robot (CT-SDR) to facilitate steerable drilling across all vertebral levels of the spinal column. Additionally, we propose a four-Phase calibration, registration, and navigation procedure to perform realistic SF procedures on a spine holder phantom by integrating the CT-SDR with a seven-degree-of-freedom robotic manipulator. The functionality of this framework is validated through planar and out-of-plane steerable drilling experiments in vertebral phantoms.

Spinal fixation procedures are currently limited by the rigidity of the existing instruments and pedicle screws leading to fixation failures and rigid pedicle screw pull out. Leveraging our recently developed Concentric Tube Steerable Drilling Robot (CT-SDR) in integration with a robotic manipulator, to address the aforementioned issue, here we introduce the transformative concept of Spatial Spinal Fixation (SSF) using a unique Flexible Pedicle Screw (FPS). The proposed SSF procedure enables planar and out-of-plane placement of the FPS throughout the full volume of the vertebral body. In other words, not only does our fixation system provide the option of drilling in-plane and out-of-plane trajectories, it also enables implanting the FPS inside linear (represented by an I-shape) and/or non-linear (represented by J-shape) trajectories. To thoroughly evaluate the functionality of our proposed robotic system and the SSF procedure, we have performed various experiments by drilling different I-J and J-J drilling trajectory pairs into our custom-designed L3 vertebral phantoms and analyzed the accuracy of the procedure using various metrics.

In robotics, acquiring new skills through Imitation Learning (IL) is crucial for handling diverse complex tasks. However, model-free IL faces challenges of data inefficiency and prolonged training time, whereas model-based methods struggle to obtain accurate nonlinear models. To address these challenges, we developed Neural ODE-based Imitation Learning (NODE-IL), a novel model-based imitation learning framework that employs Neural Ordinary Differential Equations (Neural ODEs) for learning task dynamics and control policies. NODE-IL comprises (1) Dynamic-NODE for learning the continuous differentiable task’s transition dynamics model, and (2) Control-NODE for learning a long-horizon control policy in an MPC fashion, which are trained holistically. Extensively evaluated on challenging manipulation tasks, NODE-IL demonstrates significant advantages in data efficiency, requiring less than 70 samples to achieve robust performance. It outperforms Behavioral Cloning from Observation (BCO) and Gaussian Process Imitation Learning (GP-IL) methods, achieving 70% higher average success rate, and reducing translation errors for high-precision tasks, which demonstrates its robustness and accuracy, as an effective and efficient imitation learning approach for learning complex manipulation tasks.

Data-driven approaches have shown promising results in modeling and controlling robots, specifically soft and flexible robots where developing physics-based models are more challenging. However, these methods often require a large number of real data, and gathering such data is time-consuming and can damage the robot as well. This paper proposed a novel data-efficient and non-parametric approach to develop a continuous model using a small dataset of real robot demonstrations (only 25 points). To the best of our knowledge, the proposed approach is the most sample-efficient method for soft continuum robot. Furthermore, we employed this model to develop a controller to track arbitrary trajectories in the feasible kinematic space. To show the performance of the proposed approach, a set of trajectory-tracking experiments has been conducted. The results showed that the robot was able to track the references precisely even in presence of external loads (up to 25 grams). Moreover, fine object manipulation experiments were performed to demonstrate the effectiveness of the proposed method in real-world tasks. Finally, we compared its performance with common data-driven approaches in seen/useen-before trajectory tracking scenarios. The results validated that the proposed approach significantly outperformed the existing approaches in unseen-before scenarios and offered similar performance in seen-before scenarios.

In this paper, we present the design, fabrication, and evaluation of a novel flexible, yet structurally strong, Concentric Tube Steerable Drilling Robot (CT-SDR) to improve minimally invasive treatment of spinal tumors. Inspired by concentric tube robots, the proposed two degree-of-freedom (DoF) CT-SDR, for the first time, not only allows a surgeon to intuitively and quickly drill smooth planar and out-of-plane J- and U- shape curved trajectories, but it also, enables drilling cavities through a hard tissue in a minimally invasive fashion. We successfully evaluated the performance and efficacy of the proposed CT-SDR in drilling various planar and out-of-plane J-shape branch, U-shape, and cavity drilling scenarios on simulated bone materials.

Bronchoscopy is a medical procedure that involves the insertion of a flexible tube with a camera into the airways to survey, diagnose and treat lung diseases. Due to the complex branching anatomical structure of the bronchial tree and the similarity of the inner surfaces of the segmental airways, navigation systems are now being routinely used to guide the operator during procedures to access the lung periphery. Current navigation systems rely on sensor-integrated bronchoscopes to track the position of the bronchoscope in real-time. This approach has limitations, including increased cost and limited use in non-specialized settings. To address this issue, researchers have proposed visual odometry algorithms to track the bronchoscope camera without the need for external sensors. However, due to the lack of publicly available datasets, limited progress is made. To this end, we have developed a database of bronchoscopy videos in a phantom lung model and ex-vivo human lungs. The dataset contains 34 video sequences with over 23,000 frames with odometry ground truth data collected using electromagnetic tracking sensors. With our dataset, we empower the robotics and machine learning community to advance the field. We share our insights on challenges in endoscopic visual odometry. Furthermore, we provide benchmark results for this dataset. State-of-the-art feature extraction algorithms including SIFT, ORB, Superpoint, Shi- Tomasi, and LoFTR are tested on this dataset. The benchmark results demonstrate that the LoFTR algorithm outperforms other approaches, but still has significant errors in the presence of rapid movements and occlusions.

Accurate shape estimation of concentric tube robots (CTRs) using mathematical models remains a challenge, reinforcing the need to develop techniques for accurate and real-time shape sensing of CTRs. In this paper, we develop a fusion algorithm that predicts the robot's shape by combining a mathematical model of the CTR with a measurement of the Cartesian coordinates of the robot's tip using an electro-magnetic sensor. We experimentally validated our method in static and dynamic scenarios with and without external loading. Results demonstrated that the fusion algorithm improves the error of model-based shape prediction by an average of 44.3%, corresponding to 2.43% of the robot's arc length. Furthermore, we demonstrate that our method can be used in real-time to simultaneously track the robot's tip position and predict its shape.

The Cosserat rod equations are used to model continuum and soft robots. Solving these equations are computationally expensive, particularly due to mixed boundary values and kinematic constraints. In this paper, we present a novel nonlinear observer that can rapidly estimate the solution of the Cosserat rod equations. We present details of the observer design and analyse its convergence and stability. Furthermore, we compare the accuracy and performance of the observer with common solvers used in the literature. Our results show that the proposed observer can significantly improve the computational efficiency of continuum robots’ models and estimates the solution of the Cosserat rod equations 7 times faster than common solvers.

Concentric Tube Robots (CTR) have been gaining ground in minimally-invasive robotic surgeries due to their small footprint, compliance, and high dexterity. CTRs can assure safe interaction with soft tissue, provided that precise and effective motion control is achieved. Controlling the motion of CTRs is still challenging. Commonly used model-based control approaches often employ simplified geometric/dynamic assumptions, which could be very inaccurate in the presence of unmodelled disturbances and external interaction forces. Additionally, application of emerging data-driven algorithms in real-time control of CTRs is limited due to the fact that these controllers require considerable amount of time to let the algorithm develop enough to reach a desired accuracy and relevancy. In this paper, we present a hybrid approach to overcome the aforementioned difficulties. This hybrid solution uses the solution of a kinematic model of the robot to estimate initial values for a model-free data-driven method. The proposed algorithm combines both model-based and data-driven algorithms to provide real-time motion control of CTRs interacting with an unknown external environment. Three different simulations studies were performed to thoroughly evaluate the efficacy of the proposed hybrid control approach as compared to two common model-based and data-driven control techniques. The results demonstrate superior performance of the proposed method. The root-mean-square error of the proposed hybrid approach is less than 1.1 mm, which is 9 times less than a common model-based controller.

This article presents a model predictive controller (MPC) developed for the autonomous steering of concentric tube robots (CTRs). State-of-the-art CTR control relies on differential kinematics developed by local linearization of the CTRs mechanics model and cannot explicitly handle constraints on robot's joint limits or unstable configurations commonly known as snapping points. The proposed nonlinear MPC explicitly considers constraints on the robot configuration space (i.e., joint limits) and the robot's workspace (i.e., mixed boundary conditions on robot curvature). Additionally, the MPC calculates control decisions by optimizing the model-based predictions of future robot configurations. This way, it avoids configurations it cannot recover from, i.e., joint limits, singular configurations, and snapping. The proposed controller is evaluated via simulations and experimental studies with a variety of trajectories of increasing complexity. Simulation results demonstrate the capability of MPC to avoid singularities while satisfying robot mechanical constraints. Experimental results demonstrate that our solution enables following of trajectories unattainable by state-of-the-art controllers with mean error corresponding to 1% of robot arclength.

Concentric tube robots (CTR) can traverse tightly curved paths and offer dexterity in constrained environments, making them advantageous for minimally invasive surgical scenarios that experience strict anatomical and surgical constraints. Their shape is controlled via rotation and translation of several concentrically arranged super-elastic precurved tubes that form the robot backbone. As the elastic energy accumulated in the backbone due to bending and twist of the tubes increases, robots can exhibit sudden snapping motions, which can damage the surrounding tissues. In this paper, we proposed an approach for closed-loop steering of a redundant CTR that allows for snap-free motion and enhances its force/velocity manipulability, increasing the capacity of the robot to move and/or exercise forces along any direction. First, a controller stabilizes the CTR end-effector on a desired time-variant trajectory. Next, an online optimizer uses the robot's redundant Degrees of Freedom (DoF) to reshape its manipulability in real-time and steer it away from potentially snapping configurations or increase its capacity in delivering force payloads. Simulations and experiments demonstrate the performance of the proposed control strategy. The controller can steer a generally unstable CTR along trajectories while avoiding instabilities with a mean error of 850 μm, corresponding to 0.6% of arclength, and improves robot ability to exercise forces by 55%.

The enhanced dexterity and manipulability offered by continuum manipulators makes them the robots of choice for complex procedures inside the human body. However, without tailored analytical tools to evaluate their manipulability, many capabilities of continuum robots such as safe and effective manipulation will remain largely inaccessible. This paper presents a quantifiable measure for analysing force/velocity manipulability of continuum robots. We expand classical measures of manipulability for rigid robots to introduce three types of manipulability indices to continuum robots, namely, velocity, compliance, and unified force-velocity manipulability. We provide a specific case study using the proposed method to analyse the force/velocity manipulability for a concentric-tube robot. We investigate the application of the manipulability measures to compare performance of continuum robots in terms of compliance and force-velocity manipulability. The proposed manipulability measures enable future research on design and optimal path planning for continuum robots.

Robotic-assisted needle steering can enhance the accuracy of needle-based interventions. Application of current needle steering techniques are restricted by the limited deflection curvature of needles. Here, a novel steerable needle with improved curvature is developed and used with an online motion planner to steer the needle along curved paths inside tissue. The needle is developed by carving series of small notches on the shaft of a standard needle. The notches decrease the needle flexural stiffness, allowing the needle to follow tightly curved paths with small radius of curvature. In this paper, first, a finite element model of the notched needle deflection in tissue is presented. Next, the model is used to estimate the optimal location for the notches on needle's shaft for achieving a desired curvature. Finally, an ultrasound-guided motion planner for needle steering inside tissue is developed and used to demonstrate the capability of the notched needle in achieving high curvature and maneuvering around obstacles in tissue. We simulated a clinical scenario in brachytherapy, where the target is obstructed by the pubic bone and cannot be reached using regular needles. Experimental results show that the target can be reached using the notched needle with a mean accuracy of 1.2 mm. Thus, the proposed needle enables future research on needle steering toward deeper or more difficult-to-reach targets.

In needle-based medical procedures, beveled-tip flexible needles are steered inside soft tissue with the aim of reaching pre-defined target locations. The efficiency of needle-based interventions depends on accurate control of the needle tip. This paper presents a comprehensive mechanics-based model for simulation of planar needle insertion in soft tissue. The proposed model for needle deflection is based on beam theory, works in real-time, and accepts the insertion velocity as an input that can later be used as a control command for needle steering. The model takes into account the effects of tissue deformation, needle-tissue friction, tissue cutting force, and needle bevel angle on needle deflection. Using a robot that inserts a flexible needle into a phantom tissue, various experiments are conducted to separately identify different subsets of the model parameters. The validity of the proposed model is verified by comparing the simulation results to the empirical data. The results demonstrate the accuracy of the proposed model in predicting the needle tip deflection for different insertion velocities.

This paper represents an extension to the kinematic bicycle model for beveled-tip needle motion in soft tissue, which accounts for non-constant curvature paths for the needle tip. For a tissue that is not stiff relative to the needle, the tissue deformation caused by needle insertion deviates the needle tip position from a constant curvature path. The proposed model is obtained by replacing the bicycle wheels with omnidirectional wheels that move in two orthogonal directions independently. Such wheels can move sideways, providing a means for modeling the deviations of the needle tip from a constant curvature path by incorporating new parameters in the model. Using an experimental setup, the needle is inserted into soft phantom tissue at different constant velocities and model parameters are fitted to experimental data. The model is verified by comparing the results from the model to empirical data.