Endoscopic Submucosal Dissection (ESD) is an effective minimally invasive approach to removing colon cancer, yet it is underutilized, since it is challenging to learn and perform. To promote the adoption of ESD by making it easier, we propose a system in which two small, flexible robotic manipulators are delivered through a colonoscope. Our system differs from prior robotic systems aimed at this application in that our manipulators are small enough to fit through a clinically used colonoscope. By not re-engineering the colonoscope, we maintain overall system diameter at the current clinical gold standard, and streamline the path to eventual clinical deployment. Our concentric push-pull robot (CPPR) manipulators offer dexterity and simultaneously provide a conduit for grasper or cutting tool deployment. Each manipulator in our system consists of two push-pull tube pairs, and we describe how they are actuated. We describe for the first time our approach to compensating for undesirable CPPR tip motion induced by differences in the tubes' transmission stiffness. We also evaluate the workspace of the manipulators and demonstrate teleoperation in a point-touching experiment. Lastly, we demonstrate the ability of the system to resect tissue via ex vivo animal experiments.

Surgical automation requires precise guidance and understanding of the scene. Current methods in the literature rely on bulky depth cameras to create maps of the anatomy; however, this does not translate well to space-limited clinical applications. Monocular cameras are small and allow minimally invasive surgeries in tight spaces, but additional processing is required to generate 3D scene understanding. We propose a 3D mapping pipeline that uses only RGB images to create segmented point clouds of the target anatomy. To ensure the most accurate reconstruction, we compare different structure from motion algorithms’ performance on mapping the central airway obstructions, and test the pipeline on a downstream task of tumor resection. In several metrics, including post-procedure percentage tissue charring, our pipeline performs comparably to RGB-D cameras and, in some cases, even surpasses their downstream task performance. These promising results demonstrate that automation guidance can be achieved in minimally invasive procedures with monocular cameras. This study is a step toward the complete autonomy of surgical robots.

Steerable needles are minimally invasive devices that enable novel medical procedures by following curved paths to avoid critical anatomical obstacles. We introduce a new start pose robustness metric for steerable needle motion plans. A steerable needle deployment typically consists of a physician manually placing a steerable needle at a precomputed start pose on the surface of tissue and handing off control to a robot, which then autonomously steers the needle through the tissue to the target. The handoff between humans and robots is critical for procedure success, as even small deviations from a planned start pose change the steerable needle's reachable workspace. Our metric is based on a novel geometric method to efficiently compute how far the physician can deviate from the planned start pose in both position and orientation such that the steerable needle can still reach the target. We evaluate our metric through simulation in liver and lung scenarios. Our evaluation shows that our metric can be applied to plans computed by different steerable needle motion planners and that it can be used to efficiently select plans with large safe start regions.

Mechanical metamaterials are microscale patterned structures that are designed to have specific mechanical properties at a macro-scale that are atypical of natural materials. Robotic manipulators composed of these materials can exhibit deformation and motion capabilities that can be customized and easily fabricated. However, as of now, the motion capability of such manipulators are encoded in their physical composition and cannot be changed. This paper presents multimodal metamaterial-based robot prototypes which can switch between the behaviors found in two different metamaterials. Two such robots are explored, a bending/shearing robot and a bending/twisting robot. The robot design is described in detail, including how the robots toggle between behavior modes via mechanical actuation of a sliding rod insert. Multi-modal robots are compared to their single-mode equivalents to characterize their capabilities. The single-mode behaviors are largely preserved in the multi-modal innovations. The multi-modal prototypes also demonstrate variable rigidity. We discuss the feasibility of using robots of this design as part of a robotic surgical system.

Continuum robots navigate narrow, winding passageways while safely and compliantly interacting with their environments. Sensing the robot's shape under these conditions is often done indirectly, using a few coarsely distributed (e.g. strain or position) sensors combined with the robot's mechanics-based model. More recently, given high-fidelity shape data, external interaction loads along the robot have been estimated by solving an inverse problem on the mechanics model of the robot. In this paper, we argue that since shape and force are fundamentally coupled, they should be estimated simultaneously in a statistically principled approach. We accomplish this by applying continuous-time batch estimation directly to the arclength domain. A general continuum robot model serves as a statistical prior which is fused with discrete, noisy measurements taken along the robot's backbone. The result is a continuous posterior containing both shape and load functions of arclength, as well as their uncertainties. We first test the approach with a Cosserat rod, i.e. the underlying modeling framework that is the basis for a variety of continuum robots. We verify our approach numerically using distributed loads with various sensor combinations. Next, we experimentally validate shape and external load errors for highly concentrated force distributions (point loads). Finally, we apply the approach to a tendon-actuated continuum robot demonstrating applicability to more complex actuated robots.

Steerable needles are medical devices with the ability to follow curvilinear paths to reach targets while circumventing obstacles. In the deployment process, a human operator typically places the steerable needle at its start position on a tissue surface and then hands off control to the automation that steers the needle to the target. Due to uncertainty in the placement of the needle by the human operator, choosing a start position that is robust to deviations is crucial since some start positions may make it impossible for the steerable needle to safely reach the target. We introduce a method to efficiently evaluate steerable needle motion plans such that they are safe to variation in the start position. This method can be applied to many steerable needle planners and requires that the needle's orientation angle at insertion can be robotically controlled. Specifically, we introduce a method that builds a funnel around a given plan to determine a safe insertion surface corresponding to insertion points from which it is guaranteed that a collision-free motion plan to the goal can be computed. We use this technique to evaluate multiple feasible plans and select the one that maximizes the size of the safe insertion surface. We evaluate our method through simulation in a lung biopsy scenario and show that the method is able to quickly find needle plans with a large safe insertion surface.

Steerable needles that are able to follow curvilinear trajectories and steer around anatomical obstacles are a promising solution for many interventional procedures. In the lung, these needles can be deployed from the tip of a conventional bronchoscope to reach lung lesions for diagnosis. The reach of such a device depends on several design parameters including the bronchoscope diameter, the angle of the piercing device relative to the medial axis of the airway, and the needle's minimum radius of curvature while steering. Assessing the effect of these parameters on the overall system's clinical utility is important in informing future design choices and understanding the capabilities and limitations of the system. In this paper, we analyze the effect of various settings for these three robot parameters on the percentage of the lung that the robot can reach. We combine Monte Carlo random sampling of piercing configurations with a Rapidly-exploring Random Trees based steerable needle motion planner in simulated human lung environments to asymptotically accurately estimate the volume of sites in the lung reachable by the robot. We highlight the importance of each parameter on the overall system's reachable workspace in an effort to motivate future device innovation and highlight design trade-offs.

Existing static and kinematic models of concentric tube robots are based on the ordinary differential equations of a static Cosserat rod. In this paper, we provide the first dynamic model for concentric tube continuum robots by adapting the partial differential equations of a dynamic Cosserat rod to describe the coupled inertial dynamics of precurved concentric tubes. This generates an initial-boundary-value problem that can capture robot vibrations over time. We solve this model numerically at high time resolutions using implicit finite differences in time and arc length. This approach is capable of resolving the high-frequency torsional dynamics that occur during unstable "snapping" motions and provides a simulation tool that can track the true robot configuration through such transitions. Further, it can track slower oscillations associated with bending and torsion as a robot interacts with tissue at real-time speeds. Experimental verification of the model shows that this wide range of effects is captured efficiently and accurately.

Multisection continuum arms offer complementary characteristics to those of traditional rigid-bodied robots. Inspired by biological appendages, such as elephant trunks and octopus arms, these robots trade rigidity for compliance and accuracy for safety and, therefore, exhibit strong potential for applications in human-occupied spaces. Prior work has demonstrated their superiority in operation in congested spaces and manipulation of irregularly shaped objects. However, they are yet to be widely applied outside laboratory spaces. One key reason is that, due to compliance, they are difficult to control. Sophisticated and numerically efficient dynamic models are a necessity to implement dynamic control. In this paper, we propose a novel numerically stable center-of-gravity-based dynamic model for variable-length multisection continuum arms. The model can accommodate continuum robots having any number of sections with varying physical dimensions. The dynamic algorithm is of O (n2) complexity, runs at 9.5 kHz, simulates six to eight times faster than real time for a three-section continuum robot, and, therefore, is ideally suited for real-time control implementations. The model accuracy is validated numerically against an integral-dynamic model proposed by the authors and experimentally for a three-section pneumatically actuated variable-length multisection continuum arm. This is the first sub-real-time dynamic model based on a smooth continuous deformation model for variable-length multisection continuum arms.

We introduce a method to optimize on a patient-specific basis the kinematic design of the Continuum Reconfigurable Incisionless Surgical Parallel (CRISP) robot, a needle-diameter medical robot based on a parallel structure that is capable of performing minimally invasive procedures. Our objective is to maximize the ability of the robot's tip camera to view tissue surfaces in constrained spaces. The kinematic design of the CRISP robot, which greatly influences its ability to perform a task, includes parameters that are fixed before the procedure begins, such as entry points into the body and parallel structure connection points. We combine a global stochastic optimization algorithm, Adaptive Simulated Annealing (ASA), with a motion planner designed specifically for the CRISP robot. ASA facilitates exploration of the robot's design space while the motion planner enables evaluation of candidate designs based on their ability to successfully view target regions on a tissue surface. By leveraging motion planning, we ensure that the evaluation of a design only considers motions which do not collide with the patient's anatomy. We analytically show that the method asymptotically converges to a globally optimal solution and demonstrate our algorithm's ability to optimize kinematic designs of the CRISP robot on a patient-specific basis.

Lung cancer is the deadliest form of cancer, and early diagnosis is critical to favorable survival rates. Definitive diagnosis of lung cancer typically requires needle biopsy. Common lung nodule biopsy approaches either carry significant risk or are incapable of accessing large regions of the lung, such as in the periphery. Deploying a steerable needle from a bronchoscope and steering through the lung allows for safe biopsy while improving the accessibility of lung nodules in the lung periphery. In this work, we present a method for extracting a cost map automatically from pulmonary CT images, and utilizing the cost map to efficiently plan safe motions for a steerable needle through the lung. The cost map encodes obstacles that should be avoided, such as the lung pleura, bronchial tubes, and large blood vessels, and additionally formulates a cost for the rest of the lung which corresponds to an approximate likelihood that a blood vessel exists at each location in the anatomy. We then present a motion planning approach that utilizes the cost map to generate paths that minimize accumulated cost while safely reaching a goal location in the lung.

Continuum Reconfigurable Incisionless Surgical Parallel (CRISP) robots consist of multiple needle-diameter flexible instruments that are assembled into a parallel structure inside the human body. With a camera placed at the tip of one of the instruments, the CRISP robot can be used to inspect anatomical sites in constrained body cavities in a minimally invasive manner. We introduce a motion planner for CRISP robots that computes manipulations of the flexible instruments outside the body such that the camera can visually inspect a user-specified site of clinical interest inside the body. Our sampling-based motion planner ensures avoidance of collisions with anatomical obstacles inside the body, enforces remote-center-of-motion constraints on the instrument's entry points into the body, and efficiently handles the expensive computation of CRISP robot kinematics. We also extend the motion planner to estimate the set of points inside a body cavity that can be visually inspected by the camera of a CRISP robot for a given setup. We demonstrate our method in a simulated endoscopic medical procedure in the pleural space around a lung.

Magnet-tipped, elastic rods can be steered by an external magnetic field to perform surgical tasks. Such rods could be useful for a range of new medical applications because they do not require either pull wires or other bulky mechanisms that are problematic in small anatomical regions. However, current magnetic rod steering systems are large and expensive. Here, we describe a method to guide a rod using a robot-manipulated magnet located near a patient. We solve for rod deflections by combining permanent-magnet models with a Kirchhoff elastic rod model and use a resolved-rate approach to compute trajectories. Experiments show that three-dimensional trajectories can be executed accurately without feedback and that the system's redundancy can be exploited to avoid obstacles.

When designing continuum robots for applications that require sensing, designers are faced with the problems of deciding what sensors to use, where they should be placed, and how best to use the information they provide. In this paper, we describe how a differential representation of a continuum robot's kinematic equations that govern its states (e.g., shape) can be used to simultaneously address these problems under the guidance of statistical state estimation. We identify how state estimation and sensing-system design (i.e., sensor selection and placement) are inherently coupled problems, which leads us to formulate sensing-system design as an optimization problem governed by the results of statistical estimation. As a case-study, the methods described herein are used to design a magnetic sensing system for concentric-tube robots.

Surgeons and robots typically use different approaches for bone milling. Surgeons adjust their speed and tool incidence angle constantly, which enables them to efficiently mill porous bone. Surgeons also adjust milling parameters such as speed and depth of cut throughout the procedure based on proximity to sensitive structures like nerves and blood vessels. In this paper we use image-based bone density estimates and segmentations of vital anatomy to make a robot mill more like a surgeon and less like an industrial computer numeric controlled (CNC) milling machine. We produce patient-specific plans optimizing velocity and incidence angles for spherical cutting burrs. These plans are particularly useful in bones of variable density and porosity like the human temporal bone. They result in fast milling in non-critical areas, reducing overall procedure time, and lower forces near vital anatomy. We experimentally demonstrate the algorithm on temporal bone phantoms and show that it reduces mean forces near vital anatomy by 63% and peak forces by 50% in comparison to a CNC-type path, without adding time to the procedure.

We propose a new class of robotic device for minimally-invasive surgery that lies at the intersection of continuum, parallel, and reconfigurable robotics. This Continuum Reconfigurable Incisionless Surgical Parallel (CRISP) paradigm involves the use of multiple needle-diameter devices inserted through the skin and assembled into parallel structures inside the body. The parallel structure can be reconfigured inside the patient's body to satisfy changing task requirements such as reaching initially inaccessible locations or modifying mechanical stiffness for manipulation or palpation. Another potential advantage of the CRISP concept is that many small (needle-sized) entry points into the patient may be preferable in terms of both patient healing and cosmesis to the single (or multiple) larger ports needed to admit current surgical robots. This paper presents a mechanics-based model for CRISP forward and inverse kinematics, along with experimental validation.

Concentric tube robots are needle-sized manipulators which have been investigated for use in minimally invasive surgeries. It was noted early in the development of these devices that elastic energy storage can lead to rapid snapping motion for designs with moderate to high tube curvatures. Substantial progress has recently been made in the concentric tube robot community in designing snap-free robots, planning stable paths, and characterizing conditions that result in snapping for specific classes of concentric tube robots. However, a general measure for how stable a given robot configuration is has yet to be proposed. In this paper, we use bifurcation and elastic stability theory to provide such a measure, as well as to produce a test for determining whether a given design is snap-free (i.e. whether snapping can occur anywhere in the unloaded robot's workspace). These results are useful in designing, planning motions for, and controlling concentric tube robots with high curvatures.

In designing concentric tube robots, it is often desirable to use curvatures that are as high as possible. However, with high curvatures comes the potential for elastic instabilities. This can be addressed by motion planning or by designing the robot to preclude the possibility of instability. In this paper, we pursue the latter. Our primary contribution is to show that local bifurcation theory can be used to predict the existence of elastic instability anywhere in the workspace of robots with arbitrarily many tubes, with straight transmissions. We show how to use these results to design robots that are guaranteed to remain elastically stable at all times.

Lung cancer is the most deadly form of cancer, and survival depends on early-stage diagnosis and treatment. Transoral access is preferable to traditional between-the-ribs needle insertion because it is less invasive and reduces risk of lung collapse. Yet many sites in the peripheral zones of the lung or distant from the bronchi cannot currently be accessed transorally, due to the relatively large diameter and lack of sufficient steerablity of current instrumentation. To remedy this, we propose a new robotic system that uses a tendon-actuated device (bronchoscope) as a first stage for deploying a concentric tube robot, which itself is a vehicle through which a bevel steered needle can be introduced into the soft tissue of the lung outside the bronchi. In this paper we present the various components of the system and the workflow we envision for deploying the robot to a target using image guidance. We describe initial validation experiments in which we puncture ex vivo bronchial wall tissue and also target a nodule in a phantom with an average final tip error of 0.72 mm.

Concentric tube robots are thin, tentacle-like devices that can move along curved paths and can potentially enable new, less invasive surgical procedures. Safe and effective operation of this type of robot requires that the robot's shaft avoid sensitive anatomical structures (e.g., critical vessels and organs) while the surgeon teleoperates the robot's tip. However, the robot's unintuitive kinematics makes it difficult for a human user to manually ensure obstacle avoidance along the entire tentacle-like shape of the robot's shaft. We present a motion planning approach for concentric tube robot teleoperation that enables the robot to interactively maneuver its tip to points selected by a user while automatically avoiding obstacles along its shaft. We achieve automatic collision avoidance by precomputing a roadmap of collision-free robot configurations based on a description of the anatomical obstacles, which are attainable via volumetric medical imaging. We also mitigate the effects of kinematic modeling error in reaching the goal positions by adjusting motions based on robot tip position sensing. We evaluate our motion planner on a teleoperated concentric tube robot and demonstrate its obstacle avoidance and accuracy in environments with tubular obstacles.

The needle-sized surgical tools used in arthroscopy, otolaryngology, and other surgical fields could become even more valuable to surgeons if endowed with the ability to navigate around sharp corners to manipulate or visualize tissue. We present a needle-sized wrist design that grants this ability. It can be easily interfaced with manual tools or concentric tube robots and is straightforward and inexpensive to manufacture. The wrist consists of a nitinol tube with several asymmetric cutouts, actuated by a tendon. Perhaps counter-intuitively, within this seemingly simple design concept, design optimization is challenging due to the number of parameters available and nonlinearities in material properties. In this paper, we examine a subset of possible geometries and derive kinematic and static models. Experimental results with a 1.16 mm diameter prototype validate the models. Lastly, we provide a discussion summarizing the lessons learned in our early experience designing and fabricating wrists of this type.

Several robotic systems have been proposed for removing blood from the brain in patients who have undergone a hemorrhagic stroke. In this paper we explore the use of imagebased feedback to address tissue deformation when aspirating a hemorrhage in a phantom model. This is the first time intraoperative image feedback has been used with a concentric tube robot in this application. We describe a layer by layer approach to motion planning. Computed tomography (CT) images are collected periodically during hemorrhage removal. After each CT scan, the robot's tip path is re-planned to account for the tissue deformation that has occurred since the previous scan. We compare open loop hemorrhage removal to our sequential imaging-replanning approach, illustrating that the latter has the potential to enhance the safety and efficacy of the procedure.

Lung cancer is the leading cause of cancer-related death, and early-stage diagnosis is critical to survival. Biopsy is typically required for a definitive diagnosis, but current low-risk clinical options for lung biopsy cannot access all biopsy sites. We introduce a motion planner for a multilumen transoral lung access system, a new system that has the potential to perform safe biopsies anywhere in the lung, which could enable more effective early-stage diagnosis of lung cancer. The system consists of three stages in which a bronchoscope is deployed transorally to the lung, a concentric tube robot pierces through the bronchial tubes into the lung parenchyma, and a steerable needle deploys through a properly oriented concentric tube and steers through the lung parenchyma to the target site while avoiding anatomical obstacles such as significant blood vessels. A sampling-based motion planner computes actions for each stage of the system and considers the coupling of the stages in an efficient manner. We demonstrate the motion planner's fast performance and ability to compute plans with high clearance from obstacles in simulated anatomical scenarios.

Concentric tube robots can enable new clinical interventions if they are able to pass through soft tissue, deploy along desired paths through open cavities, or travel along winding lumens. These behaviors require the robot to deploy in such a way that the curved shape of its shaft remains unchanged as the tip progresses forward (i.e., "follow-the-leader" deployment). Follow-the-leader deployment is challenging for concentric tube robots due to elastic (and particularly torsional) coupling between the tubes that form the robot. However, as we show in this paper, follow-the-leader deployment is possible, provided that tube precurvatures and deployment sequences are appropriately selected. We begin by defining follow-the-leader deployment and providing conditions that must be satisfied for a concentric tube robot to achieve it. We then examine several useful special cases of follow-the-leader deployment, showing that both circular and helical precurvatures can be employed, and provide an experimental illustration of the helical case. We also explore approximate follow-the-leader behavior and provide a metric for the similarity of a general deployment to a follow-the-leader deployment. Finally, we consider access to the hippocampus in the brain to treat epilepsy, as a motivating clinical example for follow-the-leader deployment.

Benign prostatic hyperplasia is the most common symptomatic disease in men. A new transurethral surgical intervention is available that has been shown to reduce bleeding, catheterization time, and hospitalization time in comparison to traditional Transurethral Resection of the Prostate (TURP). However, this new procedure, Holmium Laser Enucleation of the Prostate (HoLEP), is so challenging to accomplish that only a small number of expert surgeons are able to offer it. Toward facilitating broader use of HoLEP, we propose a new hand-held robotic system for the purpose of making the surgery easier to perform. In current HoLEP, the only way to aim the laser and/or manipulate tissue is to move the entire endoscope, stretching a large quantity of tissue. In contrast, our new robotic approach provides the surgeon with two concentric tube manipulators that can aim the laser and manipulate tissue simultaneously. The manipulators are deployed through a 5 mm working channel in a 26 French (8.66 mm) endoscope clinically used for transurethral procedures. This paper describes the design of the robot and experiments illustrating its ability to perform the motions expected to be useful in HoLEP.

Concentric tube robots exhibit complex workspaces due to the way their component tubes bend and twist as they interact with one another. This paper explores ways to compute and characterize their workspaces. We use Monte Carlo random samples of the robot's joint space and a discrete volumetric workspace representation, which can describe both reachability and redundancy. Experiments on two physical prototypes are provided to illustrate the proposed approach.

Steerable needles have the potential to improve the effectiveness of needle-based clinical procedures such as biopsy and drug delivery by improving targeting accuracy and reaching previously inaccessible targets that are behind sensitive or impenetrable anatomical regions. We present a new needle steering system capable of automatically reaching targets in 3-D environments while avoiding obstacles and compensating for real-world uncertainties. Given a specification of anatomical obstacles and a clinical target (e.g., from preoperative medical images), our system plans and controls needle motion in a closed-loop fashion under sensory feedback to optimize a clinical metric. We unify planning and control using a new fast algorithm that continuously replans the needle motion. Our rapid replanning approach is enabled by an efficient sampling-based rapidly exploring random tree (RRT) planner that achieves orders-of-magnitude reduction in computation time compared with prior 3-D approaches by incorporating variable curvature kinematics and a novel distance metric for planning. Our system uses an electromagnetic tracking system to sense the state of the needle tip during the procedure. We experimentally evaluate our needle steering system using tissue phantoms and animal tissue ex vivo. We demonstrate that our rapid replanning strategy successfully guides the needle around obstacles to desired 3-D targets with an average error of less than 3 mm.

Concentric tube continuum robots provide an infinite-dimensional design space, consisting of individual tube space curves and other tube parameters. Even when design choices are made to restrict the design space to a small number of discrete parameters, ad hoc selection of parameter values to achieve coverage of a desired volume, in the presence of geometric workspace constraints, is essentially impossible - even for experienced researchers. General design algorithms proposed to date have focused on reaching a discrete set of specific points, and have made non-physical approximations in the robot model (most significantly assuming infinite torsional rigidity), to speed up model computation. In this paper, we extend prior algorithms to use more accurate models and incorporate volume-based objectives. These extensions are illustrated in a case study on the design of a concentric tube robot for endonasal pituitary surgery. We show that volume-based design optimization increases the reachable percentage of the surgical workspace by an average of approximately 50%, in comparison to various sets of manually selected design parameters. We conclude that volume-based objectives should be included in future multi-objective design optimization procedures for concentric tube continuum robots.

Continuum robots have opened a broad array of applications to robotics in general, and the concentric tube continuum robots promise many benefits in medicine. Many people intuitively assume that these robots can deploy along a curved trajectory, in such a way that the curved shape of the robot's shaft remains unchanged as the tip progresses forward (i.e. “follow-the-leader” deployment). This capability would be useful in advancing along winding lumens (e.g. blood vessels, lung bronchi, etc.), as well as when the device is embedded in soft tissue and used as a steerable needle. However, in this paper we show that deploying in a follow-the-leader manner is not possible except in very special cases of tube precurvatures, combined with specific deployment sequences. We also show that follow-the-leader deployment is not possible even for many of the “simple” cases where one might intuitively expect it to be. Fortunately, useful special cases of perfect follow-the-leader behavior do exist, and we provide examples and describe the conditions that must be satisfied for this to be possible. We also study approximate follow-the-leader behavior, proposing a metric to quantify the similarity of a general deployment to a follow-the-leader deployment.

The high incidence of intracerebral hemorrhages, together with a 40% mortality rate, provide strong motivation for enhancements in the treatment methods available to physicians. To minimize the disruption to healthy brain tissue associated with gaining access to the surgical site that is imposed by traditional open or endoscopic surgical intervention, we propose a new minimally-invasive, image-guided, robotic approach that provides articulation within the lesion at the tip of a needle. In this paper we present a biocompatible and sterilizable robot, together with an image-guidance approach designed to deliver the tip of the needle accurately to the blood clot and to move it within the clot, to aspirate it. An experimental evaluation demonstrates removal of 92% of the target clot tissue in a proof-of-concept phantom study.

Steerable needles can potentially increase the accuracy of needle-based diagnosis and therapy delivery, provided they can be adequately controlled based on medical image information. We propose a novel sliding mode control law that can be used to deliver the tip of a flexible asymmetric-tipped needle to a desired point, or to track a desired trajectory within tissue. The proposed control strategy requires no a priori knowledge of model parameters, has bounded input speeds, and requires little computational resources. We show that if the standard nonholonomic model for tip-steered needles holds, then the control law will converge to desired targets in a reachable workspace, within a tolerance that can be defined by the control parameters. Experimental results validate the control law for target points and trajectory following in phantom tissue and ex vivo liver. Experiments with targets that move during insertion illustrate robustness to disturbances caused by tissue deformation.

This paper presents a versatile magnetic resonance imaging (MRI) compatible concentric tube continuum robotic system. The system enables MR image-guided placement of a curved, steerable active cannula. It is suitable for a variety of clinical applications including image-guided neurosurgery and percutaneous interventions, along with procedures that involve accessing a desired image target, through a curved trajectory. This 6 degree-of-freedom (DOF) robotic device is piezoelectrically actuated to provide precision motion with joint-level precision of better than 0.03mm, and is fully MRI-compatible allowing simultaneous robotic motion and imaging with no image quality degradation. The MRI compatibility of the robot has been evaluated under 3 Tesla MRI using standard prostate imaging sequences, with an average signal to noise ratio loss of less than 2% during actuator motion. The accuracy of active cannula control was evaluated in benchtop trials using an external optical tracking system with RMS error in tip placement of 1.00mm. Preliminary phantom trials of three active cannula placements in the MRI scanner showed cannula trajectories that agree with our kinematic model, with a RMS tip placement error of 0.61 - 2.24 mm.

We introduce a method for task-oriented design of concentric tube robots, which are tentacle-like robots with the potential to enable new minimally invasive surgical procedures. Our objective is to create a robot design on a patient-specific and surgery-specific basis to enable the robot to reach multiple clinically relevant sites while avoiding anatomical obstacles. Our method uses a mechanically accurate model of concentric tube robot kinematics that considers a robot's time-varying shape throughout the performance of a task. Our method combines a search over a robot's design space with sampling-based motion planning over its configuration space to compute a design under which the robot can feasibly perform a specified task without damaging surrounding tissues. To accelerate the algorithm, we leverage design coherence, the observation that collision-free configuration spaces of robots of similar designs are similar. If a solution exists, our method is guaranteed, as time is allowed to increase, to find a design and corresponding feasible motion plan. We provide examples illustrating the importance of using mechanically accurate models during design and motion planning and demonstrating our method's effectiveness in a medically motivated simulated scenario involving navigation through the lung.

We propose the use of a haptic touchscreen to convey graphical and mathematical concepts through aural and/or vibratory tactile feedback. We hypothesize that an important application of such a display will be in teaching visually impaired students concepts that are traditionally learned almost entirely visually. This paper describes initial feasibility studies using a commercially available haptic touchscreen to display grids, points, lines, and shapes - some of the first visual graphical entities students encounter in K-12 mathematics education, and from which more complex lessons can be constructed. We conducted user studies designed to evaluate perception of these objects through haptic feedback alone, auditory feedback alone, and combinations of the two. Our results indicate that both sensory channels can be valuable in user perception.

Kinematic models that account for deformation due to applied loads have recently been developed for a variety of continuum robots. In these cases, a set of nonlinear differential equations with boundary conditions must often be solved to obtain the robot shape. Thus, computing manipulator Jacobians and compliance matrices efficiently is not straight forward. In this paper, we propose a method for obtaining an arc length parametrized Jacobian and compliance matrix. Our approach involves obtaining an augmented Jacobian by propagating the necessary partial derivatives through the model equations, resulting in a new set of differential equations. These equations can be solved as an initial value problem, via a single numerical integration. Our method can be generally applied to various continuum robot architectures, regardless of the specific actuation system used. We provide a specific case study using this method to obtain the Jacobian for a concentric-tube robot.

We describe transnasal skull base surgery, including the current clinical procedure and the ways in which a robotic system has the potential to enhance the current standard of care. The available workspace is characterized by segmenting medical images and reconstructing the available 3D geometry. We then describe thin, ¿tentacle-like¿ robotic tools with shafts constructed from concentric tube robots, and an actuation unit designed to robotically control them in a teleoperated setting. Lastly, we discuss the results of a proof-of-concept study in a cadaveric specimen, illustrating the ability of the robot to access clinically relevant skull base targets.

The inherent flexibility of continuum robots allows them to interact with objects in a safe and compliant way. This flexibility also makes it possible to use robot deflection to estimate external forces applied to the robot. This ¿intrinsic force sensing¿ concept is particularly useful for thin continuum robots where application constraints preclude the use of traditional force sensors. This paper describes an Extended Kalman Filter approach to estimate forces applied at the tip of a continuum robot using only uncertain pose measurements and a kinematic-static model of the robot with uncertainty.

Tendons are a widely used actuation strategy for continuum robots that enable forces and moments to be transmitted along the robot from base-mounted actuators. Most prior robots have used tendons routed in straight paths along the robot. However, routing tendons through general curved paths within the robot offers potential advantages in reshaping the workspace and enabling a single section of the robot to achieve a wider variety of desired shapes. In this paper, we provide a new model for the statics and dynamics of robots with general tendon routing paths that is derived by coupling the classical Cosserat-rod and Cosserat-string models. This model also accounts for general external loading conditions and includes traditional axially routed tendons as a special case. The advantage of the usage of this coupled model for straight-tendon robots is that it accounts for the distributed wrenches that tendons apply along the robot. We show that these are necessary to consider when the robot is subjected to out-of-plane external loads. Our experimental results demonstrate that the coupled model matches experimental tip positions with an error of 1.7% of the robot length, in a set of experiments that include both straight and nonstraight routing cases, with both point and distributed external loads.

Shape control of continuum robots requires a means of sensing the the curved shape of the robot. Since continuum robots are deformable, they take on shapes that are general curves in space, which are not fully defined by actuator positions. Vision-based shape-estimation provides a promising avenue for shape-sensing. While this is often facilitated by fiducial markers, sometimes fiducials are not feasible due to either the robot's application or its size. To address this, we present a robust and efficient stereo-vision-based, shape-sensing algorithm for continuum robots that does not rely on fiducials or assume orthogonal camera placement. The algorithm employs self-organizing maps to triangulate three-dimensional backbone curves. Experiments with an object with a known shape demonstrate an average accuracy of 1.53 mm on a 239 mm arc length curve.

Intraoperative surface contour sensing can enable the registration of high-resolution three-dimensional preoperative images for precise guidance of surgical robots. This is particularly useful for guiding steerable needles in soft tissues. In this paper we combine a new minimally invasive surface scanning technique based on conoscopic holography with a steerable active cannula robot. We experimentally demonstrate cannula tip placement to multiple physical points inside phantom tissue, which correspond to points specified in preoperative images - the input an eventual clinical system would obtain from the physician. While the image-guided steerable system we propose is broadly applicable to many kinds of surgery, one particular application of interest is in ablating large liver tumors, where it is beneficial for the ablator to be repositioned to multiple locations without being withdrawn from the organ.

Medical procedures such as lung biopsy and brachytherapy require maneuvering through tubular structures such as the trachea and bronchi to reach clinical targets. We introduce a new method to plan configurations for active cannulas, medical devices composed of thin, pre-curved, telescoping lumens that are capable of following controlled, curved paths through open or liquid-filled cavities. Planning optimal configurations for these devices is challenging due to their complex kinematics, which involve both beam mechanics and space curves. In this paper, we propose an optimization-based planning algorithm that computes active cannula configurations through tubular structures that reach specified targets. Given the target location, the start position and orientation, and a geometric representation of the physical environment extracted from pre-procedure medical images, the planner optimizes insertion length and orientation angle of each lumen of the active cannula. The planner models active cannula kinematics using a physically-based simulation that incorporates beam mechanics and minimizes energy. The algorithm typically computes plans in less than 2 minutes on a standard PC. We apply the method in simulation to anatomy extracted from a human CT scan and demonstrate configurations for a 5-lumen active cannula that maneuver it through the bronchi to targets in the lung.

Continuum robots made from telescoping precurved elastic tubes enable base-mounted actuators to specify the curved shapes of robots as thin as standard surgical needles. While free space beam mechanics-based models of the shape of these `active cannulas' exist, current models cannot account for external forces and torques applied to the cannula by the environment. In this paper we apply geometrically exact beam theory to solve the statics problem for concentric-tube continuum robots. This yields the equivalent of forward kinematics for an active cannula with general tube precurvature functions and arbitrarily many tubes, under loading from a general wrench distribution. The model achieves average experimental tip errors of less than 3 mm over the workspace of a prototype active cannula subject to various tip forces.

Cochlear electrode insertion is a challenging manual procedure. One technique requires the physician to coordinate the motions of an electrode array approximately 1mm in diameter and the smaller stylet within it, using miniature forceps. A new minimally invasive access technique precludes forceps insertion because the electrode must travel through a small-diameter drilled hole to reach the cochlear access point. To address this, we present an automated insertion tool. This second generation device not only enables deployment in the minimally invasive setting, but also makes insertion velocity profiles repeatable and can sense insertion forces. Force sensing is essential because insertion forces can indicate impending damage to cochlear membranes, but are below the thresholds that can be sensed by human hands. The Automated Insertion Tool we present is designed to be compact and lightweight for straightforward integration into the operating room environment. It is able to insert an electrode with a resolution of less than 1μm, achieve velocities of up to 5 mm/sec and resolve forces as small as 0.005 N.

Continuum robots, which are composed of multiple concentric, precurved elastic tubes, can provide dexterity at diameters equivalent to standard surgical needles. Recent mechanics-based models of these "active cannulas" are able to accurately describe the curve of the robot in free space, given the preformed tube curves and the linear and angular positions of the tube bases. However, in practical applications, where the active cannula must interact with its environment or apply controlled forces, a model that accounts for deformation under external loading is required. In this paper, we apply geometrically exact rod theory to produce a forward kinematic model that accurately describes large deflections due to a general collection of externally applied point and/or distributed wrench loads. This model accommodates arbitrarily many tubes, with each having a general preshaped curve. It also describes the independent torsional deformation of the individual tubes. Experimental results are provided for both point and distributed loads. Average tip error under load was 2.91 mm (1.5%-3% of total robot length), which is similar to the accuracy of existing free-space models.

Active cannulas are a relatively new continuum robot subclass characterized by their use of preshaped tubes that transmit bending moments as they slide within one another and are axially rotated. Previous (experimentally vetted) mechanics-based models of active cannula shape assume piecewise constant precurvature of component tubes, and neglect torsion in curved sections of the device. Recently a general, coordinate-free, energy-based framework for active cannula shape has been formulated that relaxes these requirements and includes all prior models as special cases. However, only the 2-tube, constant-precurvature case has thus far been explored in detail using the framework. In this paper we consider the general case of an arbitrary number of component tubes and precurvatures that vary with arc length, deriving a set of differential equations that capture both bending and torsional effects continuously along the active cannula backbone.We then show how to solve these differential equations numerically to describe active cannula shape.

An active cannula is a medical device composed of thin, pre-curved, telescoping tubes that may enable many new surgical procedures. Planning optimal motions for these devices is challenging due to their kinematics, which involve both beam mechanics and space curves. In this paper, we propose an optimization-based motion planning algorithm that computes actions to guide the device to a target point while avoiding obstacles in the environment. The planner uses a simplified active cannula kinematic model that neglects beam mechanics, and focuses on planning for the (piecewise circular) space curves. The method is intended for use in image-guided procedures where the target and obstacles can be segmented from pre-procedure images. Given the target location, the start position and orientation, and a geometric representation of obstacles, the algorithm computes the insertion length and orientation angle for each tube of the active cannula such that the device follows a collision-free path to the target. We formulate the planning problem as a constrained nonlinear optimization problem and use a penalty method to convert this formulation into a sequence of more easily solvable unconstrained optimization problems. Simulations demonstrate optimal paths for a 3-tube active cannula with spherical obstacles. The algorithm typically computes plans in less than 1 minute on a standard PC.

We present design and experimental performance results for a novel mechanism for robotic legged locomotion at the mesoscale (from hundreds of microns to tens of centimeters). The new mechanism is compact and strikes a balance between conflicting design objectives, exhibiting high foot forces and low power consumption. It enables a small robot to traverse a compliant, slippery, tubular environment, even while climbing against gravity. This mechanism is useful for many mesoscale locomotion tasks, including endoscopic capsule robot locomotion in the gastrointestinal tract. It has enabled fabrication of the first legged endoscopic capsule robot whose mechanical components match the dimensions of commercial pill cameras (11 mm diameter by 25 mm long). A novel slot-follower mechanism driven via lead screw enables the mechanical components of the capsule robot to be as small while simultaneously generating 0.63 N average propulsive force at each leg tip. In this paper, we describe kinematic and static analyses of the lead screw and slot-follower mechanisms, optimization of design parameters, and experimental design and tuning of a gait suitable for locomotion. A series ofex vivoexperiments demonstrate capsule performance and ability to traverse the intestine in a manner suitable for inspection of the colon in a time period equivalent to standard colonoscopy.

This paper presents a new class of thin, dexterous continuum robots, which we call active cannulas due to their potential medical applications. An active cannula is composed of telescoping, concentric, precurved superelastic tubes that can be axially translated and rotated at the base relative to one another. Active cannulas derive bending not from tendon wires or other external mechanisms but from elastic tube interaction in the backbone itself, permitting high dexterity and small size, and dexterity improves with miniaturization. They are designed to traverse narrow and winding environments without relying on ldquoguidingrdquo environmental reaction forces. These features seem ideal for a variety of applications where a very thin robot with tentacle-like dexterity is needed. In this paper, we apply beam mechanics to obtain a kinematic model of active cannula shape and describe design tools that result from the modeling process. After deriving general equations, we apply them to a simple three-link active cannula. Experimental results illustrate the importance of including torsional effects and the ability of our model to predict energy bifurcation and active cannula shape.

Active cannulas are remotely actuated thin continuum robots with the potential to traverse narrow and winding environments without relying on "guiding" environmental reaction forces. These features seem ideal for procedures requiring passage through narrow openings to access air-filled cavities (e.g. surgery in the throat and lung). Composed of telescoping concentric pre-curved elastic tubes, an active cannula is actuated at its base by translation and axial rotation of component tubes. Using minimum energy principles and Lie Group theory, we present a framework for the kinematics of multi-link active cannulas. This framework permits testing of the hypothesis that overall cannula shape locally minimizes stored elastic energy. We evaluate in particular whether the torsional energy in the long, straight transmission between actuators and the curved sections is important. Including torsion in the kinematic model enables us to analytically predict experimentally observed bifurcation in the energy landscape. Independent calibration procedures based on bifurcation and tip and feature positions enable model parameter identification, producing results near ranges expected from tube material properties and geometry. Experimental results validate the kinematic framework and demonstrate the importance of modeling torsional effects in order to describe bifurcation and accurately predict active cannula shape.

Many compelling future surgical applications will be enabled by a new kind of surgical tool, capable of entering the human body through natural orifices or very small incisions and then reconfiguring into complex kinematic structures at the site of intervention. We describe a first step toward this goal - the development of a microactuator designed for use in surgical robots that are composed of large quantities of reconfigurable micro-robotic modules. The miniaturizable design proposed harnesses the Bourdon effect to convert electrolytic pressure into mechanical motion obtaining more than 400% displacement variation while consuming less than 0.5 W at less than 5 V. We describe the design, construction, and experimental results with our prototype microactuator.

In this paper we present the design of a swallowable (11mm diameter by 25mm long), 12-legged endoscopic capsule for locomotion in the lower gastro intestinal tract (large bowel). A novel slot-follower mechanism driven via lead-screw allows the capsule to be as small as current commercial pill-cameras, while simultaneously generating 2/3 N of force at each leg tip. Kinematic and static analyses of the lead screw and slot-follower mechanisms allow optimization of design parameters so that the capsule satisfies experimental and clinical design requirements for legged locomotion in the GI tract.

Needles are commonly used in medical practice as a minimally invasive means to reach subsurface targets for diagnosis or therapy delivery. Recent results indicate that steerable needles may enhance targeting accuracy and allow needles to avoid obstacles along the path to the target. This work considers teleoperation of needles made of a superelastic alloy that steer through tissue using forces generated by the standard asymmetric bevel tip. The needle may be modeled as a nonholonomic system, with inputs of insertion along and spin about the needle axis. A teleoperation system consisting of a commercial master haptic device, a custom needle-steering robot slave, and visual feedback to the operator was assembled. Human subjects experiments were performed to evaluate targeting accuracy in phantom tissue for three needle control methods: teleoperation of both insertion and spin, teleoperation of insertion with open-loop-controlled spin, and open-loop control of both insertion and spin. Targeting accuracy improved with increasing degrees of freedom of human (teleoperation) control, primarily because tissue deformation and modeling limitations result in open-loop control errors. Subjects typically performed multiple spins of the needle during insertion in order to fine tune the needle path. In addition, position, rate, and a nonlinear hybrid control were compared during teleoperation of the insertion degree of freedom. The hybrid method resulted in significantly better targeting accuracy

We have developed a new class of continuously flexible snake-like robots, called active cannulas, that consist of several telescoping pre-curved superelastic tubes. The devices derive bending actuation not from tendon wires or other external mechanisms, but from elastic energy stored in the backbone itself. This allows active cannulas to have a small diameter and a high degree of dexterity, which should enable them to navigate through complex anatomy to sites inaccessible by current surgical robotic devices. Active cannulas may also enhance patient safety because their inherent compliance mitigates potential trauma from inadvertent tool-tissue collision. A consequence of our design is that dexterity improves with miniaturization. A kinematic description of active cannula shape requires a model of the elastic interaction of telescoping pre-curved flexible tubes, and we derive a two-"link" beam mechanics-based model. Experiments using curved nitinol tubes and wires validate the model.

Many medical procedures involve the use of needles, but targeting accuracy can be limited due to obstacles in the needle’s path, shifts in target position caused by tissue deformation, and undesired bending of the needle after insertion. In order to address these limitations, we have developed robotic systems that actively steer a needle in soft tissue. A bevel (asymmetric) tip causes the needle to bend during insertion, and steering is enhanced when the needle is very flexible. An experimental needle steering robot was designed that includes force/torque sensing, horizontal needle insertion, stereo image data acquisition, and controlled actuation of needle rotation and translation. Experiments were performed with a phantom tissue to determine the effects of insertion velocity and bevel tip angle on the needle path, as well as the forces acting on the needle during insertion. Results indicate that needle steering inside tissue does not depend on insertion velocity, but does depend on bevel tip angle. In addition, the forces acting on the needle are directly related to the insertion velocity.

A novel two-degree-of-freedom tactile display reproduces the sensations of sliding contact and incipient slip through the rotation of a ball positioned under the user's fingertip. A pair of motor-driven wheels actuates the ball via contact friction. Mechanical performance requirements are used to define the dimensions and construction method of the device. Kinematic analysis shows that the drive wheel angles and their contact locations with the ball must be carefully selected in order to accurately control the axis of rotation and speed of the ball. However, psychophysical experiments indicate that some kinematic error is tolerable; errors of up to 20° in slip angle and 30% of a nominal velocity may be applied without detection from an average user. The lightweight, modular tactile display was attached to a multi-degree-of-freedom kinesthetic interface and used to display virtual environments with slip. Experimental results demonstrate that users complete a virtual paper manipulation task with lower applied forces using combined slip and force feedback in comparison with conventional force feedback alone.

The "haptic scissors" is a device that creates the sensation of cutting in virtual environments. The scissors have two degrees of freedom of motion and force feedback, one for cutting (single blade rotation) and one for translation. An algorithm was developed to simultaneously display translational and cutting forces for a realistic cutting simulation. In previous work, we use filtered data from cutting biological tissues to create "haptic recordings" of the cutting experience. Here, we consider two cutting models: one based on real tissue data and one that is analytical. The model based on real tissue is a segmented linear empirical model of the original data. Experimental results show that users cannot differentiate between these models and the haptic recordings created earlier. The analytical model uses a combination of friction, assumed material properties, and user motion (position and velocity) to determine the displayed cutting forces.

Surgical robots, including those with remote center of motion (RCM) mechanisms, have demonstrated utility in image-guided percutaneous needle placement procedures. However, widespread clinical application of these robots is hindered by not only complicated mechanical design but also the need for calibration and registration of the robot to the medical imager prior to each use. In response, we propose a Virtual RCM algorithm that requires only online tracking or registering the surgical tool to the imager, and a five degree-of-freedom (DOF) robot comprised of three prismatic DOF decoupled from two rotational DOF. The robot can be unencoded, uncalibrated, and does not require pre-operative registration. An incremental adaptive motion control cycle both guides the needle to the insertion point and orients it to align with the target. The robot executes RCM motion “virtually” without having a physically constrained fulcrum point. The proof-of-concept prototype system achieved 0.78 mm translation and 1.4 degrees rotational accuracy (within the tracker accuracy), within 17 iterative steps (0.5–1s).

In the modern world mobile robots are being utilized for many tasks that are either hazardous or unpleasant for human beings. The potential of robots to assume such tasks hinges on their ability to intelligently, efficiently, and reliably locate and interact with objects in their environment. This work focuses on the combination of intelligent sensor placement and a unique vision strategy (which itself combines several techniques) with reactive system theory to create the Electronic Ball Boy. This system is highly successful at performing the task of capture and retrieval of tennis balls, a task that is repetitious yet lacking in well-defined locations and motions for the objects of interest.