This paper presents a method for comprehensive 6D estimation of tip wrench for generally deflected static elastic rods. Current methods for load estimation are restricted to estimating lateral (point or distributed) forces for (quasi-)planar deformation; estimation of tangential force and moment, i.e., the full 6D wrench, remains largely unreliable due to the ill-posed nature of the problem. To address this challenge, this paper begins by proposing a high-fidelity static rod model that leverages Koopman Operator theory. Building on this model and utilizing shape feedback, a computationally efficient three-step wrench estimator is proposed: (i) a Koopman-UKF local moment observer, (ii) a static equilibrium solver, and (iii) a rod model propagator. Then, a 2D wrench screw system, identified as the insensible wrench in the initial estimation, elucidates error sources and informs strategies to enhance accuracy by incorporating additional feedback, such as the rod tip material frame and base axial force. Ultimately, the framework delivers accurate 6D tip wrench estimation with quantified uncertainty. Simulation and experimental evaluations validate its effectiveness, demonstrating mean errors of$53.14\,$mN (1.94%) and$2.65\,$mNm (7.18%) for a$159\,$mm-long Nitinol tube undergoing complex out-of-plane deformations, outperforming three replicated state-of-the-art methods. Additionally, its applicability to more complex continuum robots is demonstrated through load estimation on a Parallel Continuum Robot.

This paper presents an uncertainty-aware shared control and calibration method for micromanipulation using a digital microscope and a tool-mounted, multi-joint robotic arm, integrating real-time human intervention with a visual-motor policy. Our calibration algorithm leverages co-manipulation control to calibrate the hand-eye transformation without requiring knowledge of the kinematics of the microtool mounted on the robot while remaining robust to camera intrinsics errors. Experimental results show that the proposed calibration method achieves a 39.6% improvement in accuracy over established methods. Additionally, our control structure and calibration method reduces the time required to reach single-point targets from 5.74 s (best conventional method) to 1.91 s, and decreases trajectory tracking errors from 392 μm to 40 μm. These findings establish our method as a robust solution for improving reliability in high-precision biomedical micromanipulation.

Therapies targeting neurodegenerative diseases via brain ventricles and spinal parenchyma face delivery challenges. Systemic administration is ineffective due to the blood-brain barrier, while direct surgical access, especially for multi-site delivery, is highly invasive. The spinal subarachnoid space offers potential for microcatheter-based delivery, but existing robotic catheter technologies are unsuitable due to spinal anatomy constraints. This paper presents a miniaturised and sensorised steerable eversion-growing robot tailored to navigation of the subarachnoid space of the spine. The property of eversion reduces interaction forces with the anatomy, rendering our approach safer than microcatheters that need to be pushed. Our system is capable of real-time tip force estimation with three degrees of freedom (DoF) using fibre Bragg gratings (FBG). Additionally, it incorporates a micro-endoscope and a steerable tip, all within a tiny 2mm outer diameter. The system’s navigation, sensing, and imaging capabilities were evaluated using a realistic up-scaled phantom of the subarachnoid space covering the cervical spine, demonstrating interaction forces within the safe range of 2-5N during phantom navigation. Comparison study of instrument-tissue interactions further approved its clinical relevance, presenting a 73.78% decrease of the mean absolute forces to traditional insertion without the sheath in global measurements.

This paper proposes a novel, more computationally efficient method for optimizing robot excitation trajectories for dynamic parameter identification, emphasizing self-collision avoidance. This addresses the system identification challenges for getting high-quality training data associated with co-manipulated robotic arms that can be equipped with a variety of tools, a common scenario in industrial but also clinical and research contexts. Utilizing the Unified Robotics Description Format (URDF) to implement a symbolic Python implementation of the Recursive Newton-Euler Algorithm (RNEA), the approach aids in dynamically estimating parameters such as inertia using regression analyses on data from real robots. The excitation trajectory was evaluated and achieved on par criteria when compared to state-of-the-art reported results which didn’t consider self-collision and tool calibrations. Furthermore, physical Human-Robot Interaction (pHRI) admittance control experiments were conducted in a surgical context to evaluate the derived inverse dynamics model showing a 30.1% workload reduction by the NASA TLX questionnaire.

This paper presents a lumped-parameter dynamic model of a pressure driven eversion robot carrying a catheter through its hollow core. A simulation framework based on the model is developed in MATLAB and is used for understanding the underlying physics, for identifying the regions of operation, and for demonstrating that, for a range of input commands, the catheter can be used as an actuation mechanism for propelling eversion; an approach especially useful for miniaturised systems. Simulations are experimentally validated on the MAMMOBOT system, which is a miniature steerable soft growing robot for early breast cancer detection. It was demonstrated that for most regions of operation experimental results compare well with simulation exhibiting an error less than 4%. Only one region of operation demonstrated larger deviations due possibly to unmodeled dynamics, which will be investigated in future work.

This paper presents OpTaS, a task specification Python library for Trajectory Optimization (TO) and Model Predictive Control (MPC) in robotics. Both TO and MPC are increasingly receiving interest in optimal control and in particular handling dynamic environments. While a flurry of software libraries exists to handle such problems, they either provide interfaces that are limited to a specific problem formulation (e.g. TracIK, CHOMP), or are large and statically specify the problem in configuration files (e.g. EXOTica, eTaSL). OpTaS, on the other hand, allows a user to specify custom nonlinear constrained problem formulations in a single Python script allowing the controller parameters to be modified during execution. The library provides interface to several open source and commercial solvers (e.g. IPOPT, SNOPT, KNITRO, SciPy) to facilitate integration with established workflows in robotics. Further benefits of OpTaS are highlighted through a thorough comparison with common libraries. An additional key advantage of OpTaS is the ability to define optimal control tasks in the joint-space, task-space, or indeed simultaneously. The code for OpTaS is easily installed via pip, and the source code with examples can be found at github.com/cmower/optas.

This article contributes a novel method to assess robot dexterity. Existing Jacobian-based dexterity metrics, such as the manipulability index or the condition number, do not allow for comparisons between robot architectures, are local in nature, and are affected by robot dimensions (robot size). On the contrary, the introduced metric is global and allows for quantitative comparisons of robot architectures as it explicitly incorporates the orientational and positional coverage of a robot's end-effector. Experiments presented show that the proposed dexterity metric can improve the computational and precision performance of numerical inverse kinematics and showcase its suitability for use in computational dexterous robot design and, in particular, for designing concentric tube robots with high orientational and positional dexterity.

Aortic valve surgery is the preferred procedure for replacing a damaged valve with an artificial one. The ValveTech robotic platform comprises a flexible articulated manipulator and surgical interface supporting the effective delivery of an artificial valve by teleoperation and endoscopic vision. This article presents our recent work on force-perceptive, safe, semiautonomous navigation of the ValveTech platform prior to valve implantation. First, we present a force observer that transfers forces from the manipulator body and tip to a haptic interface. Second, we demonstrate how hybrid forward/inverse mechanics, together with endoscopic visual servoing, lead to autonomous valve positioning. Benchtop experiments and an artificial phantom quantify the performance of the developed robot controller and navigator. Valves can be autonomously delivered with a 2.0±0.5 mm position error and a minimal misalignment of 3.4±0.9°. The hybrid force/shape observer (FSO) algorithm was able to predict distributed external forces on the articulated manipulator body with an average error of 0.09 N. FSO can also estimate loads on the tip with an average accuracy of 3.3%. The presented system can lead to better patient care, delivery outcome, and surgeon comfort during aortic valve surgery, without requiring sensorization of the robot tip, and therefore obviating miniaturization constraints.

Continuum surgical robots can navigate anatomical pathways to reach pathological locations deep inside the human body. Their flexibility, however, generally comes with reduced dexterity at their tip and limited workspace. Building on recent work on eccentric tube robots, this paper proposes a new continuum robot architecture and theoretical framework that combines the flexibility of push/pull actuated snake robots and the dexterity offered by concentric tube robotic end-effectors. We designed and present a prototype system as a proof-of-concept, and developed a tailored quasistatic mechanics-based model that describes the shape and end-effector's pose for this new type robotic architecture. The model can accommodate an arbitrary number of arms placed eccentrically with respect to the backbone's neutral axis. Our experiments show that the error between model and experiment is on average 3.56% of the manipulator's overall length. This is in agreement with state of the art models of single type continuum architecture.

This article presents a unified mathematical approach for modeling, identifying, and solving the friction-inclusive dynamics of robotic mechanisms driven by sliding contact transmission systems. This approach is applied to the most common industrial screw-based drives: 1) the worm drive, 2) the simple lead screw drive, and 3) the antibacklash lead screw drive. The resulting dynamics, handily transferable to single and multiple degree of freedom (DoF) manipulators, are complemented with an algorithm for the solution of the forward dynamics problem as well as a framework for friction identification based on nonlinear optimization. The presented theory is experimentally tested on single-DoF screw-based drives for a variety of payloads and at different operation speeds. Simulation and parameter identification results using the classical Coulomb model, the Coulomb model with Stribeck friction, and the Armstrong model with Stribeck friction, rising static friction, and frictional memory are compared with experimental data, showcasing the effectiveness of the presented approach toward framing the concepts of friction and motion transmission into a robotics setting.

This paper presents the first absolute pose estimation approach tailored to Light Field cameras. It builds on the observation that the ratio between the disparity arising in different sub-aperture images and their corresponding baseline is constant. Hence, we augment the 2D pixel coordinates with the corresponding normalised disparity to obtain the Light Field feature. This new representation reduces the effect of noise by aggregating multiple projections and allows for linear estimation of the absolute pose of a Light Field camera using the well-known Direct Linear Transformation algorithm. We evaluate the resulting absolute pose estimates with extensive simulations and experiments involving real Light Field datasets, demonstrating the competitive performance of our linear approach. Furthermore, we integrate our approach in a state-of-the-art Light Field Structure from Motion pipeline and demonstrate accurate multi-view 3D reconstruction.

Tendrils are common stable structures in nature and are used for sensing, actuation, and geometrical stiffness modulation. In this paper, for the first time we exploit the helical geometry of a shape memory alloy (SMA) tendril as a simple to fabricate highly dexterous robotic continuum tentacle that we called Active Tendril-Backbone Robot (ATBR). This is achieved via partial (120 deg) activation of single helix turns resulting in backbone directional bendings. A 141.5 mm prototype (130 mm when fully compressed) has been fabricated and a simple theoretical framework is proposed and experimentally validated for modeling of the tentacle configuration. The manipulator has five 2-DOF joints capable of reaching bending angles of up to 54.5 deg and angular speed of up to 6.8 deg/s. The dexterity of the manipulator is showcased empirically in reaching complex configurations and simple navigation through confined space of a curving path.

This article presents a model predictive controller (MPC) developed for the autonomous steering of concentric tube robots (CTRs). State-of-the-art CTR control relies on differential kinematics developed by local linearization of the CTRs mechanics model and cannot explicitly handle constraints on robot's joint limits or unstable configurations commonly known as snapping points. The proposed nonlinear MPC explicitly considers constraints on the robot configuration space (i.e., joint limits) and the robot's workspace (i.e., mixed boundary conditions on robot curvature). Additionally, the MPC calculates control decisions by optimizing the model-based predictions of future robot configurations. This way, it avoids configurations it cannot recover from, i.e., joint limits, singular configurations, and snapping. The proposed controller is evaluated via simulations and experimental studies with a variety of trajectories of increasing complexity. Simulation results demonstrate the capability of MPC to avoid singularities while satisfying robot mechanical constraints. Experimental results demonstrate that our solution enables following of trajectories unattainable by state-of-the-art controllers with mean error corresponding to 1% of robot arclength.

This paper presents a large scale, metric Structure from Motion (SfM) pipeline for generalised cameras with overlapping fields-of-view, and demonstrates it using Light Field (LF) images. We build on recent developments in algorithms for absolute and relative pose recovery for generalised cameras and couple them with multi-view triangulation in a robust framework that advances the state-of-the-art on 3D reconstruction from LFs in several ways. First, our framework can recover the scale of a scene. Second, it is concerned with unordered sets of LF images, meticulously determining the order in which images should be considered. Third, it can scale to datasets with hundreds of LF images. Finally, it recovers 3D scene structure while abstaining from triangulating using very small baselines. Our approach outperforms the state-of-the-art, as demonstrated by real-world experiments with variable size datasets.

Concentric tube robots (CTR) can traverse tightly curved paths and offer dexterity in constrained environments, making them advantageous for minimally invasive surgical scenarios that experience strict anatomical and surgical constraints. Their shape is controlled via rotation and translation of several concentrically arranged super-elastic precurved tubes that form the robot backbone. As the elastic energy accumulated in the backbone due to bending and twist of the tubes increases, robots can exhibit sudden snapping motions, which can damage the surrounding tissues. In this paper, we proposed an approach for closed-loop steering of a redundant CTR that allows for snap-free motion and enhances its force/velocity manipulability, increasing the capacity of the robot to move and/or exercise forces along any direction. First, a controller stabilizes the CTR end-effector on a desired time-variant trajectory. Next, an online optimizer uses the robot's redundant Degrees of Freedom (DoF) to reshape its manipulability in real-time and steer it away from potentially snapping configurations or increase its capacity in delivering force payloads. Simulations and experiments demonstrate the performance of the proposed control strategy. The controller can steer a generally unstable CTR along trajectories while avoiding instabilities with a mean error of 850 μm, corresponding to 0.6% of arclength, and improves robot ability to exercise forces by 55%.

Despite several robots having been proposed for vitreoretinal surgery, there is limited information on their dynamic modeling. This gap leads to sub-optimal motor selection and hinders the application of advanced control schemes that would fulfill the goal of micro-precise surgery. This paper presents the design process and a dynamics study of a multi-Degree of Freedom (DoF) robotic system, which is inspired by established co-manipulation architectures. A rigorous kinematics and dynamics analysis of the robot's part that is responsible for manipulating the surgical tool during the retinal surgery phase is provided. In particular, the Euler-Lagrange equations of motion, which describe the dynamics of the 3-link surgical manipulator, are combined with novel analytical models of each link's corresponding transmission mechanism, including an anti-backlash lead screw assembly and a worm drive. The resulting models, transferable to existing manipulators, provide a meticulous analysis of the robot's performance that can be used both for mechanical design and control purposes.

The enhanced dexterity and manipulability offered by continuum manipulators makes them the robots of choice for complex procedures inside the human body. However, without tailored analytical tools to evaluate their manipulability, many capabilities of continuum robots such as safe and effective manipulation will remain largely inaccessible. This paper presents a quantifiable measure for analysing force/velocity manipulability of continuum robots. We expand classical measures of manipulability for rigid robots to introduce three types of manipulability indices to continuum robots, namely, velocity, compliance, and unified force-velocity manipulability. We provide a specific case study using the proposed method to analyse the force/velocity manipulability for a concentric-tube robot. We investigate the application of the manipulability measures to compare performance of continuum robots in terms of compliance and force-velocity manipulability. The proposed manipulability measures enable future research on design and optimal path planning for continuum robots.

We propose a method for geometric calibration of multi-focus plenoptic cameras using raw images. Multi-focus plenoptic cameras feature several types of micro-lenses spatially aligned in front of the camera sensor to generate micro-images at different magnifications. This multi-lens arrangement provides computational-photography benefits but complicates calibration. Our methodology achieves the detection of the type of micro-lenses, the retrieval of their spatial arrangement, and the estimation of intrinsic and extrinsic camera parameters therefore fully characterising this specialised camera class. Motivated from classic pinhole camera calibration, our algorithm operates on a checker-board's corners, retrieved by a custom micro-image corner detector. This approach enables the introduction of a reprojection error that is used in a minimisation framework. Our algorithm compares favourably to the state-of-the-art, as demonstrated by controlled and freehand experiments, making it a first step towards accurate 3D reconstruction and Structure-from-Motion.

The use of concentric tube robots has recognized advantages for accessing target lesions while conforming to certain anatomical constraints. However, their complex kinematics makes their safe telemanipulation in convoluted anatomy a challenging task. Collaborative control schemes, which guide the operator through haptic and visual feedback, can simplify this task and reduce the cognitive burden of the operator. Guaranteeing stable, collision-free robot configurations during manipulation, however, is computationally demanding and, until now, either required long periods of pre-computation time or distributed computing clusters. Furthermore, the operator is often presented with guidance paths which have to be followed approximately. This paper presents a heterogeneous (CPU/GPU) computing approach to enable rapid workspace analysis on a single computer. The method is used in a new navigation scheme that guides the robot operator towards locations of high dexterity or manipulability of the robot. Under this guidance scheme, the user can make informed decisions and maintain full control of the path planning and manipulation processes, with intuitive visual feedback on when the robot's limitations are being reached.

Tracking and shape estimation of flexible robots that navigate through the human anatomy are prerequisites to safe intracorporeal control. Despite extensive research in kinematic and dynamic modeling, inaccuracies and shape deformation of the robot due to unknown loads and collisions with the anatomy make shape sensing important for intraoperative navigation. To address this issue, vision-based solutions have been explored. The task of 2-D tracking and 3-D shape reconstruction of flexible robots as they reach deep-seated anatomical locations is challenging, since the image acquisition techniques usually suffer from low signal-to-noise ratio or slow temporal responses. Moreover, tracking and shape estimation are thus far treated independently despite their coupled relationship. This paper aims to address tracking and shape estimation in a unified framework based on Markov random fields. By using concentric tube robots as an example, the proposed algorithm fuses information extracted from standard monoplane X-ray fluoroscopy with the kinematics model to achieve joint 2-D tracking and 3-D shape estimation in realistic clinical scenarios. Detailed performance analyses of the results demonstrate the accuracy of the method for both tracking and shape reconstruction.

Bronchoscopic interventions are widely performed for the diagnosis and treatment of lung diseases. However, for most endobronchial devices, the lack of a bendable tip restricts their access ability to get into distal bronchi with complex bifurcations. This paper presents the design of a new wire-driven continuum manipulator to help guide these devices. The proposed manipulator is built by assembling miniaturized blocks that are featured with interlocking circular joints. It has the capability of maintaining its integrity when the lengths of actuation wires change due to the shaft flex. It allows the existence of a relatively large central cavity to pass through other instruments and enables two rotational degrees of freedom. All these features make it suitable for procedures where tubular anatomies are involved and the flexible shafts have to be considerably bent in usage, just like bronchoscopic interventions. A kinematic model is built to estimate the relationship between the translations of actuation wires and the manipulator tip position. A scale-up model is produced for evaluation experiments and the results validate the performance of the proposed mechanism.

Concentric tube robots comprise telescopic precurved elastic tubes. The robot's tip and shape are controlled via relative tube motions, i.e. tube rotations and translations. Non-linear interactions between the tubes, e.g. friction and torsion, as well as uncertainty in the physical properties of the tubes themselves, e.g. the Young's modulus, curvature, or stiffness, hinder accurate kinematic modelling. In this paper, we present a machine-learning-based methodology for kinematic modelling of concentric tube robots and in situ model adaptation. Our approach is based on Locally Weighted Projection Regression (LWPR). The model comprises an ensemble of linear models, each of which locally approximates the original complex kinematic relation. LWPR can accommodate for model deviations by adjusting the respective local models at run-time, resulting in an adaptive kinematics framework. We evaluated our approach on data gathered from a three-tube robot, and report high accuracy across the robot's configuration space.

Safe and effective telemanipulation of concentric tube robots is hindered by their complex, non-intuitive kinematics. Guidance schemes in the form of attractive and repulsive constraints can simplify task execution and facilitate natural operation of the robot by clinicians. The real-time seamless calculation and application of guidance, however, requires computationally efficient algorithms that solve the non-linear inverse kinematics of the robot and guarantee that the commanded robot configuration is stable and sufficiently away from the anatomy. This paper presents a multi-processor framework that allows on-the-fly calculation of optimal safe paths based on rapid workspace and roadmap pre-computation The real-time nature of the developed software enables complex guidance constraints to be implemented with minimal computational overhead. A user study on a simulated challenging clinical problem demonstrated that the incorporated guiding constraints are highly beneficial for fast and accurate navigation with concentric tube robots.

Whilst current minimally invasive surgical robots offer many advantages to the surgeon, most of them are still controlled using the traditional master-slave approach, without fully exploiting the complementary strengths of both the human user and the robot. This paper proposes a framework that provides a cooperative control approach to human-robot interaction. Typical teleoperation is enhanced by incorporating haptic guidance-based feedback for surgical tasks, which are demonstrated to and learned by the robot. Safety in the surgical scene is maintained during reproduction of the learned tasks by including the surgeon in the guided execution of the learned task at all times. Continuous Hidden Markov Models are used for task learning, real-time learned task recognition and generating setpoint trajectories for haptic guidance. Two different surgical training tasks were demonstrated and encoded by the system, and the framework was evaluated using the Raven II surgical robot research platform. The results indicate an improvement in user task performance with the haptic guidance in comparison to unguided teleoperation.

Concentric tube robots are catheter-sized robots that are ideally suited for navigating along natural anatomical pathways and treating deep-seated pathologies. Their telemanipulation in dynamic environments requires on-line computation of inverse kinematics with simultaneous avoidance of anatomical obstacles. Moreover, unstable configurations, which arise for elongated curved robots that navigate extremely tortuous paths, must be avoided. To achieve on-line computations, existing work has investigated Jacobian approximations and configuration-space precomputation. This paper leverages the state-of-the-art multi-core computer architectures to deliver real-time local inverse kinematics solutions using the established concentric tube robot mechanics models while avoiding both instabilities and anatomical collisions. Furthermore, it considers frictional active constraints for concentric tube robots, i.e. viscoelastic force fields that guide the operator away from obstacles and towards safe configurations. The value of the proposed framework is demonstrated on realistic clinical scenarios.

Concentric tube robots have shown promise for minimally invasive surgical (MIS) tasks that require navigation via tortuous anatomical paths. Despite extensive research on their kinematic and dynamic modelling, however, inaccuracies and deformations of their shape due to unknown loads and collisions with the anatomy make intraoperative shape sensing a requirement. This paper presents a vision-based shape-sensing algorithm for concentric tube robots. The proposed algorithm fuses information extracted from a standard imaging modality, monoplane X-ray fluoroscopy, with the kinematics model of the concentric tube robot, to achieve automatic, real-time, accurate and continuous robot-shape estimations despite kinematics' noise and unmodelled forces. Fusion is performed by a fast 2D/3D non-rigid registration, which combines kinematics and intraoperative tracking of the robot. Extensive simulations with a range of noise models and virtual loads acting on the robot, and experimental evaluation in air and in a skull phantom, demonstrate the clinical value of the proposed technique1.

Concentric tube robots are catheter-sized continuum robots that are well suited for minimally invasive surgery inside confined body cavities. These robots are constructed from sets of pre-curved superelastic tubes and are capable of assuming complex 3D curves. The family of 3D curves that the robot can assume depends on the number, curvatures, lengths and stiffnesses of the tubes in its tube set. The robot design problem involves solving for a tube set that will produce the family of curves necessary to perform a surgical procedure. At a minimum, these curves must enable the robot to smoothly extend into the body and to manipulate tools over the desired surgical workspace while respecting anatomical constraints. This paper introduces an optimization framework that utilizes procedureor patient-specific image-based anatomical models along with surgical workspace requirements to generate robot tube set designs. The algorithm searches for designs that minimize robot length and curvature and for which all paths required for the procedure consist of stable robot configurations. Two mechanics-based kinematic models are used. Initial designs are sought using a model assuming torsional rigidity. These designs are then refined using a torsionally-compliant model. The approach is illustrated with clinically relevant examples from neurosurgery and intracardiac surgery.

Actuators that are powered, imaged, and controlled by magnetic resonance (MR) scanners could inexpensively provide wireless control of MR-guided robots. Similar to traditional electric motors, the MR scanner acts as the stator and generates propulsive torques on an actuator rotor containing one or more ferrous particles. Generating maximum motor torque while avoiding instabilities and slippage requires closed-loop control of the electromagnetic field gradients, i.e., commutation. Accurately estimating the position and velocity of the rotor is essential for high-speed control, which is a challenge due to the low refresh rate and high latency associated with MR signal acquisition. This paper proposes and demonstrates a method for closed-loop commutation based on interleaving pulse sequences for rotor imaging and rotor propulsion. This approach is shown to increase motor torque and velocity, eliminate rotor slip, and enable regulation of rotor angle. Experiments with a closed-loop MR imaging actuator produced a maximum force of 9.4 N.

Microrobots are promising tools for micromanipulation and minimally invasive interventions. Robust electromagnetic control of microrobots can be achieved through precisely modeled magnetic steering systems and accurate localization. Error-free modeling and position information, however, are not realistic assumptions, and microrobots need to be controlled under force and localization uncertainties. In this paper, methods to account for these types of uncertainties are presented. Initially, the uncertainties in electromagnetic force generation of a new class of manipulation systems are quantified. Subsequently, a drag-force uncertainty model for linear dynamics is proposed. This model can be employed for microrobots whose fluid dynamics are not well understood. A set of performance measures is introduced in the design of controllers, and a PID and a robust H∞controller are synthesized and evaluated through simulations. To demonstrate the capabilities of the synthesized controllers under localization and force uncertainties, low update rates are considered. The H∞controller can provably respect the performance measures under higher uncertainties than the PID controller, and its performance is further quantified through experiments in a prototype electromagnetic control system.

Actuators that are powered, imaged and controlled by Magnetic Resonance (MR) scanners offer the potential of inexpensively providing wireless control of MR-guided robots. Similar to traditional electric motors, the MR scanner acts as the stator and generates propulsive torques on an actuator rotor containing one or more ferrous particles. To generate maximum motor torque while avoiding instabilities and slippage, closed-loop control of the electromagnetic field gradients, i.e., commutation, is required. This paper proposes and demonstrates a method for commutation based on interleaving pulse sequences for rotor tracking and rotor propulsion. Fast rotor tracking is achieved by a new technique utilizing radio-frequency (RF) selective excitation of a properly located fiducial marker by the ferrous particle of the rotor. Optimal marker location is derived and demonstrated to provide accurate estimates of rotor angle. In addition, closed-loop commutation control is shown to increase motor torque and also to enable regulation of rotor angle.

Concentric tube robots are continuum robots that can navigate natural pathways to reach locations deep inside the human body. Their operation is based on rotating and telescopically actuating concentric tubes to achieve robot tip pose control. During tube manipulation, the elastic energy stored in the robot structure may give rise to unstable robot configurations and loss of control. This can occur, in particular, for highly curved and elongated tubes that are required for certain surgical interventions. This paper presents a path planning methodology that allows the utilization of such generally unstable concentric tube robots by ensuring that they operate in their stable configuration regions.

Electromagnetic systems for in vivo microrobot steering have the potential to enable new types of localized and minimally invasive interventions. Accurate control of microrobots in natural fluids requires precise, high-bandwidth localization and accurate knowledge of the steering system's parameters. However, current in vivo imaging methodologies, such as fluoroscopy, must be used at low update rates to minimize radiation exposure. Low frame rates introduce localization uncertainties. Additionally, the parameters of the electromagnetic steering system are estimated with inaccuracies. These uncertainties can be addressed with robust H∞control, which is investigated in this paper. The controller is based on a linear uncertain dynamical model of the steering system and microrobot. Simulations show that the proposed control scheme accounts for modeling uncertainties, and that the controller can be used for servoing in low viscosity fluids using low frame rates. Experiments in a prototype electromagnetic steering system support the simulations.

Dexterous manipulation of intraocular microrobotic devices has the potential to significantly augment ophthalmic surgeons' capabilities. Microrobots can be employed for targeted drug delivery and for procedures such as retinal-vein cannulation that require a high degree of dexterity. For precise externally generated magnetic control of microdevices, their position in the magnetic field is needed. Since the interior of the human eye is externally observable, computer-vision techniques can be used for localization. In this paper, the complex optics of the human eye are taken into account, and an algorithm that localizes microrobotic devices based on their 3-D structure is proposed. The sensitivity of the algorithm with respect to uncertainties in optical parameters is evaluated. A human-like model eye is designed and fabricated for experiments, precision analysis is performed, and the algorithm is used for visual servoing.

The automated or semiautomated control of microrobots for targeted drug delivery, retinal surgeries, and other ophthalmic procedures has the potential to significantly augment the capabilities of human surgeons. For accurate magnetic control, the position of the microrobots is required. In this paper, we extract the intraocular projection mapping and use it in the established CAD-model-based pose-estimation framework. Our algorithm requires no focus information. This is the first work that treats a complicated refractive imaging system, such as the eye, in a model-based localization framework. We design a model-eye chamber with human-like optical elements in which we control a microrobot and estimate its position using monocular vision.

Successful ophthalmic surgeries using intraocular untethered microrobots or tethered robotic microtools require methods to robustly track the microdevices in the posterior of the human eye. The dimensions and specularities of the microdevices are major obstacles for accurate tracking. In addition, the optical structure of the human eye makes it challenging to keep the objects of interest constantly in focus, resulting in blurred images. In this paper, the advantages of using different colorspaces for intraocular tracking are examined. After selection of the appropriate colorspace, thresholds that ensure maximum separation of the device from the background are calculated. Based on trained color histograms, level sets are used to track in real time, and the use of statistical shape information is incorporated in the existing tracking framework. The efficacy of the algorithm is demonstrated by tracking a microrobot in a model eye, using a custom made ophthalmoscope and off-the-shelf ophthalmoscopy lenses. With the appropriate colorspace and threshold selection, tracking errors are minimized and are further diminished using shape information.

Future retinal therapies will be partially automated in order to increase the surgeons' ability to operate near the sensitive structure of the human eye retina. Untethered robotic devices that achieve the desired precision have been proposed, but require localization information for their control. Since the interior of the human eye is externally observable, vision can be used for localization. Previously, a focus-based paraxial localization algorithm using a mechatronic vitreoretinal ophthalmoscope(MVO) was proposed and evaluated by the authors. In this paper, the first algorithm for wide-angle intraocular localization is presented. The effectiveness of this new localization approach is demonstrated by experiments using a model eye and a customized MVO, and there is clear improvement over previously reported results.