Micro-scale robots introduce great prospective into many different medical applications such as targeted drug delivery, minimally invasive surgery and localized bio-metric diagnostics. This research presents a method for object detection and tracking system of a chain-like magnetic microsphere robots using ultrasound imaging in an in-vitro environment. The method estimates the position of the microrobot in real-time using deep learning techniques. The experiments showed that a spherical microrobot with about 500 m in diameter can be detected and tracked in real-time with a high accuracy in dynamic environments. The results exhibit a high detection and tracking accuracy for one, two and three sphere microrobots with the highest accuracy in detection and tracking around 95 % and 93% respectively.

Wireless magnetic microrobots can perform complex tasks for small-scale minimally invasive surgery (MIS) that requires high precision and dexterity. The choice of the configuration of the electromagnetic actuation (EMA) system is a key issue for reliable medical applications. This paper addresses the study of a robotic EMA platform firstly devoted to ophthalmic MIS, aiming at improving the manipulability and dexterity of the procedure. To this end, a robotic EMA system comprising four static and four mobile electromagnets is investigated. Evaluation of the magnetic force and torque, the manipulability and the dexterity indexes of EMA platforms are studied. The results demonstrate that a robotic EMA platform increases the versatility of the EMA system, and becomes resourceful to perform various tasks.

This paper introduces a two-dimensional (2-D) autonomous navigation strategy of a 750-μm steel microrobot along a complex fluidic vascular network inside the bore of a clinical 3.0T magnetic resonance imaging (MRI) scanner. To ensure successful magnetic resonance navigation of a microrobot along consecutive channels, the design of autonomous navigation strategy is needed, taking into account the major MRI technological constraints and physiological perturbations, e.g., nonnegligible pulsatile flow, limitations on the magnetic gradient amplitude, MRI overheating, and susceptibility artifacts uncertainties. An optimal navigation planning framework based on Pareto optimality is proposed in order to deal with this multiple-objective problem. Based on these optimal conditions, a dedicated control architecture has been implemented in an interventional medical platform for real-time propulsion, control, and imaging experiments. The reported experiments suggest that the likelihood of controlling autonomously untethered 750-μm magnetic microrobots is rendered possible in a complex 2-D centimeter-sized vascular phantom. The magnetic microrobot traveled intricate paths at a mean velocity of about 4 mms-1with average tracking errors below 800 μm with limited magnetic gradients ±15 mTm-1, which is compatible with clinical MRI scanners. The experiments demonstrate that it is effectively possible to autonomously guide a magnetic microrobot using a conventional MRI scanner with only a software upgrade.

This paper presents an optimal control strategy for navigation of multiple magnetic microbeads for future drug targeting applications. To transport the drugs, we use therapeutic magnetic microbeads as navigable agents controlled by magnetic gradients. The main difficulty is to control independently each therapeutic agent along a trajectory with the same magnetic gradient fields. This study proposes an optimal control methodology to control a group of different therapeutic agents at desired states. Based on a dynamic model of group of magnetic microbeads, controllability and observability conditions are formulated and simulated. Based on the proposed theoretical analysis a linear quadratic with integral action control (LQI) has been chosen to be applied to the microbeads system. Finally, an experimental investigation is carried out in millimeter-sized fluidic artery vessels to demonstrate the controllability and stability of two magnetic microbeads under different velocity and trajectory constraints with a laminar viscous fluidic environment.

This paper describes the propulsion of the catalytic tubular microjet using Navier-Stokes equations. Especially, the thrust capability of the microjet is outstanding compared with the other microsystems, but remains only partially understood. Studies have identified the internal precursory mechanisms of the propulsion of the microjet: its inner wall catalyzes the dismutation of aqueous hydrogen peroxide, and an oxygen bubble is then formed and migrates towards its widest opening. This impulses the propulsion of the microjet towards the opposite direction. However, the precise propellant role of the liquid surrounding the jet remains misunderstood. The same goes for the inner wall of the vessel in which the jet navigates, especially in narrowed environment. This article discusses these aspects in a simplified theoretical framework. Calculations are performed by explicit computation of Navier-Stokes equations. The obtained theoretical outcome are in good agreement with experimental results in the literature.

This paper presents an optimal design strategy for magnetic targeting of therapeutic drugs. In this study, to maximize the effect of the treatment and minimize adverse effects on the patient, mathematical models have been developed to find the number and the size of the boluses with respect to the growth of a tumor. Using these models, control strategies are developed to establish a schedule that allows the physician to administer the medication while respecting borne by the patient doses. To transport the drugs, we use therapeutic magnetic boluses composed of magnetic particle aggregates as navigable agents controlled by magnetic gradients. Based on the optimal design of the bolus, an experimental investigation is carried out in millimeter-sized fluidic artery vessels to demonstrate the steerability of the magnetic bolus under different velocity, shear-stress and trajectory constraints with a laminar viscous fluidic environment.

This paper has presented a new vision-based force-sensing framework that allows to characterize the forces applied on a magnetic microrobot in an endovascular-like environment. Especially, unlike common approaches used with optical microscopy where orthographic projection model are used, we consider in this paper the weak-perspective model. The proposed vision-based force characterization allows to retrieve the three dimensional (3D) translational velocities and accelerations of a microrobot viewed from a digital microscope. Hence, thanks to the dynamic model the external forces are estimated on-line. The framework was applied and validated for a magnetic microrobot navigating in a viscous flow. Experimental results in two different environments illustrate the efficiency of the proposed method.

This paper reports a control strategy of a micro-gripper based on two AFM tips for manipulation at micro/nano scale. It is composed of dual micro/nano manipulators in order to handle and to maintain a microsample through the focus of a X-ray or laser beam for material characterization and analysis. The main idea is to control and to drive in a robust way the micro/nanomanipulators by focusing the beam on the center part of the handled micro-object. To this aim, the maximum intensity of the laser beam is measured in real-time by a four-quadrant photodiode sensor. As the sample under consideration here is a superparamagnetic microsphere of 8.2 μm (focusing laser spot less than few μm2), the laser tracking system is very sensitive to light intensity variations, mechanical vibrations, microhandling force perturbations and thermal relaxation of magnetic microsamples. First, we propose to compensate the laser beam variations by estimating the position of the laser beam using a particle filter (PF) algorithm. Then, a robust control strategy based on H∞controllers ensures a robust microhandling task under the focus of the laser beam whatever the external perturbations involved and parametric model uncertainties. The dual manipulators are controlled cooperatively by combining the different actuator dynamics to track a laser beam with nanometer precision. Finally, experimental results demonstrate the robustness of the microhandling task using the proposed robust control scheme.

Magnetic microrobotics was nowadays one of the most advanced technique to reach deep locations in human body for future biomedical applications. Different magnetic microrobot designs were proposed, such as bead pulling or microswimmers. In this paper, the use of chain-like of magnetic N-microspheres was investigated to enable new kind of motions and applications. An accurate theoretical model of chain-like magnetic microbeads navigating in viscous fluidic environments is described. Thus, the behavior of such microrobot was analyzed for different number of microspheres (ranging from N = 2 to 5). The efficiency of the proposed technique was demonstrated experimentally in a microfluidic vessel phantom to mimic atherosclerosis disease leading to plaque formation that fully occluded a vasculature.

This paper presents a study of the control problem of a laser beam illuminating and focusing a microobject subjected to dynamic disturbances using light intensity for feedback only. The main idea is to guide and track the beam with a hybrid micro/nanomanipulator which is driven by a control signal generated by processing the beam intensity sensed by a four-quadrant photodiode. Since the pointing location of the beam depends on real-time control issues related to temperature variation, vibrations, output intensity control, and collimation of the light output, the 2D beam location to the photodiode sensor measurement output is estimated in realtime. We use the Kalman filter (KF) algorithm for estimating the state of the linear system necessary for implementing the proposed track-following control approach. To do so a robust master/slave control strategy for micro/nano dual-stage manipulators is presented based on sensitivity function decoupling design methodology. The decoupled feedback controller is synthesized and implemented in a 6 dof micro/nanomanipulator capable of nanometer resolution through several hundreds micrometer range. A case study relevant to tracking a laser-beam for imaging purposes is presented.

Navigating in bodily fluids to perform targeted diagnosis and therapy has recently raised the problem of robust control of magnetic microrobots under real endovascular conditions. Various control approaches have been proposed in the literature but few of them have been experimentally validated. In this paper, we point out the problem of navigation controllability of magnetic microrobots in high viscous fluids and under pulsatile flow for endovascular applications. We consider the experimental navigation along a desired trajectory, in a simplified millimeter-sized arterial bifurcation, operating in fluids at the low-Reynolds-number regime where viscous drag significantly dominates over inertia. Different viscosity environments are tested under a systolic pulsatile flow compatible with heart beating. The control performances in terms tracking, robustness and stability are then experimentally demonstrated.

MRI based nano- and microrobotics show good potential for new targeted therapies tackling e.g. cancer. In this paper, a system developed for the propulsion and navigation of small ferromagnetic objects only using clinical MRI systems is evaluated in experiments. The experiments include propulsion of an untethered ferromagnetic object against a pulsatile flow in a pipe system, navigation of an untethered object filled with ferromagnetic nanoparticles around obstacles in a free environment, and the choosing of a branch in a closed flow-less channel system when propelling an untethered ferromagnetic object. The system is found to deal with the tasks efficiently.

Microinjection is a method for the delivery of exogenous materials into cells and is widely used in biomedical research areas such as transgenics and genomics. However, this direct injection is a time-consuming and laborious task, resulting in low throughput and poor reproducibility. Here, we describe a telerobotic shared control framework for microinjection, in which a micromanipulator is controlled by the shared motion commands of both the human operator and the autonomous controller. To determine the weightings between the operator and the controller, we proposed a quantitative evaluation method using a model of speed/accuracy trade-offs in human movement. The results showed that a 40%-60% weighting on the human operator (or the controller) produced the best performance for both speed and accuracy of guiding and targeting task in microinjection suggesting that some level of both automation and human involvement is important for microinjection tasks.

Microinjection is the highly efficient delivery method of exogenous materials into cells, and it has been widely used in biomedical research areas such as transgenics and genomics. However, this direct injection task is time consuming and laborious, resulting in low throughput and poor reproducibility. This paper describes a telerobotic shared control (TSC) framework for the microinjection with high manipulation efficiencies, in which a micromanipulator is controlled by the shared motion commands of both the human operator (direct manipulation) and the autonomous controller. To determine the optimal gains between the operator and the controller, we proposed a quantitative evaluation method using Fitts' and steering laws. The results showed that a 40%-60% weighting on the human operator produced better performance for both speed and accuracy of task completion, and suggested that some level of automation or human involvement is important for microinjection tasks.

The propulsion of nano-ferromagnetic objects by means of MRI gradients is a promising approach to enable new forms of therapy. In this work, necessary techniques are presented to make this approach work. This includes path planning algorithms working on MRI data, ferromagnetic artifact imaging and a tracking algorithm which delivers position feedback for the microdevice and a propulsion sequence to enable interleaved magnetic propulsion and imaging. Using a dedicated software environment integrating path-planning methods and real-time tracking, a clinical MRI system is adapted to provide this new functionality for potential controlled interventional targeted therapeutic applications. Through MRI-based sensing analysis, this paper aims to propose a framework to plan a robust pathway to enhance the navigation ability to reach deep locations in human body. The proposed approaches are validated with different experiments.

In this paper we propose a new on-line sensor self-calibration framework. The approach is to consider the sensor/robot interaction that links the sensor signal variations to the robot velocity. By on-line calibration, we mean only the actual measurements are used to perform calibration under the condition that the interaction matrix is analytically known. This allows us to propose a very simple and versatile formulation of sensor parameter calibration. Various sensors can be considered, and calibration from different sensory data may be considered within the same process. Intrinsic and extrinsic parameters estimation are formulated as a non-linear minimization problem the Jacobian of which can be expressed analytically from the sensor model. Simulations and experiments are presented for a camera observing four points, showing good results in the case of separated intrinsic and extrinsic calibration, and illustrating the possible limitations in the case of simultaneous estimation.

This paper presents real-time MRI-based control of a ferromagnetic microcapsule for endovascular navigation. The concept was studied for future development of microdevices designed to perform minimally invasive interventions in remote sites accessible through the human cardiovascular system. A system software architecture is presented illustrating the different software modules to allow navigation of a microdevice in blood vessels, namely: (i) vessel path planner, (ii) magnetic gradient steering, (iii) tracking and (iv) closed-loop navigation control. First, the position recognition of the microrobot into the blood vessel is extracted using Frangi vesselness filtering from the pre-operation images. Then, a set of minimal trajectory is predefined, using FMM, to guide the microrobot from the injection point to the tumor area through the anarchic vessel network. Based on the pre-computed path, a GPC is proposed for robust time-multiplexed navigation along a 2D path in presence of pulsative flow. The simulation results suggest the validation of the proposed image processing and control algorithms. A series of disturbances introduced in the presence and absence of closed-loop control affirms the robustness and effectiveness of this predictive control system.

This paper deal with the problem of executing a vision-based task in an unknown environment. During such a task, two unexpected events may occur: the image data loss due to a camera occlusion and the robot collision with obstacles. We first propose a method allowing to compute the visual data when they are totally lost, before addressing the obstacle avoidance problem. Then, we design a sensor-based control strategy to perform safely vision-based tasks despite complete loss of the image. Simulation and experimental results validate our work.