Online path planning for magnetic microrobots actuated by electromagnetic system in dynamic flow field presents significant challenges due to time-varying fluid dynamics, energy constraints, and collision risks. Traditional path planning approaches, which often rely on static flow assumptions or simplified geometric models, struggle to balance energy efficiency, path continuity, and adaptability in real-world scenarios. This paper introduces an end-to-end path planner for energy-efficient and collision-free navigation of magnetic helical microrobots, integrating flow field feature extraction and reinforcement learning (RL) framework. Our method employs a transformer encoder to capture contextual correlations of flow field and uses a Soft Actor-Critic (SAC) framework to optimize energy consumption while ensuring dynamic obstacle avoidance. Simulations and experiments in dynamic flow environments validate our approach, demonstrating 14.7% lower energy consumption and robust collision avoidance in several different test scenarios.

Untethered magnetically actuated soft microrobots are promising in micromanipulation applications. Autonomous navigation and micromanipulation in dynamic environments are challenging because uncertain dynamic obstacles lead to increased planning time and reduced real-time performance. Here, we developed a novel H-shape soft magnetic microrobot with finger behaviors, including gripping and releasing. Without additional stimuli (e.g., light, temperature, etc.), the finger behaviors and movements of the microrobot can be controlled simultaneously by adjusting the strength and direction of the magnetic fields. We proposed a sliding-window-based local dynamic path planning method in dynamic environments to address the drawbacks of inefficiency and the high computational cost of global search. An enhanced optimized rapidly-exploring random tree algorithm is developed for planning avoidance paths at a safe distance from the boundary of obstacles. In order to automatically manipulate multiple objects, the manipulation process is quantitatively divided into several subtasks, and a finite state machine (FSM) based task planner is proposed to control the switching between subtasks. Experiments demonstrate the autonomous obstacle avoidance and navigation of the microrobot in dynamic environments with obstacles moving 2.5 times faster than the robot. Eight spherical objects of 2 mm diameter were automatically transported to the corresponding target position with an average error of 0.29 mm and an average transport time of 17.25 seconds. In the future, this work promises to enable automated cell sorting by microrobots. Note to Practitioners—This article was motivated by the recent interest in utilizing the small-scale microrobots to perform micromanipulation tasks. For automated manipulation of objects, effective Untethered actuation, manipulation behavior control, and navigation are required. Besides, obstacle avoidance and dynamic obstacles in the environment need to be considered. The strategy proposed here is developed for automated multi-object manipulation via a novel soft magnetic microrobot. The developed task planner quantitatively divides the operation process into several subtasks and controls the switching between them. The dynamic path planning algorithm is used to efficiently search for collision avoidance paths in dynamic obstacle environments. The motion-behavior coordination controller is used to control the robot to perform manipulation subtasks. Experimental results have validated the proposed strategy in actuation, navigation, and multiple cargoes manipulation in dynamic obstacle environments.

Magnetic microrobots have garnered significant attention and hold great potential for biomedical research applications. However, achieving precise manipulation in vivo poses significant challenges, particularly in medical image-based real-time feedback control, because it is difficult for a visual camera to track the motion of magnetic microrobots inside the body in biomedical applications. To realize the precise control of magnetic microrobots, it is also necessary to design and implement a simple and powerful control method. This approach allows for avoiding resource-intensive and complex control strategies. In this article, we present a learning-based real-time control method utilizing ultrasound images. Inspired by the ADboost concept, we use a reinforcement learning approach to integrate two simple control methods: a proportional-integral-derivative controller and a guiding vector field controller. We develop a novel$Q$-learning method called average$Q$-learning that incorporates average operation and$n$-step bootstraps. Its primary objective is to dynamically adjust the outputs of the different simple controllers. While each controller individually offers a straightforward solution, their integration contributes to a powerful control approach. To demonstrate its scalability, a nonsmooth path is utilized to investigate the integration performance of three simple controllers. In addition, we enhance a classic segmentation module, U-net, by incorporating an atrous spatial pyramid pooling module. To validate the effectiveness of the proposed control method, we conduct simulations and experiments using various planar paths. The quantitative analysis of the results demonstrates the efficacy of our approach in achieving precise manipulation, leveraging real-time control based on medical images for magnetic microrobots. Overall, this study provides a preliminary investigation into the field of medical image-based precise manipulation of magnetic microrobots in vivo applications.

The widespread application of learning-based methods in robotics has allowed significant simplifications to controller design and parameter adjustment. In this article, robot motion is controlled with learning-based methods. A control policy using a broad learning system (BLS) for robot point-reaching motion is developed. A sample application based on a magnetic small-scale robotic system is designed without detailed mathematical modeling of the dynamic systems. The parameter constraints of the nodes in the BLS-based controller are derived based on Lyapunov theory. The design and control training processes for a small-scale magnetic fish motion are presented. Finally, the effectiveness of the proposed method is demonstrated by convergence of the artificial magnetic fish motion to the targeted area with the BLS trajectory, successfully avoiding obstacles.

Microswimmer and miniswimmer toward precision-targeted medicine have attracted extensive attention recently. We have developed an autonomous manipulation approach for magnetic-driven helical miniswimmer at low Reynolds number in the horizontal plane ($H$-plane). Different from our previous work which just makes the barycenter of miniswimmer on the reference path as well as takes the swimming direction not into consideration in planar path following, our control policy in this article can make the miniswimmer to follow the reference path and, at the same time, its swimming direction is also along the reference path. A robust tracking method is employed to locate the helical miniswimmer in real time. Due to different external disturbances, an angle compensating model in the global coordinate frame is developed by radial basis function (RBF) networks trained by backpropagation algorithms, which is used to express the swimming model of the helical miniswimmer facing the gravity and lateral disturbances. A discrete-time optimal controller is formulated based on the linear-quadratic feedback control. Simulations and experiments are conducted to quantitatively validate the autonomous manipulation, and the results show the control performance with submillimeter accuracy in the$H$-plane.Note to Practitioners:This article is motivated by the potential application of precision-targeted medicine using magnetic-driven microswimmer/miniswimmer. The formulated controller employs the error model in the horizontal plane to design the control law. Simulations and experiments validate the effectiveness of the proposed discrete-time optimal control scheme using magnetic-driven miniswimmers.

Magnetically actuated small-scale robots have great potential for numerous applications in remote, confined, or enclosed environments. Multiple small-scale robots enable cooperation and increase the operating efficiency. However, independent control of multiple magnetic small-scale robots is a great challenge, because the robots receive identical control inputs from the same external magnetic field. In this article, we propose a novel strategy of completely decoupled independent control of magnetically actuated flexible swimming millirobots. A flexible millirobot shows a crawling motion on a flat plane within an oscillating magnetic field. Millirobots with different magnetization directions have the same velocity response curve to the oscillating magnetic field but with a difference of phase. We designed and fabricated a group of up to four heterogeneous millirobots with identical geometries and different magnetization directions. According to their velocity response curves, an optimal direction of oscillating magnetic field is calculated to induce a desired velocity vector for the millirobot group, one of which is nonzero and the others are approximately zero. The strategy is verified by experiments of independent position control of up to four millirobots and independent path following control of up to three millirobots with small errors. We further expect that with this independent control strategy, the millirobots will be able to cooperate to finish complicated tasks.

Steering microswimmers toward 3-D autonomous manipulation tasks has received extensive attention. Our previous works have accomplished autonomously manipulating microswimmers in the 2-D space. This article aims to extend the 2-D autonomous manipulation to 3-D autonomous manipulation. Specifically, this article addresses the problem of an autonomous system that consists of 3-D path planning and 3-D path following for magnetically driven helical microswimmers. The path-planning algorithm called optimal Bidirectional RRT* is formulated to explore the shortest route in the confined 3-D space. A proxy-based sliding mode control (PSMC) approach is developed to design stable controllers based on the error model in the Serret–Frenet frame. We transport the swimming model trained by a kind of neural network to another new helical microswimmer according to an online updating scheme. The updating scheme can identify and refine compensating angles between the swimming direction of the microswimmer and the magnetic direction in the 3-D space facing the weight disturbances of the swimmer and lateral disturbances. The experiments are conducted to quantitatively validate the 3-D autonomous manipulation system. Experimental results show the effectiveness of path planning and path following with submillimeter accuracy in a 3-D space. Future works will focus on autonomous manipulations in dynamic environments.Note to Practitioners—This article is motivated by the issue of 3-D autonomous manipulation tasks for magnetically driven helical microswimmers. The formulated path planning is responsible for finding the shortest route in the 3-D confined space. The closed-loop controller is charge of steering the helical microswimmers on a reference path based on an online updating model trained by neural networks. It is demonstrated that the helical microswimmer can find the shortest path and follow it in a 3-D space with submillimeter accuracy.

Microrobots capable of performing minimally invasive surgery, targeted drug delivery, and manipulation of micro-objects have shown great potentials in multiple application areas. However, microrobots assembled by nanoparticles with stable configuration and good performance on closed-loop control are still to be further studied. In this paper, ferromagnetic nanoparticles are used as experimental materials, rather than paramagnetic nanoparticles that require complex synthesis processes, and the advantage is demonstrated, for example, high pattern stability. The locomotion velocity as a function of the magnetic field frequency is modeled, analyzed and verified by experiment. Moreover, a path following experiment based on the arbitrary planar path following algorithm is performed. The nanoparticle microrobots are of great significance for biomedical applications. In the future works, experiments in bio-fluids, multimodal locomotion, and targeted drug delivery tasks will be investigated.

Controlling magnetic microswimmers toward 3-D manipulation tasks has received considerable attention. Although related studies on manipulating helical microswimmers have been developed, stable closed-loop controls and accuracy swimming models should be still investigated. This article addresses the problem of 3-D path following for magnetically driven helical microswimmers with an adaptive-compensation scheme. The orientation-compensation model in the global coordinate frame is learned by radial basis function (RBF) networks trained with backpropagation algorithms, which is used to express the motion of the helical microswimmer in the presence of the weight of the swimmer and lateral disturbances from the boundary effects. A proxy-based sliding-mode control (PSMC) approach is developed to design stable controllers based on the kinematic error model. The effects of variable parameters and boundary effects are also considered. Experimental results including different paths in 3-D space validated the path following with submillimeter accuracy using the helical microswimmer. Note to Practitioners-This article is motivated by the issue of the following predefined paths for magnetically driven helical microswimmers in 3-D space. The proposed closed-loop controller employs the error model in 3-D space to formulate the control law according to an orientation-compensation model learned by neural networks. It is demonstrated that the helical microswimmer is able to follow different paths in 3-D space with submillimeter accuracy using the proposed control scheme.