Microrobotic contact manipulation enables automated and precise cell capture, positioning and screening and has potential in biomedical engineering and disease detection. However, when using an optical microscope for visual positioning of targets, the poor clarity, limited cell-background contrast and lack of global 3D information of the cells in the field of view hinder global strategy making and automation, thereby affecting the accuracy and efficiency of manipulation. Here, we propose the 3D locating of multiple biological targets based on digital holography. Global–local combined visual feedback is developed for overall spatial locating and partial locating in a liquid-phase bright-field environment. By applying a filtering-based planar locating algorithm and maximum-area-based depth detection algorithm, the 3D global distribution of micro-targets in the field of view is periodically updated with a high detection rate. By applying a planar locating algorithm based on a convolutional neural network and a depth detection algorithm based on a gradient descent, the 3D fast locating of targets is performed precisely. Experiments show that the detection rate of the global positioning is 95.1%, the mean average precision of the local planar positioning is 90.53%, and the deviation of the local depth positioning is$1.22~\mu \text{m}$. When capturing cells, this method reaches an average speed of 7.4 cells/min and a collection rate of 90.5%. We anticipate that our method will support the research in cell-based bioengineering including cell screening and early disease diagnosis. Note to Practitioners—Automated cell manipulation is one of the most significant techniques in cell-based biomedical applications. This paper introduces a three-dimensional multi-target visual positioning method for automated cell manipulation. Combining with holographic imaging technique, the global information for the cells in the limited field of view is provided with improved imaging clarity and extended depth of field. The visual recognition algorithm can screen and extract the rare cells in the cell population, which will support the research in the field of early disease diagnosis and biomedicine. The research outcome provides an effective and precise solution to achieve biological targets positioning, capture, and screening within 3D liquid-phase bright-field environment.

Micron-range untethered, magnetic helical robots have great potential for biomedical applications due to their desirable performance with high flexibility and accuracy in unstructured and confined environments. However, at the microscale, time-varying uncertain disturbances in the environment and electromagnetic system greatly hinder helical microrobot tracking control performance. When a microrobot is replaced or even a derivative version with a slight helical body structure change is used for different tasks, the performance of the original control scheme remarkably decreases or even becomes ineffective. Here, we propose a data-driven optimal integrated controller (D2-OIC) that realizes precise tracking and transfer control among a series of helical microrobots with derived structures in different situations. The control approach has a parallel structure with nonlinear feedforward and linear feedback controllers. The nonlinear feedforward controller inversely maps the relationship between the electromagnetic field state and the helical microrobot motion state, allowing the helical microrobot to quickly approach the desired motion state. The linear feedback controller effectively adjusts the controller parameters using the virtual reference feedback tuning (VRFT) method, thus eliminating any residual motion errors arising from nonlinear control. By retraining on newly acquired and collected cumulative data with assigned weights, the nonlinear feedforward controller is updated to achieve transfer control among various helical microrobot types. In the experiment, two helical microrobot types performed arbitrary path tracking and obstacle avoidance tasks with tracking errors consistently less than 4% of the microrobot body length, demonstrating the feasibility of the proposed method.

Many previous works have facilitated muscle cell (C2C12) alignment to form fiber-like cell structures. However, there still remains a challenge how to induce C2C12 myoblasts in the cell structures to differentiate into matured myocytes to form a functional muscle tissue, while external mechanical stimulation has been proved to have good effects on proliferation and differentiation of myoblasts. In this paper, we proposed a vision-based micro robotic manipulation system to achieve automatic mechanical stimulation for one single muscle fiber-like cell structures (MFCS). A tube, which is attached to a three degree-of-freedom (DOF) manipulator, and a probe are employed to apply the uniaxial mechanical stimulation to train the MFCS. To measure the force applied on MFCS, a vision-based measuring and correction method is utilized, which decrease the error by 74%. Moreover, based on the viscoelastic property of the MFCS, a feedback control algorithm has been applied to compensate for the force loss to realize the force stimulation. And the final value of force remains 699 ± 1μN after 110s experiment.

The research on rare cells makes a significant contribution to biology research and medical treatment for the application of diagnostic operation as well as prognoses treatment. Therefore, sorting them from heterogeneous mixtures is crucial and valuable. Traditional cell sorting methods featured with poor purity and recovery rate as well as limited flexibility, which are not ideal approaches for rare type. In this paper, we proposed a cell screening method based on automated microrobotic aspiration-and-placement strategy under fluorescence microscope. An innovative autofocusing visual feedback (AVF) method is proposed for precise three-dimensional (3D) locating of target cells. For depth detection, multiple depth from defocus (MDFD) method is adopted to solve symmetry problem and attain an average accuracy of 97.07%. For planar locating, Markov random field (MRF) based locating method is utilized to separate and locate the overlapped cells. The end actuator locating and real-time tracking are performed relying on normalized cross-correlation (NCC) method. Experiential results show that our system collects rare cells (100 cells ml-1) at a speed of 5 cells min-1with 90% purity and 75% recovery rate, which is valuable for biological and medical application.

Microgrippers play a significant role in manipulation of micro-objects. To achieve dexterous and precise manipulation, a microgripper is required to be compactly designed and embedded with sensing feedback. Meanwhile, to convert the sensor position into displacement of the microgripper, the embedded sensors should be calibrated by additional equipment like laser sensor. However, a microgripper always needs to be calibrated during manipulation (online calibration), which is still a big challenge with current technology. In this paper, we proposed a highly compact microgripper integrated with position sensors, and a visual-based calibration method to handle such challenge. Moreover, to enhance grasping accuracy, flexure hinges are employed to achieve a low impedance grasping mechanism and to avoid the backlash in traditional bearing. Furthermore, kinematics analysis and Fine Element Analysis (FEA) are implemented to improve the design efficiency. Finally, fibrous micro-rings are successfully assembled, and the results reveal that the calibrated microgripper can be well employed to operate micro-objects.

Study of cellular behaviors, especially the ultra-rare cell type, can aid in the accuracy of clinic diagnoses as well as the development of bioresearch engineering, thus the importance of isolating them from heterogeneous mixtures. However, current methods may fail in purity, versatility or cause contamination to cell targets, which is fatal drawback to rare cells. To address this issue, we propose a versatile method to automatically select and capture fluorescent stained target cells with high purity and recovery rate, through developing a novel 3D image processing algorithm under dark field. With the automated pick-and-place strategies, the micro-robotic system achieves cell screening even in an environment with ultra-sparse cells. In the proposed visual method, Markov Random Field (MRF) separation is adapted into the fluorescent environment to attain real-time planar location of micropipette and target cells. A reformative method derived from Depth from Defocus (DFD) is brought up to acquire 3D information. The basic system for this method mainly consists of a camera mounted on motorized fluorescent microscope and a micromanipulator for cell capture. The fluorescent label help to screen out most of the undesired cells while also bring extra constraints and requisition to our visual method. Finally, experiments of collecting 3 T3 cells are performed to verify the feasibility and validity of the designed method, achieving average 98% purity and 80% recovery rate within the time limits. This study indicates that proposed visual processing method can not only provides reliable location feedback for micro-manipulation in rare cell sorting, but also can be easily extended to satisfy other automated micro-robotics manipulation.

In micromanipulation, lots of methods have been developed to manipulate objects in microscale. However, few of them can be applied in both the transportation and the rotation of the micro objects. In this paper, we present a novel method to realize the non-contact transportation and rotation of the micro objects based on the vibration-induced swirl flow. A piezo actuator is set between the glass needle and a metal rod. The sine wave with controlled frequency and amplitude is input into the piezo actuator to drive the glass needle to move circularly, which is caused by resonance of the actuator and the metal rod. We place the glass needle under water and keep a limited distance to the bottom. The circular vibration of the glass needle can generate a swirl flow and low pressure around it. The low pressure can trap and transport the micro objects vertically to the glass needle, and the swirl flow can rotate the objects continuously. Finally, we realize the trap and rotation of micro object with only one piezo actuator. Experiments of transportation and rotation of microbeads are carried out, and the results demonstrate it is a simple, low-cost, effective micromanipulation method.

Helical structure assembled by hydrogel microfibers is significant for culture of smooth muscle cells. However, the helical structure is only fabricated at the macroscale, while the fabrication of helical microstructure is still a challenge due to the lack of assembly method. In this paper, we propose a robotics-based assembly method to handle such challenge. An electromagnetic needle (EMN) is employed as end-effector to magnetically reel the microfiber encapsulating magnetic nanoparticles around a micropillar, and a dual-ring structure is designed to keep the microfiber being attracted at the EMN tip. For enhancing the stability of tip attraction, the manipulation mode of anticlockwise pushing microfiber is established. Moreover, the interaction mechanism between EMN tip and microfiber is analyzed by developing a static force model, and then the key condition of stably reeling microfiber is concluded. Furthermore, a robotics-based motion trajectory of EMN tip is planned to achieve a smooth reeling process. Based on such planning, the size of dual-ring structure is further optimized to improve the success rate of reeling. Finally, the helical microstructure with there-turn coils is successfully fabricated.

Due to the poor controllability in hydrogels, Hydrogels-based assembly to form larger 3D complex shapes is still a big challenge. In this paper, we have reported a novel “bottom-up” method to fabricate three-dimensional (3D) magnetic alginate microfibers (MAMs) assemblies with complex shapes. Specifically, Alginate microfibers encapsulating Fe3O4magnetic nanoparticles (MNs) and fibroblasts (NIH/3T3) have been spun using microfluidic method with “pinch-off” scheme. Experimental results show that the MAMs can respond quickly to the magnetic field, enabling their enhanced controllability. The magnetic assembly system is constructed by PDMS microfluidic device, 3D supporting model adhered on the bottom surface of dish filled with deionized water and magnet. The microfluidic “printing“ and magnetic deposition in magnetic assembly process are experimentally demonstrated, respectively. Because of magnetic field, the complex assembly shapes can be fabricated just by moving the microfluidic device in a plane. To match well with the shape of supporting model and to keep a stable assembly structure, the secondary cross-linking method is employed. From the LIVE/DEAD assay, cells can survive well during the magnetic assembly process.

Fabrication of artificial blood vessels in micro scale significantly benefits the regeneration of functional human vascular networks. In this paper, we develop an efficient multi-microrobotic system with an innovative motorized sample holder (MSH) and two manipulators. Air is injected into the solution through a glass pipette fixed on one manipulator to create bubbles. These bubbles conduct a regular rising movement, which is utilized to assemble the 2D ring-shaped microgels fabricated in a simple micro fluidic device. With this novel bubble-based method and the robotic system, we achieve the automation of the assembly. A 1.2 mm long vascular-like microtube with an outer diameter of 200 μm is fabricated. The whole process of the bubble-based assembly is visually observed and analyzed with side view. Key parameters are characterized to improve the assembly. Results show that the automated bubble-based assembly success rate is 100% and average time cost of assembling every microgel is as low as 3.25s.

3D assembly of cellular structures is important for the fabrication of biological substitute in tissue engineering. In this paper, a novel rail-guided multi-microrobotic system was proposed for the assembly of cellular structure. The cellular 2-dimensional (2D) module was fabricated by UV illumination of the crosslinkable hydrogel. The coordinated manipulation among the micromanipulators was performed with newly designed concentric movement along the rail, which realized the arbitrary change of micromanipulator posture. Through the rotation of the end-effectors around the specimen without swapping out the visual field, the manipulation flexibility was improved. The distance information between the micromanipulator and the module was acquired from vision feedback system and utilized for the automatic pick-up of the microstructure. Through the cooperation among multi-manipulators with hybrid motors, the micromanipulation to assemble the 3D structure with 30 nm operation resolution was achieved. Finally, the rail-guided DeSCom system realized the bottom-up fabrication of cellular vascular-like microtube with vision feedback.