Magnetic microrobotic swarms are increasingly studied for their potential in precise motion control under complex and uncertain environments. One of the core challenges in swarm-level control lies in achieving accurate trajectory tracking in the presence of time-varying dynamic parameters, such as fluid resistance. To address such problem, we incorporate error integration into the swarms control framework and derive a kinematic model through model transformation. Based on this model, a Lyapunov-based adaptive robust control strategy is developed to ensure closed-loop stability and compensate for environmental uncertainties. The proposed controller dynamically adapts to unknown environmental variations, and rigorous theoretical analysis establishes the asymptotic stability of the system. A series of experiments are conducted on an electromagnetic actuation platform using silicone oil environments with different viscosities (5 cSt, 10 cSt, and 15 cSt) as well as under a rapidly varying viscosity generated by syringe pump to validate the method. Experimental results confirm that the proposed strategy enables the microrobotic swarm to achieve stable and precise trajectory tracking under varying resistance conditions, demonstrating its robustness and adaptability.

Current numerical methods for solving kinematic identification (KI) and inverse kinematics (IK) are limited in accuracy, convergence rates, and robustness, necessitating further enhancement. This paper presents a modified Halley method for solving the KI and IK problems of serial robots based on the product of exponentials formula, achieving quintic convergence. Specifically, a general error model is first established based on exponential coordinates, and KI and IK are reformulated as root-finding problems. Next, the modified Halley method, which we prove to be a fifth-order method and incorporates a damping strategy, is proposed to resolve the singularity issue and enhance robustness. Subsequently, the Jacobian and Hessian matrices required for the proposed method are analytically derived based on the time differential of exponentials. Furthermore, highly simplified explicit formulas for these matrices are presented for the IK problem. Simulations on serial robots with various configurations validate the proposed method's accuracy, convergence rates, and robustness in solving KI and IK problems, as well as its advantages over the state-of-the-art. Additionally, experimental validation of KI on two physical robots further demonstrates the effectiveness of the proposed method. Our custom-written MATLAB and C++ codebases are made publicly available for download.

The aspiration to replicate the capabilities of the human hand has driven innovations in the design of soft robotic hands. Despite these advancements, many existing designs of soft hands still lack effective in-hand vision and the ability for each finger to achieve active multi-degree-of-freedom motion. This paper proposes a cable-driven soft robotic hand that can achieve dexterous grasping and manipulation, vision-guided grasping, vision-based slip detection and compensation, as well as visually servoed in-hand manipulation. The hand has five soft fingers, each capable of independent flexion/extension motion and bidirectional ad/abduction motion. A red-green-blue-depth (RGB-D) camera is integrated into the palm of the soft hand to enable in-hand vision capability. Modeling of the soft hand is established to analyze its kinematics, statics, and manipulability. A series of experiments are conducted to demonstrate its dexterous grasping and manipulation capabilities on a variety of objects. Using 3D point cloud data from the in-palm camera, an effective vision-guided grasping strategy is developed to grasp objects on a table. The in-hand vision also enables slip detection and compensation during grasping to maintain the grasp stability. Furthermore, a hierarchical, visually servoed controller is developed to perform closed-loop in-hand object manipulation. With its high dexterity and visual feedback capabilities, the soft hand will find important applications such as household object manipulation and food picking/sorting, and may also be used as a prosthetic hand or an auxiliary hand for humans.

The collaboration of microrobot teams has attracted considerable attention, particularly in the field of micro/nano manipulation. Achieving independent control and motion planning of multiple magnetic microrobots for coordinated movements is one of the most important tasks that is still unsolved. In this paper, a$12\times 12$coil array system is developed to generate a series of localized magnetic fields that enable simultaneous control of multiple identical magnetic microrobots, allowing teams of microrobots to collaborate in parallel for micromanipulation tasks. First, the structure of the microcoil is optimized based on the finite element model to increase the strength and gradient of the magnetic field, which in turn enhances the driving performance of the system. Meanwhile, an improved multi-target tracking algorithm that utilizes kernel correlation filtering (KCF) and image contour detection (ICD) techniques is proposed to improve the tracking accuracy of microrobots. In addition, collaborative planning for multiple magnetic microrobots is also achieved with the combination of the conflict-based search (CBS) algorithm. Finally, the developed system is tested with extensive physical experiments. Especially, experiments on magnetic droplet transport with two microrobots are also conducted. The results impressively demonstrated the effectiveness of the devised system and the proposed methods. Note to Practitioners—This article is motivated by the recent wide interest in magnetic microrobots. Actuated by external magnetic field, magnetic microrobots can wirelessly perform targeted delivery/therapy and other micro-assembly tasks. To facilitate collaboration between microrobots, independent control of each microrobot is desirable. However, due to the interaction between magnetic microrobots and the global magnetic field, the collaboration of multiple microrobots presents great challenges. Therefore, several coil-array-based systems have been developed. In this paper, we develop a magnetic actuation system from both hardware and software aspects for the collaborative motion of multiple magnetic microrobots. The coil structure is optimized to enhance the driving performance of the devised system, and a fused multi-target tracking algorithm is proposed to improve the tracking accuracy. In combination with the CBS algorithm, collision-free paths are planned for multiple identical microrobots. The experimental results show that the constructed system and proposed methods can realize coordinated motion of multiple identical magnetic microrobots, which has enormous potential for some biomedical applications.

Magnetic field-driven microrobots have shown high potential in the field of medical applications. Autonomous navigation is a crucial concern for magnetic microrobots, however, the path planning, actuation and control of magnetic microrobots still remain challenging, especially for complex and large-workspace human body environments. Depending on the specific task and environmental conditions, it is important to employ appropriate planning and control architectures for the magnetic navigation systems. In light of this objective, this paper introduces a novel navigation framework, using an improved path planning and following control method. An evolutionary strategy based RRT (ES-RRT) planner is designed to achieve a shorter, smoother and safer path. Furthermore, an extended state observer (ESO)-based controller is specifically designed for the path tracking process of the microrobots. This controller enables the microrobots to accurately follow the computed path. Experiments demonstrate the effectiveness of the proposed strategy: Feasible path in different conditions and environments can be obtained with short and smooth enough, and autonomous navigation following of microrobots is realized with satisfactory path tracking control accuracy.Note to Practitioners—In contrast to macroscale robots, microrobots face challenges when it comes to integrating onboard components such as processors and power sources. Consequently, alternative methods have been developed, including optical, chemical, and biological actuation. Among these approaches, the utilization of magnetic fields is particularly favorable due to its ability to penetrate deep tissues while ensuring high safety. Additionally, magnetic fields offer diverse propulsion options, such as rotating fields and oscillating fields, along with excellent controllability. Despite significant advancements in the fabrication, functionalization, and locomotion of magnetic microrobots, autonomous navigation remains an area that requires further development. In the medical application, the planning path of the microrobots needs to be short and smooth enough. Besides, considering the unknown dynamics and external disturbances of the system, it is extremely important for microrobots to complete the precise path following control of the planned path. The motivation of this work is to develop an effective navigation scheme for microrobots which consisting of a path planner and motion controller. A shorter and smoother path will be planned based on a novel improved RRT planner. The precise control of the path following then will be achieved by adopting an extended state observer (ESO) controller. The proposed methods would enable microrobots to act safely and greatly enhance robots’ capabilities.

Magnetic microrobots demonstrate significant potential in medical applications by providing innovative solutions for precise treatment through targeted drug delivery, minimally invasive surgery, and vascular cleaning. However, within biological organisms, there are various complex obstacle environments that require a navigation technology prioritizing safety and emphasizing smoothness. This paper proposes a safety-enhanced navigation planning (SENP) algorithm to achieve multiple objectives such as safety, path smoothness, and short distance, enabling collision-free navigation in complex medical environments. Unlike traditional methods that require multiple heuristic cost functions to guide the navigation planning algorithm, our approach leverages the safety-enhancing features of the safe artificial potential field (SAPF) to promote collision-free navigation in complex medical environments. By adopting a goal-biased strategy to guide the generation of random sampling points, the number of iterations is reduced, and the convergence speed of the algorithm is improved. In addition, the traditional artificial potential field planning method often leads to the oscillation of the corridor, our method effectively addresses this issue along with the uncertainty in the quality of the initial path and the lengthy convergence time to the optimal path. Comparative analysis with various algorithms in different environments shows that our proposed method excels in terms of smoothness and path length under the premise of safety, making it suitable for magnetic microrobots in complex environments.Note to Practitioners—The motivation for this work lies in advancing safe, efficient, and rapid navigation strategies for magnetic microrobots in medical applications. While significant progress has been made in the development of magnetic microrobots, navigating through complex environments such as human blood vessels remains a substantial challenge. The ability to perform safe and effective motion planning within narrow and intricate channels is crucial for medical applications. In response to this need, we propose a path planning method specifically designed for magnetically actuated microrobots, based on SENP. Our approach emphasizes achieving a path that is not only short and smooth but also prioritizes safety throughout the navigation process. Compared to traditional sampling-based algorithms, our method effectively overcomes limitations such as initial path quality uncertainty and prolonged convergence to optimal paths. This allows for the rapid generation of a high-quality initial path with a focus on safety while significantly accelerating convergence. Moreover, our approach provides a robust solution for navigating narrow channels, making it highly suitable for challenging medical environments.

Point cloud clustering is a promising method for 3D surface defect diagnosis in manufacturing but requires manual clustering parameter selection, reducing usability. This paper proposes an automatic point cloud clustering method to address this issue. It employs a strategy that progresses from coarse to fine. In the coarse searching stage, a K-Nearest Neighbor (KNN) graph analysis technique is developed to recognize potential defective regions in parallel. Moving on to the fine stage of extracting detailed defects, a modified DBSCAN algorithm is proposed, in which the clustering parameters are calculated automatically from the KNN graph analysis results. Experimental results showed that the proposed method achieved cloud clustering with automatically calculated clustering parameters for surface defect diagnosis. The proposed method outperformed the traditional region growing algorithm in accuracy (0.942 vs. 0.680) and processing speed (21500 points/sec vs. 8740 points/sec) without requiring manual intervention.Note to Practitioners—This paper presents a method for diagnosing defects on automobile and flat steel surfaces. Current 3D point cloud techniques for surface defect diagnosis require manual parameter adjustments, reducing usability. This paper proposes an automatic method without manual intervention. The proposed method uses a coarse-to-fine strategy. The 3D point cloud is divided into sub-blocks to locate potential defects, and a clustering algorithm then extracts detailed defects with automatically determined parameters. We mathematically characterize changes in point density caused by surface defects and show how these features can be used for clustering parameter calculation. Experimental results demonstrate the method’s efficiency on flat as well as some curved surfaces, but it has yet to be evaluated on complex structures. Future work will aim to broaden its application to include a more extensive variety of surfaces and integrate it with robotic vision systems.

Soft robotic grippers possess high structural compliance and adaptability, allowing them to grasp objects with unknown and irregular shapes and sizes. To enable more dexterous manipulation, soft sensors that are similar in mechanical properties to common elastomer materials are desired to be integrated into soft grippers. In this paper, we develop ionic hydrogel-based strain and tactile sensors and integrate these sensors into a three-finger soft gripper for learning-based object recognition and force-controlled grasping. Such hydrogel-based sensors have excellent conductivity, high stretchability and toughness, good ambient stability, and unique antifreezing property; they can be readily attached to a soft gripper at desired locations for strain and tactile sensing. By using a deep-learning model, the sensory soft gripper is demonstrated to be capable of grasping and recognizing objects at both room and freezing temperatures, and achieving close to 100% recognition accuracy for ten typical objects. Moreover, the capacitive tactile feedback of the gripper is utilized to develop a closed-loop force controller and realize force-controlled grasping of fragile or highly deformable objects. A new slip detection and compensation strategy is also proposed and validated for the sensory gripper for adjusting the grasping force in real time upon detecting slippage. Note to Practitioners—The multimodal sensation of a soft robotic gripper could enrich its grasping functionalities and improve its manipulation performance. This research integrates novel antifreezing ionic hydrogel-based strain and tactile sensors into a three-finger soft robotic gripper for learning-based object recognition and force-controlled grasping. Constructed from a highly stretchable, ambient-stable, and antifreezing ionic hydrogel, the strain and tactile sensors can be readily integrated at the desired locations on the soft gripper, and can reliably operate at both ambient and freezing temperatures with excellent mechanical and electrical properties. Based on the feedback of the strain and tactile sensors, a deep learning model is employed to enable high-accuracy object recognition while grasping, which can be useful for manipulation in vision-free environments. Closed-loop force control and slip compensation strategies are also demonstrated for reliably grasping fragile/deformable objects and handling slip events during the manipulation of heavy objects. The sensory soft gripper and the associated object recognition and force control methods could find practical applications in a variety of robotic manipulation tasks.

Nanomaterials possess superior mechanical, electrical, and optical properties suitable for device applications in different fields such as nanoelectronics, photonics, and sensors. Characterizing the multiphysical properties of single nanomaterials and nanostructures provides experimental guidelines for synthesis and device applications of functional nanomaterials. Nanomanipulation techniques under scanning electron microscopy (SEM) have enabled the testing of mechanical and electrical properties of various nanomaterials. However, the introduction of micro-photoluminescence ($\mu $-PL) measurement into an SEM setup for in-situ single nanomaterial characterization is still experimentally challenging; in particular, the seamless integration of the mechanical, electrical, and$\mu $-PL testing techniques inside an SEM for multi-field-coupled characterization of single nanostructures is still unexplored. In this work, we report the first SEM-based nanomanipulation system for multiphysical characterization of single nanomaterials. A custom-made, optical-microfiber-based$\mu $-PL setup is integrated onto a nanomanipulation system with four nanomanipulators inside an SEM. The system is also equipped with a conductive nanoprobe and a conductive atomic force microscopy (AFM) probe for electrical nanoprobing and electroluminescence (EL) measurement of single nanomaterials with contact force feedback. Using the system, field-coupled characterization (i.e., optomechanical, optoelectronic, electromechanical, and mechano-optoelectronic testing) of single InGaN/GaN nanowires (NWs) are conducted; and, for the first time, the effect of mechanical compression applied to individual InGaN/GaN NWs on its optoelectronic property is revealed. Note to Practitioners—With the rapid advances of nanophotonics and nanoelectronics, the optical and optoelectronic characterization of semiconductive nanomaterials becomes widely used for guiding the material synthesis and improving the nanodevice performance. However, few studies on optical-relevant characterization were carried out in SEM, mainly due to the limited space of an SEM chamber, making it challenging to integrate optical components for effective optical excitation and luminescence measurement. To address this issue, space-saving optical microfibers were integrated into the SEM chamber for in-situ optoelectronic characterization of semiconductor NWs, along with the seamless integration of mechanical and electrical nanoprobing tools for electromechanical characterization. The developed nanomanipulation system will greatly facilitate the multiphysical testing of semiconductor nanomaterials, and thus expedite their synthesis optimization processes and broaden their optoelectronic device applications.

Soft robotic grippers possess high structural compliance and adaptability for grasping objects with unknown and irregular shapes and sizes. To enable more dexterous manipulation, soft sensors with similar mechanical properties to common elastomer materials are desired to be integrated into soft grippers. In this paper, we develop ionic hydrogel-based strain and tactile sensors and integrate these sensors into a three-finger soft gripper for learning-based object recognition. Such hydrogel-based sensors have excellent conductivity, high stretchability and toughness, good ambient stability, and unique anti-freezing property, and can be readily attached to a soft gripper at desired locations for strain and tactile sensing. Based on a deep-learning model, we demonstrate the capability of the sensory soft gripper for object grasping and recognition at both room and freezing temperatures, and achieve high recognition accuracy close to 100% for 10 typical objects. With these abilities, our gripper can find interesting applications such as sorting food or chemicals in low temperature storage and cold chain transportation, or manipulating equipment in polar area.

The nematode worm Caenorhabditis elegans is a model organism widely used in biological research on genetics, development, neuroscience, and aging. Microinjection is an effective and widely adopted method to create transgenetic worms, perform ribonucleic acid (RNA) interference of certain genes, and introduce different types of molecules into specific locations inside a worm body. Based on microfluidics and robotic micromanipulation techniques, we develop a robotic system for automated microinjection of C. elegans with greatly improved injection speed and success rate over traditional manual microinjection. A double-layer microfluidic device with computer-controlled pneumatic valves is developed for automated on-chip loading, immobilization, injection, and downstream sorting of single worms. A new autofocusing-based contact detection algorithm is proposed to find the optimal injection position along the depth direction of the microscope field of view. The direction and location of the needle tip are reliably identified using an image processing algorithm. Through experiments on 240 worms, the system demonstrates automated injection at a speed of 6 worms/min (9.97 s/worm) with a presorting operation success rate of 78.8% (postsorting operation success rate: 100%), which are more than 23 times faster and 1.6 times higher than the speed (0.25 worm/min) and success rate (30%) of a proficient human operator, respectively. With the superior performance, this system will enable new large-scale gene- and molecule-screening studies on C. elegans that cannot be fulfilled by the conventional microinjection technique.

Functional nanomaterials possess exceptional multi-physical (e.g., mechanical, electrical and optical) properties compared with their bulk counterparts. To facilitate both synthesis and device applications of these nanomaterials, it is highly desired to characterize their multi-physical properties with high accuracy and efficiency. The nanomanipulation techniques under scanning electron microscopy (SEM) has enabled the testing of mechanical and electrical properties of various nanomaterials. However, the seamless integration of mechanical, electrical, and optical testing techniques into an SEM for triple-field-coupled characterization of single nanostructures is still unexplored. In this work, we report the first SEM-based nanomanipulation system for high-resolution mechano-optoelectronic testing of single semiconductor InGaN/GaN nanowires (NWs). A custom-made optical measurement setup was integrated onto a four-probe nanomanipulator inside an SEM, with two optical microfibers actuated by the nanomanipulator for NW excitation and emission measurement. A conductive tungsten nanoprobe and a conductive atomic force microscopy (AFM) cantilever probe were integrated onto the nanomanipulator for electrical nanoprobing of single NWs for electroluminescence (EL) measurement. The AFM probe also served as a force sensor for quantifying the contact force applied to the NW during nanoprobing. Using this unique system, we examined, for the first time, the effect of mechanical compression applied to an InGaN/GaN NW on its optoelectronic properties.

This paper reports a vision-based microfluidic system for automated, high-speed sorting of the nematode worm C. elegans. Exceeding the capabilities of conventional worm sorting microfluidic devices purely relying on passive sorting mechanisms, our system is capable of accurate measurement of the worm body length/width and active sorting of worms with the desired sizes from a mixture of worms at different developmental stages. This feature is enabled by the combination of vision-based worm detection and sizing algorithms and automated on-chip worm manipulation. A double-layer microfluidic device with computer-controlled pneumatic valves is developed for sequential loading, trapping, imaging, and sorting of single worms based on vision-based worm size measurement results. To keep the system operation robust, vision-based algorithms for detecting multi-worm loading and worm size measurement failure have also been developed. We conducted sorting experiments on 319 worms and achieve an average sorting speed of 10.4 worms per minute (5.8 s/worm) with an operation success rate of 90.3%. This system will facilitate worm biology studies where body size measurement and size-based sorting of many worms are needed.

This paper reports the electrostatic regulation of surface traction of a quadruped soft robot to improve its locomotion efficiency. The soft robot, containing five pneumatic channel networks (PneuNets) in different parts of its body, is actuated to achieve undulated locomotion. Electrostatic adhesion is applied to the bottom surface of each robot leg, by using a thin elastomeric adhesion pad embedded with interdigitated comb electrodes. The adhesion pad is fully compatible with the soft robot structure, and is able to adjust the level of surface traction on the robot leg during locomotion. We calibrate the adhesion force generated by the pad as a function of its size and the applied electrostatic voltage. We demonstrate the control of the moving direction and speed of the soft robot on horizontal surfaces with different frictional and electrical characteristics, by adjusting the level of electrostatic adhesion. With the electrostatic traction control, the robot can also climbing up an inclined metal surface with a low coefficient of friction, which cannot be achieved by the same robot without adhesion pads. This work illustrates the important role of surface friction on locomotion of the soft robot, and provides an efficient solution to surface traction control of soft robots.

This paper reports a microelectromechanical systems (MEMS)-based microgripper, integrating two-axis actuators and force sensors, for microscale compression and shear testing of soft materials. The device employs V-beam electrothermal actuators to drive an active gripping arm and compress or shear a microscale sample grasped at the gripping tips, and two triplate differential capacitive sensors to measure the compression and shear forces applied to the sample with nanonewton resolution (compressive force resolution: 7.7 nN, and shear force resolution: 57.5 nN). Using the microgripper, we demonstrate, for the first time, on-chip compression and shear testing of polydimethylsiloxane (PDMS) microstructures prepared at different crosslinking levels. We believe that this device will be useful for accurately characterizing mechanical properties of a variety of microscale soft materials.

Many micromanipulation systems employ sensorless actuators and possess unknown modeling errors, feedback measurement noise, and time delays. Conventional modelbased control schemes ignore some of these characteristics, and thus sacrifice the control performance of the system. This paper presents a new model compensation-prediction scheme for control of micromanipulation systems, which estimates the unknown modeling errors from single noisy feedback measurement and predicts and compensate the system time delay. This approach combines two modeling errors into a single equivalent error through mathematical transformation, and estimates the combined term using a noise-insensitive extended high-gain observer (EHGO). After removing the unknown term, the system is then transformed into a time invariant form, and a Smith predictor is implemented to predict and compensate the time delay. The effectiveness of the proposed compensation-prediction scheme is demonstrated by both numerical simulation and experiments of two typical micromanipulation systems. The results show that this method is able to significantly improve the control performance of a conventional PID controller by simultaneously reducing the settling time and overshoot of the system.

The capability of accurately applying millinewton-level touch stimuli to Drosophila larvae and simultaneously observing their resultant fluorescence responses in mechanosensitive neuron transmission will enable novel studies of mechanotransduction neural circuitry. This paper presents an automated robotic micromanipulation system capable of force-controlled mechanical stimulation and quantitative fluorescence imaging of Drosophila larvae, which significantly improves the force regulation accuracy and operation consistency over conventional manual operations. An elastomeric microdevice is developed for efficient immobilization of an array of larvae for subsequent force-controlled touching. A microelectromechanical systems (MEMS) based force sensor is integrated into the robotic system for closed-loop force control of larva touching at a resolution of 50 μN. Two micromanipulators are coordinately servoed using orchestrated position and force control laws for automatic operations. The system performs simultaneous force-controlled larva touching and fluorescence imaging at a speed of four larvae per minute, with a success rate of 92.5%. This robotic system will greatly facilitate the dissection of mechanotransduction mechanisms of Drosophila larvae at both molecular and cellular levels.

The tiny nematode worm Caenorhabditis elegans has long been a popular model organism for genetic, developmental, and biochemical studies in which worm microinjection plays a critical role. This paper presents an automated robotic system for high-speed injection of C. elegans with an efficiency more than 10 times faster than that of a proficient injection technician. To facilitate the injection process, a multilayer, hydraulically-controlled polydimethylsiloxane (PDMS) microfluidic device is developed to rapidly load, immobilize, flush, sort and collect individual worms. In addition, a newly proposed contact detection algorithm is adopted to find the optimal injection position along the z axis within the microscope view field. The direction and location of the needle tip are identified online based on an effective image processing algorithm. According to continuous injection of 40 worms, our system is able to perform worm injection at a speed of 6.6 worms per minute with a pre-sorting success rate of 77.5% (post sorting: 100%). The superior performance provided by the system will significantly facilitate large-scale transgenic studies and biomolecule screening on C. elegans.

Force sensing and control are of paramount importance in robotic micromanipulation. A contact force regulator capable of accurately applying mechanical stimuli to a live Drosophila larva could greatly facilitate mechanobiology research on Drosophila and may eventually lead to novel discoveries in mechanotransduction of neuron circuitry. In this paper, we present a novel contact force scheme implemented in an automated Drosophila larvae micromanipulation system, featuring a switched fuzzy-PD controller and a noise-insensitive extended high gain observer. The switched fuzzy-PD controller inherits the fast convergence from fuzzy controller and overcomes the drawbacks (overshoot and oscillation) of a conventional fuzzy controller. The observer provides precise estimation to compensate for system modeling errors despite force measurement noise, which overcomes the disadvantage of traditional high gain observer. Force control experiments show that, compared to an conventional PID controller, this new controller-observer scheme has significantly enhanced dynamic performance in terms of rising time, overshoot, and oscillation. The developed robotic system and the force control scheme will be applied to mechanical stimulation and fluorescence imaging of Drosophila larvae for identifying new mechanotransduction mechanisms.

This paper presents a robotic system that is capable of both picking up and releasing microobjects with high accuracy, reliability, and speed. Due to force-scaling laws, large adhesion forces at the microscale make rapid, accurate release of microobjects a long-standing challenge in micromanipulation, thus representing a hurdle toward automated robotic pick-and-place of micrometer-sized objects. The system employs a novel microelectromechanical systems (MEMS) microgripper with a controllable plunging structure to impact a microobject that gains sufficient momentum to overcome adhesion forces. The performance was experimentally quantified through the manipulation of 7.5-10.9 ¿m borosilicate glass spheres in an ambient environment. Experimental results demonstrate that the system, for the first time, achieves a 100% success rate in release (which is based on 700 trials) and a release accuracy of 0.45 ± 0.24 ¿m. High-speed, automated microrobotic pick-and-place was realized by visually recognizing the microgripper and microspheres, by visually detecting the contact of the microgripper with the substrate, and by vision-based control. Example patterns were constructed through automated microrobotic pick-and-place of microspheres, achieving a speed of 6 s/sphere, which is an order of magnitude faster than the highest speed that has been reported in the literature.

This paper presents the use of a monolithic, force-feedback MEMS (microelectomechanical systems) microgripper for characterizing both elastic and viscoelastic properties of highly deformable hydrogel microcapsules (15–25µm) at wet state during micromanipulation. The single-chip microgripper integrates an electrothermal microactuator and two capacitive force sensors, one for contact detection (force resolution: 38.5nN) and the other for gripping force measurements (force resolution: 19.9nN). Through nanoNewton force measurements, closed-loop force control, and visual tracking, the system quantified Young's modulus values and viscoelastic parameters of alginate microcapsules, demonstrating an easy-to-operate, accurate compression testing technique for characterizing soft, micrometer-sized biomaterials.

Taking a different architecture than manual operation and existing microrobotic systems, this paper presents the first automated system that employs novel microfabricated cell holding devices and vision-position based control of multiple motion control devices to achieve easy sample immobilization, rapid cell orientation, and fast injection of mouse embryos. The system requires minimal human involvement through a maximum of three computer mouse clicking per mouse embryo, is human operator skill independent, and is immune from fatigue. While no robotic systems have provided performance close to manual operation, according to the preliminary experimental results (n = 90) from this study, this microrobotic system demonstrated: (i) an injection speed of 9 embryos/min vs. 2 embryos/min in typical manual operation, (ii) a success rate of 98.9%, a higher rate than the best success rate (90%) achieved by proficient injection technicians with over 10 years' experience, and (iii) a high survival rate of 82.1%, a rate comparable with the best survival rate (~80%) achieved by proficient injection technicians. Further improvement of the automated system will change the way of how mouse embryos are injected and promise its practical use in biology laboratories and mouse facilities.

This paper presents an automated cell contour visual measurement technique and a data synchronization mechanism for real-time, high-accuracy mechanical characterization of individual cells with micropipette aspiration. A computer vision tracking algorithm is developed for automatically measuring cell deformation parameters in real time (30 Hz) with a resolution down to 0.21 pixel, significantly enhancing the accuracy and efficiency of micropipette aspiration. To achieve a high characterization accuracy, the cell deformations and applied pressure changes are precisely synchronized using a data synchronization mechanism. Experimental results on both solid-like cells (interstitial cells) and liquid-like cells (neutrophils) quantitatively demonstrate that the visual tracking algorithm is capable of significantly increasing the efficiency and accuracy of micropipette aspiration. Among several characterized mechanical parameters, the viscoelastic properties of porcine aortic valve interstitial cells were, for the first time, quantified in this study.

The ability of efficiently delivering soluable/insoluable drug compounds or biomolecules into individual biological cells and quantifying their cellular responses is important for genetics, proteomics, and drug discovery. This paper presents a fully automated system for zebrafish embryo injection, which overcomes the problems inherent in manual injection, such as human fatigue and large variations in success rates due to poor reproducibility. Based on ldquolooking-then-movingrdquo control, the microrobotic system performs injection at a speed of 15 zebrafish embryos (chorion unremoved) per minute. Besides a high injection speed that compares favorably with that of a highly proficient injection technician, a vacuum-based embryo holding device enables fast immobilization of a large number of zebrafish embryos, shortening the embryo patterning process from minutes to seconds. The recognition of embryo structures from image processing identifies a desired destination inside the embryo for material deposition, together with precise motion control resulting in a success rate of 100%. Carefully tuning suction pressure levels as well as injection and retraction speeds produced a high survival rate of 98%. The quantitative performance evaluation of the automated system was based on the continuous injection of 250 zebrafish embryos. The technologies can be extended to other biological injection applications such as the injection of mouse embryos,Drosophilaembryos, andC.elegansto enable high-throughput biological and pharmaceutical research.

This paper presents the first demonstration of force-controlled micrograsping at the microNewton force level. The system manipulates highly deformable biomaterials (hydrogel microcapsules and biological cells) in an aqueous environment using a MEMS-based microgripper with integrated force feedback along two axes. The microgripper integrates an electrothermal V-beam microactuator and two capacitive force sensors, one for contact detection (force resolution: 38.5 nN) and the other for gripping force measurements (force resolution: 19.9 nN). The MEMS-based microgripper and the force control system experimentally demonstrate the capability of rapid contact detection and reliable force-controlled micrograsping to accommodate variations in sizes and mechanical properties of objects with a high reproducibility. Cell viability testing validated that the temperature at gripping arm tips does not exceed 50degC.

Injection of foreign materials (e.g., DNA, RNAi, sperm, protein, and drug compounds) into individual cells has significant implications in genetics, transgenics, assisted reproduction, and drug discovery. This video presents a microrobotic system for fully automated zebraflsh embryo injection, which overcomes the problems inherent in manual operation, such as human fatigue and large variations in success rates due to poor reproducibility. Based on computer vision and motion control, the automated microrobotic system is capable of immobilizing a large number of zebraflsh embryos into a regular pattern within seconds and injecting 15 embryos (chorion unremoved) per minute with a success rate, survival rate, and phenotypic rate all close to 100%.

This paper presents a micropipette aspiration system and a cell contour visual tracking algorithm for realtime, high-accuracy mechanical characterization of individual cells. The computer vision tracking algorithm measures cell deformation parameters in real time (30Hz) with a resolution down to 0.21 pixel, significantly enhancing the accuracy and efficiency of the micropipette aspiration technique. Representing another advantage over manual measurements in terms of characterization accuracies, the micropipette aspiration system features precise synchronization between cell deformations and applied pressure changes. Experimental results on both solid-like cells (interstitial cells) and liquid-like cells (neutrophils) demonstrate the effectiveness of the system and the visual tracking algorithm. Among several characterized mechanical parameters, the viscoelastic properties of porcine aortic valve interstitial cells were, for the first time, quantified in this study.

Manipulation and characterization of individual biological cells require cellular forces be precisely measured in real time. This paper presents a computer vision-based cellular force measurement platform that allows for the use a single vision sensor to simultaneously obtain two forms of feedback (i.e., vision and force). A novel silicone elastomer-based cell holding device and a sub-pixel visual tracking algorithm are developed. Deflections of elastic, low-stiffness structures are visually tracked, and material deflections are subsequently transformed into cellular forces. Experimental results demonstrate that the current vision-based force sensing system is capable of performing robust cellular force measurements at a full 30 Hz with a 3.7 muN resolution. Importantly, the vision-based cellular force sensing framework established in this study is not scale or cell line dependent. The device design, visual tracking algorithm, and experimental technique form a powerful framework that permits visually resolving cellular forces in real time with a picoNewton (26 pN) resolution for applications in single cell manipulation and characterization