In the field of micromanipulation, accurate identification and localization of micro-targets are crucial. This article presents a robust high-precision micromanipulation targets localization and motion tracking algorithm to address the challenges posed by uneven illumination, shadows, object occlusion, and image noise in micromanipulation. The proposed method employs improved bilateral filtering and the Sobel operator for illumination correction and edge contour detection in the horizontal view, achieving identification and Z-directional localization of micro-targets. In the vertical view, the improved grayscale template and the feature point matching algorithms accurately identify micro-targets and locate manipulation points in the X/Y directions, even under occlusion and rotation. Experimental results demonstrate that the algorithm can identify and localize micro-targets with an average error less than 1 μm and an average time of 120 ms, significantly improving the success rate, accuracy, and efficiency compared to existing methods. This article provides a novel approach for the precise automation of micromanipulation tasks.

Docking is a common step in magnetic helical microrobot operations. However, the boundary effect causes helical microrobots to drift laterally near the container bottom. The mismatch between orientation and movement direction further complicates manual steering. In this paper, an automatic control method for on-plane docking of magnetic helical microrobots is proposed, which divides the docking process into preparation, alignment, and insertion stages. First, the docking path is planned according to the position and orientation of the helix and the target. Second, a steering-based path-following controller guides the helix swim to the ready position for alignment. Third, another path-following controller realizes the position alignment through rotational-frequency-controlled movement direction while locking the helix orientation aligned with the target. Finally, the docking is completed in the insertion stage. Experiments verified the automatic docking of helical microrobots with static targets, including inserting into a slender gap/tube, assembling with a micro-shaft axially, and leaning against a rod radially. This method can eliminate the negative impact of operator’s fatigue and inexperience on docking operations and has the potential to be applied in biomedical and microoperations.Note to Practitioners—This paper is motivated by the problem of docking magnetically driven helical-shaped swimming microrobots with target objects on Petri dishes, which is a common step in capture, loading, assembly, and delivery operations. Here, two closed-loop path-following controllers based on steering and rotational frequency control guide the microrobot to approach and align with the target respectively along a calculated path. In alignment, the non-ideal lateral drift of helical microrobots is controlled by rotational frequency for position alignment. Experiments verified the automatic docking of micro-helixes with static targets, including inserting into a slender gap/tube, connecting with a micro-shaft, and leaning against a rod.

Scanning Electron Microscopy (SEM) is an ideal observation tool for small scales robotics. It has the potential to achieve automated nano-robotic tasks such as nano-handling and nano-assembly. Path following control of nano-robot end effectors using SEM vision feedback is a key for an intuitive programming of elementary robotic tasks sequences. It requires the ability to track end effectors under various SEM scan speeds. SEM suffers however from tricky issues that limits robotic tracking capabilities. This paper focuses on one specific issue related to the compromise between the scan speed and the image quality. This restriction seriously limits the performance of conventional vision tracking algorithms when used with electron images. At high scan speed, the image quality is very noisy making very difficult to differentiate the robot end effector from the background, hence limiting the tracking capabilities. The work related in this paper explores for the first time the potential value of Convolutional Neural Networks (ConvNet) in the context of nano-robotic vision tracking inside SEM. The aim is to localize an end-effector, AFM cantilever in the case of the study, from SEM images for any scan speed configuration and despite of low images quality. For that purpose, a data set of AFM tip images is build up from SEM images for the learning algorithm. Network performances are estimated under different SEM scan speeds. Thanks to the learning algorithm, experimental results show robust AFM tip tracking capabilities inside the SEM under various scan speed conditions.

Microobject transporting is an important task in numerous micromanipulation applications. In this paper, we propose a novel method to transport microobjects using a magnetically actuated microrobot at water surfaces. The microrobot is a pentagram-shaped magnetized sheet with typical diameter of$800\boldsymbol{\mu}\mathbf{m}$. By applying magnetic gradient generated by electromagnetic coils, the microrobot can move controllably at water surfaces. Microobjects at water surfaces can be trapped by the microrobot due to the capillary attraction. Microobject transporting is realized by direct push of the microrobot. A releasing strategy by accelerating the microrobot is also presented to separate the microobject from the microrobot. The experimental results show that the maximum transporting speed for solder balls achieved 3.3mm/s with a magnetic gradient of 79mT/m.

In the die bonding process of MEMS microphone chip, the exact center and rotate angle of MEMS microphone chip is corrected after the vacuum nozzle has adsorbed it. In order to achieve high speed and precision placement function, the flying vision system based on rotating mirror is proposed, the mirror is driven to rotate by the power drawn from the nozzle rise and fall movement through a rack and pinion mechanism, the nozzle pick and place functions is not interfered by mirror through designing the rack and pinion mechanism parameters. The vision system can capture an static clear MEMS chip image when the mirror rotated to a 45 degrees position referenced with horizontal position, the center position and rotating angle of MEMS chip is fast calculated by the fine and coarse Hough Transform Algorithm, the center location and angle will be corrected by the control system during the mounted head is flying to placement position. The experimental results show that the system can obtain high-quality MEMS microphone chip image, the chip's center position and rotating angle correction algorithm takes no more than 0.1s, so it can meet the high-speed placement requirements. In this proposed flying vision system, mechanical structure is simple and compact, MEMS chip's image is clear without distortion, and there is no restriction on the mounted head movement trajectory. The flying vision system significantly enhances the quality and efficiency of SMD Chip; it can be widely used in the surface mounted system.

Due to scale effects, the releasing of micro objects has been a long-standing challenge in micromanipulation applications. In this paper a micromanipulation system is presented based on the adhesion control with compound vibration. This adhesion control technique employs inertia force to overcome adhesion force achieving 100% repeatability with releasing accuracy of 4±0.5μm, which was experimentally quantified through the manipulation of 20-100μm polystyrene spheres under an optical microscope. The micromanipulation system consists of a microgripper and a piezoelectric ceramics module. The compound vibration comes from the electrostatic actuator and the piezoelectrically driven actuator. Surface and bulk micromachining technology is employed to fabricate the microgripper used in the system from a single crystal silicon wafer. Experimental results confirmed that this adhesion control technique is independent of substrate. Theoretical analyses were conducted to understand the releasing mechanism. Based on this preliminary study, the micromanipulation system proves to be an effective solution for active releasing of micromanipulation.

In biomedical areas, such as protein crystallization, drug discovery and medical diagnostics, thousands of reagents with different viscosities need to be transferred or dispensed in submicroliter range. Most of the commercial automated liquid dispensing systems need to be calibrated through experiment to accurately dispense liquids when the reagents viscosities change, which is time consuming and less accurate. In this paper, an intelligent precise liquid handling system was developed, in which a high speed MEMS flow sensor was used. Benefiting from the feedback of sensor information, the system can self-adjust the open time of the solenoid vale to accurately dispensing desired reagent volume without pre-calibration. Firstly, the system construction was introduced in detail. Secondly, a novel closed-loop control strategy was proposed to calculate valve open-time for each dispensing cycle, which makes the system immune to liquid viscosity, pressure fluctuation and some other disturbances. Finally, experiments results are presented with different dispensing volumes, coefficient of variance (CV) has been shown to be below 3% at 1ldquol and approach 4% at 100 nl.

Batch microassembly of MEMS sensors is limited by the manual manipulation required specially trained technicians. To reduce the production costs and simultaneously obtain high production quality, a flexible microassembly system for automated bonding of MEMS sensors is developed. The system consists of a set of autonomous modules that can adapt their structures and functions to various sizes of MEMS sensors, including positioning stages, a microscopy imaging system, a flexible micromanipulator, a heater, a fixture, a supply station and auxiliary systems. Optomechatronic design is essential to the development of integrated systems due to the basic importance of microscope optics to microassembly. In this paper, major methodology issues in optomechatronic design of this system are introduced. A wavelet-based microscopic focus measure and a control scheme with a modified Smith predicator to decrease the inherent time delay of vision system are presented. A smart force sensor with one dimension is employed to sense and control the interactive force. To perform manipulations automatically, a control system, including a task planning level and a real-time execution level, is developed. The productivity of the flexible microassembly system is validated by further experiments

Microscopy imaging can not achieve both high resolution and wide image space simultaneously. Image fusion and 3-D surface reconstruction is of fundamental importance to micromanipulation in providing high lever task understanding, task planning and real time control. Purpose of image fusion is to combine those images into one single image with an extended depth-of-field. Previously, real valued wavelet transforms based image fusion suffers from shift variance, poor directionality, and lack of phase information. One promising fusion method based on complex wavelet transforms to overcome these problems is proposed. Combined 2-D position data of "in focus" pixels with a height map obtained from the proposed image fusion method, a 3-D surface reconstruction algorithm of microparts is developed. Experimental results validate performances of the proposed image fusion and 3-D surface reconstruction methods.

Microscopy imaging can not achieve both high resolution and wide image space simultaneously. Autofocusing and 3-D surface reconstruction techniques are of fundamental importance to automated micromanipulation in providing high lever task understanding, task planning and real time control. In this paper, a new wavelet-based focus measure is developed, which provides significantly better depth resolution accuracy, and robustness than previous ones. A complex valued wavelet-based microscopic image fusion method and 3-D surface reconstruction scheme were proposed. Purpose of image fusion is to combine those multi-focus images into one single clear composite image with an extended depth-of-field. Combined 2-D position data of "in focus" pixels with a height map obtained from the proposed image fusion method, a 3-D surface reconstruction algorithm of microparts is developed. Experimental results validate the performances of the proposed image fusion and 3-D surface reconstruction methods

This paper presents a Task-reconfigurable system developed for efficient and reliable MEMS assembly. The system consists of a set of autonomous modules that can adapt their shape and function to various assembly tasks, including a distributed 6 DOF coarse positioning module, a distributed 5 DOF fine positioning module, grippers designed for different manipulation tasks with same mechanical interface, a multi-view imaging system, and control software. In order to achieve high precision and dexterity in microassembly, a hybrid vision-force control method is proposed. For easier reconfiguration, a semi-automated calibration method is applied to quick calibrating of the new reconfigurations. Analysis and 3 cases of microassembly are presented to demonstrate that this system has high reconfigurability and adaptability.

This paper presents the automatic microassembly of miniaturized gear system by hybrid vision-force control with the vision system and the 3-DOF force sensor feedback data. The assembly process consists of three phases, including visual positioning, searching meshing state, and inserting. Visual feedback is used to achieve the coarse positioning and guide the grasping and transporting of microparts during the visual positioning phase. During the searching phase, a fuzzy PID controller is used to control the contact force on z axis and fuzzy logic strategy is developed to search the meshing state by the force feedback on x and y axes. The tolerance compensations movement is used to support the inserting task to avoid the case of blocking and prevent the micro parts from damaging. The assembly experiment of three planetary gears validates the hybrid force control strategy.