Noncontact manipulation of soft micro-fibers has great potential in advanced manufacturing, materials science, and biomedical engineering. However, current noncontact manipulation techniques primarily focus on objects with regular shapes, e.g., solid particles, cells, or droplets, with fewer solutions available for manipulating flexible and elongated structures. In this paper, an automated ultrasonic manipulation system is introduced for in-plane soft micro-fiber manipulation, which mainly consists of an ultrasonic transducer array and a microscope. A real-time trap generation algorithm is designed to manipulate the micro-fibers by the visual feedback from microscope. An adequate theoretical analysis is also provided for explanation of the deformation behavior of micro-fiber under external forces. The system is capable of precise in-plane positioning and motion trajectory planning to micro-fiber end, and in-plane morphological reshaping to the micro-fiber. Experiments validated the effectiveness of the proposed system for the in-plane manipulation of soft micro-fibers. Finally, the system was showcased by the practical application of material property characterization.

Acoustic tweezers have been a valuable tool across various fields, from nano-microfabrication to biology. Their unique characteristics enable three-dimensional particle manipulation, where acoustic trapping serves as a fundamental requirement. However, traditional methods struggle to maintain steady particle positioning due to nonlinear forces and complex dynamic coupling effects. As a result, particle oscillations are inevitable and cannot be effectively compensated by predesigned acoustic trapping. To address these challenges, this study introduces a novel visual feedback control approach that dynamically adjusts the acoustic field distribution to mitigate oscillations along the z-axis of the acoustic trapping. A binocular microscopic vision system is employed for precise particle localization, while a disturbance observer estimates the effects of strong nonlinearity and uncertainties of the acoustic trapping. The proposed methodology is validated through simulations and experiments, demonstrating a significant reduction in z-axis oscillations from 1.33× wavelength to within 0.03× wavelength. This advancement marks a step forward in achieving precise and complex acoustic manipulation using traveling-wave acoustic tweezers.

Deep-learning-based object detection has gained widespread application in surface defect inspection, with anchor-based detectors achieving remarkable success by utilizing dense anchors to align with defects. Determining the optimal anchor configuration, i.e., sizes and aspect ratios of anchor boxes, remains a critical challenge, particularly when addressing defects with significant shape variations. While previous studies have predominantly focused on developing more efficient network architectures and learning strategies, the problem of anchor configuration determination has not been thoroughly explored. To address this gap, this paper proposes the IoU-Aware Clustering (IAC) algorithm, which autonomously learns suitable anchor configurations by extracting shape priors from diverse defects. IAC takes the training bounding boxes as potential clustering centers and selects a subset that aligns with the shape distribution of the training samples. The algorithm involves only a single hyper-parameter, the anchor number k, making it highly adaptable to various scenarios. Experimental results demonstrate that IAC can effectively generate anchor configurations tailored to defect shapes, significantly improving the mean Average Precision (mAP) by 6.9% and 14.4% on two industrial defect datasets with substantial shape variations.

Visual tracking is crucial in nanomanipulation inside scanning electron microscopy (SEM), especially for complex 3D manipulation tasks. However, tracking the micro- and nanoscale objects and manipulators under different rotational angles, especially out-of-plane rotation, remains challenging due to significant changes in their appearance in the image space. In this paper, we propose a template matching-based method for nanoscale tracking, particularly addressing challenges from out-of-plane rotation. By leveraging the image Jacobian matrix, we establish the relationships between image and Cartesian coordinates, enabling dynamic generation of templates for specified rotation angles. Then, a visual tracking pipeline is proposed, consisting of an offline preparation stage and an online tracking stage. Based on the proposed template generation method, the pipeline dynamically generates appropriate templates as rotational angles vary and performs accurate tracking using template matching. Further, the templates can be conveniently generated for specified magnifications using the image Jacobian matrix, enabling adaptation to changes in SEM magnification. Extensive experiments, including tracking under various SEM magnifications, complex trajectories, and different types of end-effectors, are conducted to demonstrate the effectiveness of the proposed method. Comparisons with widely used tracking approaches further highlight its superiority in both tracking accuracy and real-time performance. Finally, the successful deployment of the proposed method in a real nanomanipulation task confirms its practical applicability.

Noncontact trapping of micro objects has great application potential in fields like material science and biomedical engineering due to its label-freeness and biocompatibility. In this paper, an automated acoustic micro-particle trapping system implemented with phased transducer array (PTA) is prototyped. The system is incorporated with a stereo vision to provide visual feedback benefited from localization of the invisible acoustic field through hydrophone scanning. Binocular vision calibration and stereo matching are realized using image Jacobian matrix. An efficient phase modulation algorithm is proposed for the calculation of desired PTA phase profile in real-time and a spatiotemporal multiplexing control strategy is adopted to dynamically generate multiple trappings. Experimental results well demonstrated that the stable trapping of multiple particles can be robustly realized by the system, leading to the improvements of robotic noncontact manipulation with invisible acoustic end-effector. Note to Practitioners—This paper is motivated by the problem that previous classic acoustic trapping was achieved as a physical phenomenon that particles within the trapping zone would be automatically trapped and thus required people to place the particle into the invisible trapping zone, which is neither precision nor efficient. Such problem is a crucial factor that limits acoustic tweezer to be further readily usable in bioengineering, surface manufacturing, and quantitative micromechanical characterization. In this work, automated acoustic trapping is presented in the context of robotics, as grasping task in conventional industrial robots, that can generate the acoustic trap exactly in the location where particles are detected (by microscopic vision, or micro-CT or acoustic imaging, etc.). This paper proposes a full pipeline to automatically trap multiple particles using ultrasonic transducer array and binocular microscopic vision. The experiments verified the ability of proposed method in simultaneously trapping three micro particles with opposite acoustic properties. Such trapping method is the foundational technology for further acoustic manipulation such as arraying and sorting, which will be the tasks in our future work.

The noncontact acoustic manipulation of particles, biosamples, droplets, and air bubbles has emerged as a promising technology in the fields of biology, chemistry, medicine, etc. The noncontact nature offers significant advantages in terms of biocompatibility, contamination free, and material versatility. However, current noncontact acoustic manipulation techniques still lack adequate selectivity, robustness, and precision controllability in complex environments. To this end, in this article, we propose an automated noncontact manipulation system that leverages a high-density ultrasonic phased transducer array in combination with a microscope to further optimize and enhance the controllability and flexibility of noncontact particle manipulation. This work presents several notable contributions. First, we successfully realized selective particle manipulation, allowing instantaneous interaction with users to perform user-designated and objective-oriented manipulation tasks. Second, we integrated a closed-loop control strategy into the system that effectively mitigates misalignment errors induced by the trapping stiffness heterogeneity of acoustic trap and enables automated precision position control of particles in complex environments (in 30-mm-wide workspace, positioning precision is 1/40 of the wavelength). Third, we proposed a reconfigurable acoustic trap design method, named pseudovortex trap, featuring real-time computing and trapping particles larger than the wavelength. The system setup, the calibration specifics, the acoustic trap design methodology, and the corresponding visual servo control scheme (in terms of selective trapping, precision positioning, and dynamic trajectory planning) are given in detail in the article. Meanwhile, the trapping stiffness and the manipulation stability are also analyzed in this work. Experimental results well demonstrated the effectiveness of the proposed system.

Knot tying is an essential task for robotic surgery, which is routinely realized by dual-arm robotic manipulation. Despite the well-established protocol and progress at macro scale so far, there remain challenges to further advance robotic knot tying technique, particularly in terms of decreasing space consumption with better dexterity, higher precision, and well biomechanical compatibility. In this paper, we propose a novel dual-arm nanorobotic system setup for automated knot tying performed on mini-incision under stereo microscope, featured by an additional rotation degree of freedom mounted on each arm. With this setup, an optimized motion trajectory planning under standard knot-tying protocol is also presented in order to support tying knots with shorter and thinner suture. Leveraging the natural advantage of nanorobotics and microscope, the proposed system is capable of tying consecutive throws with micro-suture on mini-incision, like in vascular anastomosis or microsurgery. We successfully evaluated the knot tying system on 2.0 mm wide bionic blood vessel with 30 mm long #8-0 micro-suture. We finally tested the mechanical strength of the knots for potential medical assessment.

Non-contact particle cluster manipulation holds significant promise in the realms of advanced manufacturing, chemistry, and pharmacy. However, achieving precise and dynamic control over the spatial kinematics of particle clusters remains a significant challenge, necessitating real-time and accurately programmable robotic end-effector. To this end, we develop an innovative non-contact, precise particle cluster manipulation system with ultrasonic phased array transducer (PAT) under microscope. This system combines a physics-based deep learning algorithm for real-time calculation of phase-only holograms (POHs), supporting PAT to dynamically form acoustic fields, namely holographic acoustic end-effector (HAE). Leveraging the dynamically and accurately generated HAEs by our system, kinematics control of particle clusters including aggregation, rotation, and translation is yielded. The extensive experiments well demonstrated the effectiveness of proposed system for particle cluster manipulation.

The pursuit of 3D reconstruction from 2D images for nanomanipulation under scanning electron microscopy stands as a critical research endeavor. Previous methods either necessitates additional lighting which is difficult in standard SEM devices or relies on feature matching with low resolution and precision, further constraining reconstruction performance. In this paper, we propose a novel robot-assisted nanoscale 360° reconstruction approach, which simplifies SEM setups and maximizes the utilization of robot motion and feedback. By harnessing a nanorobotic system, we capture 360°multi-view images automatically with precise mapping information and camera postures. Sequentially, neural radiance field reconstruct the pixel-wise structure and synthesizing images from diverse perspectives. Experimental results using two real datasets demonstrates our approach’s efficacy, achieving PSNR of 28.1 and SSIM of 0.93 for nanotube reconstruction, and PSNR of 32.8 and SSIM of 0.98 for AFM cantilever reconstruction. These results validate the reliability and robustness of our proposed robot-assisted reconstruction method.

Landing on a vertically oscillating platform poses a significant challenge for multi-rotor unmanned aerial vehicle (UAVs) due to the time-varying ground effect (GE). In this work, we formulated a data-driven GE dynamic model that accurately describes the complex interactions between UAVs and both stationary and oscillating platforms. Integrating this model with a feedforward controller effectively compensates for GE, resulting in improved landing performance. The proposed GE model elucidates the relationship between GE and factors such as UAVs’ velocity, throttle magnitude, and the motion of the landing platform. It highlights that the GE experienced during the landing process of UAVs is not only contingent on the current state but also related to past states. The resulting GE model is parsimonious and suitable for onboard computers with limited computational power, and its accuracy has been confirmed through a series of flight experiments. To demonstrate the effectiveness of the developed UAVs landing scheme, we compared our approach with robust control and internal model control methods. Experimental results indicate that the proposed landing strategy achieves faster and smoother landings, with at least a 22% improvement in smoothness and a 25% reduction in landing time.

Dynamic model is essential for robust and reliable robotic fish motion control. Despite considerable efforts in robotic fish dynamic modeling, background flow has not been well considered yet, leading to the deterioration of applying robotic fish to practice. In this paper, we propose a novel dynamic model, termed Flow-Aware Robotic fish Model (FARM), that with well consideration to background flow using Koopman operators without increasing computation complexity. Specifically, we first collect motion data of the robotic fish in different background flow fields, and then obtain a linear approximation (the dynamic model) of nonlinear dynamics through carefully selected lifted functions. The obtained model can predict the next state based on the current state, control input, and average flow velocity of the local flow field. We evaluate the effectiveness of obtained model by comparing the Root Mean Square Error (RMSE) of predicted motion trajectories with real trajectories in various flow field environments. The results indicate that FARM is highly promising for obtaining a reliable dynamic model and can achieve comparable prediction accuracy even in unseen flow field environments with rough flow maps.

Acoustic holography has emerged as a cutting-edge technique for constructing a micro-robot acoustic end-effector for non-contact manipulation. As one of typical implementations of acoustic holography, Binary Amplitude-Only Hologram (BAOH) featured with a simple structure provides an efficient alternative for modulating acoustic fields that support micro-robotic manipulation. In the present study, we propose a deep learning based BAOH generation method for constructing precise and high-resolution end-effector based on acoustic field. Specifically, we model the BAOH generation problem into an optimization framework. The framework combines an acoustic wave propagation model with the deep neural network, in favor of bypassing the laborious collection of labeled data and facilitating the model to learn the inverse mapping. Additionally, to address the issues of gradient invalidation and information loss caused by binarization, the framework uses an adaptive binarization layer consisting of differentiable binarization and adaptive threshold automatically learned during training, which facilitates to realize end-to-end optimization and increase the non-linear capacity of the model. The simulation experiments show that the proposed method is capable to predict BAOH that supports precise, robust, versatile and real-time construction of acoustic end-effector, enjoying broad prospects in various applications related to micro-robotic manipulation.

Despite of the promising advancements in the last decade, the majority of scanning electron microscope (SEM)-based nanomanipulation tasks remain manually performed. Even in automated tasks, human intervention is required, at least during the task preparatory stage, where both the object of interest and robot end-effector are adequately enclosed in the field of view (FOV) of SEM at moderate magnification. This paper proposes a fully automated visual servo control method for workspace navigation of nanorobot end-effector that actively maintains the end-effector position offset from the center of FOV during passive working scene zooming and translation operations. We also propose using the scaling image Jacobian matrix theory to adaptively establish the hand-eye relationship of the nanorobotic system at uncalibrated magnifications, without the need for hardware regulation. This proposed method for workspace navigation is applicable to almost all commercial nanomanipulation systems. To the authors’ best knowledge, there has been no dedicated research on this problem in the literature. Experiments show that the proposed method significantly improves workspace navigation efficiency by about two-thirds, even for a skilled operator.Note to Practitioners—Workspace navigation refers to the process of moving the FOV to visually enclose the object or feature of interest at a moderate magnification, which is prerequisite for initializing either an automated or manual nanomanipulation task. This process involves consecutive zooming of the SEM in and out, as well as translating the sample stage, which can cause the end-effector to move out of the FOV. Therefore, it is necessary to move the robot end-effector with visual assistance along workspace navigation to avoid unexpected collisions. Typically, skilled operators perform this process manually in almost all nanomanipulation tasks, which is tedious and time-consuming. Therefore, developing an automated visual servo control method for workspace navigation of end-effector will significantly contribute to the nanomanipulation community.

Acoustic holography (AH) is a promising technique for precise noncontact micro-nano robotic manipulation. It encodes a three-dimensional (3D) acoustic field acting as a virtual end-effector into a two-dimensional (2D) hologram, whereby the desired acoustic field reconstruction is made possible. Most traditional methods to implement AH, such as 3D printed holographic lens and phased array of transducers (PAT), have limitations of dynamic and dexterous manipulation. Furthermore, existing iterative optimization algorithms to calculate 2D holograms have inadequate accuracy and real-time performance. To address these issues, this paper proposes a physics-based deep learning method with a novel training framework for phase-only hologram (POH) calculation enabling further pushing forward the PAT-based AH for noncontact robotic manipulation. By implementing independent control of each channel on PAT referring real-time calculated POH by a well-trained network, the desired acoustic field can be reconstructed in real-time with high fidelity. The results both on a simulated dataset and a real dataset demonstrate that our method supports accurate and dynamic reconstruction of desired acoustic field with distinct morphologies, with an average reconstruction error of 0.085 and average POH computing time of 47 milliseconds on GPU. Indeed, this work shows the future potential of AH in the field of noninvasive medical therapy, exogenous material delivery, and miniaturized industrial assembly.Note to Practitioners—This paper addresses the challenge of noncontact micro-nano robotic manipulation by PAT-based AH, an intriguing technique in bioengineering, micro-assembly, and material characterization. However, existing approaches have limited precision and real-time performance. To overcome these limitations, this paper proposes a physics-based deep learning method with a novel training framework. Our method achieves excellent accuracy and real-time performance, enabling efficient reconstruction of various complicated acoustic field morphologies for precise and dynamic acoustic manipulation. Experimental results demonstrate its high manipulation flexibility due to the independent modulation of each channel of PAT and real-time precise control due to the ultrafast calculation of the proposed deep learning method, though the method has not yet been deployed into an acoustic manipulation system and tested in practice. Future research will focus on designing physical experiments for further evaluation. Overall, the proposed method provides a novel and promising basis for desired acoustic field generation.

Noncontact particle manipulation (NPM) shows great application potential than its conventional counterpart particularly in terms of non-invasiveness, and thus has significantly extended robotic manipulation capacity into bio- medical engineering, material science, etc. As NPM by means of electric, magnetic, and optical field has successfully demonstrated powerful strength in both academia and industry, NPM boosted by acoustic field, however, still faces staggering challenges. It is indeed in the very recent years that controllable dynamic airborne or waterborne acoustic field modulation technology emerged in academia. In this paper, we report our latest research regarding dexterous and dynamic noncontact micro-particle manipulation on water surface effected by acoustic field in terms of automated trapping, closed-loop positioning, and real-time motion planning, which can be applied to scenarios such as parallel 3D printing, cell assembly, etc. The main contribution of this work is we demonstrated the feasibility of objective-oriented and fully automated acoustic manipulation of micro-particle in precision scale based on robotic approach in 2D plane. Experiment results showed that the repetitive positioning accuracy can reach as high as 16 μm, which is essentially the pixel scale factor.

Phase-only acoustic holography is a fundamental and promising technique for contactless robotic manipulation. Through independently controlling phase-only hologram (POH) of phase array of transducers (PAT) and simultaneously driving each channel by sophisticated circuits, a certain acoustic field is dynamically generated in working medium (e.g., air, water or biological tissues) at certain moment. The phase profile of PAT is required dynamically and precisely as per arbitrary expected acoustic field for the sake of versatile and stable robotic manipulation. However, the most conventional methods rely on iterative optimization algorithms which are inevitably time-consuming and probably non-convergent, moreover hindering versatility and fidelity of acoustic robotic manipulation. To address these issues, this paper reports a real-time phase-only acoustic holography algorithm by virtue of iterative unsupervised learning. Using a physics model to construct two queues, which we refer to as experience pools, data pairs consisting of a target acoustic amplitude hologram in expected acoustic field and corresponding POH of PAT are collected on-the-fly, circumventing costly preparation of annotated dataset in advance. With iterative learning between neural network training and experience pools update, both the solution of objective inverse mapping and the adaptation for arbitrary desired acoustic field are mutually enhanced. The experiments and results validated that the proposed approach surpasses previous algorithms in terms of real time and precision.

Emotion detection presents challenges to intelligent human-robot interaction (URI). Foundational deep learning techniques used in emotion detection are limited by information-constrained datasets or models that lack the necessary complexity to learn interactions between input data elements, such as the the variance of human emotions across different contexts. In the current effort, we introduce 1) MoEmo (Motion to Emotion), a cross-attention vision transformer (ViT) for human emotion detection within robotics systems based on 3D human pose estimations across various contexts, and 2) a data set that offers full-body videos of human movement and corresponding emotion labels based on human gestures and environmental contexts. Compared to existing approaches, our method effectively leverages the subtle connections between movement vectors of gestures and environmental contexts through the use of cross-attention on the extracted movement vectors of full-body human gestures/poses and feature maps of environmental contexts. We implement a cross-attention fusion model to combine movement vectors and environment contexts into a joint representation to derive emotion estimation. Leveraging our Naturalistic Motion Database, we train the MoEmo system to jointly analyze motion and context, yielding emotion detection that outperforms the current state-of-the-art.

Ultrafast acoustic holography (AH) enabling dynamic contactless micro-nano robotic manipulation has recently attracted wide attention. As an advanced technique, AH encodes specific three-dimensional (3D) acoustic field on a two-dimensional (2D) hologram whereby realizing holographic reconstruction with high fidelity. However, current approaches face the limitation of encoding time, accuracy and flexibility, thus, leading to inapplicability for dynamic and precise robotic manipulation. Here, we develop an approach to overcome these issues. Its basic idea is to use a convolutional neural network trained in a self-supervised manner with iterative interaction with virtual physical environment. Energy conservation is incorporated to access the physical constrain during wave propagation. The experimental results demonstrate that the proposed method circumvents laborious annotated dataset preparation and boosts the reinforcement from physics model. By the validation and comparison on distinct acoustic fields with various patterns, the accuracy and real-time performance of the proposed method are confirmed supporting dynamic and precise robotic manipulation.

This paper proposes an efficient robot assembly skill learning framework based on only a few demonstrations. The assembly skill learning process consists of two phases, e.g., the pre-training phase and the self-learning phase. In pre-training phase, the assembly networks are initialized from demonstration data. A novel data augmentation model based on state transition model is designed, which only requires several human demonstrations to learn the parameters of the model and reduces the cost of demonstration data collection in pre-training. In self-learning phase, the pre-trained assembly networks are further optimized by a hybrid exploration strategy in assembly environment, which explores both parameter space and action space to increase exploration efficiency. On par with the learning framework, a fuzzy reward function balancing the efficiency and compliance of assembly is elaborately designed to evaluate action’s performance in assembly skill self-learning process. Series of physical experiments were well conducted on a sophisticated assembly platform to verify the effectiveness of the proposed efficient robot assembly skill learning framework. Experimental results demonstrate that the learning efficiency of the proposed framework is at least three times as efficient as the state-of-art methods, while the skill performance outperforms the state-of-art by more than 50%. Note to Practitioners—The motivation of this paper is to develop a skill learning framework to achieve efficient assembly skill learning for robot in real-world system. The proposed framework is only based on several demonstrations, which shows high practicability for robot skill learning. The self-learning method is dexterously used to optimize the learned assembly skill from demonstrations since the limited space of demonstrations. Besides, the hybrid exploration strategy and fuzzy reward strategy are well designed to improve the behavior of self-learning. The proposed framework can improve the efficiency and performance of robot assembly learning skill. It is extremely important for robot working in practical applications.

Acoustic holography is a newly emerging and promising technique to dynamically generate arbitrary desired holographic acoustic field in 3D space for contactless robotic manipulation. The latest technology supporting complex dynamic holographic acoustic field reconstruction is through phased transducer array (PTA), where the phase profile of emitted acoustic wave from discrete transducers is controlled independently by sophisticated circuits to modulate the acoustic interference field. While the forward kinematics of a phased array based robotic manipulation system is simple and straightforward, the inverse kinematics of the required holographic acoustic field is mathematically non-linear and unsolvable, which substantially limits the application of dynamic holographic acoustic field for robot manipulation. In this work, we propose a physics-based deep learning framework for this phase retrieval inverse kinematics problem so that the target complex hologram could be reconstructed precisely with average MAE of 0.022 and in real time with prediction time of 47 milliseconds on GPU. The accuracy and real time of the proposed method for dynamic holographic acoustic field reconstruction from PTA are demonstrated experimentally.

Simultaneous assembly of multiple objects is a key technology to form solid connections among objects to get compact structures in precision assembly and micro-assembly. Dramatically different from traditional assembly of two objects, the interaction among multiple objects is more complicated on analysis and control. During simultaneous assembly of multiple objects, there are multiple mutually effected contact surfaces, and multiple force sensors are needed to perceive the interaction status. In this paper, a coordinated micro-robot manipulation strategy is proposed for simultaneous assembly problem, which is based on microscopic vision and force information. Taking simultaneous assembly of three objects as an instance, the proposed method is well articulated, including calibration of assembly system, force analysis for each contacting surface, and insertion control strategy for assembly process. The proposed method is applicable also to case with more objects. Experiment results demonstrate effectiveness of the proposed method.

This article proposes a two-finger-based micrograsping system with high compliant borosilicate 3.3 glass probes and the corresponding sensing and control algorithms, which enables the orientation manipulation of microparts in three-dimensional (3-D) space. Compared with the existing research, the novelty of this article relies on three aspects: 1) the end-effector of the microgripper is designed to be with high compliance so that the squeeze force exerted on microparts can be more accurately regulated and the proposed microgripper is capable of manipulating fragile microparts; 2) the micrograsping system is endowed the capability to fully manipulate microparts' orientation without recurring to auxiliary probes or rotary stages; and 3) the vibration characteristic of the grasping arm is investigated as cantilever beam for gasping stability analysis and squeeze force maintaining. In specific, taking spherical microparts with dimensions in the range from tens to hundreds of micrometers as target, the grasping system configuration, and the contact model between the probe and the microparts are first presented. Afterward, kinematics-based motion control strategy for position adjustment and orientation manipulation of microparts is clarified. Then, squeeze force regulation strategy is proposed, including adhesive force evaluation, vision-based squeeze force estimation, and the micropart releasing method. Finally, the vibration characteristic of the grasping arm is investigated as cantilever beam for grasping stability analysis and the squeeze force maintaining. The reliability and availability of the proposed micrograsping system is validated by well-designed experiments.

The problem of simultaneous precision assembly of multiple objects is quite practical one to form compact physical structures and functionalities in mechatronics and advanced robotics. The core research aspects facing the problem are the contact status perception between each two object and the motion planning of each separate object. These two aspects mutually affect each other and cannot be discussed separately. In this paper, we first strategically discuss the possible approaches to solve the simultaneous assembly problem and analyze their advantages and drawbacks. Then, a probabilistic control method is developed based on the incomplete perceived information of the assembly process, which can achieve the highest assembly efficiency from the strategic perspective. Specifically, by fully utilizing the mechanical properties of materials in micrometer scale, the interaction between objects is first characterized as stochastic state-transition process. Second, adopting the simultaneous feeding strategy instead of serial feeding, the current contact status between each two object is determined based on the state-transition equation as a probability distribution along a hyperline. Finally, the motion planning technique is designed taking all possible radial forces on every contact surface into consideration. The experimental results demonstrate the effectiveness of the proposed method.

This paper proposes a wire wrapping approach, which enables high-precision, fully automatic, quality controllable, and visually monitored wire wrapping by actively rotating the rod and coordinately translating the wire based on microscopic vision and force information. Viewing the wire as a one-dimensional object, the proposed paper contributes to both the precision manipulation field and the engineering utilities to fabricate precision helical structures used in many fields. The basic technical contents involved are the active rotation of the rod and the coordinated translation of the manipulator, both of which are designed to keep the relative spatial relationship and the interactive force between the rod and the wire. Extensive experiments were conducted to validate the effectiveness of the proposed method to achieve high-precision wire wrapping. Experimental results show that with discretized active rotation increment about the rod axis of 6°, the local tensile deformation of the wire can be controlled within ±2 μm error range, while the average local helical angle can be controlled within ±0.3° error range with standard deviation less than 1.5°.

This paper proposes an efficient strategy for the insertion of the partially flexible object in precision assembly. The partially flexible object refers to the component with unevenly distributed flexibilities: coupling rigidity and flexibility. This paper focuses on the insertion of one class of partially flexible objects: rigid shapes connected by a compliant mechanism. We first analyze the horizontal compliance of the compliant mechanism and build a model to relate its state and force. The insertion is separated into two stages according to the insertion type. The first stage is a compliant insertion and we estimate the insertion direction based on the built model, horizontally compensate resorting to the horizontal compliance and the updated direction, and efficiently plan the vertical insertion in an aggressive strategy regarding the uncertainties caused by the compliant mechanism and predicting the future insertion. The second stage is a hybrid insertion with both rigid and compliant gripping and its features include that the object states are not precisely measurable and the motion of a part of the object is not directly controllable. To solve it, we qualitatively and quantitatively analyze all possible configurations and, taking advantage of the insertion property, conclude one insertion posture based on which a control strategy is proposed. Additionally, a conservative insertion strategy is planned resorting to the past execution performance and the current state evaluation. Experiments are carried out to demonstrate the validation of the proposed method.

This paper proposes an efficient strategy to simultaneously insert multiple objects, which are parallel connected by passive compliant mechanisms, in precision assembly. The distinctions of this task include: each object is held compliantly; multiple objects are parallel connected to a manipulator; not all the peg-in-hole has the same insertion condition; and high accuracy is required for each insertion. This configuration can provide sufficient compliance and improve insertion efficiency for massive precision assembly. We model the relationship between the state and force of a single compliant mechanism, and analyze the horizontal compliance of parallel mechanisms. Based on the model, with a fitting and optimization method the states of all but one compliant mechanisms are acquired from microscopic views and the remaining states are optimized with resultant forces provided by a force sensor. To efficiently plan the parallel insertion, we propose a strategy to horizontally compensate according to the resultant force and the horizontal compliance, and to vertically insert based on the insertion ratio expectation, the horizontal offsets of each individual insertion, and the horizontal force. Experiments are carried out to demonstrate the validation of the proposed method.

This paper proposes an efficient and accurate method for collision detection between convex prisms in precision alignment and a detach controller based on the contact location for the collision occurrence. We project the objects onto an appropriate plane and detect the collision status considering the relationship between their planar contours. Efficient methods are presented for the overlap checking of several elementary contours, and the way to obtain the vertices of the projected shapes is also introduced. The detection is then accelerated by classifying into the corresponding case based on the relative configuration and using the efficient planar checking to replace the cubic calculation. A detach controller is presented to immediately separate the objects according to the contact location once collision occurs. The computational efficiency and comparison are demonstrated in simulations, and experiments are carried out to validate the detection and the controller.

An automatic system is developed to realize high precision assembly of two components in the size of mm level with an interference fit in 3-dimensional (3-D) space with 6-degree-of- freedoms (DOF), which consists of a manipulator, an adjusting platform, a sensing system and a computer. The manipulator is employed to align component B to the component A in position. The adjusting platform aligns the component A to component B in orientations and inserts A into B. The sensing system includes three microscopes and a force sensor. The three microscopes are mounted approximately orthogonal to observe components from different directions in the aligning stage. The force sensor is introduced to detect the contact force in assembly process. In the aligning stage, a pose control method based on image Jacobian matrix is proposed. In the insertion stage, a position control method based on the contact force is proposed. The calibration of image Jacobian matrix is also presented. Experimental results demonstrate the effectiveness of the proposed system and methods.