Magnetic localization, owing to its immunity to line-of-sight occlusion and non-contact nature, is considered a promising technology for medical applications. While it is intuitive that localization performance degrades as the target moves farther from the sensor array, uncertainty analysis has long been overlooked, which is essential for quantifying localization quality. In this work, we present a pose estimation and uncertainty analysis framework on SE(3) for magnetic source localization using sensor arrays, which enables concise formulation of the problem and quantitative assessment of the results. The volume of the uncertainty ellipsoid is used to characterize localization confidence, while the surface shape in Cartesian space is used for visualization. This also provides insight into the effective workspace of the magnetic localization system prior to deployment. Guided by this analysis, we designed a movable magnetic sensor array to expand the limited sensing volume and mitigate localization uncertainty, thereby enhancing overall localization performance. Simulations and pose tracking experiments validate the effectiveness of this framework. By dynamically moving the sensor array to minimize the volume of the uncertainty ellipsoid, localization errors are reduced compared with static and projection-based strategies by 67.04%, 16.51% in position and 43.87%, 20.73% in orientation, respectively. Furthermore, phantom experiments on distal locking screw alignment and magnetic capsule endoscope tracking demonstrate the system's capability in improving localization accuracy (reducing alignment errors by 71.88%) and expanding the effective workspace by several folds.

Nowadays, laparoscopic surgery procedures face a trade-off between expensive, complex robotic systems and manual instruments with limited functionality. Fully robotic solutions offer precision but lack portability and intuitive control, while manual tools rely solely on the surgeon’s dexterity, limiting maneuverability and depth perception in confined spaces. To bridge this, we propose a Human-Guided Robotic-Assistance Handheld Continuum Medical Robot System (HRHC). This system simulates intuitive manual operation with robotic precision, extending the surgeon’s capabilities while maintaining portability. Additionally, a stereo vision system enhances real-time depth perception, improving spatial awareness in minimally invasive procedures.

Significant progress has been made in enhancing the motion capabilities of quadruped robots in unstructured environments due to advancements in hardware and control algorithms. However, limited research has been conducted on the fault-tolerant control of quadruped robots, which is crucial for their operation in remote or extreme environments like disaster sites. In this paper, we primarily focus on fault-tolerant strategies for common joint-stuck situations. By leveraging the static stability of quadruped robots, it becomes possible to adjust their control policies and enable them to continue following predetermined trajectories. We introduce a contextual meta-reinforcement learning (Meta-RL) method to design fault-tolerant policies. This method infers task-related latent vectors from the context to assist in training the policy network, ensuring both conciseness and optimality in various situations. Additionally, to expedite algorithm training, we propose a reference action generator (RAG). To validate the proposed algorithm, extensive simulations and physical experiments are conducted. The results demonstrate that our method allows the robot to maintain its trajectory even when faced with motor locking. Furthermore, our method outperforms all baseline algorithms, highlighting its superiority in terms of fault tolerance. Note to Practitioners—The motivation of this article is to provide fault-tolerant policies for quadruped robots, specifically referring to the policies for joint-stuck situations. Previous fault-tolerant strategies either require individually designing control strategies for each joint stuck task, which brings a significant workload to designers, or adopting a unified strategy that cannot provide the optimal strategy for each task. In this article, we utilize the Meta-RL method to handle the joint stuck issue in robots for the first time. By combining the context encoder and RAG, we can provide more suitable policies for various motor-stuck tasks. Both the simulation and physical experiments validate the effectiveness and applicability of this method.

Visual localization plays a critical role in the functionality of low-cost autonomous mobile robots. Contemporary leading methods for precise visual localization are predominantly 3D scene-specific, necessitating extra computational and memory overhead to construct a 3D scene model in novel environments. An alternative approach of directly using a database of 2D images for visual localization offers more flexibility. However, such methods currently suffer from limited localization accuracy. In this paper, we propose an accurate and robust multiple checking-based 3D model-free visual localization system to address the aforementioned issues. To ensure high accuracy, our focus is on estimating the pose of a query image relative to the retrieved database images using 2D-2D feature matches. Theoretically, by incorporating the local planar motion constraint into both the estimation of the essential matrix and the triangulation stages, we reduce the minimum required feature matches for absolute pose estimation, thereby enhancing the robustness of outlier rejection. Additionally, we introduce a multiple-checking mechanism to ensure the correctness of the solution throughout the solving process. The efficacy of our approach is substantiated through both qualitative and quantitative assessments on simulated and two real-world datasets evidencing significant improvements in accuracy and robustness provided by our 3D model-free visual localization system.Note to Practitioners—The motivation of this article stems from the need to develop an accurate visual localization system with simplicity and flexibility of map construction and easy adaption to new environments. Such a system holds great practical value for a range of applications, including warehouse robots, service robots, and countless others. Existing visual localization systems that achieve high accuracy are dependent on a pre-built accurate 3D scene map, which pose challenges in terms of map construction and consume significant storage resources onboard, particularly for large scenes. And the aforementioned efforts need to be repeated when changing to a new scene. In this article, an accurate and robust 3D model-free visual localization system is proposed to handle this problem. The map construction is simplified to build a set of database images with associated camera poses, which is trivial as it amounts to adding posed images to a database. The core idea for achieving high accuracy and robustness is to model the local planar motion characteristic of general ground-moving robots into both essential matrix estimation and triangulation stages to obtain two minimal solutions. The proposed localization system simplifies the task of switching between different application scenarios for the robot, reducing additional workload and lowering the difficulty of use.

This study developed a multisensor asynchronous fusion method for vision cameras and laser sensors for real-time tracking of a millimeter-scale piezoelectric microrobot with high sampling rates. The design, manufacture, and realization of the proposed microrobot using a monolithic integrated manufacturing process is realized with a size of 36×36×18 mm and a weight of 4.9 g. Two local estimators for the vision camera and laser sensors are established independently for 2-D tracking of the millimeter-scale microrobot. An asynchronous fusion method for the camera and laser sensors is developed by optimally weighting the two local estimators in a linear minimum variance framework using the Lagrange multiplier method at the synchronized update time. High-precision 2-D trajectory tracking of the proposed microrobot was achieved with an average velocity of 301.6 mm/s and a sampling rate of up to 4000 Hz. Experiments tracking different trajectories of the microrobot at different velocities and sampling rates demonstrated the effectiveness of the proposed fusion method.

Parallel micro/nano robots hold great potential in micro-manufacturing/assembly, microsurgery, and precision engineering because of their high precision and stiffness. However, it is challenging to develop a millimeter-scale robot with a large workspace, high bandwidth, and high precision. In this paper, we present the design, fabrication, tests, and potential applications of a piezoelectric planar parallel microrobot. The developed microrobot consists of a parallel mechanism, three amplification mechanisms, and three independently controlled piezoelectric actuators. The microrobot is miniaturized to the millimeter scale through a monolithic integrated manufacturing process, and achieves a dimension of 36$\times$36$\times$34 mm, a weight of 4.9 g, and a static workspace of 33.9 mm$^{2}$. The resonant frequencies reach 55-65 Hz in x and y directions, and 95 Hz in rotation. The microrobot exhibits high positioning accuracy in the trajectory tracking experiment on three different lines, circle, and triangle trajectories at a board bandwidth. Moreover, the microrobot can repeat 50 periodic circle trajectories in one second, with a velocity of 628.3 mm/s and a precision of 10.9$\upmu$m. Furthermore, we conducted three validation experiments to demonstrate the potential applications of the microrobot in tremor compensation for micromanipulation, 3D printing electronics, and minimally invasive surgery.Note to Practitioners—This work is motivated by the need to design a microrobot with a large workspace, high bandwidth, high precision, and compact structure. Such capabilities are essential across various fields of micromanipulation, including micro-manufacturing/assembly, microsurgery, and precision engineering. This paper presents a detailed manufacturing process for this type of microrobot, followed by comprehensive experimental tests. The resonant frequency, workspace, quasi-static and dynamic trajectory tracking experiments of the microrobot are thoroughly tested. The experimental results demonstrate the excellent performance of the microrobot in executing various operational tasks. Additionally, the potential applications in the fields of micromanipulation, 3D printing electronics, and minimally invasive surgery are demonstrated. By assembling various end effectors on the robot platform, the microrobot could have greater potential for application in more fields.

Continuum robots, owing to their inherent compliance, have become essential in endoscopic surgical procedures, such as mucosal ablation. However, the prevalent design of endoscopic manipulators, which typically features only a single active bending segment, often results in limited dexterity and accessibility. Additionally, the incorporation of variable stiffness in these robots has attracted significant interest, with the aim to improve manipulation capabilities in confined spaces. In the paper, we propose a novel variable-length continuum robot with variable stiffness for endoscopic surgery. The robot’s stiffness can be altered either by modifying the catheter’s length or solid-liquid transition of low-melting-point alloy (LMPA). The design and fabrication methods of the robot are meticulously detailed. Additionally, a quasi-static stiffness model along with a learning-based stiffness compensation approach for accurate stiffness estimation are proposed. Leveraging this model, a contact force controller is designed for ablation procedure. The experimental results show that our robot possesses good flexibility and accessibility, making it highly adept at manipulating in confined spaces. Its variable stiffness feature significantly enhances its ability to counteract external disturbance and prevent tip deformation (with a average position change of 1.1mm). Finally, through force control experiments and a surgical demonstration in a gastrointestinal model, we have further validated the robot’s applicability in surgical contexts. Note to Practitioners—This paper proposed a variable-length continuum robot with variable stiffness for endoscopic surgery. The robot can achieve axial elongation and omnidirectional bending motion, having better dexterity and accessibility than traditional medical continuum robots with one active bending segment. The robot’s stiffness can be adjusted by the length changes or solid-liquid transition of low-melting-point alloy (LMPA). Besides, an accurate stiffness model and a contact force controller are proposed for endoscopic ablation surgery. By experimental results, the robot shows high flexibility and accessibility, allowing access to confined spaces for manipulation, and good control accuracy and variable stiffness capability for endoscopic surgery.

Global localization using onboard perception sensors, such as cameras and light detection and ranging (LiDAR) sensors, is crucial in autonomous driving and robotics applications when Global Positioning System (GPS) signals are unreliable. Most approaches achieve global localization by sequential place recognition (PR) and pose estimation (PE). Some methods train separate models for each task, while others employ a single model with dual heads, trained jointly with separate task-specific losses. However, the accuracy of localization heavily depends on the success of PR, which often fails in scenarios with significant changes in viewpoint or environmental appearance. Consequently, this renders the final PE of localization ineffective. To address this, we introduce a new paradigm,PR-by-PE localization, which bypasses the need for separate PR by directly deriving it from PE. We propose RING#, an end-to-endPR-by-PE localizationnetwork that operates in the bird's-eye-view (BEV) space, compatible with both vision and LiDAR sensors. RING# incorporates a novel design that learns two equivariant representations from BEV features, enabling globally convergent and computationally efficient PE. Comprehensive experiments on the north campus long-term vision and LiDAR (NCLT) and Oxford datasets show that RING# outperforms state-of-the-art methods in both vision and LiDAR modalities, validating the effectiveness of the proposed approach.

We focus on the task of unknown object rearrangement, where a robot is supposed to reconfigure the objects into a desired goal configuration specified by an RGB-D image. Recent works explore unknown object rearrangement systems by incorporating learning-based perception modules. However, they are sensitive to perception error, and pay less attention to task-level performance. In this article, we aim to develop an effective system for unknown object rearrangement amidst perception noise. We theoretically reveal that the noisy perception impacts grasp and place in a decoupled way, and show such a decoupled structure is valuable to improve task optimality. We propose grasp, see, and place (GSP), a dual-loop system with the decoupled structure as prior. For the inner loop, we learn a see policy for self-confident in-hand object matching. For the outer loop, we learn a grasp policy aware of object matching and grasp capability guided by task-level rewards. We leverage the foundation model CLIP for object matching, policy learning, and self-termination. A series of experiments indicate that GSP can conduct unknown object rearrangement with higher completion rates and fewer steps.

Knot tying is an essential task for robotic surgery, which is routinely realized by dual-arm robotic manipulation. Despite the well-established protocol and progress at macro scale so far, there remain challenges to further advance robotic knot tying technique, particularly in terms of decreasing space consumption with better dexterity, higher precision, and well biomechanical compatibility. In this paper, we propose a novel dual-arm nanorobotic system setup for automated knot tying performed on mini-incision under stereo microscope, featured by an additional rotation degree of freedom mounted on each arm. With this setup, an optimized motion trajectory planning under standard knot-tying protocol is also presented in order to support tying knots with shorter and thinner suture. Leveraging the natural advantage of nanorobotics and microscope, the proposed system is capable of tying consecutive throws with micro-suture on mini-incision, like in vascular anastomosis or microsurgery. We successfully evaluated the knot tying system on 2.0 mm wide bionic blood vessel with 30 mm long #8-0 micro-suture. We finally tested the mechanical strength of the knots for potential medical assessment.

Non-contact particle cluster manipulation holds significant promise in the realms of advanced manufacturing, chemistry, and pharmacy. However, achieving precise and dynamic control over the spatial kinematics of particle clusters remains a significant challenge, necessitating real-time and accurately programmable robotic end-effector. To this end, we develop an innovative non-contact, precise particle cluster manipulation system with ultrasonic phased array transducer (PAT) under microscope. This system combines a physics-based deep learning algorithm for real-time calculation of phase-only holograms (POHs), supporting PAT to dynamically form acoustic fields, namely holographic acoustic end-effector (HAE). Leveraging the dynamically and accurately generated HAEs by our system, kinematics control of particle clusters including aggregation, rotation, and translation is yielded. The extensive experiments well demonstrated the effectiveness of proposed system for particle cluster manipulation.

The pursuit of 3D reconstruction from 2D images for nanomanipulation under scanning electron microscopy stands as a critical research endeavor. Previous methods either necessitates additional lighting which is difficult in standard SEM devices or relies on feature matching with low resolution and precision, further constraining reconstruction performance. In this paper, we propose a novel robot-assisted nanoscale 360° reconstruction approach, which simplifies SEM setups and maximizes the utilization of robot motion and feedback. By harnessing a nanorobotic system, we capture 360°multi-view images automatically with precise mapping information and camera postures. Sequentially, neural radiance field reconstruct the pixel-wise structure and synthesizing images from diverse perspectives. Experimental results using two real datasets demonstrates our approach’s efficacy, achieving PSNR of 28.1 and SSIM of 0.93 for nanotube reconstruction, and PSNR of 32.8 and SSIM of 0.98 for AFM cantilever reconstruction. These results validate the reliability and robustness of our proposed robot-assisted reconstruction method.

Active reconstruction technique enables robots to autonomously collect scene data for full coverage, relieving users from tedious and time-consuming data capturing process. However, designed based on unsuitable scene representations, existing methods show unrealistic reconstruction results or the inability of online quality evaluation. Due to the recent advancements in explicit radiance field technology, online active high-fidelity reconstruction has become achievable. In this paper, we propose GS-Planner, a planning framework for active high-fidelity reconstruction using 3D Gaussian Splatting. With improvement on 3DGS to recognize unobserved regions, we evaluate the reconstruction quality and completeness of 3DGS map online to guide the robot. Then we design a sampling-based active reconstruction strategy to explore the unobserved areas and improve the reconstruction geometric and textural quality. To establish a complete robot active reconstruction system, we choose quadrotor as the robotic platform for its high agility. Then we devise a safety constraint with 3DGS to generate executable trajectories for quadrotor navigation in the 3DGS map. To validate the effectiveness of our method, we conduct extensive experiments and ablation studies in highly realistic simulation scenes.

X-ray to CT registration is an essential technique to provide on-site guidance for clinicians and medical robots by aligning preoperative information with intraoperative images. Current methods focus on local registration with small capture ranges and necessitate a manual initial alignment before precise registration. Some existing global methods are likely to fail in thoracic surgeries because of the respiratory motion and the nearly colinear nature of vertebrae landmarks. In this study, we propose a vertebrae-based global X-ray to CT registration method with the assistance of clinical setups for thoracic surgeries. Firstly, vertebrae centroids are automatically localized by CNN-based networks in CT and X-ray for establishing 2D/3-D correspondences. Then, inspired by clinical setup, we address the degradation of colinear landmarks of 6-DoF pose estimation by introducing a 4-DoF solver. Considering the inaccurate priori and landmark mislocalization, the solver is embedded into the Adaptive Error-Aware Estimator (AE2) to simultaneously estimate weights and aggregate candidate poses. Finally, the whole method is trained in an end-to-end manner for better performance. Evaluations on both the public LIDC-IDRI dataset and clinical dataset demonstrate that our method outperforms existing optimization-based and learningbased approaches in terms of registration accuracy and success rate. Our code: https://github.com/LiuLiluZJU/2P-AE2

Magnetic continuum robots are subject to external magnetic fields and deformed remotely, simplifying the robot’s transmission mechanism and providing it with significant potential for miniaturization and operational flexibility. However, modeling magnetic field distribution generated by permanent magnets is complex and requires time-consuming pre-calibrations. Moreover, it is highly susceptible to environments with ferromagnetic materials, posing significant challenges for the control of magnetic continuum robots. In response, we propose an approach that does not overly focus on the magnetic field distribution but instead directly learns the inverse kinematics of magnetic continuum robots end-to-end. Binding the robot’s configuration to the pose of external magnets, precise control of continuum robots is facilitated. Additionally, we leverage teleoperation techniques to broaden the applicability of this method. By mounting magnets on a robotic arm and directly utilizing the target pose of the external magnet predicted by a multi-layer perceptron (MLP), we achieve the operation and navigation of magnetic continuum robots in complex environments. Experiments demonstrate that the mean control accuracy along the robot using our learning-based inverse kinematics is about half of the robot’s diameter.

Small-scale continuum robot-assisted minimally invasive surgery has received crucial attention due to its smaller incisions and high dexterity. In medical scenarios such as radiofrequency ablation and nasal/throat swab sampling, monitoring and controlling the forces applied to human tissue can help improve the safety and comfort level of the procedure. However, the tip-sensor-based force detection method can barely be deployed due to the miniature size of the continuum robot; meanwhile, the mechanical modeling-based high-precision force estimation cannot be realized on account of the continuum robots’ complex structure with high nonlinear properties. To address the high-precision force estimation challenge for further compliant control during minimally invasive interventions, a learning-based high-precision force estimation method via long short-term memory (LSTM) is proposed in this paper. On this basis, compliance control and high-precision force tracking can be further realized for small-scale continuum robot. The compliance control ensures a smooth and stable transition during the interaction between the robot and the environment, and force tracking can be utilized for maintaining or precisely controlling the force applied to the human tissue. Finally, the contact force sensing and control experiments are carried out on a small-scale continuum robot system prototype, and a demonstration using a human nasal cavity model is conducted. The results validate that the proposed LSTM neural network fits the mechanical model of the continuum robot well with the root mean square error of 3.44mN, and the control method can significantly compensate for the instantaneous impact during contact with an attenuation of 59.3$\%$and rapidly respond to keep the force accurately at the expected value with the mean absolute error of 2.41mNNote to Practitioners—This research is motivated by the increasing number of applications for small-scale continuum robot-assisted minimally invasive interventional surgery, such as radiofrequency ablation, biopsy, and endoscopic submucosal dissection. These surgeries require high-precision contact force sensing and control. However, due to the small size and complex structure of the continuum robot, traditional methods such as installing force sensors and mechanical modeling are not effective. Therefore, this paper utilizes a neural network to infer contact force through more accessible information about the robot, such as the tension of the actuators. This method allows for compliant force control during the dynamic contact between the small-scale continuum robot and human tissue. Experiments conducted on a human nasal cavity model demonstrate that the proposed method can improve the safety and reliability of small-scale continuum robot-assisted minimally invasive surgery.

With the increasing computing power, using data-driven approaches to co-design a robot's morphology and controller has become a promising way. However, most existing data-driven methods require training the controller for each morphology to calculate fitness, which is time-consuming. In contrast, the dual-network framework utilizes data collected by individual networks under a specific morphology to train a population network that provides a surrogate function for morphology optimization. This approach replaces the traditional evaluation of a diverse set of candidates, thereby speeding up the training. Despite considerable results, the online training of both networks impedes their performance. To address this issue, we propose a concurrent network framework that combines online and offline reinforcement learning (RL) methods. By leveraging the behavior cloning term in a flexible manner, we achieve an effective combination of both networks. We conducted multiple sets of comparative experiments in the simulator and found that the proposed method effectively addresses issues present in the dual-network framework, leading to overall algorithmic performance improvement. Furthermore, we validated the algorithm on a real robot, demonstrating its feasibility in a practical application.

Pose registration is critical in vision and robotics. This article focuses on the challenging task of initialization-free pose registration up to 7DoF for homogeneous and heterogeneous measurements. While recent learning-based methods show promise using differentiable solvers, they either rely on heuristically defined correspondences or require initialization. Phase correlation seeks solutions in the spectral domain and is correspondence-free and initialization-free. Following this, we propose a differentiable solver and combine it with simple feature extraction networks, namely DPCN++. It can perform registration for homo/hetero inputs and generalizes well on unseen objects. Specifically, the feature extraction networks first learn dense feature grids from a pair of homogeneous/heterogeneous measurements. These feature grids are then transformed into a translation and scale invariant spectrum representation based on Fourier transform and spherical radial aggregation, decoupling translation and scale from rotation. Next, the rotation, scale, and translation are independently and efficiently estimated in the spectrum step-by-step. The entire pipeline is differentiable and trained end-to-end. We evaluate DCPN++ on a wide range of tasks taking different input modalities, including 2D bird's-eye view images, 3D object and scene measurements, and medical images. Experimental results demonstrate that DCPN++ outperforms both classical and learning-based baselines, especially on partially observed and heterogeneous measurements.

Common medical image segmentation tasks require large training datasets with pixel-level annotations which are very expensive and time-consuming to prepare. To overcome such limitation and achieve the desired segmentation accuracy, a novel Weakly-Interactive-Mixed Learning (WIML) framework is proposed by efficiently using weak labels. On one hand, utilize weak labels to reduce annotation time for high-quality strong labels by designing a Weakly-Interactive Annotation (WIA) part of the WIML which prudently introduces interactive learning into the weakly-supervised segmentation strategy. On the other hand, utilize weak labels and very few strong labels to achieve desired segmentation accuracy by designing a Mixed-Supervised Learning (MSL) part of the WIML which can boost the segmentation accuracy by providing strong prior knowledge during training. Besides, a multi-task Full-Parameter-Sharing Network (FPSNet) is proposed to better implement this framework. Specifically, to further reduce annotation time, attention modules (scSE) are integrated into FPSNet to improve the class activation map (CAM) performance for the first time. To further improve segmentation accuracy, a Full-Parameter-Sharing (FPS) strategy is designed in FPSNet to alleviate the overfitting of the segmentation task supervised by very few strong labels. The proposed method is validated on the BraTS 2019 and LiTS 2017 datasets, and experiments demonstrate that the proposed method WIML-FPSNet outperforms several state-of-the-art segmentation methods with minimal annotation efforts.

Global localization plays a critical role in many robot applications. LiDAR-based global localization draws the community's focus with its robustness against illumination and seasonal changes. To further improve the localization under large viewpoint differences, we propose RING++ that has roto-translation-invariant representation for place recognition and global convergence for both rotation and translation estimation. With the theoretical guarantee, RING++ is able to address the large viewpoint difference using a lightweight map with sparse scans. In addition, we derive sufficient conditions of feature extractors for the representation preserving the roto-translation invariance, making RING++ a framework applicable to generic multichannel features. To the best of our knowledge, this is the first learning-free framework to address all the subtasks of global localization in the sparse scan map. Validations on real-world datasets show that our approach demonstrates better performance than state-of-the-art learning-free methods and competitive performance with learning-based methods. Finally, we integrate RING++ into a multirobot/session simultaneous localization and mapping system, performing its effectiveness in collaborative applications.

Simultaneous assembly of multiple objects is a key technology to form solid connections among objects to get compact structures in precision assembly and micro-assembly. Dramatically different from traditional assembly of two objects, the interaction among multiple objects is more complicated on analysis and control. During simultaneous assembly of multiple objects, there are multiple mutually effected contact surfaces, and multiple force sensors are needed to perceive the interaction status. In this paper, a coordinated micro-robot manipulation strategy is proposed for simultaneous assembly problem, which is based on microscopic vision and force information. Taking simultaneous assembly of three objects as an instance, the proposed method is well articulated, including calibration of assembly system, force analysis for each contacting surface, and insertion control strategy for assembly process. The proposed method is applicable also to case with more objects. Experiment results demonstrate effectiveness of the proposed method.

Coupling the endface of an optical fiber to an integrated optical component is currently a low-throughput and costly manual process in the fabrication of the optical devices. In order to meet the high-volume demand for commercial optoelectronic devices, coupling must be automated. This paper presents a robotic positioning system and corresponding path planning strategy based on both the position and light intensity feedback. In this work, a micro-robotic positioning system with 3 degrees of freedoms (DOFs) is developed and integrated with an optical microscopy. Then the fuzzy controller is developed to design the trajectory. Lastly, simulation and experimental results demonstrate the accuracy and efficiency of the proposed system. Compared with the traditional manual method, the robotic positioning system can realize the coupling within 40 seconds. This method will have a significant impact on the automatic process of the micro manufacture field.

Accurate alignment of the watch hand is a critical procedure in the industrial production of watches. However, compared with the traditional ideal laboratorial case, the watch-hand alignment suffers various external disturbances from the aged equipment and industrial operating conditions, which seriously influence the alignment precision. To achieve accurate watchhand alignment in the complex industrial environment, this paper develops a robotic micromanipulation system and proposes a corresponding robust control strategy. First, a micromanipulation system with five degrees of freedom is set up and integrated with an optical microscope. Then, a set of proper machine vision methods is adopted to obtain accurate position information in disregard of the interference terms. Third, the dynamic model of the controlled watch hand is built with consideration of the external disturbances from the production environment. Since the velocity information cannot be observed accurately through the machine vision process, we design a state observer to acquire the position and velocity values for the following control strategy. After that, a sliding mode controller with the radical basis function neural network is proposed to achieve high-precision alignment while resisting the external disturbance. Finally, simulation and experimental results prove that the proposed method can realize the alignment of the watch hand within 6 s with the accuracy of 2 × 10-3rad, which is enhanced at least two times compared with the traditional manual method.

Copper microwire is an essential metal widely used in micro-electron mechanical systems. Since micro/nano material usually demonstrates unique mechanical properties due to scaling effect, copper microwire mechanical properties need to be investigated for better adhibition. Herein, we propose a nanorobotics manipulation system for copper microwire insitu twisting test. Firstly, a system with six degree-of-freedoms (DOFs) nanorobotic manipulator integrated inside scanning electron microscope (SEM) is introduced. Secondly, a positioning and assembly method for copper microwire specimen are proposed to solve the mismatching problem. Finally, the copper microwire is twisted in-situ and its properties are investigated and analyzed. The copper microwire sample fracture morphology shows a severe plastic deformation and being along with the emergence of deformation twin and intertwine, which exhibit strong scaling effects. This system provides a new method for in-situ twisting test, which paves the way for mechanical characterization inside SEM and benefits the fundamental nanomaterial research immensely.

High-precision positioning is an essential requirement for sample operation at a small scale. At the current stage, although nanometer-scale accuracy has been achieved for the linear positioning, the rotational positioning (attitude control) is still very challenging and rarely addressed. This paper presents a rotatable nanorobotic system with rotational degrees of freedom first. Then, the system error, i.e., nonaxisymmetrical eccentricity error of the mechanism, is investigated dynamically and its fault model is established. After that, a double-loop servo repetitive controller is accordingly designed based on the circle interpolation strategy. The theoretical analysis and experimental results verify that the rotational positioning accuracy can be controlled up to submicrometers stably, which improves at least one order of magnitude than the current static method. Finally, two application cases are given to highlight the significance of this approach, i.e., surface defect detection from 360° and in situ twisting characterization of 1-D micro/nanomaterial. This research paves a new avenue for the ultrahigh rotational positioning at microscopy environment, which is expected to generate a long-term impact on the micro/nanofields, such as microscopy imaging, material characterization, and so on.

Scanning electron microscope (SEM) has been widely used for characterization and manipulation in micro/nano field. However, existing SEMs can only provide images for samples from one single direction. This limitation sometimes would lead to misleading results. In order to provide multidirectional SEM imaging, a nanorobotic manipulation system with a rotation robot is designed and integrated with SEM. During the robot's rotation, samples fixed on the robot can be viewed by SEM from multi-direction, instead of one direction. The rotation robot is able to rotate 360° along one rotation axis. To facilitate the multidirectional imaging, an automatic alignment method is put forward. This automatic alignment is to align samples to the rotation robot's rotation axis, so that during rotation, samples wouldn't move out of SEM's field of view (FOV). Alignment results are given to justify the proposed alignment method. Magnetic wires have been observed from multi-direction to demonstrate the nanorobotic manipulation system's potential application in mic.ro/nano defect study.

Microscopy has been an indispensable tool for micro/nanosample imaging, manipulation, and characterization. However, viewing the micro/nanosample from multidirection is still a big challenge for current microscopy. To address the above issue, this paper proposes a novel nanorobotic manipulation system for the automatic alignment and multidirectional imaging under microscopes. First, a miniature rotation robot with three degrees of freedom is designed and integrated with a microscope. Then, a forward-backward alignment strategy containing three loops, i.e., position shift loop, angle loop, and magnification loop, is proposed to align the sample to the rotation axis of the robot automatically. After that, the sample is imaged from multidirection by rotating the robot with one revolution (360°). Finally, the alignment accuracy is evaluated and multi-directional images of various samples are implemented. This study provides a new way for the microscopic imaging, which is expected to exert a significant impact in multiple fields on a small scale, including microscopy imaging, microdefect detection, micromanipulation, in situ characterization, and so on.