At small scales, automating robotic tasks such as assembly, force/displacement characterization, positioning, etc., appear to be particularly limited. This is due to the lack of sufficiently performing and easy-to-implement multi-degrees-of-freedom measurement systems able to measure the relative pose between micro-parts. In order to address this issue, a measurement method based on High-Precision fiducial markers (named HP code) is proposed. This measurement method combines a periodic pattern (providing high resolution by phase-based computation) with more regular QR codes (bringing versatile implementations and a quick detection). The design and method to efficiently locate these HP codes are presented in this paper. Experimental investigations demonstrate ultra-high resolution: 2 nm and$5 ~\mu $rad along$X,Y$and$\Theta $respectively (i.e. one thousandth of a pixel typically). The method is designed to be scalable as well as self-calibrated and to provide high robustness and high versatility. Two typical challenging applications in the field of microrobotics are automated to demonstrate these disruptive performances and the easy-to-implement capability of the method: (1) the automated assembly of two micro-fluidic chips through visual servoing with an achieved positioning accuracy below 50 nm, and (2) the automated micromechanical characterization of single fibers achieved by the integration of HP codes into a compliant structure enabling simultaneous micro-force and displacement sensing capabilities. These achievements highlight the versatility of the method and open the door to the rapid automation of high-quality robotic tasks at the micro scale. Note to Practitioners—The motivation for this work/study is based on the fact that many application areas are extensively orienting towards microrobotic systems to perform precise tasks with versatility. However, at the micro scale, many disturbances such as the effects of climate change strongly affect this precision. This problem is amplified by the fact that sensors cannot be easily integrated, either by lack of space or by the lack of measurement systems available. Vision-based approaches are widespread at this scale and appear very promising to measure the relative pose between micro-parts. Nevertheless, existing vision-based approaches like digital image correlation are both scale and texture dependent. Due to the lack of space, they are also difficult to use in practice at small scales for high resolution measurement. The main contribution of this paper lies in the capability to achieve ultra-high resolution measurements. For that, a structure based on High-Precision fiducial markers (named HP codes) is proposed and requires few and simple settings while achieving very high resolution both in position and orientation, typically down to one thousandth of a pixel and a few micro radians, respectively. It provides an off-the-shelf solution, versatile, easy to implement and achieves high resolution measurements in the plane (XY$\Theta $). HP codes are applicable to a wide range of applications such as tracking of a component/part of a mobile or deformable system, visual servoing of microrobots, positioning of samples, assembly of components or even mechanical characterization. A free distribution of the library is available online athttps://projects.femto-st.fr/vernier/.

The demand in the biomedical field for fast and precise devices forin vitroapplications has increased in recent years. Mobile microrobots are significantly suitable for such applications and are developing rapidly. These microrobots offer untethered actuation toward a contamination-free environment while allowing for fast and precise handling of biological entities for applications such as positioning, sensing, delivery, and cell surgery that are highly effective for new drug discoveries and improving our understanding of cells’ behavior on the single-cell level. Here, we present a review of the recent state of the art in the actuation and implementation of mobile microrobots forin vitroapplications. We first explore the widely used methods of wireless actuation. Next, we address the challenge of implementing an on-board interaction technique to handle the target biological entity without affecting the actuation of the microrobot. Finally, we discuss the future directions that would draw the basic outline for the next generation of mobile microrobots forin vitroapplications.

This paper proposes a new control-oriented model to compute the magnetic field created by a coil. A major challenge for untethered microscale mobile robotics is the control of objects for precise and fast displacements. In this work, we propose to use an alternative implementation of a model based on elliptic integral functions to control magnetically actuated micro-robots. It allows to compute the magnetic field even in the area close to the coil quickly and accurately. This model is evaluated numerically and compared to classical approaches - dipole approximation, map-based interpolation and classical elliptic integral models - in terms of accuracy, computation time and memory requirement. Simulation results show that this works allows to have an accurate model in the whole workspace by avoiding numerical issues encountered in previous works. It can be computed in a few milliseconds, making it the right candidate for closed-loop control of magnetically actuated micro-robots.

Several approaches are proposed in the literature to calculate the drag force, the electric field and the induced dielectrophoretic force. This paper analyzes the performances of various models for closed loop control of dielectrophoretic systems in comparison with experiments. This article compares their performance in terms of accuracy, computation time, and memory consumption. Four classical approaches are available to calculate the electric field. Their performances are analyzed in the paper. We have shown that combining the dipolar model of dielectrophoresis force with an anisotropic drag force (integrating the wall-effect) provides an interesting ratio precision/computation time. This paper provides an original comparison of several models described in literature whose performances have been compared with experiments.

This paper deals with the open-loop characterization of a micromanipulation system actuated by thermocapillary convective flows. Micrometric size objects placed at the air/liquid interface are actuated by heating the surface of the liquid using a laser. The heat generates a surface tension gradient at the interface which induces thermocapillary convective flows that are used to move the objects. In this paper, the performances of this approach are analyzed based on open-loop experiments. Several actuation strategies are proposed and discussed. The experimental results highlight the potential of this approach since velocities up to several millimeters per second are obtained. However the precision of the positioning is not ensured by open-loop actuation, so closed-loop control will be necessary in future works. As a first step towards closed-loop control, this paper proposes a model of the system. This model is based on the open-loop experimental results, but the proposed methodology can be applied to any setup that use thermocapillary convective flows for particle manipulation.

One of the greatest challenges in microrobotics is the development of robotic devices for high-speed transportation and precise positioning of microcomponents. This paper proposes to use non contact magnetic actuation in which objects are placed at the air/liquid interface and are actuated through magnetic field gradients. A physical model is developed and identified to perform closed-loop control. This approach is validated through several experiments in 1-D. Precise positioning and high-speed trajectory tracking of objects smaller than 100 μm are achieved. The position error of an object of 60 × 50 × 25 μm3is less than 10% of its size and the maximum velocity reached is about 6 mm/. The closed-loop control has been tested on objects as small as 30 × 20 × 25 μm3and demonstrates its ability to perform precise positioning (the position error is less than 7% of the size of the object). This approach represents a promising solution to design devices for high throughput transportation and precise positioning of micro-objects, which will lead to magnetic smart surfaces at micrometer scale.

Microscale teleoperation with haptic feedback requires scaling gains in the order of 104−107. These high gains impose a trade-off between stability and transparency. Due to the conservative approach used in most designs, transparency is reduced since damping is added to the system to guarantee stability. Starting from the fact that series, negative feedback and parallel connection of passive systems is a passive system, a new approach is addressed in this work. We propose here a complete teleoperation chain designed from the ground up for full transparency and stability, including a novel self-sensing probe and a high fidelity force-feedback haptic interface. By guaranteeing the passivity of each device and assuming that the human operator and the environment are passive systems, a homothetic direct coupling can be used without jeopardizing the stability and provides best transparency. The system is experimentally demonstrated in the complex case of a probe interacting with a water droplet under human control, while accurately transcribing the interaction back to operator.

One of the greatest challenge in microrobotics is the development of miniaturized smart surfaces for a high speed conveying and positioning of micro-objects. This paper proposes a new approach where objects are situated at the air/liquid interface and are manipulated through magnetic fields. It demonstrates that a good repeatability and a high speed can be obtained. A physical modeling is presented to analyze the dynamic behavior of the micro-object. Experiments are performed to determine the physical parameters of the model and to attest the good repeatability of the motion for an object of size 100×90×25μm3. A good agreement between the physical model and the experimental measurement is demonstrated. Since the velocity of the micro-object can be 10 times higher at the air/liquid interface than in the liquid this approach represents a promising solution to design smart surfaces for a high throughput conveying of micro-objects.

This paper presents a review of the major haptic feedback teleoperation systems for micromanipulation. During the last decade, the handling of micrometer-sized objects has become a critical issue. Fields of application from material science to electronics demonstrate an urgent need for intuitive and flexible manipulation systems able to deal with small-scale industrial projects and assembly tasks. Two main approaches have been considered: fully automated tasks and manual operation. The first one require fully pre determined tasks, while the later necessitates highly trained operators. To overcome these issues the use of haptic feedback teleoperation where the user manipulates the tool through a joystick whilst feeling a force feedback, appears to be a promising solution as it allows high intuitiveness and flexibility. Major advances have been achieved during this last decade, starting with systems that enable the operator to feel the substrate topology, to the current state-of-the-art where 3D haptic feedback is provided to aid manipulation tasks. This paper details the major achievements and the solutions that have been developed to propose 3D haptic feedback for tools that often lack 3D force measurements. The use of virtual reality to enhance the immersion is also addressed. The strategies developed provide haptic feedback teleoperation systems with a high degree of assistance and for a wide range of micromanipulation tools. Based on this expertise on haptic for micromanipulation and virtual reality assistance it is now possible to propose microassembly systems for objects as small as 1 to 10 micrometers. This is a mature field and will benefit small-scale industrial projects where precision and flexibility in microassembly are required.

This work presents a stable vision based haptic feedback for micromanipulation using both an asynchronous Address Event Representation (AER) silicon retina and a conventional frame-based camera. At this scale, most of the grippers used to manipulate objects lack of force sensing. High frequency vision detection thus provides a sound solution to get information about the position of the object and the tool to provide virtual haptic guides. Artificial retinas present high update rates, which enables to address one of the major challenge of haptic feedback teleoperation systems, namely stability. However static objects are not detected. The haptic feedback is thus based on an asynchronous silicon retina to provide a high update rate of moving objects and a frame-based camera to retrieve the position of the target object. This approach is validated by pick-and-place of microspheres (diameter: around 50 micrometers) using a piezoelectric microgripper. The displacement of the tool, as well as the opening and closing of the gripper are controlled by the haptic device. Haptic virtual guides are transmitted to users to assist them in the different steps of the pick-and-place task: a virtual stiffness ensures the correct alignment of the tool with respect to the object, a repulsive haptic force enables users to monitor the gripping step, and operators are assisted while picking and placing the object.

Micromanipulation systems have recently been receiving increased attention. Teleoperated or automated micromanipulation is a challenging task due to the need for high-frequency position or force feedback to guarantee stability. In addition, the integration of sensors within micromanipulation platforms is complex. Vision is a commonly used solution for sensing; unfortunately, the update rate of the frame-based acquisition process of current available cameras cannot ensure-at reasonable costs-stable automated or teleoperated control at the microscale level, where low inertia produces highly unreachable dynamic phenomena. This paper presents a novel vision-based microrobotic system combining both an asynchronous address event representation silicon retina and a conventional frame-based camera. Unlike frame-based cameras, recent artificial retinas transmit their outputs as a continuous stream of asynchronous temporal events in a manner similar to the output cells of a biological retina, enabling high update rates. This paper introduces an event-based iterative closest point algorithm to track a microgripper's position at a frequency of 4 kHz. The temporal precision of the asynchronous silicon retina is used to provide a haptic feedback to assist users during manipulation tasks, whereas the frame-based camera is used to retrieve the position of the object that must be manipulated. This paper presents the results of an experiment on teleoperating a sphere of diameter around 50 μm using a piezoelectric gripper in a pick-and-place task.

Molecular interactions typically have a high dynamic range (HDR), combining short-range stiff repulsive effects with long-range, soft attractive and repulsive terms. As a result, faithful haptic rendering of such molecular interactions is both important and difficult, in particular in applications where the precise perception of molecular forces is necessary (e.g. in molecular docking simulations). Traditionally, teleoperation coupling using constant gain control schemes have limited applications since they are unable to transmit to users low attractive forces without truncating repulsive ones. Furthermore, constant scaling displacement induces either instability or time-consuming experiments (displacements are slow), which deteriorates the ease of manipulation. In this paper, we describe a variable gain haptic coupling method specifically designed to render high dynamic range (molecular) forces. The proposed method is evaluated by user tests on an experiment involving two water molecules. We observe that variable force amplification is widely appreciated, whereas variable displacement scaling is appropriated only for users familiar with haptic manipulation. A complex experiment on a HIV molecule is carried out using this variable gain system. Advantages and limitations of this approach are discussed.

This paper reports the remote handling of microscale objects, between two sites approximately 630 km distant. To manipulate objects less than 10 ¿m, specific equipments such as AFM (Atomic Force Microscope) cantilevers integrated into a SEM (Scanning Electron Microscope) are generally required. Enabling remote access to such a system would benefit any micro/nanoresearcher. However, vision feedback and sensor data of a micromanipulation system are generally limited, hence the implementation of a teleoperation scenario is not straightforward. Specific tools are proposed here for an intuitive manipulation in a wide range of applications. To ensure ease of manipulation, both a 3D virtual representation of the scene and haptic feedback are provided. Force sensor feedback is limited since only two measures are available. In order to extend this information, vision algorithms are developed to estimate the respective positions of the tool and objects, which are then used to calculate the haptic feedback. The stability of the overall scheme is very sensitive to time delays. This requirement is taken into account in vision algorithms and the communication module which transfers the data between the two remote sites. In addition, the proposed robotic control architecture is modular so that the platform can be used for a wide range of applications. First results are obtained on a teleoperation between Paris, France, and Oldenburg, Germany.

In this paper, a fully teleoperated 3D micro assembly task with haptic feedback is presented. Microspheres (diameter: 4-6μm) are manipulated by pick-and-place. The setup is composed of a dual-tip gripper controlled through a haptic interface. To grasp the spheres, the tips must be correctly positioned with respect to the objects. The approach proposed to align the gripper is based on a user-driven exploration of the to-be-manipulated object. During this step, the haptic feedback is based on amplitude measurements from cantilevers in dynamic mode. Hence, the operator perceives the contact while freely exploring the manipulation area. A virtual guide is generated to pull the user to the optimum contact point, allowing correct positioning of dual tips. For the pick-and-place operation, the haptic feedback provides the user with information about the microscale interactions occurring during the operation. As experimental validation, a two-layer pyramid composed of four nylon microspheres is built in ambient conditions.

This paper deals with the problem of bilateral haptic control in nanorobotics. At this scale, a human operator cannot interact directly with objects. He needs special tools manipulated through robotic systems. Therefore, force feedback devices are the only solution to provide him a sense of touch. However, the quality of the rendering strongly influences his ability to perform a given task. Stability is the main requirement that the system must fulfil to be usable. As the choice of the controller and its tuning are critical issues, a general method to tune the parameters of two haptic controllers is presented. A theoretical study is carried out and the methodology is validated with an experiment composed of several phases with high dynamic phenomena. Intrinsic limitations of the two controllers are also pointed out.

This paper presents a 2D teleoperation task at microscales with force feedback. At this scale, two major problems arise while performing manipulation tasks: the lack of 3D real time visual feedback, and the difficulty to determine the interaction forces. Therefore, indications must be provided to help the user perform a given task. In this paper, we provide the user with intuitive force feedback, to improve objects' manipulation using a haptic device. Our platform is composed of a tipless beam manipulator, which is deformed when forces are applied to it. These deformations are measured using a laser. The force information we provide to the user is based on the raw beam's deformation measurement, and mechanical properties of the probe. It does provide the operator with indications about the interaction forces. This approach is validated by performing lateral and longitudinal rolling operations using microspheres with a radius of 25-micrometers. 2D rolling telemanipulation at microscale with force feedback is successfully demonstrated.

In this paper, a new tool dedicated to the analysis and the conception of molecules is presented. It is composed of an adaptive simulation software and a haptic device used to interact with molecules while feeling either the forces applied by the environment or the internal forces. The adaptive articulated body algorithm allows fast simulations of complex flexible molecules. To handle the coupling with the force feedback device, two different control schemes designed for nanoscale applications and providing high transparency rendering are proposed and compared.