This paper presents a functionality that has been developed for the home-made AFM-in-SEM robotic system at the ISIR laboratory. The method allows extending the range of an Atomic Force Microscope (AFM) and dealing with drift issues by fusing multiple individually AFM topography patches. The merging of the patches into a single image is done through a Generalized Procrustes Analysis Iterative Closest Point (GPA-ICP) algorithm. To validate the effectiveness of the approach, an AFM image of a TGX1 calibration grid and a 3.4billion-year-old organic-walled microfossil are reconstructed by automatically merging 50 AFM elementary topography patches of dimension 0.9 µm × 1.2 µm based on feature matching. The overlap between two adjacent patches is 50 % and 33 % in the X and Y axes respectively. The result is a coherent 3.2 µm × 3.0 µm drift-free long range AFM topography without significant artifacts. The method is tested using an AFM-in-SEM system based on a 3-DOF cartesian robot equipped with inertial piezoelectric actuators. This method can be used to extend the range of any type of AFM with a dual XY stage setup. Thus, it opens the door for high-resolution long-range AFM by adding a long-range coarse resolution stage to a preexisting AFM system all without needing to actuate both stages simultaneously.

This article proposes a method for the correction of angular deviations caused during the fixing process of samples prepared for Atomic Force Microscopy (AFM). The correction is done using the angular control of a 6-DOF PPPS parallel platform were the sample is placed, while the AFM scan is performed by a 3-DOF serial cartesian robot with a tuning fork probe designed to perform FM-AFM. The method uses the generic x, y, and z data provided by the AFM after performing a scan on a free surface of the sample substrate. This is used to calculate the plane that closest approximates the points by solving a system of linear equations. This plane is then used to estimate the angular corrections that the 6-DOF parallel robot has to do in order to compensate the deviations. The proposed algorithm can be performed iteratively in order to refine the correction. The method also does not require any special preparation of the substrate. It only requires to have a free surface to scan. Experiments are performed using this algorithm to correct the orientation deviation of a substrate of V1 High-grade mica. The results show that the method is able to correct the angular deviation of the sample relatively to the AFM probe with an error of 0.2° after only two iterations of the algorithm.

The spherical joint is an effective solution to design parallel micro-robotic systems with rotation capabilities in the three-dimensional space. This type of joint has however some non-linear characteristics, such as the clearance, which affect the positioning accuracy in micro-robotic tasks. The starting point of this study lies in experimental observations of rotation errors from a 3-PPPS 6-DOF parallel micro-robotic systems operating inside a scanning electron microscope. The objective of the paper is to assess the role of the spherical joints in the rotation errors and to evaluate whether the joints non-linearities can cause errors with the same order of magnitude as those observed experimentally. To this end, the first part of the study addresses the modeling of 3-PPPS 6-DOF parallel micro-robotic systems with spherical joints including the clearance. This model allows for analysing the effect of the clearance on position and rotation accuracies of the micro-robotic system. It is found by simulations that the same positioning behavior as in the experiments occurs when the clearance of the spherical joint is included in the model, supporting the hypothesis. Therefore, it is concluded that clearance in spherical joints has a significant effect on the precision of parallel type micro-robotic systems which opens new challenges in the control of poly-articulated micro-robotic systems with clearance compensation.

Active force sensors are key instruments to get around the tradeoff between the sensitivity and the measurement range of conventional passive force sensors. Thanks to their quasi-infinite stiffness in closed loop, active sensors can be applied for force measurements on samples with a wide range of stiffness without interference with the mechanical parameters of the sensor. MEMS (Micro-Electro Mechanical Systems) active force sensors have been wildly developed in the literature but they are ill adapted for force measurements at the Newton level needed in meso-scale robotics. In this article, a novel structure for a meso-scale active force sensor is proposed for the measurement of forces from the milli-newton to the newton.This novel meso-scale sensor is based on a nil-stiffness guidance and an electromagnetic actuation. This paper deals with its design, identification, calibration and closed loop control. The sensor exhibits nil-stiffness characteristic in open loop and an almost infinite stiffness in closed loop. This allows measuring forces with a large range of gradients. First experiments shows the ability of this new sensor architecture to measure low frequency forces up to 0.8N with a precision of 0.03 N and a closed loop -20 dB cutoff frequency of 73.9Hz.

Micromanipulation is challenging due to the specific physical effects governing the microworld. Interactive approaches using only visual feedback are limited to the 2D image of the microscope, and have forcibly lower bandwidth. Recently, haptic feedback teleoperation systems have been developed to try to overcome those difficulties. This paper explores the case of an optical tweezers platform coupled to an haptic device providing transparent force feedback. The impact of haptic feedback regarding user dexterity on tactile exploration tasks is studied using 3 μm microbeads and a test bench with micro sized shapes. The results reveal a consistent improvement in both users' trajectory tracking and their control of the contact forces. This also validates the experimental setup which performed reliably on 140 different trials of the evaluation.

In this paper, we introduce novel miniature swimmers with multiple rigid tails based on spherical helices. The tail distribution of these prototypes enhances its swimming features as well as allowing to carry objects with it. The proposed swimmers are actuated by a rotating magnetic field, generating the robot rotation and thus producing a considerable thrust to start self-propelling. These prototypes achieved propulsion speeds up to 6 mm/s at 3.5 Hz for a 6-mm in size prototypes. We study the efficiency of different tail distribution for a 2-tailed swimmer by varying the angular position between both tails. Moreover, it is demonstrated that these swimmers experience great sensibility when changing their tail height. Besides, these swimmers demonstrate to be effective for cargo carrying tasks since they can displace objects up to 3.5 times their weight. Finally, wall effect is studied with multi-tailed swimmer robots considering 2 containers with 20 and 50-mm in width. Results showed speeds' increments up to 59% when swimmers are actuated in the smaller container.

Collaborative robotics allows merging the best capabilities of humans and robots to perform complex tasks. This allows the user to interact with remote and directly inaccessible environments such as the micro-scale world. This interaction is made possible by the bidirectional exchange of information (displacement - force) between the user and the environment through a haptic interface. The effectiveness of the human/robot interaction is highly dependent on how the human feels the forces. This is a key point to enable humans to make the right decisions in a collaborative task. This paper discusses the design of a dynamic observer to estimate the forces applied by a human operator on a class of parallel pantograph-type haptic interfaces used to control small-scale robotic systems. The objective is to reject disturbances in order to improve the human force perception capability over a wide frequency range. A dynamic pantograph model is proposed and experimentally validated. The observer is designed on the basis of the proposed dynamic model and its efficiency in estimating the applied human force is demonstrated for the first time with pantograph-type interfaces. Experimental validation first shows the effectiveness of the perturbation observer for external human force estimation with a response time of less than 0.2 s and a mean error of less than 7 mN and then the effectiveness of the controller in improving the quality of human sensation of forces down to 10 mN.

As part of an evaluation process of user experience realism in a Virtual Reality (VR) system, we focus in this paper on one of the core characteristics of vision: the relationship between contrast and luminance. The experiment aims at validating in VR reaction time predictions given by Rea and Ouellette's model. The subjects have to distinguish, as fast as they can, a target object from an uniform background. Our results did not match the predictions of the model. Our subjects showed higher performance in performing the task than expected. At low level of contrast, our subjects could easily perceive a target they should not have been able to see at all. This is explained by the size of the visual field surrounding the target: at low level of visibility, the larger the surrounding, the easier perception the is. We conclude that the Rea and Ouellette's model could be applied in VR if a specific visual field size factor was added.

This paper proposes a new control-oriented model to compute the magnetic field created by a coil. A major challenge for untethered microscale mobile robotics is the control of objects for precise and fast displacements. In this work, we propose to use an alternative implementation of a model based on elliptic integral functions to control magnetically actuated micro-robots. It allows to compute the magnetic field even in the area close to the coil quickly and accurately. This model is evaluated numerically and compared to classical approaches - dipole approximation, map-based interpolation and classical elliptic integral models - in terms of accuracy, computation time and memory requirement. Simulation results show that this works allows to have an accurate model in the whole workspace by avoiding numerical issues encountered in previous works. It can be computed in a few milliseconds, making it the right candidate for closed-loop control of magnetically actuated micro-robots.

Scanning Electron Microscopy (SEM) is an ideal observation tool for small scales robotics. It has the potential to achieve automated nano-robotic tasks such as nano-handling and nano-assembly. Path following control of nano-robot end effectors using SEM vision feedback is a key for an intuitive programming of elementary robotic tasks sequences. It requires the ability to track end effectors under various SEM scan speeds. SEM suffers however from tricky issues that limits robotic tracking capabilities. This paper focuses on one specific issue related to the compromise between the scan speed and the image quality. This restriction seriously limits the performance of conventional vision tracking algorithms when used with electron images. At high scan speed, the image quality is very noisy making very difficult to differentiate the robot end effector from the background, hence limiting the tracking capabilities. The work related in this paper explores for the first time the potential value of Convolutional Neural Networks (ConvNet) in the context of nano-robotic vision tracking inside SEM. The aim is to localize an end-effector, AFM cantilever in the case of the study, from SEM images for any scan speed configuration and despite of low images quality. For that purpose, a data set of AFM tip images is build up from SEM images for the learning algorithm. Network performances are estimated under different SEM scan speeds. Thanks to the learning algorithm, experimental results show robust AFM tip tracking capabilities inside the SEM under various scan speed conditions.

Flexible and helical magnetic microswimmers have been well reviewed in the literature because they could be exploited for envisaged applications such as targeted drug delivery, material removal, and micromanipulation. In this article, scaled-up versions of those robots are introduced to study in detail their maneuverability and dexterity while swimming. The robots were immersed in pure glycerol, thus, reproducing a low Reynolds scenario. The proposed robots were previously optimized, achieving their best performances. The experiments assess the performances of these two kinds of robots in terms of rapidity, and steering error following 3D trajectories in environments with high viscous variations.

Micro-assembly is a challenging issue for automation due to particularities of micro-world physics and limitations on sensors. Consequently, most applications are human-operated often with basic joystick-like interfaces. Beside being nonintuitive, these solutions do not provide their users with a meaningful insight into the microworld. This paper proposes a novel intuitive remote handling interface, using a classical hand-held assembly tool as a paradigm. The master device is a portable instrumented tweezers with one active degree of freedom. Its spatial motion, tracked by optical means, controls the slave kinematics while its pinch commands the slave robot's microgripper and provides haptic feedback. Different coupling strategies using position or speed variables are demonstrated.

Nano-robotic systems based on Piezoelectric StickSlip (PSS) actuators have become increasingly popular in research and industry for semi-automated and automated tasks at small scales. For an efficient use of PSS actuators, a series of research have been fulfilled on design process, dynamic modeling, driving methods and position control. However, there have been very few investigations on velocity control of PSS actuators. Velocity control is important to enable the nano-robotic system to generate a smooth and efficient motion and to avoid the undesired inertial shock of the end effector. This paper deals with velocity characterization and control strategies for nano-robotic systems based on PSS actuators. The range of achievable velocities on PSS actuators is studied in air and vacuum environments. This analysis allows the definition of a detailed map of the velocity characteristics in forward and backward directions of motion. Velocity control strategies are then studied based on an instantaneous velocity feedback and an average velocity feedback. Results of the proposed method show the first experimental demonstration of velocity control for PSS actuators in medium and high speed configurations opening new perspectives on the use of nano-robotic systems in dynamic automated tasks.

Previously, we developed a 3D path following algorithm to improve the microrobot performances to overcome the modeling errors and environmental disturbances in order to perform real tasks, tested experimentally in a scaled-up helical microswimmer. In this paper, we show that adapting the propulsion mode, the general path following algorithm can be used for any microswimmer behaving as a nonholonomic system. For that purpose, we study a magnetic robot with a flexible tail that mimics the spermatozoa locomotion mechanism. A frequency and amplitude characterization of the flexible swimmer using an oscillating magnetic field is shown. To adapt the propulsion mode, we develop a 3D magnetic field control based on the steering angular velocities which are computed from the path following algorithm in order to propel and steer the flexible robot to reach a desired location in space and achieve for the first time a 3D closed-loop motion control using a swimmer with flexible flagella.

Optical Tweezers are considered one of the most suitable techniques for biological tasks, however the lack of automation make this technology less accessible. We present here a new 3D force sensing method with high bandwidth (up to 10Khz) which can allow implementing complex robotic approaches. Proposed technique uses high speed image tracking with nano-metric resolution in 3 directions. Its capabilities are demonstrated in a teleoperated 3D manipulation scenario with a haptic user interface, where naive users performed direct in vitro haptic exploration of isolated Red Blood Cells inside a Petri dish.

One of the greatest challenges in microrobotics is the development of robotic devices for high-speed transportation and precise positioning of microcomponents. This paper proposes to use non contact magnetic actuation in which objects are placed at the air/liquid interface and are actuated through magnetic field gradients. A physical model is developed and identified to perform closed-loop control. This approach is validated through several experiments in 1-D. Precise positioning and high-speed trajectory tracking of objects smaller than 100 μm are achieved. The position error of an object of 60 × 50 × 25 μm3is less than 10% of its size and the maximum velocity reached is about 6 mm/. The closed-loop control has been tested on objects as small as 30 × 20 × 25 μm3and demonstrates its ability to perform precise positioning (the position error is less than 7% of the size of the object). This approach represents a promising solution to design devices for high throughput transportation and precise positioning of micro-objects, which will lead to magnetic smart surfaces at micrometer scale.

This paper addresses the problem of 3D path following of magnetic helical microswimmers in closed-loop. An error kinematic model in a local frame with sideslip and attack angles is used to express the motion of the helical microswimmer. A new derivation of the chained form with three inputs and five states is used to linearize the kinematic model in order to design a decoupled stable control. In experimentation, the 3D path following is validated using a scaled-up magnetic helical microswimmer with visual servo control by following first a spatial straight line, then a helix trajectory and finally an inclined sinusoidal trajectory. The closed-loop control is also compared with the open-loop control to illustrate the robustness and the accuracy of both controllers to the disturbances.

In order to define trajectory-tracking strategies for nano-robotic systems using piezoelectric stick-slip actuators, the dynamics of the elementary actuator must be studied and well modeled. The modeling of this class of actuators is complex because several nonlinear parameters are involved. In this paper, we propose a systematic modeling methodology of piezoelectric stick-slip actuators for nano-robotic systems control. The main idea is the proposition of an augmented voltage/frequency rate dependent modeling of the friction force based on a multi-state elasto-plastic formulation. Experimental and simulation results demonstrate the efficiency of the model in the time and the frequency domains. As a case of study, the proposed model is used to define a control strategy in order to detect collisions when the nano-robotic system is operating inside a Scanning Electron Microscope (SEM). This application demonstrates the need of a voltage/frequency rate dependent modeling.

Visual tracking and estimation of the 3D posture of a micro/nano-object is a key issue in the development of automated manipulation tasks using the visual feedback. The 3D posture of the micro-object is estimated based on a template matching algorithm. Nevertheless, a key challenge for visual tracking in a scanning electron microscope (SEM) is the difficulty to observe the motion along the depth direction. In this paper, we propose a template-based hybrid visual tracking scheme that uses luminance information to estimate the object displacement on x-y plane and uses defocus information to estimate object depth. This approach is experimentally validated on 4-DoF motion of a sample in a SEM.

Direct photometric visual servoing uses only the pure image information as a visual feature, instead of using classic geometric features such as points or lines. It was demonstrated efficiently in 6 degrees of freedom (DoF) positioning. However, in micro-scale, using only image intensity as a visual feature performs unsatisfactorily in cases where the photometric variation is low, such as motions along vision sensor's focal axis under a high magnification. In order to improve the performance and accuracy in those cases, an approach using hybrid visual features is proposed in this paper. Image gradient is employed as a visual feature on z axis while image intensity is used on the other 5 DoFs to control the motion. A 6-DoF micro-positioning task is accomplished by this hybrid visual servoing scheme. The experimental results obtained on a parallel positioning micro-stage under a digital microscope show the robustness and efficiency of the proposed method.

This paper addresses modeling issues for a class of nano-robotic systems using piezoelectric stick-slip actuators. The work focuses on the friction force modeling to describe the dynamics of a stick-slip actuator in a wide operating range needed in nano-robotics. Based on the theory of the single state elasto-plastic model and on an experimental analysis, necessary conditions on presiding modeling are highlighted. The conditions allow describing the dynamics of stick-slip type actuators for both scanning mode and stepping mode in the time and the frequency domains and for backward and forward directions of the motion. The proposed dynamic model opens new perspective for closed loop control of nano-robotic system.

This letter presents a novel computationally efficient and robust pattern tracking method based on a time-encoded, frame-free visual data. Recent interdisciplinary developments, combining inputs from engineering and biology, have yielded a novel type of camera that encodes visual information into a continuous stream of asynchronous, temporal events. These events encode temporal contrast and intensity locally in space and time. We show that the sparse yet accurately timed information is well suited as a computational input for object tracking. In this letter, visual data processing is performed for each incoming event at the time it arrives. The method provides a continuous and iterative estimation of the geometric transformation between the model and the events representing the tracked object. It can handle isometry, similarities, and affine distortions and allows for unprecedented real-time performance at equivalent frame rates in the kilohertz range on a standard PC. Furthermore, by using the dimension of time that is currently underexploited by most artificial vision systems, the method we present is able to solve ambiguous cases of object occlusions that classical frame-based techniques handle poorly.

Helical microswimmers that are capable of propulsion at low Reynolds numbers have great potential for numerous applications. Several kinds of artificial magnetic-actuated helical microswimmers have been designed by researchers. However, they are primarily open-loop controlled. This paper aims to investigate methods of closed-loop control of a magnetic-actuated helical swimmer at low Reynolds number by using visual feedback. For many in-vitro applications, helical swimmers should pass through a defined path, for example along channels with no prerequisite on the velocity profile along the path. Therefore, the main objective of this paper is to achieve a velocity-independent planar path following task. Since the planar path following is based on 3-D steering control of the helical swimmer, a 3-D pose estimation of a helical swimmer is introduced based on the real-time visual tracking with a stereo vision system. The contribution of this paper is in two parts: The 3-D steering of a helical swimmer is demonstrated by visual servo control; and the path following of a straight line with visual servo control is achieved, then compared with open-loop control. We further expect that with this visual servo control method, the helical swimmers will be able to follow reference paths at the microscale.

Microscale teleoperation with haptic feedback requires scaling gains in the order of 104−107. These high gains impose a trade-off between stability and transparency. Due to the conservative approach used in most designs, transparency is reduced since damping is added to the system to guarantee stability. Starting from the fact that series, negative feedback and parallel connection of passive systems is a passive system, a new approach is addressed in this work. We propose here a complete teleoperation chain designed from the ground up for full transparency and stability, including a novel self-sensing probe and a high fidelity force-feedback haptic interface. By guaranteeing the passivity of each device and assuming that the human operator and the environment are passive systems, a homothetic direct coupling can be used without jeopardizing the stability and provides best transparency. The system is experimentally demonstrated in the complex case of a probe interacting with a water droplet under human control, while accurately transcribing the interaction back to operator.

Helical microswimmers capable of propulsion at low Reynolds numbers have been proposed for many applications. However, closed-loop controlled helical swimmers are still challenging because of the limits of optical tracking, and a lack of control parameters lying on the swimming characteristics of both linear propulsion and steering. Although the linear propulsion characteristics of helical swimmers were extensively studied, the steering characteristics have not yet been clearly shown. Helical microswimmers are efficient in propulsion, whereas their high surface-to-volume ratio limits the steering performances. In this paper, we characterized both the direction and inclination steering using a real-time visual tracking of orientation. The direction steering efficiency could be increased in terms of response time with a higher inclination angle both in floating conditions and on a sticky substrate. We thus developed a 3D steering strategy by combining direction and inclination steering to improve the steering performance. We further expect that the characterization of steering performance can contribute to defining the control parameters for future closed-loop control.

One of the greatest challenge in microrobotics is the development of miniaturized smart surfaces for a high speed conveying and positioning of micro-objects. This paper proposes a new approach where objects are situated at the air/liquid interface and are manipulated through magnetic fields. It demonstrates that a good repeatability and a high speed can be obtained. A physical modeling is presented to analyze the dynamic behavior of the micro-object. Experiments are performed to determine the physical parameters of the model and to attest the good repeatability of the motion for an object of size 100×90×25μm3. A good agreement between the physical model and the experimental measurement is demonstrated. Since the velocity of the micro-object can be 10 times higher at the air/liquid interface than in the liquid this approach represents a promising solution to design smart surfaces for a high throughput conveying of micro-objects.

In order to characterize the mechanical behavior of fragile resonant microelectromechanical systems (MEMS)/nanoelectromechanical systems (NEMS), nondestructive measurements are required. In this paper, a cartography of local stiffness variations on a suspended micromembrane is established for the first time, by a tuning-fork-based dynamic force sensor inside a scanning electron microscope (SEM). Experiments are conducted individually on a batch of InP membranes 200 nm thin, using a 9-degree-of-freedom (dof) nanomanipulation system, complemented with virtual reality and automation tools. Results provide stiffness values in the range of a few newton per meter, with variations in a single sample depending on the membrane models.

Recent studies have been carried out for rotating single flexible flagellum: a possible propelling mechanism that has been adopted by several artificial microswimmers due to its relatively simple structure yet considerable propulsive force generation. In this paper, we introduce a miniature swimming robot design with multiple flexible artificial flagella that benefits from the increased number of flagella. The characteristic length of the robot body is less than 1 mm. Experimental characterization of swimming of the robot shows that swimming speed can be linearly improved solely by increasing the number of attached flagella, suggesting a new way for speed enhancement besides flagellum geometry optimization. In addition, a numerical model modified from the single, straight flexible flagellum case is further established to study propulsive force generation by nonstraight, flexible flagellum. A robot with multiple, sinusoidal flagella design is fabricated to demonstrate the capability of the proposed two-step photolithography-based microfabrication method to handle more complex flagella designs, which may enhance swimming performance.

The optical trap is a powerful non-contact approach for manipulating micron sized objects. Teleoperation of optical tweezers can be performed by coupling with a haptic interface, which allows an efficient robotic device to control positions and get force feedback. This provides users direct and intuitive microscopic interactions. The major difficulty in order for haptic devices to generate a reliable tactile sensation lies in its high frequency requirement of more than 1 kHz. This paper presents a fast force feedback teleoperation system for optical tweezers that attains this high frequency. The used force sensor is a novel event-based camera that transmits output as a continuous stream of asynchronous temporal events thus enabling high speed event-based visual processing. This new sensor is compared to a conventional frame based one to show advantages of our setup. A complex task of exploiting three dimensional target surface is performed demonstrating the robustness and efficiency of the presented method. This is the first time microspheres are used to touch targets of arbitrary form and color, which may interest broad-reaching biological and physical applications.

Typical methods of measuring mechanical properties at the micro-scale are destructive, and do not allow proper characterisation on resonant MEMS/NEMS. In this paper, a cartography of local stiffness variations on a suspended micromembrane is established for the first time, by a tuning-fork-based dynamic force sensor inside a SEM. Experiments are conducted on InP membranes 200nm thin, using a 9-DoF nano-manipulation system, complemented with virtual reality and automation tools. Results provide stiffness values ranging from 0.6 to 3 N/m on a single sample.

This paper presents a review of the major haptic feedback teleoperation systems for micromanipulation. During the last decade, the handling of micrometer-sized objects has become a critical issue. Fields of application from material science to electronics demonstrate an urgent need for intuitive and flexible manipulation systems able to deal with small-scale industrial projects and assembly tasks. Two main approaches have been considered: fully automated tasks and manual operation. The first one require fully pre determined tasks, while the later necessitates highly trained operators. To overcome these issues the use of haptic feedback teleoperation where the user manipulates the tool through a joystick whilst feeling a force feedback, appears to be a promising solution as it allows high intuitiveness and flexibility. Major advances have been achieved during this last decade, starting with systems that enable the operator to feel the substrate topology, to the current state-of-the-art where 3D haptic feedback is provided to aid manipulation tasks. This paper details the major achievements and the solutions that have been developed to propose 3D haptic feedback for tools that often lack 3D force measurements. The use of virtual reality to enhance the immersion is also addressed. The strategies developed provide haptic feedback teleoperation systems with a high degree of assistance and for a wide range of micromanipulation tools. Based on this expertise on haptic for micromanipulation and virtual reality assistance it is now possible to propose microassembly systems for objects as small as 1 to 10 micrometers. This is a mature field and will benefit small-scale industrial projects where precision and flexibility in microassembly are required.

Despite a large number of proofs of concept in nanotechnologies (e.g., nanosensors), nanoelectromechanical systems (NEMS) hardly come to the market. One of the bottlenecks is the packaging of NEMS which require handling, positioning, assembling and joining strategies in the mesoscale (from 100 nm to 10$\ \mu{\rm m}$, between nanoscale and microscale). It requires models of the interaction forces and adhesion forces dedicated to this particular scale. This paper presents several characteristics of the mesoscale in comparison with nanoscale and microscale. First, it is shown that the distributions of charges observed on the micro-objects and meso-objects would have negligible effects on the nano-objects. Second, the impact of both chemical functionalization and physical nanostructuration on adhesion are presented. Third, the van der Waals forces are increased by local deformations on the mesoscale contrary to the nanoscale where the deformation is negligible. This paper shows some typical characteristics of the mesoscale.

Micro and nanorobots can change many aspects of medicine by enabling targeted diagnosis and therapy, and minimal invasive surgery. A helical nanobelt with a magnetic head was proposed as a microrobot driven by rotating magnetic field in prior works. Magnetically coated tails were already shown in some works. However the control of such surface magnetic tails is not clearly realized yet. This paper aims to obtain control parameters for the modeling and simulation of the influence of surface magnets onto the swimming performances. For this, we created scaled-up helical nanobelts and the experimental testbed to get the control parameters and to prepare future closed-loop control.

This work presents a stable vision based haptic feedback for micromanipulation using both an asynchronous Address Event Representation (AER) silicon retina and a conventional frame-based camera. At this scale, most of the grippers used to manipulate objects lack of force sensing. High frequency vision detection thus provides a sound solution to get information about the position of the object and the tool to provide virtual haptic guides. Artificial retinas present high update rates, which enables to address one of the major challenge of haptic feedback teleoperation systems, namely stability. However static objects are not detected. The haptic feedback is thus based on an asynchronous silicon retina to provide a high update rate of moving objects and a frame-based camera to retrieve the position of the target object. This approach is validated by pick-and-place of microspheres (diameter: around 50 micrometers) using a piezoelectric microgripper. The displacement of the tool, as well as the opening and closing of the gripper are controlled by the haptic device. Haptic virtual guides are transmitted to users to assist them in the different steps of the pick-and-place task: a virtual stiffness ensures the correct alignment of the tool with respect to the object, a repulsive haptic force enables users to monitor the gripping step, and operators are assisted while picking and placing the object.

Micromanipulation systems have recently been receiving increased attention. Teleoperated or automated micromanipulation is a challenging task due to the need for high-frequency position or force feedback to guarantee stability. In addition, the integration of sensors within micromanipulation platforms is complex. Vision is a commonly used solution for sensing; unfortunately, the update rate of the frame-based acquisition process of current available cameras cannot ensure-at reasonable costs-stable automated or teleoperated control at the microscale level, where low inertia produces highly unreachable dynamic phenomena. This paper presents a novel vision-based microrobotic system combining both an asynchronous address event representation silicon retina and a conventional frame-based camera. Unlike frame-based cameras, recent artificial retinas transmit their outputs as a continuous stream of asynchronous temporal events in a manner similar to the output cells of a biological retina, enabling high update rates. This paper introduces an event-based iterative closest point algorithm to track a microgripper's position at a frequency of 4 kHz. The temporal precision of the asynchronous silicon retina is used to provide a haptic feedback to assist users during manipulation tasks, whereas the frame-based camera is used to retrieve the position of the object that must be manipulated. This paper presents the results of an experiment on teleoperating a sphere of diameter around 50 μm using a piezoelectric gripper in a pick-and-place task.

Molecular interactions typically have a high dynamic range (HDR), combining short-range stiff repulsive effects with long-range, soft attractive and repulsive terms. As a result, faithful haptic rendering of such molecular interactions is both important and difficult, in particular in applications where the precise perception of molecular forces is necessary (e.g. in molecular docking simulations). Traditionally, teleoperation coupling using constant gain control schemes have limited applications since they are unable to transmit to users low attractive forces without truncating repulsive ones. Furthermore, constant scaling displacement induces either instability or time-consuming experiments (displacements are slow), which deteriorates the ease of manipulation. In this paper, we describe a variable gain haptic coupling method specifically designed to render high dynamic range (molecular) forces. The proposed method is evaluated by user tests on an experiment involving two water molecules. We observe that variable force amplification is widely appreciated, whereas variable displacement scaling is appropriated only for users familiar with haptic manipulation. A complex experiment on a HIV molecule is carried out using this variable gain system. Advantages and limitations of this approach are discussed.

The paper documents the principle and experiments of the "2mm dash" winner at NIST IEEE Mobile Microrobotics Challenge held at ICRA2010 in Alaska [1]. Submission is made for the special session "ICRA Robot Challenge: Advancing Research Through Competitions". The new MagPieR microrobot was specially designed for breaking the speed record, providing a planar magnetic actuation with an optimised coils setup and a subsequent piezoelectric actuation for improved sliding condition. The paper describes the principle of actuation, the microrobot manufacturing flowchart and the assembly setup. Some simulations are provided with a first series of experimental data and conclusions.

To understand robotic grasping at the nanoscale, contact mechanics between nano grippers and nano samples was studied. Contact mechanics models were introduced to simulate elastic contacts between various profiles of flat surface, sphere and cylinder for different types of nano samples and nano grippers. Analyses and evaluation instances indicate that friction forces, commonly used in macro grasping to overcome the gravity, at nanoscale is often not enough to overcome relatively strong adhesion forces to pick up the nano sample deposited on a substrate due to tiny contact area of the grasping. Two-finger grippers are proposed for the stable nanoscale grasping and a nonparallel gripper with a 'V' configuration was demonstrated with better grasping capabilities than a parallel one. To achieve the robotic nanoscale grasping, a nano gripper constructed from two individually actuated and sensed tips is presented. Pick-and-place manipulation of silicon nanowires validate the theoretical analyses and capabilities of the proposed nano gripper.

This paper reports the remote handling of microscale objects, between two sites approximately 630 km distant. To manipulate objects less than 10 ¿m, specific equipments such as AFM (Atomic Force Microscope) cantilevers integrated into a SEM (Scanning Electron Microscope) are generally required. Enabling remote access to such a system would benefit any micro/nanoresearcher. However, vision feedback and sensor data of a micromanipulation system are generally limited, hence the implementation of a teleoperation scenario is not straightforward. Specific tools are proposed here for an intuitive manipulation in a wide range of applications. To ensure ease of manipulation, both a 3D virtual representation of the scene and haptic feedback are provided. Force sensor feedback is limited since only two measures are available. In order to extend this information, vision algorithms are developed to estimate the respective positions of the tool and objects, which are then used to calculate the haptic feedback. The stability of the overall scheme is very sensitive to time delays. This requirement is taken into account in vision algorithms and the communication module which transfers the data between the two remote sites. In addition, the proposed robotic control architecture is modular so that the platform can be used for a wide range of applications. First results are obtained on a teleoperation between Paris, France, and Oldenburg, Germany.

We present a novel force sensor for cell indentation and cell injection. This force sensor is a monolithic structure that integrates two resonators. It provides a surface sensitive to out-of-plane forces where a living cell can be conveniently placed for manipulation. Normal forces applied upon the cell under study are estimated via frequency shifts of the resonators. In this paper, we develop a theoretical study for predicting and optimizing the structure's sensitivity. As a proof of concept, we also report the fabrication and experimental characterization of a first prototype. In ambient conditions, our prototype presently offers a quality factor of ~700, and a linear sensitivity of ~5.75 Hz/¿m. In addition, we report the implementation of a compact and low-cost optical fiber setup to monitor the resonators' frequency. Potential applications are illustrated with the measurement of forces applied on lobster eggs.

We propose a method to increase propulsion of a micro-scale swimming robot powered by an artificial flagellum through the use of multiple helices while retaining the simple actuation method of a single rotation axis. Scaled up experiments with similar Reynolds number are carried out to compare the performance of five different propulsion designs with pairs of stiff or flexible flagella. The designs feature stiff helices, straight flexible rods, and flexible helices inspired by bacterial flagella. Results indicate that for a given rotation frequency, thrust is proportional to the number of helices, but that the torque required to drive a flagellum offset from the common rotation axis is increased. Furthermore, shape deformation of flexible helices due to bending forces can positively affect thrust under certain conditions. Therefore, given the ease of fabrication, the use of multiple offset flexible flagella is a potential method to achieve increased thrust force in artificial bacteria flagella.

In this article, a nanorobotic manipulation system under Scanning Electron Microscope (SEM) is developed for mechanical property characterization of ultra flexible nano-structures. Frequency modulated quartz tuning fork is proposed as gradient force sensing. Helical Nanobelts (HNB) were used as example to demonstrate the capabilities of the proposed system. The stiffness of HNBs were obtained in full tensile elongation experiments, ranging from 0.009 N/m at rest position to 0.297 N/m at full elongation before breaking with a resolution of 0.0031 N/m. The non-linear behavior of the HNB's measured stiffness is clearly revealed for the first time in full range. Furthermore, the stiffness could be transformed into force measurement that ranges from 14.5 nN to 2.96 μN.

In this paper, a fully teleoperated 3D micro assembly task with haptic feedback is presented. Microspheres (diameter: 4-6μm) are manipulated by pick-and-place. The setup is composed of a dual-tip gripper controlled through a haptic interface. To grasp the spheres, the tips must be correctly positioned with respect to the objects. The approach proposed to align the gripper is based on a user-driven exploration of the to-be-manipulated object. During this step, the haptic feedback is based on amplitude measurements from cantilevers in dynamic mode. Hence, the operator perceives the contact while freely exploring the manipulation area. A virtual guide is generated to pull the user to the optimum contact point, allowing correct positioning of dual tips. For the pick-and-place operation, the haptic feedback provides the user with information about the microscale interactions occurring during the operation. As experimental validation, a two-layer pyramid composed of four nylon microspheres is built in ambient conditions.

This paper deals with the problem of bilateral haptic control in nanorobotics. At this scale, a human operator cannot interact directly with objects. He needs special tools manipulated through robotic systems. Therefore, force feedback devices are the only solution to provide him a sense of touch. However, the quality of the rendering strongly influences his ability to perform a given task. Stability is the main requirement that the system must fulfil to be usable. As the choice of the controller and its tuning are critical issues, a general method to tune the parameters of two haptic controllers is presented. A theoretical study is carried out and the methodology is validated with an experiment composed of several phases with high dynamic phenomena. Intrinsic limitations of the two controllers are also pointed out.

This paper presents a 2D teleoperation task at microscales with force feedback. At this scale, two major problems arise while performing manipulation tasks: the lack of 3D real time visual feedback, and the difficulty to determine the interaction forces. Therefore, indications must be provided to help the user perform a given task. In this paper, we provide the user with intuitive force feedback, to improve objects' manipulation using a haptic device. Our platform is composed of a tipless beam manipulator, which is deformed when forces are applied to it. These deformations are measured using a laser. The force information we provide to the user is based on the raw beam's deformation measurement, and mechanical properties of the probe. It does provide the operator with indications about the interaction forces. This approach is validated by performing lateral and longitudinal rolling operations using microspheres with a radius of 25-micrometers. 2D rolling telemanipulation at microscale with force feedback is successfully demonstrated.

In this paper, a new tool dedicated to the analysis and the conception of molecules is presented. It is composed of an adaptive simulation software and a haptic device used to interact with molecules while feeling either the forces applied by the environment or the internal forces. The adaptive articulated body algorithm allows fast simulations of complex flexible molecules. To handle the coupling with the force feedback device, two different control schemes designed for nanoscale applications and providing high transparency rendering are proposed and compared.

Applications of the conventional atomic force microscope (AFM) succeeded in manipulating nanoparticles, nanowires or nanotubes by widely used pushing or pulling operations on a single plane. However, pick-and-place nanomanipulation is still a challenge in the air. In this paper, a modified AFM, called three-dimensional (3D) manipulation force microscope (3DMFM) was developed, aiming to achieve the pick-and-place in the air. This system mainly consists of two microcantilevers and each is quipped with a nanopositioning device and an optical lever, constructing a nanotweezer with capabilities of picking and releasing nanoobjects with force sensing. Before the 3D manipulation, one of the cantilevers is employed to position nanoobjects and locate the tip of another cantilever by image scanning, then these two cantilevers fit together as a nanotweezer to grasp, transport and place the nanoobjects with real-time force sensing. In pick-and-place experiments, silicon nanowires (SiNMs) with different diameters were manipulated and 3D nanowire crosses were achieved. 3D nanomanipulation and nanoassembly in the air could become feasible through the newly developed 3DMFM.

Three-dimensional (3-D) automated micromanipulation at scale of several micrometers using a nanotip gripper is presented. The gripper is constructed from protrudent tips of two independently actuated atomic force microscope (AFM) cantilevers and each cantilever. A protocol allows these two cantilevers to form a gripper for grasping and releasing the microspheres to target positions without obstacle of adhesive forces in air. For grasping, amplitude feedback from the dithering cantilevers is employed to locate the grasping points by laterally scanning the side of the microspheres. Real time force sensing is used to monitor the whole process of the pick-and-place with steps of pickup, transport and release. For trajectory planning, an algorithm based on the shortest path solution is used to obtained 3-D micropatterns with high efficiencies. In experiments, microspheres with diameters from 3 ¿m to 4 ¿m were manipulated and 3-D micropyramids with two layers were achieved. 3-D micromanipulation and 3-D microassembly at the scale of several microns to submicron could become feasible through the newly developed nanotip gripper.

Laser induced convection flows is a new and promising method to achieve better manipulation of mesoscale objects (above 1 mum and below 500 mum) in a liquid medium. The temperature gradient created by laser absorption generates natural and thermocapillary (or Marangoni) convection flows. These flows are used to perform the manipulation itself. In this paper, we demonstrate for the first time that large and heavy particles can be dragged using the Marangoni convection flows. Experiments based on these phenomena show that fast and accurate underwater micromanipulation of particles up to 280 mum is possible using only a convergent 1480 nm laser beam.

Both the extent and accuracy of force application in atomic force microscope (AFM) nanomanipulation are significantly limited by the nonlinearity of the commonly used optical lever with a nonlinear position-sensitive detector (PSD). In order to compensate the nonlinearity of the optical lever, a nonlinear calibration method is presented. This method applies the nonlinear curve fit to a full-range position-voltage response of the photodiode, obtaining a continuous function of its voltage-related sensitivity. Thus, Interaction forces can be defined as integrals of this sensitivity function between any two responses of photodiode voltage outputs, instead of rough transformation with a single conversion factor. The lateral position-voltage response of the photodiode, a universally acknowledged puzzle, was directly characterized by an accurately calibrated force sensor composed of a tippless piezoresistive force sensor, regardless of any knowledge of the cantilevers and laser measuring system. Experiments using a rectangular cantilever (normal force constant 0.24 N/m) demonstrated that the proposed nonlinear calibration method restrained the sensitivity error of normal position-voltage responses to 3.6% and extended the force application range.

This paper presents a new method for a six degrees of freedom haptic feedback in molecular docking simulations in virtual reality. The proposed method allows real-time haptic interaction even in the case of classical molecular simulation which implies notoriously long computation time. These simulations are classically used by the pharmaceutical industry (Sanofi-Aventis) and are based on the energetic description of atoms to estimate the interaction between a ligand and a protein. The haptic control scheme uses wave variables for a stable and robust teleoperation, and a transcription of the calculated energy into forces and torques for the manipulation of a flexible ligand around the binding site of a flexible molecule. This method can then be used with any energetic force field using a minimization process, thus avoiding the fastidious optimization of molecular simulation programs.

This paper presents a new method allowing haptic feedback in molecular docking simulations using a minimization process. These simulations, classically used by the pharmaceutical industry, for example Sanofl-Aventis, are based on the energy description of atoms to estimate the interactions between a ligand and a protein. The main drawback is that forces and torques cannot be calculated by the means of a simple derivation. The proposed method is to locally build an energy model, the shape of which is correctly predetermined, depending on parameters to be estimated, themselves functions of the energy of the interatomic interactions and of the displacement of the haptic device. The interaction's wrench can be obtained using an analytic derivation of the energy model. The molecular simulator does not need to be optimized or modified, only the calculated interaction energy is used to build a model which will interact with the haptic device. This new method can then be used with any force field using a minimization process, ensuring stable manipulation, and a low- force dynamic, therefore allowing comprehensive and stable force feedback.

This paper deals with the design of a micro-force sensing device for biomechanical characterization of biological samples. This device combines (SPM) techniques and advanced robotics approaches and allows to carry out in vitro prolonged observations as well as biomechanical characterization experiments. Elastic properties of biological samples are reflected to the macroscale during the mechanical characterization process by means of a haptic sensing device. Non-linear elasticity theory formalism is used in order to achieve realistic elastic rendering. Mechanical characterization experiments are conducted on human tumoral Epithilial Hella cells in order to demonstrate the efficiency and viability of the proposed system

The development of a mechanical force sensing device system based on force/vision feedback control for exploring in vitro the contact mechanics of human adherent cervix Epithelial Hela cells is presented in this paper. The design of the prototype combines scanning probe microscopy (SPM) techniques with advanced robotics approaches. Some important issues in the design process, such as in vitro environment constraints and calibration of the force sensing probe are also addressed in this paper. The system is then used for accurate and non-destructive mechanical characterization based on soft contact interactions on biological samples. Finally, some mechanical properties of the studied biological samples are estimated using two appropriate models describing the contact mechanism taking into account adhesion forces

This paper presents a remote handling force feedback coupling for micromanipulation systems. In the literature, the most generally used coupling mode is 'force position'. This kind of control scheme is not portable and instability is an often occurring problem. The coupling scheme proposed in this paper is based on the passivity considerations on the teleoperated systems. It is independent of the used haptic interface and the manipulator and unconditionally stable regarding scaling ratios. It is experimented using the LRP's (Laboratoire de Robotique de Paris) micromanipulator, which is based on AFM architecture and uses the adhesion forces for pickup and release tasks. A comparison between the force position coupling and proposed coupling is presented. Experimental results show the good performances in terms of stability.

This paper presents our work in developing an autonomous micromanipulation system. The originality of our system is that it takes advantage of adhesion to grip micro-objects by using a single fingered gripper. This is in fact a tipless cantilever previously designed for atomic force microscopy applications. We describe vision techniques employed to process images provided by an optical microscope, allowing to position accurately the end-effector for a gripping task. A theoretical study of the direct force measurement device and an experimental validation show how we can improve the measurement of impact and contact forces. Then we explain the strategy used to bring the gripper into contact with the object, based on force control and kinematic redundancy. Finally, a simplified model of the release task is proposed in order to determine conditions that allow to roll the object, and then to place it with precision.

A micro-manipulation method based on adhesion forces and dynamic effects has been proposed in our previous papers. A prototype manipulator, called [mu]MAD, has been constructed and successfully experimented. This paper describes the advanced capabilities of [mu]MAD, especially two new interesting applications: sorting of micro-objects and mechanical characterizations.

Describes a dynamical strategy for releasing micro objects picked-up by means of adhesion forces. While sticking effects are used in order to capture an object by adequately choosing a high surface energy constitutive material for the end-effector, these same effects handicap considerably the release. We propose to take advantage of the inertial effects of both the end-effector and the manipulated object to overbalance adhesion forces and to achieve the release. Simulations show that for this purpose, accelerations as high as 10/sup 5/ m/s/sup 2/ are needed. Successful manipulation of a 40 /spl mu/m radius glass sphere is experimented.

Compliant mechanisms are particularly well to applications with small range of motions such as in microsystems. This paper focuses on the design of compliant mechanisms based on a compliant beam network for mechanical transmission problems. The aim is to define an set of mechanical structures subject to technological constraints, by integrating fabrication process particularities which here lead to bending efforts and preload. Here a method is described to determine by an optimization process adapted structures to mechanical tasks such as rigidification tasks or motion transmission. Basic examples illustrate this method.

In this paper, we propose an original design for a manipulation system of rigid micro-objects by adhesion (50 /spl mu/m/spl sim/200 /spl mu/m) in open air. The design is based on a precise analysis of the mechanical conditions for manipulation by adhesion. Simulations using a dynamic model of a canonical manipulation (capture and release of micro-objects) have shown that it exists an "end-effector initial acceleration" windows for which manipulations are possible. The end-effector we have developed integrates a highly sensitive contact sensor and two piezo-accelerators (/spl sim/10/sup 6/ m.s/sup -2/). Successful manipulations by adhesion of silicon chips with a gold coated piezoresistive silicon cantilever have been carried out and are presented as conclusion of this paper.

This paper reports current researches lead at Laboratoire de Robotique de Paris in the collective microrobotics field. The approach is based on multiagents behaviours adapted to microrobotics framework. A global problem is described concerning the management of colonies of microrobots and several elementary problems are brought out from it. Then, a solution to one of this elementary problems is presented. This solution explains how a colony of microrobots with very reactive individual behaviour can generate a path between two points (one of them is unknown at initial time). We avoid, with this solution, the problem of marking the environment in a microrobotics field.