Positioning of micro-nanoobjects inside a scanning electron microscope (SEM) for manipulation is a key and challenging task to perform. Often it is performed by skilled operators via teleoperation, which is tedious and lacks repeatability. In this paper, rendering this task as an image-guided problem, we present a frequency domain scheme for automatic control of positioning platform movements. The designed controller uses the relative global image motion computed using the frequency spectral information of the images as visual signal and can provide control up to five degrees of freedom. The proposed approach is validated in simulations as well as experimentally using a high-resolution piezo-positioning platform mounted inside a SEM vacuum chamber. The obtained results quantify the performance of the proposed nanopositioning scheme.

Estimation of 3D object position is a crucial step for a variety of robotics and computer vision applications including 3D reconstruction and object manipulation. When working in microscale, new types of visual sensors are used such as Scanning Electron Microscope (SEM). Nowadays, micro- and nanomanipulation tasks, namely components assembly, are performed in teleoperated mode in most of the cases. Measuring object position and orientation is a crucial step towards automatic object handling. Current methods of pose estimation in SEM allow recovering full object movement using its computer-aided design (CAD) model. If the model is not known, most methods allow to estimate only in-plane translations and rotation around camera optical axis. In the literature, SEM is considered as a camera with parallel projection or an affine camera, which means image invariance to z-movement and bas-relief ambiguity. In this paper, authors address the problem of measuring full 3D rotation of the unknown scene for uncalibrated SEM without additional sensors. Rotations are estimated from image triplets by solving a spherical triangle from fundamental matrices only, without need of intrinsic calibration, allowing to avoid parallel projection ambiguities. The presented results, obtained in simulation and on real data, allow validating the proposed scheme.

Recent studies show that the handling of thin wafers in the photovoltaic industry can lead to unacceptable yields due to cell scratching and breaking. This paper presents the concept and design of a novel modular conveyor that is intended for handling planar fragile objects at a high speed without contact. Each element of the conveyor is a square block that is able to generate tilted air jets that lift and push the object in a single direction. Various handling functions can thus be achieved by the assembly of several blocks. To manage the complexity of assembling systems composed of hundreds of blocks, an automatized design methodology is proposed. This process gives the best topology of the conveyor that meets expected specifications such as trajectory, speed, and travel time. The optimization relies on the physical model of the modular system describing the motion of the object pushed by directed air jets. Experimental comparisons show that the simulation predicts accurately the motion of a glass wafer according to the arrangement of the blocks and the volume of air flow. A maximal speed of 0.3 m/s was experimentally observed and, on larger simulated conveyors, the speed of the wafer could theoretically reach 2.9 m/s.

Recently, high resolution visual methods based on direct-phase measurement of periodic patterns has been proposed with successful applications to microrobotics. This paper proposes a new implementation of direct-phase measurement methods to achieve 3-DoF (degrees of freedom) visual servoing. The proposed algorithm relies on a single frequency tracking rather than a complete 2D discrete Fourier transform that was required in previous works. The method does not require any calibration step and has many advantages such as high subpixelic resolution, high robustness and short computation time. Several experimental validations (in favorable and unfavorable conditions of use) were performed using a XYθ microrobotic platform. The obtained results demonstrate the efficiency of the frequency-based controller, this in term of accuracy (micrometric error), convergence rate (30 iterations in nominal conditions) and robustness.

This paper presents preliminary results of a new robotic wrist for minimally invasive surgery. This wrist is a high-dexterity miniature robot, able to provide simultaneously the grasping/cutting (1-DoF) and rotations capabilities with 3-DoF. The grasping function is insured by the folding of the top platform of a parallel structure. The grasping capability of the wrist is part of the mechanical structure itself and can be fully controlled by external actuators. In order to validate this original approach, an experimental prototype has been fabricated using 3D printing technology at a larger scale. The inverse kinematic model has been developed and the workspace analysis was accomplished to assess the capabilities of the realized system. Finally, experimental tests have been also carried out for validating the proposed structure.

In the photovoltaic solar cell industry as in the semiconductor industry, efforts to reduce the thickness of silicon wafers are in progress. Wafer damage and breakage during handling can lead to unacceptable yields and alternative solutions have to be proposed. This paper presents a modular contact-free wafer handling system that responds to the industrial requirements in terms of throughput and flexibility. The system is based on simple unidirectional modules that can be assembled together to form the desired trajectory. A complete and accurate physical model of the modular system describing the motion of the wafer transported by directed air-jets is proposed. A decentralized control at the block level is realized to damp the object motion. The experimental results show a great reduction of the response time compared to free motion and a standard deviation of the servo error below the millimeter. In addition, simulations show that a 150 mm wafer could reach a speed up to 2.9 m/s on large conveyors.

This paper presents two visual servoing approaches for nanopositioning in a scanning electron microscope (SEM). The first approach uses the total pixel intensities of an image as visual measurements for designing the control law. The positioning error and the platform control are directly linked with the intensity variations. The second approach is a frequency domain method that uses Fourier transform to compute the relative motion between images. In this case, the control law is designed to minimize the error i.e. the 2D motion between current and desired images by controlling the positioning platform movement. Both methods are validated at different experimental conditions for a task of positioning silicon microparts using a piezo-positioning platform. The obtained results demonstrate the efficiency and robustness of the developed methods.

Fast and reliable autofocusing methods are essential for performing automatic nano-objects positioning tasks using a scanning electron microscope (SEM). So far in the literature, various autofocusing algorithms have been proposed utilizing a sharpness measure to compute the best focus. Most of them are based on iterative search approaches; applying the sharpness function over the total range of focus to find an image in-focus. In this paper, a new, fast and direct method of autofocusing has been presented based on the idea of traditional visual servoing to control the focus step using an adaptive gain. The visual control law is validated using a normalized variance sharpness function. The obtained experimental results demonstrate the performance of the proposed autofocusing method in terms of accuracy, speed and robustness.

This paper demonstrates accurate micropositioning scheme based on a direct visual servoing process. This technique uses only the pure image signal (photometric information) to design the control law. With respect to traditional visual servoing approaches that use geometric visual features (points, lines ...), the visual features used in the control law are the pixel intensity. The proposed approach was tested in term of accuracy and robustness in several experimental conditions. The obtained results have demonstrated a good behavior of the control law and very good positioning accuracy. The obtained accuracies are estimated to 14 nm, 89 nm, and 0.001 degrees in the x, y and θ axes of positioning platform, respectively.

Many industries require contactless transport and positioning of delicate or clean objects such as silicon wafers, glass sheets, solar cell or flat foodstuffs. The authors have presented a new form of contactless distributed manipulation using induced air flow. Previous works concerned the evaluation of the maximal velocity of transported objects and one degree-of-freedom position control of objects. This paper introduces an analytic model of the velocity field of the induced air flow according to the spatial configuration of vertical air jets. Then two degrees-of-freedom position control is investigated by exploiting the linearity property of the model. Finally the model is validated under closed-loop control and the performances of the position control are evaluated.

This paper introduces a new aerodynamic-traction principle to handle delicate and clean products, such as silicon wafers, glass sheets, or flat foodstuffs. The product is carried on a thin air cushion and transported along the system by induced airflows. This induced airflow is an indirect effect of strong vertical air jets that pull the surrounding fluid. This paper provides a qualitative explanation of the operating principles and a description of the experimental device. First experimental results with active control are presented. The maximum velocity and acceleration that can be obtained for the considered device geometry meet the requirements for industrial applications.

This paper presents a distributed control architecture to perform part recognition and closed-loop control of a distributed manipulation device. This architecture is based on decentralized cells able to communicate with their four neighbors thanks to peer-to-peer links. Various original algorithms are proposed to reconstruct, recognize and convey the object levitating on a new contactless distributed manipulation device. Experimental results show that each algorithm does a good job for itself and that all the algorithms together succeed in sorting and conveying the objects to their final destination. In the future, this architecture may be used to control MEMS-arrayed manipulation surfaces in order to develop Smart Surfaces, for conveying, fine positioning and sorting of very small parts for micro-systems assembly lines.

In this paper, a new contactless conveyor system based on an original aerodynamic traction principle is described and experimented. This device is able to convey without any contact flat objects like silicon wafer, glass sheets or foodstufff thanks to an air cushion and induced air flows. A model of the system is established and the identification of the parameters is carried out. A closed-loop control is proposed for one dimension position control and position tracking. The PID-controller gives good performances for different reference signals. Its robustness to object change and perturbation rejection are also tested.

Recently, a great deal of interest has been developed in learning in multi-agent systems to achieve decentralized control. Machine learning is a popular approach to find controllers that are tailored exactly to the system without any prior model. In this paper, we propose a semi-decentralized reinforcement learning control approach in order to position and convey an object on a contact-free MEMS-based distributed-manipulation system. The experimental results validate the semi-decentralized reinforcement learning method as a way to design control laws for such distributed systems.

This paper describes the vision-based methods developed for assembly of complex and solid 3D MEMS (micro electromechanical systems) structures. The microassembly process is based on sequential robotic operations such as planar positioning, gripping, orientation in space and insertion tasks. Each of these microassembly tasks is performed using a pose-based visual control. To be able to control the microassembly process, a 3D model-based tracker is used. This tracker is able to directly provide the 3D micro-object pose at real-time and from only a single view of the scene. The methods proposed in this paper are validated on an automatic assembly of fives silicon microparts of 400 ¿m × 400 ¿m × 100 ¿m on 3-levels. The insertion tolerance (mechanical play) is estimated to 3 ¿m. The precision of this insertion tolerance allows us to obtain solid and complex micro electromechanical structures without any external joining (glue, wending. Promising positioning and orientation accuracies are obtained who can reach 0.3 ¿m in position and 0.2° in orientation.

We improve and adjust usual weak calibration techniques to the case of stereo video microscopes: Harris detector using a simplex optimization method for feature points detection, a "cha" window based ZNSSD correlation for points matching. Images of a pattern made with a water drop covered with nickel fillings are used. The result is validated by constructing 3D view of a micromanipulation work field.

Multi-agent systems (MAS) are a field of study of growing interest in a variety of domains such as robotics or distributed controls. The article focuses on decentralized reinforcement learning (RL) in cooperative MAS, where a team of independent learning robots (IL) try to coordinate their individual behavior to reach a coherent joint behavior. We assume that each robot has no information about its teammates' actions. To date, RL approaches for such ILs did not guarantee convergence to the optimal joint policy in scenarios where the coordination is difficult. We report an investigation of existing algorithms for the learning of coordination in cooperative MAS, and suggest a Q-learning extension for ILs, called hysteretic Q-learning. This algorithm does not require any additional communication between robots. Its advantages are showing off and compared to other methods on various applications: bi-matrix games, collaborative ball balancing task and pursuit domain.

The vision system for micromanipulation and microassembly usually includes at least two cameras allowing top and lateral views of the work field. The top view is used to control the xy position of the microgripper. As for the lateral view, it allows the control of the z position. The paper describes how the lateral camera can be replaced by a virtual one using the trifocal transfer. Then, the work field is set free. The novel view synthesis method used requires only a weak calibration to recover the geometry of the scene, then, it is well suited for the vision at the microscale

Reinforcement learning (R-L) is an intuitive way of programming well-suited for use on autonomous robots because it does not need to specify how the task has to be achieved. However, RL remains difficult to implement in realistic domains because of its slowness in convergence. In this paper, we develop a theoretical study of the influence of some RL parameters over the learning speed. We also provide experimental justifications for choosing the reward function and initial Q-values in order to improve RL speed within the context of a goal-directed robot task

This paper presents a new reinforcement learning algorithm named Short Term Model-based learning (STM-Q) able to learn quickly to control stochastic systems. Our objective is to control the positioning of small mechanical parts with a micromanipulation system for microfactory applications. The micropositioning platform is made of three linear actuators that move points to push the object toward a goal position. At the micrometric scale, it is really difficult to model the behaviour of manipulated objects. Moreover, due to the gap between real world and discrete world perceived by the camera, the motion of parts can be seen as a little stochastic. To overcome these problems and design an automated device, we use reinforcement learning methods and develop a new model-based algorithm taking into account the stochastic behaviour of the positioning task.

This paper presents a hybrid control architecture based on subsumption and schemas motors principles in order to achieve complex and cooperative tasks. The control architecture implemented is constituted by a set of independent and elementary behaviors organized in layers of skills. Specific low-level behaviors, called altruistic behaviors and inspired by societies of insects (attractive or repulsive signals), are used to improve the efficiency of the control. Therefore, competitive and cooperative mechanisms are used in a unique hybrid architecture of control to perform a complex box-pushing task by a set of mini-robots. The analysis of an elevated number of simulations allows us to have statistical results (time to complete the task was chosen as performance criteria) which show the existence of an optimal number of robots to achieve the box-pushing task and underline the importance of the use of altruistic behaviors to enhance the cooperative task.

Deals with the definition of safe tasks for displacement missions of a non-holonomic mobile robot in an indoor environment. The goal of this study is to plan actions (path following) as well as observations (local maps), leading the robot to configurations where pertinent features can be sensed, thus allowing the robot's best localization relative to its environment. Firstly, the local map concept is introduced as a set of the best landmarks used for planning and executing robust motion movements. Secondly, a path planning method dealing with uncertainties is proposed, where both uncertainties in localization and in control of a non-holonomic mobile robot are managed. The safeness of the proposed method is due to the mixing between the planning phase and the navigation phase.

Deals with the design of local maps and their associated validity region. A local map is introduced as a set of the best landmarks used for planning and executing robust motions. An automatic local map design is studied for a non-holonomic mobile robot equipped with a ring of telemetric sensors and navigating in an indoor environment. The goal of the work is to define observations and their associated sensing points, which can be both used by a planner and an execution controller. The result is a simplification of the matching process and a robust navigation completion. At execution time local navigation instead of global navigation is performed using local maps in validity region for the robot localization.

This paper addresses the problem of planning a path following, robust with respect to small localization errors, allowing a mobile robot to reach safely its goal. A path planning method dealing with uncertainties is proposed where both uncertainty in localization and in control of a nonholonomic mobile robot are managed. The safeness of this method is due to the combination of the planning phase and the navigation phase by using the same localization and control processes. The localization process realized the multisensor fusion of data provided by a ring of telemetric sensors in combination with a dead-reckoning system. The robustness is due to a stabilizing controller on which the system remains stable when estimated values are used for feedback instead of exact ones. By determining a compact attractive domain bounding the regulation error, we dynamically compute a security margin during the path planning. This security margin ensures the safeness of the planned path.

This paper is mainly concerned with task function planning in the context of mobile robotics. A task function is basically defined as a closed loop regulation mechanism referenced with respect to a coherent set of map primitives called local map. Task function planning is practically carried out in two steps, First, given an environment model and a sensor model, a geometric path is computed from both the intermediate concept of "configuration uncertainty potential" and the list of all the map primitives used by the vehicle to estimate its localization. Such a path represents a compromise between the length, the simplicity of vehicle control and the tracking accuracy. Then, using an efficient data analysis method, a local map sequence is dynamically computed from the previous path. Finally, task function planning consists in attaching to each local map a closed loop regulation mechanism (task function) so that the complete robot mission can be ensured by successively refering to each local map.>