Robotic handling inside the SEM is well known, but mostly a teleoperated task, but automation of in-situ handling would be highly advantageous and could enable serious production and assembly in the SEM. However, reliable techniques who facilitate an automation are still lacking. Especially the positioning along the optical axis of the SEM remains as a major challenge. In this paper, an universal approach addressing this particular issue is proposed, allowing to assist and improve handling and assembly task. The approach exploits typical charging effects in the SEM and derives information about the positions of objects and end-effectors as well as about assembly statuses. Assembly accuracies better than 100nm are achieved and controlled approaching speeds of 10 μm/s.

Piezo-actuated stages have become more and more promising in nanopositioning applications due to the excellent advantages of the fast response time, large mechanical force, and extremely fine resolution. Modeling and control are critical to achieve objectives for high-precision motion. However, piezo-actuated stages themselves suffer from the inherent drawbacks produced by the inherent creep and hysteresis nonlinearities and vibration caused by the lightly damped resonant dynamics, which make modeling and control of such systems challenging. To address these challenges, various techniques have been reported in the literature. This paper surveys and discusses the progresses of different modeling and control approaches for piezo-actuated nanopositioning stages and highlights new opportunities for the extended studies.

Micro- and nanosized objects aligned in specific spatial order are of great interest for applications in photonics and nanoelectronics. In particular, piezo-actuated robotic setups are promising tools to arrange and manipulate these objects individually. However, automated robotic processing on the sub-micron scale remains challenging due to the force scaling laws and the limited possibilities in terms of control. This paper presents the current progress on fully-automated pick-and-place routines of individual colloidal particles using a dedicated dual-probe setup inside a scanning electron microscope. Applying tailored probes in combination with image processing of the visual feedback provided by the microscope allow for complex automation sequences. The limits of the current technique are highlighted and the challenges for automated processing of progressively smaller particles are discussed.

The AFM is a common tool for ultra-precise surface characterization and a standard instrument a variety of research and development disciplines. However, the characterization of three dimensional high-aspect ratio and sidewall structures remains a hardly accomplishable task. Novel exchangeable and customizable scanning probe tips - NanoBits - can be attached to standard AFM cantilevers offering unprecedented freedom in adapting the shape and size of the tips. These NanoBits of few μm size have to be assembled into micro-cartridges. This challenging assembly task is performed inside the SEM by a micro-gripper. A powerful automation framework has been developed facilitating image based automation and visual servoing for this task. Template matching, BLOB-detection, and special SEM-based detection approaches are used to achieve the automated assembly.

This paper presents reproducible pick and place handling of particles within the sub micron range. The handling strategy is based on a robotic dual probe setup that is integrated into a high resolution scanning electron microscope. By purposeful utilization of the predominant adhesive forces on the nanoscale, this setup facilitates the assembly of overall complex arrangements of different nanoparticles using haptic devices. The paper discusses control issues of the setup as well as the advantages and restrictions of the proposed technique.

This paper deals with the nanorobotic and nondestructive assessment of the stiffness properties of nanopaper made of microfibrillated cellulose. Back-calculations of the Young's modulus show the agreement of the newly found results with conventional tensile testing results, therewith proving nanorobotics as a reasonable complement for conventional testing.

In this paper, a nanorobotic system containing a custom-made nanohandling robot, atomic force microscope, scanning electron microscope and focused ion beam is presented. The nanorobotic atomic force microscope system facilitates the analysis of nanostructured surfaces as well as the handling of nanoobjects. Furthermore, a gas injection system is available that enables particle beam-based structuring and deposition of nanomaterials. This combination provides an enabling technology for the hybrid processing, manipulation and characterization of nanomaterials.

Modern processes of micro- and nanofabrication imply several metrology steps on critical dimensions in order to assure and assess device perfomences. Particularly, manufacturing processes of novel disruptive photonic devices and nanoelectronic circuit architectures require improved three dimensional acquisition and visualization techniques for their metrology. Amongst others line width and sidewall roughness are the most important parameters for nanooptical components but the determination of these parameters becomes increasingly demanding, since the continuous shrinkage of these devices demand an even higher lateral resolution of the measurements. The atomic force microscope (AFM) is a common tool for this characterization and a standard instrument for all kinds of research and development disciplines. However, the characterization of three dimensional high-aspect ratio and sidewall structures remains a hardly accomplishable task. Novel exchangeable and customizable scanning probe tips, so-called NanoBits, can be attached to standard AFM cantilevers offering unprecedented freedom in adapting the shape and size of the tips to the surface topology of the specific application. The ultimate goal is the realization of an in-situ exchange of NanoBits within the regular AFM environment. For this, NanoBits have to be provided in a freestanding way, making them accessible for the AFM cantilever. The direct fabrication of such structures is still challenging, hence the robotic preassembly of NanoBits cartridges is proposed, where every cartridge can carry several different NanoBits. Within this paper all components required for the realization of this concept are presented.

This paper presents the usage of a nanorobotic handling technique for the transfer of individual copper-nanowires with diameters between 150nm and 400nm and length of about 10 μm. Handling and assembly are inside the SEM and use electron beam induced deposition as well as focused ion beam milling. The handled nanowires are used to assembly a NEMS device, namely an electrostatic switch design, where the nanowire acts as switching contact. Working principle, reproducibility and specific values of the switches are investigated, as well as the conductivity of the nanowires themself.

The importance of nanoelectronics for the future is well-recognized. Next-generation nanoelectronic technologies, for the usage in intelligent implants, intelligent drugs or even ICs for the coupling of destroyed nerves, are sensitive to dimensional change. Therefore, an appropriate packaging is essential to the success or failure of these technologies. In this paper current work to use DNA as a template for bonding at the nanoscale and for novel nanoelectronic components is presented. Moreover, a method is presented, which enables the handling and manipulation of DNA at dry conditions, thus enabling the feasible usage for industrial purposes as well as for science. For this the necessary steps, starting with the immobilization and choice of useable nanowires, followed by the extraction and separation of these wires, the coarse positioning, the immobilization onto the target substrates as well as a proper fine tuning at the target are presented.

MRI based nano- and microrobotics show good potential for new targeted therapies tackling e.g. cancer. In this paper, a system developed for the propulsion and navigation of small ferromagnetic objects only using clinical MRI systems is evaluated in experiments. The experiments include propulsion of an untethered ferromagnetic object against a pulsatile flow in a pipe system, navigation of an untethered object filled with ferromagnetic nanoparticles around obstacles in a free environment, and the choosing of a branch in a closed flow-less channel system when propelling an untethered ferromagnetic object. The system is found to deal with the tasks efficiently.

This paper presents a nanorobotic approach facilitating the transfer and characterization of individual graphene flakes that are grown by different fabrication techniques. The approach makes use of a nanorobotic atomic force microscope system that is integrated into a high resolution scanning electron microscope and focused ion beam device. This combination is used to perform both, the nanorobotic transfer and the mechanical characterization of the graphene flake allowing to systematically analyze different sample areas and to optimize the fabrication processes. Furthermore, the nanorobotic system enables the reliable pick-and-place handling and processing of graphene flakes to realize more comprehensive analysis steps or even the prototyping of graphene-based devices.

In this paper, a focused ion beam processing technique is presented that facilitates the modification of carbon nanotubes (CNTs) in terms of length, diameter, and orientation. The CNTs are mounted onto an atomic force microscope (AFM) probe by using a nanorobotic microgripper-based pick-and-place handling strategy. Such CNT-enhanced AFM probes are needed for metrology measurements of nanostructures with critical dimensions and high aspect ratios. The complete process of assembly and processing is realized inside a nanorobotic dual beam scanning electron microscope (SEM) and focused ion beam (FIB) machine.

This paper reports the remote handling of microscale objects, between two sites approximately 630 km distant. To manipulate objects less than 10 ¿m, specific equipments such as AFM (Atomic Force Microscope) cantilevers integrated into a SEM (Scanning Electron Microscope) are generally required. Enabling remote access to such a system would benefit any micro/nanoresearcher. However, vision feedback and sensor data of a micromanipulation system are generally limited, hence the implementation of a teleoperation scenario is not straightforward. Specific tools are proposed here for an intuitive manipulation in a wide range of applications. To ensure ease of manipulation, both a 3D virtual representation of the scene and haptic feedback are provided. Force sensor feedback is limited since only two measures are available. In order to extend this information, vision algorithms are developed to estimate the respective positions of the tool and objects, which are then used to calculate the haptic feedback. The stability of the overall scheme is very sensitive to time delays. This requirement is taken into account in vision algorithms and the communication module which transfers the data between the two remote sites. In addition, the proposed robotic control architecture is modular so that the platform can be used for a wide range of applications. First results are obtained on a teleoperation between Paris, France, and Oldenburg, Germany.

Microrobotics is a field of high interest in the recent past. Commonly used imaging modalities in this field include optical microscopes and scanning electron microscopes (SEM). Due to the scanning principle of the SEMs, objects in motion may not be represented in the SEM images the same way as they really are. Movement leads to movement artifacts. This effect strongly limits the possibilities for the use of image based methods for tracking and recognition and it also limits the possible speeds achievable for automated procedures. In this work, a tracking algorithm is proposed which is robust to artifacts and distortions generated by object motion. The algorithm is tested and validated in experiments.

The propulsion of nano-ferromagnetic objects by means of MRI gradients is a promising approach to enable new forms of therapy. In this work, necessary techniques are presented to make this approach work. This includes path planning algorithms working on MRI data, ferromagnetic artifact imaging and a tracking algorithm which delivers position feedback for the microdevice and a propulsion sequence to enable interleaved magnetic propulsion and imaging. Using a dedicated software environment integrating path-planning methods and real-time tracking, a clinical MRI system is adapted to provide this new functionality for potential controlled interventional targeted therapeutic applications. Through MRI-based sensing analysis, this paper aims to propose a framework to plan a robust pathway to enhance the navigation ability to reach deep locations in human body. The proposed approaches are validated with different experiments.

In this paper, a novel nanorobotic strategy for non-destructive and direct electrical characterization of as-grown bundles of single-walled carbon nanotubes (SWCNTs) is presented. For this purpose, test patterns of SWCNT bundles having different diameters are grown on a silicon substrate by chemical vapor deposition. A new design of microstructured four-point-probes is proposed and fabricated allowing for direct contacting of vertically aligned bundles of SWCNTs. A nanorobotic setup is upgraded into a dual endeffector system to achieve good electrical contact between four-point-probe and SWCNT bundle and to perform electrical measurements. First experimental results of non-destructive electrical characterization are presented and discussed.

In this paper, the concept and first results of a novel toolbox for nanoscale characterization are presented. A nanorobotic AFM system is being developed and integrated into a high resolution SEM/FIB system allowing nanoanalysis, -manipulation and -structuring. The compact and modular AFM setup enables probe- as well as sample-scanning and uses self-sensing AFM cantilevers. Image fusion algorithms are developed to merge SEM and AFM information for hybrid analysis of nanoscale objects. A commercial AFM controller is embedded into a special control system architecture that allows for automation of nanomanipulation sequences.

The integration of biomaterials into micro/nano-sensors or micro/nano-systems is expected to improve the properties of such systems or even lead to the development of novel innovative systems. A key problem to be solved beforehand is the development and realization of proper preparation, handling and manipulation methods with respect to an industrial usage. To enable such a usage, the methods have to be automatable, robust to environmental changes as well as feasible in a scanning electron microscope (SEM). According to these points, the target of the presented efforts is to develop these methods for a future design of nanoelectronic parts and to solve packaging problems at the nanoscale. As a consequence, the paper presents a novel concept for the usage of biomaterials, such as DNA and wood fibers/fibrils, for the packaging at the nanoscale. Novel methods for the DNA-handling with an atomic force microscope (AFM) at dry conditions, which can also be used in the vacuum chamber of a SEM will be presented as well as wood fibers/fibrils manipulation methods in the SEM.

Carbon nanotubes (CNTs) are one of the most promising materials for nanoelectronic applications. Before bringing CNTs into large-scale production, a reliable nanorobotic system for automated handling and characterization as well as prototyping of CNT-based components is essential. This paper presents the NanoLab setup, a nanorobotic system that combines specially developed key components such as electrothermal microgrippers and mobile microrobots inside a scanning electron microscope. The working principle and fabrication of mobile microrobots and electrothermal microgripper as well as their interaction and integration is described. Furthermore, the NanoLab is used to explore novel key strategies such as automated locating of CNTs for pick-and-place handling and methods for electrical characterization of CNTs. The results have been achieved within the framework of a European research project where the scientific knowledge will be transfered into an industrial system that will be commercially available for potential customers.

This paper analyzes the disturbance sources acting on nano-assembly systems inside the scanning electron microscope, which complicate the automation of assembly processes in the scanning electron microscope. The influence of intrinsic sources, i.e. thermal drift, actuator offset and end effector vibrations due to actuator movements are examined and approaches are suggested. The electron-beam interaction with the assembly system has been identified as another disturbance source and its impact on automated assembly processes is qualified and quantified. Solutions for disturbance-resistant assembly processes are suggested.

This paper presents the detailed design of a nanohandling robot cell that can work inside an SEM's vacuum chamber and incorporates miniature video microscopes in order to enable fully automated nanohandling and -assembly. The geometrical and mechanical requirements are defined and addressed in a modular implementation. Image processing techniques can be used to recognize and track objects and three dimensional information can be obtained by stereo vision as well as the microscope's focus. To control this highly heterogeneous system, different low-level controllers are used, challenges for cooperatively controlling the multi-robot system are outlined, and high-level automation is discussed.

Automated robot-based nanomanipulation is one of the key challenges of microsystem technology and nano-technology. Controlled, reproducible operation on the nano-scale will enable high-throughput manufacturing of revolutionary products and open up new application fields. The ultimate goal of these research activities is the development of automated nanomanipulation processes to build a bridge between existing precise handling strategies for micro- and nano-scale objects and aspired high-throughput fabrication of micro- and nanosystems. This paper describes our current work on different problems of nanohandling automation, both by nanorobots inside a scanning electron microscope and by an atomic force microscope applied as a nanorobot.

A nanorobotic manipulation setup for the handling and characterization of carbon nanotubes (CNTs) is presented. The nanorobotic setup can be integrated into a scanning electron microscope (SEM) and various endeffectors may be attached to the manipulator either for CNT handling or characterization. The pick-and-place task is carried out by using an electrothermal actuated microgripper, designed for controlled manipulation of nanotubes. The nanotube is picked up from an array of multiwalled carbon nanotubes (MWCNTs) and transferred to the tip of an atomic force microscope (AFM) probe in order to assemble a high-aspect ratio AFM supertip. Another application of the nanorobotic setup considered in this paper is the nondestructive mechanical characterization of CNTs. A piezoresistive AFM probe is used to bend MWCNTs, while the bending force is measured, in order to estimate the Young's modulus of the investigated MWCNTs.

Current research work on the development of an automated nanohandling cell in a scanning electron microscope (SEM) is presented. An experimental setup is shown, in which two micro robots cooperate in the vacuum chamber of an SEM. A client-server control system that can integrate various microrobots and sensors has been developed and evaluated by automatic handling of TEM (transmission electron microscope) lamellae. The robots are controlled in a closed-loop way by using images from several CCD cameras and from the SEM. Algorithms for real-time processing of noisy SEM images have been implemented and tested. Finally, the experiment on automatic handling of TEM lamellae inside an SEM is described

In this paper a new assembly process inside a scanning electron microscope (SEM) is demonstrated. The process includes the separation and handling of a silicon nanowire and its bonding to an atomic force microscope (AFM) probe using electron beam induced deposition (EBiD). EBiD is carried out by using the electron beam of the SEM and a flexible mobile microrobot as transporter of the precursor gas. In addition, experiments are presented to investigate mechanical and chemical properties of the bonded nanowire. Real-time object tracking and image processing are used to measure the force applied to the Si-nanowire. In this way, the load and tension stress on the bonds can be calculated. Finally, energy dispersive X-ray (EDX) analysis is applied to the bonds to estimate their chemical composition

Mobile microrobots with piezo slip-stick actuation and more than one degree of freedom mostly do not have internal pose sensors. One possibility for fast pose estimation with high accuracy is the application of external visual sensors like a video camera in combination with a light microscope or a scanning electron microscope (SEM). In particular, the use of a SEM makes high demands on the image processing. High update rates of the pose data enforce a short image acquisition time of the SEM images. Hence, the image noise increases, because frame averaging or averaging of the detector signal is time consuming. This paper presents a method to calculate the x, y position and the orientation of a micro gripper in a strongly noised SEM image stream with cross-correlation in real-time. To make real-time pose estimation possible, only a region-of-interest (ROI) is correlated with the target pattern. The SEM is almost predestined to work with ROI's, because the scan area of the electron beam can be chosen arbitrarily. At the beginning of the paper, the setup of the used mobile microrobot based nanohandling station was described briefly.

This paper presents newly developed mobile microrobots with high precision in the sub-micrometer range, while offering a macroscopic workspace. The main parts of the mobile microrobots, a mobile platform and a manipulator unit, are explained. The mobile platform's actuation is based on the slip-stick-principle. Because of new arrangement of the components promising characteristics have been investigated. Especially of interest is the low voltage needed to drive the actuators, because it makes possible the integration of batteries. A manipulation unit with actuators based on a combination of electric motors and piezoelectric actuators is described as well. Finally the future effort and outlook is discussed.

Nanohandling robots are the result of increasing research activities at the border between microsystem technology and robotics. Today, robots with dimensions of a few cm/sup 3/ can be developed. Like conventional robots, microrobots represent a complex system that usually contains several different types of actuators and sensors. The measurement of gripping forces is the most important sensor application in micro-handling besides visual servoing to protect the parts from too high surface pressures and thereby damage the object during the assembly process. In this paper, suitable force sensing techniques are described, and a concept for the measurement of gripping forces in nanohandling robotics with AFM cantilever-based sensors is introduced. Furthermore, a haptic interface which allows the operator to feel and control the forces in the micro world is presented.

Microrobots are the result of increasing research activities at the border between microsystem technology and robotics. Today, robots with dimensions of a few cm/sup 3/ can be developed. Like conventional robots, microrobots represent a complex system that usually contains several different types of actuators and sensors. The measurement of gripping forces is the most important sensor application in micro-manipulation besides visual servoing to protect the parts from excessive surface pressures and thereby damage the object during the assembly process. In this paper suitable force sensing methods are described, and a concept for the measurement of gripping forces in micro-robotics with AFM cantilevers is introduced. Furthermore, the signal processing and haptic interface used are presented.

After introducing a flexible microrobot cell that has been developed at the University of Karlsruhe, we present the assembly planning system of this cell, which is tailored to the specific needs of microassembly. Important differences between conventional (macro-) assembly and microassembly are discussed. These must to be taken into account at the assembly planning level to be able to perform an efficient microassembly. A formal description of a planning procedure is presented and planning algorithms are described. The planning system has been tested by an example of an automatic assembly planning of the worldwide smallest commercially available micromotor made by Faulhaber, German. The test results achieved are demonstrated.

There is an increasing interest in performing assembly of microsystems by flexible microrobots. A microrobot-based microassembly desktop station has been developed at the University of Karlsruhe. In this paper, new implementation results of a computer vision based automatic control system of a mobile microrobot with an integrated flexible manipulator are discussed. The operations of the robot manipulator take place under a microscope equipped with a CCD-camera. A major drawback of this system is its lack of depth information. For this reason, the sensor system has been enhanced by a line laser, using laser triangulation. The set-up of the modified sensor system is described. The measuring principle is based on a method called sheet of light triangulation, a variation of standard triangulation. This approach is described in detail, and its advantages and limitations are discussed.

Automation of highly precise microassembly steps will make it easier to realize low cost microsystems. For this, a microrobot-based microassembly desktop station is being developed at the University of Karlsruhe. In this paper, we suggest a common microassembly model for a computer-aided microassembly planning-which is based on geometric reasoning-and discuss its components in detail. The feasibility criteria for the generation of feasible assembly sequences and the optimization criteria for selecting the optimal assembly plan are described. For both these planning steps, the algorithms for an automatic procedure have been developed and implemented. For stations employing several microrobots, a method for decomposition and allocation of an assembly plan is suggested. Finally, we discuss the structure of the station's planning system.

Micromanipulation by microrobots has become an issue of primary importance in industry and biomedicine, since human manual capabilities are restricted to certain tolerances. The manipulation of biological cells or the assembly of a microsystem composed of several microcomponents are good examples. An automated microrobot-based micromanipulation desktop station has been developed at the University of Karlsruhe. The process of assembly takes place in the field of view of a light optical microscope. This paper focuses on motion control problems of the piezo-driven microrobots employed by the station. The ability to adapt itself to the process requirements is of great importance for micromanipulation robots. They must be able to operate in a partially defined environment and to ensure reasonable behaviour in unpredicted situations. A neural control concept based on a reference model is proposed as a solution. It is shown, that the neural controller is able to learn the desired behaviour. It considerably outperforms an analytically designed linear controller in the real environment.

Presents a control system for the intelligent force control of multifingered robot grips. The multilevel system architecture combines both a fuzzy-based adaptation level and a neural-based one with a conventional PID-controller that allows autonomous performance of fine manipulations of an object. Two kinds of fuzzy controllers are presented. They use a decision making logic which expresses the a priori knowledge about the grasp force behaviour inside the friction cones and the necessary reactions regarding the criterion for grip stability. A neural controller, based on a Hooke-Jeeves optimisation approach, has been developed as well. The neural control algorithm is implemented by a three-layered backpropagation neural network. The neural and fuzzy controllers can be used separately or in parallel. In the last case the neural controller can be on-line trained using the input-output information from the fuzzy one. A computer based simulation system for the peg-in-hole insertion task is developed to analyse and to compare the capabilities of both control algorithms. The demands of flexibility and real-time control of the implementation of the control system are suited by a tailorable parallel computer concept. The two basic ideas of the concept are to set up each processing element individually for its application and connect these elements with different communicational methods according to the applicational demands. As this happens before runtime the concept is called static flexibility and is implemented using a new mechanical computer structure.>

The authors present an advanced control concept for microrobotic systems, which is based on the combination of a neural network approach for the adaptation of manipulation parameters and a fuzzy logic approach for the correction of parameter values given to a conventional controller. This multilevel system architecture is suitable for the intelligent control of microrobots that can operate autonomously in changing environments. Typical tasks for these robots are exploration and fine manipulation, which demand intelligent task planning and motion/force control capabilities. The planning component deals with the successive determination of initial manipulation parameters, whereas the neural system performs during manipulation, computing suboptimal grasp forces and learning inference rules used for parameter adjustment.