This work focuses on identifying the best performing loss functions for denoising scanning electron microscopy (SEM) images. Recent studies on the impact and effect of loss functions in image denoising are multiplying, particularly in the fields of low-light and microscopic imaging. Most studies have focused on hybrid loss functions, which are a mixture of multiple loss functions, and have shown their significance. However, manual experiments with different loss functions to build a hybrid function are generally very time-consuming. Thus, we propose a framework named LFRanker (Loss Function Ranker) that automatically and iteratively experiments loss function from a given set. LFRanker automatically calculates combinations using set theory from a given set of loss functions. The findings indicated significant variations across the different assessment criteria (such as PSNR, SSIM and keypoints preservation). In this work, we focus on SEM image denoising, which may be subject to reconstruction in the next step. Thus, it is necessary to preserve the maximum keypoints for point-cloud reconstruction during denoising, along with high quality. Based on these criteria, we found a hybrid loss function, combining LMSE, LMAE, LPERand LSIMperforms best. Finally, LFRanker demonstrates the ability to automate manual experimental efforts in identifying appropriate loss functions for denoising.

International audience

The paper deals with the analysis of noise in Scanning Electron Microscopy (SEM) images with respect to the dwell time, i.e. the time to acquire a pixel. Identifying the type and parameters of noise in images is crucial for denoising, as most denoising methods rely on prior knowledge of noise. Therefore, analyzing image noise is a significant undertaking and a window-wise based image segmentation of a single image has been proposed to find the smoothest region of the image. In fact, automating the identification of the flat region, noise type estimation, and statistical verification are essential aspects of this study. All the experiments were performed on a set of images obtained from a Zeiss Auriga field effect SEM and . It is shown that the noise types and levels change with respect to dwell time. The fast (dwell time 100–500 nanoseconds) and faster scan speed (dwell time less than 100 nanoseconds) changes the noise type to Gamma distribution, while for the slow one (few microseconds or more than 500 nanoseconds) it follows a Gaussian distribution. Moreover, the level of the noise are subject to higher values with fast and very fast scan speeds. The study suggests that utilizing its findings can establish a basis for prior knowledge in deep learning denoisers, which can assist in guiding the process of SEM image denoising.

Positioning of micro-nanoobjects inside a scanning electron microscope (SEM) for manipulation is a key and challenging task to perform. Often it is performed by skilled operators via teleoperation, which is tedious and lacks repeatability. In this paper, rendering this task as an image-guided problem, we present a frequency domain scheme for automatic control of positioning platform movements. The designed controller uses the relative global image motion computed using the frequency spectral information of the images as visual signal and can provide control up to five degrees of freedom. The proposed approach is validated in simulations as well as experimentally using a high-resolution piezo-positioning platform mounted inside a SEM vacuum chamber. The obtained results quantify the performance of the proposed nanopositioning scheme.

Estimation of 3D object position is a crucial step for a variety of robotics and computer vision applications including 3D reconstruction and object manipulation. When working in microscale, new types of visual sensors are used such as Scanning Electron Microscope (SEM). Nowadays, micro- and nanomanipulation tasks, namely components assembly, are performed in teleoperated mode in most of the cases. Measuring object position and orientation is a crucial step towards automatic object handling. Current methods of pose estimation in SEM allow recovering full object movement using its computer-aided design (CAD) model. If the model is not known, most methods allow to estimate only in-plane translations and rotation around camera optical axis. In the literature, SEM is considered as a camera with parallel projection or an affine camera, which means image invariance to z-movement and bas-relief ambiguity. In this paper, authors address the problem of measuring full 3D rotation of the unknown scene for uncalibrated SEM without additional sensors. Rotations are estimated from image triplets by solving a spherical triangle from fundamental matrices only, without need of intrinsic calibration, allowing to avoid parallel projection ambiguities. The presented results, obtained in simulation and on real data, allow validating the proposed scheme.

This paper presents two visual servoing approaches for nanopositioning in a scanning electron microscope (SEM). The first approach uses the total pixel intensities of an image as visual measurements for designing the control law. The positioning error and the platform control are directly linked with the intensity variations. The second approach is a frequency domain method that uses Fourier transform to compute the relative motion between images. In this case, the control law is designed to minimize the error i.e. the 2D motion between current and desired images by controlling the positioning platform movement. Both methods are validated at different experimental conditions for a task of positioning silicon microparts using a piezo-positioning platform. The obtained results demonstrate the efficiency and robustness of the developed methods.

Fast and reliable autofocusing methods are essential for performing automatic nano-objects positioning tasks using a scanning electron microscope (SEM). So far in the literature, various autofocusing algorithms have been proposed utilizing a sharpness measure to compute the best focus. Most of them are based on iterative search approaches; applying the sharpness function over the total range of focus to find an image in-focus. In this paper, a new, fast and direct method of autofocusing has been presented based on the idea of traditional visual servoing to control the focus step using an adaptive gain. The visual control law is validated using a normalized variance sharpness function. The obtained experimental results demonstrate the performance of the proposed autofocusing method in terms of accuracy, speed and robustness.

This paper describes the vision-based methods developed for assembly of complex and solid 3D MEMS (micro electromechanical systems) structures. The microassembly process is based on sequential robotic operations such as planar positioning, gripping, orientation in space and insertion tasks. Each of these microassembly tasks is performed using a pose-based visual control. To be able to control the microassembly process, a 3D model-based tracker is used. This tracker is able to directly provide the 3D micro-object pose at real-time and from only a single view of the scene. The methods proposed in this paper are validated on an automatic assembly of fives silicon microparts of 400 ¿m × 400 ¿m × 100 ¿m on 3-levels. The insertion tolerance (mechanical play) is estimated to 3 ¿m. The precision of this insertion tolerance allows us to obtain solid and complex micro electromechanical structures without any external joining (glue, wending. Promising positioning and orientation accuracies are obtained who can reach 0.3 ¿m in position and 0.2° in orientation.

We improve and adjust usual weak calibration techniques to the case of stereo video microscopes: Harris detector using a simplex optimization method for feature points detection, a "cha" window based ZNSSD correlation for points matching. Images of a pattern made with a water drop covered with nickel fillings are used. The result is validated by constructing 3D view of a micromanipulation work field.

The vision system for micromanipulation and microassembly usually includes at least two cameras allowing top and lateral views of the work field. The top view is used to control the xy position of the microgripper. As for the lateral view, it allows the control of the z position. The paper describes how the lateral camera can be replaced by a virtual one using the trifocal transfer. Then, the work field is set free. The novel view synthesis method used requires only a weak calibration to recover the geometry of the scene, then, it is well suited for the vision at the microscale