SUMMARY: We present a fast particle fusion method for particles imaged with single-molecule localization microscopy. The state-of-the-art approach based on all-to-all registration has proven to work well but its computational cost scales unfavorably with the number of particles N, namely as N2. Our method overcomes this problem and achieves a linear scaling of computational cost with N by making use of the Joint Registration of Multiple Point Clouds (JRMPC) method. Straightforward application of JRMPC fails as mostly locally optimal solutions are found. These usually contain several overlapping clusters that each consist of well-aligned particles, but that have different poses. We solve this issue by repeated runs of JRMPC for different initial conditions, followed by a classification step to identify the clusters, and a connection step to link the different clusters obtained for different initializations. In this way a single well-aligned structure is obtained containing the majority of the particles. RESULTS: We achieve reconstructions of experimental DNA-origami datasets consisting of close to 400 particles within only 10 min on a CPU, with an image resolution of 3.2 nm. In addition, we show artifact-free reconstructions of symmetric structures without making any use of the symmetry. We also demonstrate that the method works well for poor data with a low density of labeling and for 3D data. AVAILABILITY AND IMPLEMENTATION: The code is available for download from https://github.com/wexw/Joint-Registration-of-Multiple-Point-Clouds-for-Fast-Particle-Fusion-in-Localization-Microscopy. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

In this paper, we present a multichannel cross-modal fusion algorithm to combine two complementary modalities in electron tomography: X-ray spectroscopy and scanning transmission electron microscopy (STEM). The former reveals compositions with high elemental specificity but low signal-to-noise ratio (SNR), while the latter characterizes the structure with high SNR but little chemical information. We use a multivariate regression to build a cross-modal fusion framework for these two modalities to simultaneously achieve high elemental specificity and high SNR for a target element chosen from the sample under study. Specifically, we first compute three-dimensional tomograms from tilt-series datasets of X-ray and STEM using different reconstruction algorithms. Then, we generate many feature images from each tomogram. Finally, we adopt partial least squares regression to assess the connection between these feature images and the reconstruction of the target element. Based on the simulated and experimental datasets of semiconductor devices, we demonstrate that our algorithm can not only produce continuous edges, homogeneous foreground, and clean background in its element-specific reconstructions but also can more accurately preserve fine structures than state-of-the-art tomography techniques. Moreover, we show that it can deliver results with high fidelity even for X-ray datasets with limited tilts or low counts. This property is highly desired in the semiconductor industry where acquisition time and sample damage are essential.

Bimodal tomography introduces a weighting factor α to incorporate X-ray data into projection images acquired from scanning transmission electron microscope (STEM) for achieving an atom-specific three-dimensional (3D) reconstruction of an object on the nanoscale. Currently its value is chosen by computing reconstructions for a large range of α ∈ (0,1) and comparing them to a hand-segmented ground truth with the mean square error (MSE). Since this is infeasible for an industrial application, in this paper we propose an image quality metric to quantify the quality of tomograms in terms of cross-atomic contamination and noise for selecting the weighting factor without a ground truth. Numerical results demonstrate that our framework can determine close-to-optimal weighting factor within an accuracy of ±0.03. Moreover, approximating the shape of the minimum by a parabola effectively reduces the computational time by 90%.

With electron tomography, we can reconstruct a three-dimensional (3D) volume of a specimen from a series of its two-dimensional (2D) projection images on the nanoscale. In a scanning transmission electron microscope (STEM), element-specific maps and mass-contrast projections can be simultaneously acquired from the X-ray spectrometer and electron detector. The X-ray tomogram has high chemical specificity but low signal-to-noise ratio (SNR), while the electron tomogram has poor compositional information but high SNR. In this paper, we adopt and modify a regression-based image fusion algorithm to combine these two complementary modalities, so that the fused version would maintain both high chemical specificity and high SNR. We demonstrate that our method improves reconstruction quality on an experimental dataset of a core-shell nanoparticle. Specifically, it delivers tomograms with sharper edges and smoother fore- and background, and hence can enable easier and more accurate 3D characterization of such nanostructures.

Anisotropy imaging can be used to image resonance energy transfer between pairs of identical fluorophores and, thus, constitutes a powerful tool for monitoring protein homo-association in living single cells. The requirement for only a single fluorophore significantly simplifies biological preparation and interpretation. We use quantitative methods for the acquisition and image processing of anisotropy data that return the expected error of the anisotropy per pixel based on photon statistics. The analysis methods include calibration procedures and allow for a balance in spatial, anisotropy, and temporal resolution. They are featured here with anisotropy images of fluorescent calibration beads and enhanced green fluorescent protein complexes in live cells.

In this paper, we present a novel method to estimate curvature of iso gray-level surfaces in gray-value images. Our method succeeds where standard isophote curvature estimation methods fail. There is neither a segmentation of the surface needed nor a parametric model assumed. Our estimator works on the orientation field of the surface. This orientation field and a description of local structure is obtained by the Gradient Structure Tensor. The estimated orientation field has discontinuities mod pi. It is mapped via the Knutsson mapping to a continuous representation. The principal curvatures of the surface, a coordinate invariant property, are computed in this mapped representation. From these curvatures, locally the bending energy is computed to describe the surface shape. An extensive evaluation shows that our curvature estimation is robust even in the presence of noise, independent of the scale of the object and furthermore the relative error stays small.

Local curvature represents an important shape parameter of space curves which are well described by differential geometry. We have developed an estimator for local curvature of space curves embedded in n-dimensional (n-D) grey-value images. There is neither a segmentation of the curve needed nor a parametric model assumed. Our estimator works on the orientation field of the space curve. This orientation field and a description of local structure is obtained by the gradient structure tensor. The orientation field has discontinuities; walking around a closed contour yields two such discontinuities in orientation. This field is mapped via the Knutsson mapping to a continuous representation; from a n-D vector to a symmetric n2-D tensor field. The curvature of a space curve, a coordinate invariant property, is computed in this tensor field representation. An extensive evaluation shows that our curvature estimation is unbiased even in the presence of noise, independent of the scale of the object and furthermore the relative error stays small.