Accurate segmentation of neurons in 3D fluorescence microscopy images is essential for advancing neuroscience. Prevalent methods split a volume into patches and process each patch separately due to computational resource limitations. However, they fail to capture global neuronal morphology across multiple patches, which results in discontinuous segmentation and poses a challenge for subsequent neuronal reconstruction. In this paper, we propose a dual U-Net architecture termed "Glancing Beyond Patch" Network (GBP-Net) to incorporate contextual information into segmentation. Specifically, GBP-Net encodes contextual and high-resolution information using two U-Nets, respectively, and facilitates their integration through a cross-scale context module (CSCM) and a cross-resolution fusion module (CRFM). CSCM utilizes a cross-attention mechanism to enable interaction of features from diverse fields of view after encoders, and CRFM employs Mamba to adaptively fuse high-resolution features after decoders. These two modules complement each other at different levels, enabling to capture both global neuronal structures and fine-grained details. Additionally, the proposed cross-network loss guides to focus on challenging samples by penalizing misclassified voxels from both networks, which further promotes the exploitation of contextual information. Experimental results on three datasets demonstrate that our method outperforms other advanced segmentation methods while maintaining computational efficiency. It keeps the global structure of neurons and achieves the highest F1 scores.

Superpixel aggregation is a powerful tool for automated neuron segmentation from electron microscopy (EM) volume. However, existing graph partitioning methods for superpixel aggregation still involve two separate stages-model estimation and model solving, and therefore model error is inherent. To address this issue, we integrate the two stages and propose an end-to-end aggregation framework based on deep learning of the minimum cost multicut problem called DeepMulticut. The core challenge lies in differentiating the NP-hard multicut problem, whose constraint number is exponential in the problem size. With this in mind, we resort to relaxing the combinatorial solver-the greedy additive edge contraction (GAEC)-to a continuous Soft-GAEC algorithm, whose limit is shown to be the vanilla GAEC. Such relaxation thus allows the DeepMulticut to integrate edge cost estimators, Edge-CNNs, into a differentiable multicut optimization system and allows a decision-oriented loss to feed decision quality back to the Edge-CNNs for adaptive discriminative feature learning. Hence, the model estimators, Edge-CNNs, can be trained to improve partitioning decisions directly while beyond the NP-hardness. Also, we explain the rationale behind the DeepMulticut framework from the perspective of bi-level optimization. Extensive experiments on three public EM datasets demonstrate the effectiveness of the proposed DeepMulticut.

MOTIVATION: The registration of serial section electron microscope images is a critical step in reconstructing biological tissue volumes, and it aims to eliminate complex nonlinear deformations from sectioning and replicate the correct neurite structure. However, due to the inherent properties of biological structures and the challenges posed by section preparation of biological tissues, achieving an accurate registration of serial sections remains a significant challenge. Conventional nonlinear registration techniques, which are effective in eliminating nonlinear deformation, can also eliminate the natural morphological variation of neurites across sections. Additionally, accumulation of registration errors alters the neurite structure. RESULTS: This article proposes a novel method for serial section registration that utilizes an unsupervised optical flow network to measure feature similarity rather than pixel similarity to eliminate nonlinear deformation and achieve pairwise registration between sections. The optical flow network is then employed to estimate and compensate for cumulative registration error, thereby allowing for the reconstruction of the structure of biological tissues. Based on the novel serial section registration method, a serial split technique is proposed for long-serial sections. Experimental results demonstrate that the state-of-the-art method proposed here effectively improves the spatial continuity of serial sections, leading to more accurate registration and improved reconstruction of the structure of biological tissues. AVAILABILITY AND IMPLEMENTATION: The source code and data are available at https://github.com/TongXin-CASIA/EFSR.

BACKGROUND: Nanoscale connectomics, which aims to map the fine connections between neurons with synaptic-level detail, has attracted increasing attention in recent years. Currently, the automated reconstruction algorithms in electron microscope volumes are in great demand. Most existing reconstruction methodologies for cellular and subcellular structures are independent, and exploring the inter-relationships between structures will contribute to image analysis. The primary goal of this research is to construct a joint optimization framework to improve the accuracy and efficiency of neural structure reconstruction algorithms. RESULTS: In this investigation, we introduce the concept of connectivity consensus between cellular and subcellular structures based on biological domain knowledge for neural structure agglomeration problems. We propose a joint graph partitioning model for solving ultrastructural and neuronal connections to overcome the limitations of connectivity cues at different levels. The advantage of the optimization model is the simultaneous reconstruction of multiple structures in one optimization step. The experimental results on several public datasets demonstrate that the joint optimization model outperforms existing hierarchical agglomeration algorithms. CONCLUSIONS: We present a joint optimization model by connectivity consensus to solve the neural structure agglomeration problem and demonstrate its superiority to existing methods. The intention of introducing connectivity consensus between different structures is to build a suitable optimization model that makes the reconstruction goals more consistent with biological plausible and domain knowledge. This idea can inspire other researchers to optimize existing reconstruction algorithms and other areas of biological data analysis.

Due to the limitation of the satellite attitude measurement accuracy and the system servo control error of the payload scanning mechanism, an optimal use of Micro-Wave Radiation Imager (MWRI) observations requires high geolocation accuracy. In the operational system, the MWRI geolocation accuracy reaches 1 pixel, and there still exists room for improvement. In this article, we improve upon the coastline inflection point method (CIM) and propose to assign the accurate correspondence by employing a nonrigid point set registration method. First, the method identifies a set of latent variables to recognize outliers and then applies nonparametric geometric constraints to the correspondence asa prioridistribution. Second, the maximuma posteriori(MAP) estimation is applied by the expectation–maximization (EM) algorithm to obtain correct inliers. The comparison with other methods demonstrates that the proposed method can provide more accurate estimation of geolocation bias. In addition, the pixel error and changes in spacecraft attitude with the long-term geolocation data in FY-3C MWRI before and after correction were analyzed during the period from April 1 to August 30, 2018. The results have shown that the geolocation errors are reduced from [0.50, 0.60] pixels to [0.20, 0.33] pixels in the along- and cross-track directions after the attitude correction. In addition, the reduction of the standard deviation shows that the geolocation quality of MWRI is improved.

Known as input in the numerical weather prediction (NWP) models, microwave radiation imager (MWRI) data have been widely distributed to the user community. Nevertheless, the current operational geolocation accuracy is still on the pixel scale due to the presence of geolocation uncertainty. In this letter, we propose a new method to estimate the geolocation errors in MWRI data. Compared to the traditional coastline inflection method (CIM), the proposed method has two innovations. First, we establish a surface fitting interpolation model by involving more observations to detect the coastline. Second, we employ the iterative closest point (ICP) algorithm to determine the correspondences between the detected coastline and the actual coastline. Simulated experimental results demonstrate that the proposed method can provide a more accurate geolocation error estimation than the CIM. By applying our method, we have processed an MWRI data set from January 1 to February 28 in 2016. The experimental results have shown that the operational FY-3C MWRI geolocation errors are 0.4813 and 0.4909 pixels in the along-track and cross-track directions, respectively, which can be significantly reduced to 0.1299 and 0.1497 pixels after the attitude correction. It means that the geolocation accuracy has an average improvement up to 70%.

The microwave radiation imager (MWRI) located onboard the FengYun-3C (FY-3C) satellite provides a considerable amount of critical information for numerical weather predictions. Obtaining accurate geolocation results from the FY-3C MWRI data is of great importance. In this letter, we improve the traditional coastline inflection method (CIM) and propose an$\ell _{0}$sparse approximation model for geolocation error estimation and correction. Specifically, we propose using the jump point of the step function to estimate the true coastline point. This approach can characterize the geolocation errors more accurately than the CIM, which further improves the geolocation accuracy. In the theoretical part, we provide a complete solution to obtain the step function through an iterative blind deconvolution. For a practical use, we demonstrate the effectiveness of the proposed method for geolocation error estimation through quantitative results obtained on the FY-3C MWRI data. The experimental results show that the proposed method can achieve an improvement of up to 33.33% in the standard deviation of geolocation errors (approximately 0.00030) compared to the traditional CIM (approximately 0.00045). Furthermore, we also apply the proposed method to the FY-3C satellite and improve the geolocation accuracy of the MWRI data through geolocation error correction.

Image registration of serial electron microscopy (EM) sections is a feasible way to reveal 3D structure of the biological tissue. However, the image registration proves difficult, as it is hard to find reliable correspondences between the adjacent sections and the section distortion may occur during the sample preparation. In this paper, we propose a non-linear image registration method for serial EM sections, which is composed of pairwise correspondences extraction, correspondences position adjustment and image warping. The proposed method is highly automatic, and retains the morphology of the original electron microscopic images as much as possible. We demonstrate that our method outperforms the state-of-the-art approaches on several datasets of serial EM sections images including a synthetic test case.