Accurate segmentation of neurons in 3D fluorescence microscopy images is essential for advancing neuroscience. Prevalent methods split a volume into patches and process each patch separately due to computational resource limitations. However, they fail to capture global neuronal morphology across multiple patches, which results in discontinuous segmentation and poses a challenge for subsequent neuronal reconstruction. In this paper, we propose a dual U-Net architecture termed "Glancing Beyond Patch" Network (GBP-Net) to incorporate contextual information into segmentation. Specifically, GBP-Net encodes contextual and high-resolution information using two U-Nets, respectively, and facilitates their integration through a cross-scale context module (CSCM) and a cross-resolution fusion module (CRFM). CSCM utilizes a cross-attention mechanism to enable interaction of features from diverse fields of view after encoders, and CRFM employs Mamba to adaptively fuse high-resolution features after decoders. These two modules complement each other at different levels, enabling to capture both global neuronal structures and fine-grained details. Additionally, the proposed cross-network loss guides to focus on challenging samples by penalizing misclassified voxels from both networks, which further promotes the exploitation of contextual information. Experimental results on three datasets demonstrate that our method outperforms other advanced segmentation methods while maintaining computational efficiency. It keeps the global structure of neurons and achieves the highest F1 scores.

Accurate segmentation of organelles in electron microscopy (EM) volumes is essential for understanding intracellular organization. While promising, deep learning-based methods could be unstable and unreliable without sufficient annotations. Masked image modeling (MIM), a powerful pretraining technique, has proven effective in enhancing segmentation by extracting meaningful representations from large-scale unlabeled data. However, random masking strategies in classic MIMs could overlook the unique structural patterns of organelles and the spatial redundancy inherent in EM volumes, thus limiting pretraining efficiency. To address this issue, we propose OrgMIM, a dual-branch MIM framework that integrates complementary masking strategies to capture critical subcellular semantics and learn organelle-specific representations from EM data. Specifically, one branch is guided by static structural priors, leveraging visual foundation models to generate affinity maps that indicate organelle membranes as masking candidates. The other is driven by dynamic reconstruction feedback, using a self-guidance mechanism to compute average loss maps that highlight intricate organelle patterns for heuristic masking. Moreover, cross-branch consistency regularization is introduced for reliable representation learning across sparse semantic contexts. To support large-scale pretraining, we construct IsoOrg-1K, the first organelle-centric 3D EM dataset, comprising 928 informative volumes and over 120 billion voxels. Extensive evaluations on three public EM datasets with varied resolutions and appearances validate the superior performance of OrgMIM. Notably, OrgMIM pretraining on IsoOrg-1K boosts mIoU by 28.78% over training from scratch on the CMCC dataset with a Transformer-based model. All datasets, source codes, and pretrained weights are available at https://github.com/yanchaoz/OrgMIM.

Despite advances in ultrathin cutting, serial sections in electron microscopy (EM) still exhibit noticeable anisotropy, with much lower z-axis resolution compared with the other two axes. As a result, the imaged biovolume suffers from low connectivity smoothness in contextual structures, which makes the subcellular ultrastructural segmentation challenging. The recent 2.5D hybrid convolutions allow the direct learning of asymmetric semantics from anisotropic features. However, plain representations without the isotropic scale prior limit the performance of the upper bound. This paper presents a novel framework, referred to as ReIsoSeg, which aims at incorporating an isotropic scaling prior into anisotropic biovolumes. More precisely, ReIsoSeg consists of an anisotropic primary encoder, a pseudo-isotropic auxiliary module, and a weight-shared decoder. The auxiliary module implicitly deforms the anisotropic features from the primary encoder to align with the isotropic prior. The re-isotropic loss squeezes the pseudo-isotropic representations into the anisotropic space to reuse the anisotropic labels. The shared decoder ensures that the outputs of the anisotropic encoder converge towards the isotropic representations. During the inference process, the auxiliary module is excluded. Comprehensive experiments were conducted on the AC3/AC4, CREMI, and MitoEM subcellular ultrastructure datasets. The obtained results demonstrate the high performance of the proposed ReIsoSeg.

With an explosion in the uptake of volume electron microscopy (vEM) across neuroscience and fast-paced advances in imaging protocols, it is timely to introduce general-purpose automation for newly generated large-scale vEM datasets. Recent vision foundation models (e.g., SAM) set a new benchmark for the generalization of 2D segmentation. However, SAM has difficulty handling neurons that are densely packed into 3D volumes. To overcome this obstacle, we consider solutions from both data and model aspects. In terms of data, we introduce a data engine to optimize manual labeling, including (i) human-in-the-loop data cleansing and (ii) model-in-the-loop data unification. In terms of model, we present the SAEM2-SAvEM3with strategies, including (i) auxiliary learning, which predicts complementary representations for SAM masks and improves performance on dense instances; (ii) full-stage distillation, which integrates ViT embeddings into a 3D U-Net, achieves 2D-to-3D lifting and model slimming at the same time; and (iii) prompt-based graph partitioning, which reuses SAM prompts to assign weights of nodes and edges in the oversegmentation graph. According to evaluations of dense and large-scale sparse neurons, the out-of-distribution performance of our pretrained-distilled models is on par with the state-of-the-art supervised and semi-supervised methods. The overall pipeline provides a possible general-purpose solution for 3D neuron reconstruction in any new vEM data. Our code are available at https://github.com/JackieZhai/SAvEM3.

Accurate and robust segmentation of mitochondria from electron microscopy images plays a critical role in understanding cellular functions. While effective, learning-based approaches require vast quantities of expert annotations. Manual efforts can be alleviated by transferring the knowledge learned from the source domain to unseen target domains, as known as unsupervised domain adaptation. In this work, we propose a two-stage pipeline for cross-dataset mitochondria segmentation, aiming to mitigate domain shift in a coarse-to-fine manner. In the first stage, we integrate the style transfer block and segmentation network into an end-to-end image alignment framework. Specifically, patch-wise contrastive learning is employed to guarantee the semantic fidelity of mitochondria, providing more reliable translated images for the segmentation network. In the second stage, a novel online self-training network updated with target images and co-evolving pseudo labels is proposed to fine-tune the segmentation network trained beforehand. Furthermore, to effectively utilize unlabeled data, consistency regularization is introduced in both stages to enforce stable predictions under various perturbations. Experimental results on public datasets demonstrate that the proposed approach outperforms the existing methods by a large margin in bidirectional adaptation for mitochondria segmentation and achieves comparable results with supervised methods.