Accurate segmentation of organelles in electron microscopy (EM) volumes is essential for understanding intracellular organization. While promising, deep learning-based methods could be unstable and unreliable without sufficient annotations. Masked image modeling (MIM), a powerful pretraining technique, has proven effective in enhancing segmentation by extracting meaningful representations from large-scale unlabeled data. However, random masking strategies in classic MIMs could overlook the unique structural patterns of organelles and the spatial redundancy inherent in EM volumes, thus limiting pretraining efficiency. To address this issue, we propose OrgMIM, a dual-branch MIM framework that integrates complementary masking strategies to capture critical subcellular semantics and learn organelle-specific representations from EM data. Specifically, one branch is guided by static structural priors, leveraging visual foundation models to generate affinity maps that indicate organelle membranes as masking candidates. The other is driven by dynamic reconstruction feedback, using a self-guidance mechanism to compute average loss maps that highlight intricate organelle patterns for heuristic masking. Moreover, cross-branch consistency regularization is introduced for reliable representation learning across sparse semantic contexts. To support large-scale pretraining, we construct IsoOrg-1K, the first organelle-centric 3D EM dataset, comprising 928 informative volumes and over 120 billion voxels. Extensive evaluations on three public EM datasets with varied resolutions and appearances validate the superior performance of OrgMIM. Notably, OrgMIM pretraining on IsoOrg-1K boosts mIoU by 28.78% over training from scratch on the CMCC dataset with a Transformer-based model. All datasets, source codes, and pretrained weights are available at https://github.com/yanchaoz/OrgMIM.

Despite advances in ultrathin cutting, serial sections in electron microscopy (EM) still exhibit noticeable anisotropy, with much lower z-axis resolution compared with the other two axes. As a result, the imaged biovolume suffers from low connectivity smoothness in contextual structures, which makes the subcellular ultrastructural segmentation challenging. The recent 2.5D hybrid convolutions allow the direct learning of asymmetric semantics from anisotropic features. However, plain representations without the isotropic scale prior limit the performance of the upper bound. This paper presents a novel framework, referred to as ReIsoSeg, which aims at incorporating an isotropic scaling prior into anisotropic biovolumes. More precisely, ReIsoSeg consists of an anisotropic primary encoder, a pseudo-isotropic auxiliary module, and a weight-shared decoder. The auxiliary module implicitly deforms the anisotropic features from the primary encoder to align with the isotropic prior. The re-isotropic loss squeezes the pseudo-isotropic representations into the anisotropic space to reuse the anisotropic labels. The shared decoder ensures that the outputs of the anisotropic encoder converge towards the isotropic representations. During the inference process, the auxiliary module is excluded. Comprehensive experiments were conducted on the AC3/AC4, CREMI, and MitoEM subcellular ultrastructure datasets. The obtained results demonstrate the high performance of the proposed ReIsoSeg.

Superpixel aggregation represents a highly effective approach for automated neuron segmentation from electron microscopy (EM) volumes, which can be considered as a graph partitioning task on the region adjacency graph (RAG) of extracted superpixels. However, existing graph partitioning models for superpixel aggregation suffer from the modeling error due to insufficient model capacity. More specifically, the modeling error is caused by the simplification in formulating the real-world graph partitioning task (i.e., superpixel aggregation) into a mathematically well-defined optimization problem. To address this issue, we sidestep the explicit formulation and propose a fully end-to-end superpixel aggregation method based on deep learning of the graph partitioning task, called DeepPartitioning. The central challenge lies in characterizing the partitioning task involving combinatorial complexity. Hence, our method incorporates a line graph neural network (LGNN) to capture higher-order relational structures in RAGs. Specifically, the LGNN enables the propagation of second-order superpixel-pair features among adjacent edges in RAGs. In this way, the partitioning task can be implicitly transformed into the vanilla second-order multicut problem while maintaining higher-order structural information. Overall, our method integrates a second-order feature extractor, a higher-order feature integrator (i.e., the LGNN), and a differentiable approximation to a multicut solver into a unified, learnable framework. Extensive experiments on three public EM datasets demonstrate the effectiveness of the proposed DeepPartitioning within the neuron segmentation pipeline.

With an explosion in the uptake of volume electron microscopy (vEM) across neuroscience and fast-paced advances in imaging protocols, it is timely to introduce general-purpose automation for newly generated large-scale vEM datasets. Recent vision foundation models (e.g., SAM) set a new benchmark for the generalization of 2D segmentation. However, SAM has difficulty handling neurons that are densely packed into 3D volumes. To overcome this obstacle, we consider solutions from both data and model aspects. In terms of data, we introduce a data engine to optimize manual labeling, including (i) human-in-the-loop data cleansing and (ii) model-in-the-loop data unification. In terms of model, we present the SAEM2-SAvEM3with strategies, including (i) auxiliary learning, which predicts complementary representations for SAM masks and improves performance on dense instances; (ii) full-stage distillation, which integrates ViT embeddings into a 3D U-Net, achieves 2D-to-3D lifting and model slimming at the same time; and (iii) prompt-based graph partitioning, which reuses SAM prompts to assign weights of nodes and edges in the oversegmentation graph. According to evaluations of dense and large-scale sparse neurons, the out-of-distribution performance of our pretrained-distilled models is on par with the state-of-the-art supervised and semi-supervised methods. The overall pipeline provides a possible general-purpose solution for 3D neuron reconstruction in any new vEM data. Our code are available at https://github.com/JackieZhai/SAvEM3.

BACKGROUND: Nanoscale connectomics, which aims to map the fine connections between neurons with synaptic-level detail, has attracted increasing attention in recent years. Currently, the automated reconstruction algorithms in electron microscope volumes are in great demand. Most existing reconstruction methodologies for cellular and subcellular structures are independent, and exploring the inter-relationships between structures will contribute to image analysis. The primary goal of this research is to construct a joint optimization framework to improve the accuracy and efficiency of neural structure reconstruction algorithms. RESULTS: In this investigation, we introduce the concept of connectivity consensus between cellular and subcellular structures based on biological domain knowledge for neural structure agglomeration problems. We propose a joint graph partitioning model for solving ultrastructural and neuronal connections to overcome the limitations of connectivity cues at different levels. The advantage of the optimization model is the simultaneous reconstruction of multiple structures in one optimization step. The experimental results on several public datasets demonstrate that the joint optimization model outperforms existing hierarchical agglomeration algorithms. CONCLUSIONS: We present a joint optimization model by connectivity consensus to solve the neural structure agglomeration problem and demonstrate its superiority to existing methods. The intention of introducing connectivity consensus between different structures is to build a suitable optimization model that makes the reconstruction goals more consistent with biological plausible and domain knowledge. This idea can inspire other researchers to optimize existing reconstruction algorithms and other areas of biological data analysis.