Accurate segmentation of neurons in 3D fluorescence microscopy images is essential for advancing neuroscience. Prevalent methods split a volume into patches and process each patch separately due to computational resource limitations. However, they fail to capture global neuronal morphology across multiple patches, which results in discontinuous segmentation and poses a challenge for subsequent neuronal reconstruction. In this paper, we propose a dual U-Net architecture termed "Glancing Beyond Patch" Network (GBP-Net) to incorporate contextual information into segmentation. Specifically, GBP-Net encodes contextual and high-resolution information using two U-Nets, respectively, and facilitates their integration through a cross-scale context module (CSCM) and a cross-resolution fusion module (CRFM). CSCM utilizes a cross-attention mechanism to enable interaction of features from diverse fields of view after encoders, and CRFM employs Mamba to adaptively fuse high-resolution features after decoders. These two modules complement each other at different levels, enabling to capture both global neuronal structures and fine-grained details. Additionally, the proposed cross-network loss guides to focus on challenging samples by penalizing misclassified voxels from both networks, which further promotes the exploitation of contextual information. Experimental results on three datasets demonstrate that our method outperforms other advanced segmentation methods while maintaining computational efficiency. It keeps the global structure of neurons and achieves the highest F1 scores.

Accurate segmentation of organelles in electron microscopy (EM) volumes is essential for understanding intracellular organization. While promising, deep learning-based methods could be unstable and unreliable without sufficient annotations. Masked image modeling (MIM), a powerful pretraining technique, has proven effective in enhancing segmentation by extracting meaningful representations from large-scale unlabeled data. However, random masking strategies in classic MIMs could overlook the unique structural patterns of organelles and the spatial redundancy inherent in EM volumes, thus limiting pretraining efficiency. To address this issue, we propose OrgMIM, a dual-branch MIM framework that integrates complementary masking strategies to capture critical subcellular semantics and learn organelle-specific representations from EM data. Specifically, one branch is guided by static structural priors, leveraging visual foundation models to generate affinity maps that indicate organelle membranes as masking candidates. The other is driven by dynamic reconstruction feedback, using a self-guidance mechanism to compute average loss maps that highlight intricate organelle patterns for heuristic masking. Moreover, cross-branch consistency regularization is introduced for reliable representation learning across sparse semantic contexts. To support large-scale pretraining, we construct IsoOrg-1K, the first organelle-centric 3D EM dataset, comprising 928 informative volumes and over 120 billion voxels. Extensive evaluations on three public EM datasets with varied resolutions and appearances validate the superior performance of OrgMIM. Notably, OrgMIM pretraining on IsoOrg-1K boosts mIoU by 28.78% over training from scratch on the CMCC dataset with a Transformer-based model. All datasets, source codes, and pretrained weights are available at https://github.com/yanchaoz/OrgMIM.

Superpixel aggregation represents a highly effective approach for automated neuron segmentation from electron microscopy (EM) volumes, which can be considered as a graph partitioning task on the region adjacency graph (RAG) of extracted superpixels. However, existing graph partitioning models for superpixel aggregation suffer from the modeling error due to insufficient model capacity. More specifically, the modeling error is caused by the simplification in formulating the real-world graph partitioning task (i.e., superpixel aggregation) into a mathematically well-defined optimization problem. To address this issue, we sidestep the explicit formulation and propose a fully end-to-end superpixel aggregation method based on deep learning of the graph partitioning task, called DeepPartitioning. The central challenge lies in characterizing the partitioning task involving combinatorial complexity. Hence, our method incorporates a line graph neural network (LGNN) to capture higher-order relational structures in RAGs. Specifically, the LGNN enables the propagation of second-order superpixel-pair features among adjacent edges in RAGs. In this way, the partitioning task can be implicitly transformed into the vanilla second-order multicut problem while maintaining higher-order structural information. Overall, our method integrates a second-order feature extractor, a higher-order feature integrator (i.e., the LGNN), and a differentiable approximation to a multicut solver into a unified, learnable framework. Extensive experiments on three public EM datasets demonstrate the effectiveness of the proposed DeepPartitioning within the neuron segmentation pipeline.

The multicut problem, also known as correlation clustering, is a classic combinatorial optimization problem that aims to optimize graph partitioning given only node (dis)similarities on edges. It serves as an elegant generalization for several graph partitioning problems and has found successful applications in various areas such as data mining and computer vision. However, the multicut problem with an exponentially large number of cycle constraints proves to be NP-hard, and existing solvers either suffer from exponential complexity or often give unsatisfactory solutions due to inflexible heuristics driven by hand-designed mechanisms. In this article, we propose a deep graph reinforcement learning method to solve the multicut problem within a combinatorial decision framework involving sequential edge contractions. The customized subgraph neural network adapts to the dynamically edge-contracted graph environment by extracting bilevel connected features from both contracted and original graphs. Our method can learn to infer feasible multicut solutions end-to-end toward optimization of the multicut objective in a data-driven manner. More specifically, by exploring the decision space adaptively, it implicitly gains heuristic knowledge from topological patterns of instances and thereby generates more targeted heuristics overcoming the short-sightedness inherent in the hand-designed ones. During testing, the learned heuristics iteratively contract graphs to construct high-quality solutions within polynomial time. Extensive experiments on synthetic and real-world multicut instances show the superiority of our method over existing combinatorial solvers, while also maintaining a certain level of out-of-distribution generalization ability.

Superpixel aggregation is a powerful tool for automated neuron segmentation from electron microscopy (EM) volume. However, existing graph partitioning methods for superpixel aggregation still involve two separate stages-model estimation and model solving, and therefore model error is inherent. To address this issue, we integrate the two stages and propose an end-to-end aggregation framework based on deep learning of the minimum cost multicut problem called DeepMulticut. The core challenge lies in differentiating the NP-hard multicut problem, whose constraint number is exponential in the problem size. With this in mind, we resort to relaxing the combinatorial solver-the greedy additive edge contraction (GAEC)-to a continuous Soft-GAEC algorithm, whose limit is shown to be the vanilla GAEC. Such relaxation thus allows the DeepMulticut to integrate edge cost estimators, Edge-CNNs, into a differentiable multicut optimization system and allows a decision-oriented loss to feed decision quality back to the Edge-CNNs for adaptive discriminative feature learning. Hence, the model estimators, Edge-CNNs, can be trained to improve partitioning decisions directly while beyond the NP-hardness. Also, we explain the rationale behind the DeepMulticut framework from the perspective of bi-level optimization. Extensive experiments on three public EM datasets demonstrate the effectiveness of the proposed DeepMulticut.

Accurate and robust segmentation of mitochondria from electron microscopy images plays a critical role in understanding cellular functions. While effective, learning-based approaches require vast quantities of expert annotations. Manual efforts can be alleviated by transferring the knowledge learned from the source domain to unseen target domains, as known as unsupervised domain adaptation. In this work, we propose a two-stage pipeline for cross-dataset mitochondria segmentation, aiming to mitigate domain shift in a coarse-to-fine manner. In the first stage, we integrate the style transfer block and segmentation network into an end-to-end image alignment framework. Specifically, patch-wise contrastive learning is employed to guarantee the semantic fidelity of mitochondria, providing more reliable translated images for the segmentation network. In the second stage, a novel online self-training network updated with target images and co-evolving pseudo labels is proposed to fine-tune the segmentation network trained beforehand. Furthermore, to effectively utilize unlabeled data, consistency regularization is introduced in both stages to enforce stable predictions under various perturbations. Experimental results on public datasets demonstrate that the proposed approach outperforms the existing methods by a large margin in bidirectional adaptation for mitochondria segmentation and achieves comparable results with supervised methods.