Using image-level weakly supervised semantic segmentation (WSSS) techniques to segment tissue regions in giga-pixel histopathological whole slide images (WSI) has garnered widespread attention, as it can reduce many annotation workloads for pathologists. Most recent studies are based on class activation mapping (CAM) to generate pseudo masks, which are then used to train segmentation model in a fully supervised manner. However, it is still a challenge to accurately segment non-predominant tissue categories due to the existence of long-tailed and inter-class homogeneity matters. For these matters, we propose three designs to solve them: 1) Diffusion-based Data Generation to synthesis new images of tail class to expand data distribution; 2) Feature Recalibration to reassign the logits in CAM to narrow the feature-level prediction gap between predominant and non-predominant classes; 3) Grade-skip Learning to correct the under-fitting tendency of hard samples during the segmentation phase. Moreover, we also design a powerful pipeline LoHo for histopathology tissue segmentation. Extensive experiments demonstrate that our method not only achieves new state-of-the-art performances but also significantly improves segmentation of tail classes. In addition, our methods are plug-and-play, making it easily integrable into many mainstream WSSS frameworks.

Semi-supervised segmentation (S3) is one of the preferred choices for histopathological image segmentation tasks, while how to improve model’s learning capability for unlabeled data remains a key challenge in S3. The remarkable feature extraction abilities of Segment Anything Model (SAM) offers a potential opportunity. However, SAM’s performance on contextual complex histopathological images is not so desirable due to its limitations in finely capture structural relationships. To address this issue, we propose a novel SAM-based S3framework QuPaS, which consists of Quantum Force Field (QFF) Finetuning and Adversarial Estimation (AE). QFF covers the shortage of SAM’s limited understanding of spatial structure by simulating intermolecular forces to explore the structural topological relationships between pixel-level features. AE introduces an adversarial estimation network to align the consistency of confidence distributions between different outputs, thereby reducing the interference of incompatible semantic features on the model. Extensive experiments across three challenging histopathological segmentation scenarios have demonstrate that our QuPaS completely outperforms the state-of-the-art S3methods. Furthermore, QuPaS is able to maintain stable generalization performance on previously unseen domains. The code will be released at: https://github.com/director87/QuPaS.

Weakly supervised learning based on image-level labels can effectively reduce annotation costs, making it a popular choice for histopathological tissue segmentation. However, this pattern still face some challenges: 1) inaccurate class activation maps (CAM) make pseudo masks quality insufficient; 2) noisy pixels in pseudo masks will mislead the segmentation model's decision-making. To deal with these problems, we propose a novel weakly supervised semantic segmentation (WSSS) framework. First, we introduce Local Spatial Affine Perturbation to strengthen the model's utilization of weak supervision signals and improve its robustness to noisy regions within CAM. Second, we propose Wave-aware Dynamic Feature Aggregation to adaptively enhance the information-aware representation of target regions to obtain fine-grained pseudo masks enriched with positive semantic information. Third, we train a segmentation model with a noise-suppression scheme called Cross-scale Logits Distillation to reduce the inevitable false positive pixels in pseudo masks. We conduct extensive experiments to validate our method and set new state-of-the-art segmentation performances on five histopathological tissue segmentation datasets. Moreover, we will introduce a new dataset, GCSS-WSSS for gastric cancer, to promote the diversification for the research community of computational pathology. Code and data will be released at: https://github.com/director87/WaWeHis.

Cancer is a leading cause of death worldwide due to its aggressive nature and complex variability. Accurate prognosis is therefore challenging but essential for guiding personalized treatment and follow-up. Previous research often relied on single data sources, missing the opportunity to combine various types of patient information for more comprehensive survival predictions. To address these challenges, we propose a two-stage fusion method named Cross-Attention and Multimodal Low-Rank Interaction Fusion Framework (CA-MLIF). In the first stage, we propose a CA mechanism for real-time feature updates and cross-modal mutual learning to capture rich semantic information. In the second stage, we design a novel multimodal low-rank interaction fusion method for survival prediction. Specifically, we present modal attention mechanism (MAM) for feature filtration, low-rank multimodal fusion (LMF) for model complexity reduction, and optimal weight concatenation (OWC) for maximizing feature integration. Extensive experiments on two public datasets TCGA-GBMLGG and TCGA-KIRC, as well as a multi-center in-house lung adenocarcinoma (LUAD) dataset validate the effectiveness of CA-MLIF, which demonstrate that our method outperforms existing approaches in survival prediction under both pathology-gene fusion and CT-pathology fusion scenarios.

Image-level weakly supervised semantic segmentation (WSSS) reduces the dependence on high-quality data annotation, which plays a crucial role in computational pathology. Benefit from the ability to localize the objects with only binary labels, Class Activation Map (CAM) is a widely used method to initial pseudo masks. However, due to the low contrast among different tissues in histopathological images, most existing CAM-based methods perform poorly in gland segmentation. We retrospect this process and find that class consistency and semantic consistency can guide the network to effectively distinguish confusing pixels and generate fine-grained pseudo masks. Specifically, for class consistency, we propose Consistency Correlation Attention (CCA) to encourage the network to focus on the contribution of class features to semantic dependencies. For semantic consistency, we propose Multi-scale Pyramid Fusion Pooling (MPFP) to aggregate coarse-to-fine global semantic information from CAMs at multiple spatial resolutions, thus identifying class localization. Additionally, we introduce a Purified Labels Filtration (PLF) strategy during the segmentation phase to mitigate the noisy supervision signal and improve the segmentation quality of the model. Extensive experiments show that the our method achieves new state-of-the-art results on three publicly available gland datasets. Furthermore, our method demonstrates impressive domain adaptation capability, achieving satisfactory results with only a small portion of samples when faced with unseen domain data.

Nuclei segmentation is a fundamental and critical step in digital pathological image analysis. Fully supervised nuclei segmentation requires a lot of pixel-by-pixel manual annotation by pathologists, which is very time-consuming and laborious. To minimize the labeling burden of pathologists, this paper uses only point annotations of nuclei data for weakly supervised learning. Specifically, a two-stage model named EOFD-Net with feature denoising and edge optimization is proposed. In the first stage, three weak labels (K-means cluster labels, Voronoi labels, and superpixel labels) with complementary information are used to train the encoder-decoder network to achieve coarse segmentation of nuclei. A feature denoising module(FDM) is designed in the encoder part, which can effectively reduce noise interference. In the second stage, we designed an edge optimization strategy using the prior knowledge of the trained model in the first stage. Confident learning is employed to denoise pseudo-label and rectify the mislabel. These optimized labels are input into the second stage to obtain the final segmentation results. The performance of our method outperforms current state-of-the-art methods on two publicly nuclei segmentation datasets, MoNuSeg and TNBC.