Cytological images originate from exfoliated cells, collected via liquid-based slides and digitized into whole slide images (WSIs). Unlike histological WSIs that exhibit continuous and well-structured tissue, cytological WSIs are sparse in spatial distribution and unstructured in cellular relationships. Typically, the nucleus serves as the primary diagnostic feature, while surrounding cytoplasmic information plays a supportive role. These unique characteristics limit the development of effective foundation models and hinder the transferability of histology-based models for cytopathology. To address this, we propose **Cyto-SSL**, the first self-supervised pretraining framework for cytological images. It introduces **Nuclei-Centered Perturbation**, which highlights individual nuclei by perturbing non-nuclear regions. We also design an SR-Transformer module, which complements this by using sparse attention to concentrate on diagnostically relevant scattered cells, while iRPE helps model to capture local spatial relationships and avoids unnecessary attention to irrelevant global structures. Experimental results show that **Cyto-SSL** enhances performance across diverse cytological datasets and Multiple Instance Learning (MIL) methods. On a WSI-level dataset, it achieved 95.67% accuracy and outperformed ImageNet-pretrained ResNet-50 by 11.33%, demonstrating superior feature representation for cytological analysis. Additionally, **Cyto-SSL** modules are plug-and-play, easily integrated into other pretraining frameworks, yielding a 2.6% accuracy gain across different SSL methods.

Integrating multimodal data of pathological image and gene expression for cancer survival analysis can achieve better results than using a single modality. However, existing multimodal learning methods ignore fine-grained interactions between both modalities, especially the interactions between biological pathways and pathological image patches. In this article, we propose a novel Pathway-Aware Multimodal Transformer (PAMT) framework for interpretable cancer survival analysis. Specifically, the PAMT learns fine-grained modality interaction through three stages: (1) In the intra-modal pathway-pathway / patch-patch interaction stage, we use the Transformer model to perform intra-modal information interaction; (2) In the inter-modal pathway-patch alignment stage, we introduce a novel label-free contrastive loss to aligns semantic information between different modalities so that the features of the two modalities are mapped to the same semantic space; and (3) In the inter-modal pathway-patch fusion stage, to model the medical prior knowledge of "genotype determines phenotype", we propose a pathway-to-patch cross fusion module to perform inter-modal information interaction under the guidance of pathway prior. In addition, the inter-modal cross fusion module of PAMT endows good interpretability, helping a pathologist to screen which pathway plays a key role, to locate where on whole slide image (WSI) are affected by the pathway, and to mine prognosis-relevant pathology image patterns. Experimental results based on three datasets of bladder urothelial carcinoma, lung squamous cell carcinoma, and lung adenocarcinoma demonstrate that the proposed framework significantly outperforms the state-of-the-art methods.

The integration of multi-omics data, including DNA methylation, mRNA expression, and miRNA profiles, is crucial for accurate disease subtyping and outcome prediction in complex disorders such as Alzheimer's disease and various cancers. However, the inherent heterogeneity and inconsistency among omics views present significant challenges for reliable data fusion. To address these issues, we propose KANTrust, a novel framework for trustworthy multi-omics classification that explicitly models both epistemic and aleatoric uncertainties. Our method combines a Kolmogorov-Arnold Network (KAN)enhanced robust representation module, a contrastive evidence consistency module, and an evidence-theoretic fusion module to achieve reliable multi-view integration. KANTrust adaptively highlights informative features within each omics modality, promotes semantic alignment across views, and quantifies uncertainty through a Dempster-Shafer framework. Experimental evaluations on four real-world biomedical datasets demonstrate that KANTrust consistently outperforms state-of-the-art methods in both binary and multi-class classification tasks. Code is available at https://github.com/wcj6/KANTrust.

Anomaly detection (AD) identifies outliers for applications like defect and lesion detection. While CLIP shows promise for zero-shot AD tasks due to its strong generalization capabilities, its inherent Anomaly-Unawareness leads to limited discrimination between normal and abnormal features. To address this problem, we propose Anomaly-Aware CLIP (AA-CLIP), which enhances CLIP's anomaly discrimination ability in both text and visual spaces while preserving its generalization capability. AA-CLIP is achieved through a straightforward yet effective two-stage approach: it first creates anomaly-aware text anchors to differentiate normal and abnormal semantics clearly, then aligns patch-level visual features with these anchors for precise anomaly localization. This two-stage strategy, with the help of residual adapters, gradually adapts CLIP in a controlled manner, achieving effective AD while maintaining CLIP's class knowledge. Extensive experiments validate AA-CLIP as a resource-efficient solution for zero-shot AD tasks, achieving state-of-the-art results in industrial and medical applications. The code is available at https://github.com/Mwxinnn/AA-CLIP.

Microsatellite instability (MSI) is a crucial biomarker for guiding the efficacy of immunotherapy and adjuvant chemotherapy, making its detection essential for effective cancer treatment and prognosis. Traditional MSI prediction methods encounter challenges including high costs and limited accuracy under low tumor purity conditions. Recent advancements have explored deep learning for MSI prediction from pathological images, yet these approaches often overlook the multi-scale nature of pathological images and specific pathological features critical for MSI diagnosis. In this study, we proposed MSIscope, a novel Transformer-based method for detecting MSI from pathological images by fusing multi-scale pathological image information. Our approach consists of three key components: 1) ROI selection: we design a region of interest (ROI) selector based on convolutional neural networks and attention mechanisms, selecting tumor regions and important non-tumor regions as our focus; 2) Multi-scale vision expansion and feature extraction: we develop an algorithm that captures a broader view centered on a specified area to obtain a multi-scale field of view. The CTransPath feature extractor is then used to extract features from the image; 3) Multi-scale fusion Transformer: we propose a multi-scale feature aggregator (MS-Transformer) to aggregate contextual features across regions and scales. Our method was experimentally validated on public datasets, achieving an AU-ROC of 0.911 on the TCGA pan-cancer dataset and 0.887 on the TCGA-CRC dataset, surpassing existing methods. Additionally, it maintains high AUROC on datasets with lower tumor purity, outperforming current approaches. These results highlight the potential of MSIscope as an robust method for MSI prediction.

The Transformer-based methods provide a good opportunity for modeling the global context of gigapixel whole slide image (WSI), however, there are still two main problems in applying Transformer to WSI-based survival analysis task. First, the training data for survival analysis is limited, which makes the model prone to overfitting. This problem is even worse for Transformer-based models which require large-scale data to train. Second, WSI is of extremely high resolution (up to 150,000 x 150,000 pixels) and is typically organized as a multi-resolution pyramid. Vanilla Transformer cannot model the hierarchical structure of WSI (such as patch cluster-level relationships), which makes it incapable of learning hierarchical WSI representation. To address these problems, in this paper, we propose a novel Sparse and Hierarchical Transformer (SH-Transformer) for survival analysis. Specifically, we introduce sparse self-attention to alleviate the overfitting problem, and propose a hierarchical Transformer structure to learn the hierarchical WSI representation. Experimental results based on three WSI datasets show that the proposed framework outperforms the state-of-the-art methods.

Survival analysis is a significant study in cancer prognosis, and the multi-modal data, including histopathological images, genomic data, and clinical information, provides unprecedented opportunities for its development. However, because of the high dimensionality and the heterogeneity of histopathological images and genomic data, acquiring effective predictive characters from these multi-modal data has always been a challenge for survival analysis. In this article, we propose a transformer-based survival analysis model (TransSurv) for colorectal cancer that can effectively integrate intra-modality and inter-modality features of histopathological images, genomic data, and clinical information. Specifically, to integrate the intra-modality relationship of image patches, we develop a multi-scale histopathological features fusion transformer (MS-Trans). Furthermore, we provide a cross-modal fusion transformer based on cross attention for multi-scale pathological representation and multi-omics representation, which includes RNA-seq expression and copy number alteration (CNA). At the output layer of the TransSurv, we adopt the Cox layer to integrate multi-modal fusion representation with clinical information for end-to-end survival analysis. The experimental results on the Cancer Genome Atlas (TCGA) colorectal cancer cohort demonstrate that the proposed TransSurv outperforms the existing methods and improves the prognosis prediction of colorectal cancer.

Survival analysis is crucial to the evaluation of cancer treatment options and deep learning-based methods integrating pathological images and genomic data have been used for prognosis prediction. However, the most methods are based on the analysis of pathological image patches, thus ignoring the morphological structure information at larger field-of-view and intrinsic relationships between patches. Meanwhile, the existing models fail to exploit the powerful representation learning capabilities of the neural networks for effective multimodal feature fusion of pathological images and genomic data. In this paper, we propose a novel transformer-based fusion network integrating pathological images and genomic data (PGTFNet) for cancer survival analysis. Specifically, we present a transformer-based feature fusion module for multi-scale pathological slides to fully exploit the intra-modality relationships between image patches at various fields of view. Moreover, in order to make effective inter-modality feature fusion of pathological images and genomic data, we introduce a cross-attention transformer module that can exchange feature representations of different modalities between two transformers branches. The PG-TFNet is performed on the colorectal cancer dataset from the Cancer Genome Atlas (TCGA), which contains paired whole-slide images and genomic data with ground truth survival data. The experimental results from a 10-fold cross validation demonstrate that the proposed PG-TFNet facilitates the prognosis prediction of colorectal cancer and shows superiority over the existing methods.

The segmentation of cancer regions is a key step in pathological image analysis. Although traditional methods (such as U-Net) have achieved good results in general medical image segmentation, the segmentation performance of the tumor region is still unsatisfactory because the boundary of the tumor is too blurred. Moreover, most tumor region segmentation methods focus on the learning of image content features while ignoring learning relationship among pixels on tumor contours. In this paper, we developed a multi-task learning technique to enhance the importance of contours and increase the weight of pixels relationship learning for the tumor segmentation. Different from the traditional single-decoder network, a parallel contour decoder with LRLM (location relationship learning module) is introduced as an auxiliary decoder to learn the relationship-based features of tumor contours, which forms a two-decoder network. To promote the information fusion of the two tasks, the two decoders share a same encoder with bidirectional skip connections between the auxiliary contour decoder and the main content decoder. Experimental results show that LR-Net is superior to many popular approaches, such as CE-Net and U-Net.

Automatic breast cancer grading methods based on HE stained pathological images can be summarized into two categories. The first category is to use learning-based methods to directly extract the features of the pathological image for breast cancer grading. However, unlike the coarse-grained problem of breast cancer classification, grading of breast Invasive Ductal Carcinoma (IDC) is a fine-grained classification problem. Only using general methods cannot classify IDC well. The second category is to conduct the three evaluation criteria of Nottingham Grading System (NGS) separately, and then integrate the results of the three criteria to obtain the final IDC grading result. However, NGS is only a semi-quantitative evaluation method. The inherent medical motivation of NGS is to grade IDC with the help of nuclei-related features. In this paper, we proposed a nuclei-aware network for IDC grading in pathological images. The entire network achieves an effect similar to the attention mechanism in end-to-end learning, so as to learn fine-grained and nuclei-related feature representations for IDC grading. It should to be pointed out that our method can emphasize custom areas, thus providing a way to model medical knowledge into the network structure. This is different from the general attention mechanism that cannot artificially control the area of attention. Experimental results show that the performance of proposed method is better than the state-of-the-art.

Segmenting the cerebral vessels precisely from the time-of-flight magnetic resonance angiography (TOF-MRA) images is important for the diagnosis and therapy of the cerebrovascular diseases. Since the complex structures of cerebral vessels, the current cerebrovascular segmentation algorithms based on statistical model have less accuracy for stenotic vessels and are quite time-consuming. In this paper, we propose a novel automatic cerebrovascular segmentation algorithm based on focused Multi-Gaussians (FMG) model and weighted 3D Markov Random Field. As far as our knowledge, this is the first time to adopt multi-Gaussians distributions as vascular model with the purpose of modeling the vascular tissue more accurately. Furthermore, the fitting range is narrowed to local region related to vessels in order to make the model focus on the vascular tissue and simplify the finite mixture model. To incorporate precise local character of images to the model, we design a new weighted 3D MRF by a weighted neighborhood system (W-NBS). Finally, the particle swarm optimization (PSO) algorithm of parameter estimation has been implemented parallelly based on GPUs and the execution speed was improved by about 70 times. The experimental results show that the algorithm can produce detailed segmentation results especially for stenotic vessels.

Tumor mutational burden (TMB) is the most important and most promising biomarker in the era of tumor immunotherapy, and it can predict the immunotherapy efficiency of patients in various cancers including liver cancer. TMB is mainly obtained by next generation sequencing technology such as whole exome sequencing (WES). However, conditions such as excessive testing costs, lengthy detection cycles, and tissue sample dependence severely limit the clinical application of TMB. Inspired by the inner link between the intrinsic characteristics of the tumor cell genome and the pathological features of tumor cells and their microenvironment-related cells, we propose a deep learning method for predicting the level of TMB (high or low) directly from pathological images. This study found that the feature scale (receptive field) is the biggest factor affecting the classification effect of TMB prediction, and further determined the best receptive field through a series of experiments. Experimental results show that our method is far more out performance of the commonly used panel sequencing (99.7% VS 79.2%). To the best of our knowledge, this is the first research to predict TMB and the highest level of accuracy of genomic characteristic predicted by pathological images. The proposed method has the potential to provide immunotherapy to a much broader subset of patients with liver cancer.