The majority of spatial transcriptomics datasets are characterized by low resolution, wherein each spot generally encompasses multiple cells. This limitation poses challenges for exploring biological insights at the cellular level. Consequently, the development and application of robust deconvolution methods for spatial transcriptomics data are imperative to address this challenge. Addressing the limitations of previous deconvolution methods—such as the lack of consideration cell type labels from single-cell sequencing data and the inability to adaptively capture local relationship among points—we propose a novel spatial transcriptomics data deconvolution model based on label-guided Multi-Head Dynamic Graph Attention Networks with Optimal Transport(MHDGATOT). Our approach leverages an advanced multi-head dynamic graph attention network to adaptively capture inter-data relationships and generate effective low-dimensional embeddings. Subsequently, we employ optimal transport based on fused gromov-wasserstein to derive the transport matrix between spatial transcriptomics data and single-cell sequencing data, facilitating the accurate deconvolution of spatial transcriptomics datasets. Experimental validation substantiates the effectiveness of our model.

Cervical cancer is one of the fastest growing and most dangerous cancers, seriously threatening women’s health and lives. Cervical cytopathology image classification is a very important approach for diagnosing cervical cancer. The advent of the automatic computer-aided diagnosis system can tackles this issue. However, cervical cell images of different classes exhibit similar appearances, posing a challenge for accurate classification. To address this challenge, this work proposes a framework named MSCCNet. In our MSCCNet, the cross-layer attention-based feature fusion module is used to obtain multi-scale discriminative features. Meanwhile, the spatial relationship modeling module is utilized to encode the relative relationship between objects and capture more slight differences between cervical cells, further strengthening the representation ability of features. We also introduce the joint loss to enhance the penalty for misclassified samples. The model training and evaluation are performed on our developed DSCC dataset and publicly available SIPaKMeD datasets. The proposed MSCCNet achieves overall accuracies of 87.88% and 97.90% on these two datasets, respectively, outperforming several existing classification methods.

In order to address the issue that previous deep learning methods for classifying pathological images cannot adaptively learn features, we propose an end-to-end attention pooling method based on a multi-instance learning patch scoring model. Our method integrates feature extraction and classification into a unified framework that is conducive to extracting the most valuable features. In this model, a patch scoring method is constructed by a multi-instance learning method firstly and then the partial patches selected by the patches scoring model are classified using an end-to-end classification model that incorporates an attention pooled mechanism. To make the pathological image classification mechanism more compatible with the pathologist diagnosis method, we use the squared average normalization function instead of the softmax function to optimize the feature extraction and fusion process, so that the high score patches in positive pathological images receive more attention weights, thus giving better interpretability to the classification results. Experiments on publicly available datasets TCGA_BRCA show a significant improvement in the performance of our approach over other work.

Convolutional Neural Networks (CNNs) have become the de facto approach for medical image classification in recent years. However, the deficiency of convolutional operations in extracting global features has limited the further improvement of this task. Vision Transformers (ViTs) can model long-range dependencies via self-attention mechanism but unfortunately lose local feature details. In this paper, we propose a dynamic aggregation enhanced dual-stream network termed DDN to take the advantage of ViT and CNN to enrich the feature representation of medical images. Specifically, our proposed DDN is built by stacking several Dynamic Dual-stream Units (DDU). In DDU, local and global features are learned by the CNN branch and Transformer branch respectively whilst complementing each other via a bi-directional propagation strategy, then features of both branches are aggregated in a dynamic manner and the integrated information is used to enhance the feature representations of the two branches simultaneously. Extensive experiments show that our proposed DDN performs best compared with other state-of-the-art models on the public Kvasir dataset and ISIC2018 dataset.

Breast cancer histopathological image classification has made great progress with the use of Convolutional Neural Networks (CNNs). However, due to the limited receptive field, CNNs have difficulty in learning the global information of breast cancer histopathological images, hindering the further improvement of this task. To solve this problem, we reasonably apply self-attention mechanism to this task and propose a new network called Local-Global Vision Transformer (LGViT) which utilizes CNNs to capture local features and self-attention mechanism to learn global features of histopathological images. LGViT has several advantages: (1) We propose Local-Global Multi-head Self-attention, a new mechanism that models long-range dependencies with low computational cost. In this mechanism, self-attention is first performed separately within each window. Then, Multiple Instance Learning scheme is utilized to obtain a representative token for each window. Finally, we compute self-attention among these representative tokens to capture global information. (2) We propose Ghost Feed-forward Network, which compensates for the deficiency of Vision Transformer in capturing local features via a locality mechanism. (3) We use a CNN stem to effectively capture low-level information. Experiments on the PatchCamelyon dataset show that LGViT is better than other state-of-the-art methods.

Lots of researchers have studied for classifying histopathological whole slide images (WSIs). Since a WSI is too large to be processed directly, researchers usually cut it into many small-sized patches and then integrate the discriminative features extracted from the patches to obtain a slide-level feature of the WSI. The integration strategy generating the slide-level features is crucial for the WSI classification model. Lots of attention-based methods have been proposed for such purpose. However, most attention-based methods do not take the patches relationship into consideration, which affects the classification performance of the models. In this work, we propose a novel Context-Guided attention (CGattention) method to integrate the patch-level features, which constructs a context vector to simulate the global context information of the whole WSI and implicitly characterizes the relationship between patches in the WSI. When evaluated on two publicly available datasets, the CGattention based model obtained the better performance than other attention-based models.

Tumor mutation burden(TMB) is a biomarker for predicting immunotherapy responses, which can be used to filter out Triple-Negative Breast Cancer(TNBC) patients who benefit from immunotherapy. It is generally measured by whole-exome sequencing (WES) in clinical practice. However, WES has the disadvantage of being expensive, time-consuming, and operational complexity so that it is not available in most hospitals. To solve these issues, we developed a machine learning algorithm that predicts the TMB of TNBC based on the nuclei score of histopathological images, which can obtain high accuracy without manually labeling tumor regions. We verify the effectiveness of patches filtered by nuclei score to TMB classifier by comparing the performance of the model that trained with all patches and trained with selected patches. Experiments results show that the accuracy of the model trained by patches selected with nuclei score is 87.5% and F1 is 80%, which are much higher than training with all patches(87.5% vs 56.25%, 80% vs 58.82%). The time of testing a sample using our approach is only 1 in 26, compared with the test time with all patches. To the best of our knowledge, this is the first research to predict TMB from TNBC histopathological images. The proposed approach has the potential to provide immunotherapy to a much broader subset of patients with TNBC.

The cervical histopathology analysis result is the gold standard for cervical cancer diagnosis. Conventional histopathological examination depends on pathologists’ observation under microscope, which is notoriously labor-intensive and subjective. The popularization of digital pathology technology makes the collection of the cervical histopathological whole slide images (WSIs) more convenient, so it has become possible to develop computer-aided diagnosis methods for cervical cancer. In this work, we first collected the cervical histopathological WSIs from 917 patients with pathological diagnosis through a retrospective study, of which 286 WSIs contained annotations of several lesion areas that were manually outlined by the pathologists. Then we proposed a method for classifying cervical histopathological WSIs by combining deep multi-instance transfer learning (DMITL) and support vector machine (SVM). The DMITL aimed for learning the representations of the WSIs, and the SVM was used for building the classification model of the WSIs. We generated the training and test sets based on our collected WSIs to train and evaluate our method. The validation results have shown that the good performance of our proposed method.

Breast cancer has become one of the most common cancers in the world, and it is also the most lethal cancer in women. As a non-invasive imaging modality, ultrasonography can diagnose the degree of breast lesions and be used for large-scale screening. However, since the lesions in breast ultrasound(BUS) images are morphologically diverse, accompanied by relatively low contrast and complex textures, BUS image recognition faces greater challenges than natural images. In this study, We propose a novel network architecture that combines convolutional neural network(CNN) with vision transformer(ViT) to aggregate local feature details and long-range feature dependencies. Moreover, in order to perform multi-scale feature fusion, we introduce cross attention between the deep feature map and the shallow feature map in the network block to carry out the interaction between the deep feature and the shallow feature information. To verify the effectiveness of the model, we constructed a large-scale dataset and conducted extensive experiments. The results show that our method achieves an accuracy of 85.33%, under the comparable parameter complexity, which outperforms most convolutional neural networks(CNNs) and vision transformers (ViTs).