Joint analysis of multiple biomarker images and tissue morphology plays an important role in disease diagnosis, treatment planning, and drug development. This necessitates precise cross-staining comparison among Whole Slide Images (WSIs) of immunohistochemical (IHC) and hematoxylin and eosin (H&E) stained microscopic slides. In this paper, we propose a novel three-step, feature-based, automatic cross-staining WSI alignment method to support clinical practice. The comparative and ablation experiments have shown the good performance of our method, suggesting that it can offer a simple and effective cross-staining WSIs alignment contributing to pathological diagnosis.

Cervical cytologic whole slide image (WSI) multiple classificaton (grading) is a challenging task. Current studies typically ignore the unbalanced data distribution and require multi-class annotations to learn cell features for WSI grading, which largely suffers from label noise. In this paper, we design a three-stage framework to solve these problems. The first stage uses a binary detector and classifier to screen abnormal cells from the gigapixel WSI. By focusing on binary tasks, we alleviate the effects of label noise and data imbalance. To explore the intrinsic characteristics of cervical cells, we use self-supervised learning to acquire comprehensive cell features for subsequent analysis. In the third stage, we propose a well-designed supervised contrastive learning (SCL) framework for WSI grading. To handle the data-imbalance problem, we pre-compute the optimal positions of class centers which are uniformly distributed on the feature space. During training, we perform SCL whilst matching WSIs to their corresponding class centers, which fosters a class-balanced feature space for WSI representations. Extensive experiments on a large-scale dataset demonstrate that our method achieves state-of-the-art performance.

Cervical cancer is one of the fastest growing and most dangerous cancers, seriously threatening women’s health and lives. Cervical cytopathology image classification is a very important approach for diagnosing cervical cancer. The advent of the automatic computer-aided diagnosis system can tackles this issue. However, cervical cell images of different classes exhibit similar appearances, posing a challenge for accurate classification. To address this challenge, this work proposes a framework named MSCCNet. In our MSCCNet, the cross-layer attention-based feature fusion module is used to obtain multi-scale discriminative features. Meanwhile, the spatial relationship modeling module is utilized to encode the relative relationship between objects and capture more slight differences between cervical cells, further strengthening the representation ability of features. We also introduce the joint loss to enhance the penalty for misclassified samples. The model training and evaluation are performed on our developed DSCC dataset and publicly available SIPaKMeD datasets. The proposed MSCCNet achieves overall accuracies of 87.88% and 97.90% on these two datasets, respectively, outperforming several existing classification methods.

The automatic and effective cervical cell classification technique is critical for cervical cytology screening and cervical cancer prevention. We notice that cervical cell classification is a fine-grained classification task. The difference between classes is small while the difference within a class is large, so it is difficult to capture the discriminative features between different classes of cells for classification. To address this problem, this paper proposes an attention pyramid model (Masked-AP) used for cervical cell classification. Our Masked-AP effectively combines high-level semantic features with low-level detailed features extracted from images, which are both important for classification. Further, we also use the attention mask to drive our model to focus more on the cell nuclei containing substantial discriminative information. For model evaluation, we built a cervical cell dataset named LDCC including 17476 images from 507 subjects. LDCC is available at http://ldcc3.biodwhu.cn/LDCC3/. Our method yielded 74.11% accuracy, 74.11% recall, 74.19% precision, and 74.07% F1-score, and outperforms the state-of-the-art methods.

Convolutional Neural Networks (CNNs) have become the de facto approach for medical image classification in recent years. However, the deficiency of convolutional operations in extracting global features has limited the further improvement of this task. Vision Transformers (ViTs) can model long-range dependencies via self-attention mechanism but unfortunately lose local feature details. In this paper, we propose a dynamic aggregation enhanced dual-stream network termed DDN to take the advantage of ViT and CNN to enrich the feature representation of medical images. Specifically, our proposed DDN is built by stacking several Dynamic Dual-stream Units (DDU). In DDU, local and global features are learned by the CNN branch and Transformer branch respectively whilst complementing each other via a bi-directional propagation strategy, then features of both branches are aggregated in a dynamic manner and the integrated information is used to enhance the feature representations of the two branches simultaneously. Extensive experiments show that our proposed DDN performs best compared with other state-of-the-art models on the public Kvasir dataset and ISIC2018 dataset.

Breast cancer histopathological image classification has made great progress with the use of Convolutional Neural Networks (CNNs). However, due to the limited receptive field, CNNs have difficulty in learning the global information of breast cancer histopathological images, hindering the further improvement of this task. To solve this problem, we reasonably apply self-attention mechanism to this task and propose a new network called Local-Global Vision Transformer (LGViT) which utilizes CNNs to capture local features and self-attention mechanism to learn global features of histopathological images. LGViT has several advantages: (1) We propose Local-Global Multi-head Self-attention, a new mechanism that models long-range dependencies with low computational cost. In this mechanism, self-attention is first performed separately within each window. Then, Multiple Instance Learning scheme is utilized to obtain a representative token for each window. Finally, we compute self-attention among these representative tokens to capture global information. (2) We propose Ghost Feed-forward Network, which compensates for the deficiency of Vision Transformer in capturing local features via a locality mechanism. (3) We use a CNN stem to effectively capture low-level information. Experiments on the PatchCamelyon dataset show that LGViT is better than other state-of-the-art methods.

Lots of researchers have studied for classifying histopathological whole slide images (WSIs). Since a WSI is too large to be processed directly, researchers usually cut it into many small-sized patches and then integrate the discriminative features extracted from the patches to obtain a slide-level feature of the WSI. The integration strategy generating the slide-level features is crucial for the WSI classification model. Lots of attention-based methods have been proposed for such purpose. However, most attention-based methods do not take the patches relationship into consideration, which affects the classification performance of the models. In this work, we propose a novel Context-Guided attention (CGattention) method to integrate the patch-level features, which constructs a context vector to simulate the global context information of the whole WSI and implicitly characterizes the relationship between patches in the WSI. When evaluated on two publicly available datasets, the CGattention based model obtained the better performance than other attention-based models.

Tumor mutation burden(TMB) is a biomarker for predicting immunotherapy responses, which can be used to filter out Triple-Negative Breast Cancer(TNBC) patients who benefit from immunotherapy. It is generally measured by whole-exome sequencing (WES) in clinical practice. However, WES has the disadvantage of being expensive, time-consuming, and operational complexity so that it is not available in most hospitals. To solve these issues, we developed a machine learning algorithm that predicts the TMB of TNBC based on the nuclei score of histopathological images, which can obtain high accuracy without manually labeling tumor regions. We verify the effectiveness of patches filtered by nuclei score to TMB classifier by comparing the performance of the model that trained with all patches and trained with selected patches. Experiments results show that the accuracy of the model trained by patches selected with nuclei score is 87.5% and F1 is 80%, which are much higher than training with all patches(87.5% vs 56.25%, 80% vs 58.82%). The time of testing a sample using our approach is only 1 in 26, compared with the test time with all patches. To the best of our knowledge, this is the first research to predict TMB from TNBC histopathological images. The proposed approach has the potential to provide immunotherapy to a much broader subset of patients with TNBC.

The cervical histopathology analysis result is the gold standard for cervical cancer diagnosis. Conventional histopathological examination depends on pathologists’ observation under microscope, which is notoriously labor-intensive and subjective. The popularization of digital pathology technology makes the collection of the cervical histopathological whole slide images (WSIs) more convenient, so it has become possible to develop computer-aided diagnosis methods for cervical cancer. In this work, we first collected the cervical histopathological WSIs from 917 patients with pathological diagnosis through a retrospective study, of which 286 WSIs contained annotations of several lesion areas that were manually outlined by the pathologists. Then we proposed a method for classifying cervical histopathological WSIs by combining deep multi-instance transfer learning (DMITL) and support vector machine (SVM). The DMITL aimed for learning the representations of the WSIs, and the SVM was used for building the classification model of the WSIs. We generated the training and test sets based on our collected WSIs to train and evaluate our method. The validation results have shown that the good performance of our proposed method.

BACKGROUND: Via counting the different kinds of white blood cells (WBCs), a good quantitative description of a person's health status is obtained, thus forming the critical aspects for the early treatment of several diseases. Thereby, correct classification of WBCs is crucial. Unfortunately, the manual microscopic evaluation is complicated, time-consuming, and subjective, so its statistical reliability becomes limited. Hence, the automatic and accurate identification of WBCs is of great benefit. However, the similarity between WBC samples and the imbalance and insufficiency of samples in the field of medical computer vision bring challenges to intelligent and accurate classification of WBCs. To tackle these challenges, this study proposes a deep learning framework by coupling the pre-trained ResNet and DenseNet with SCAM (spatial and channel attention module) for accurately classifying WBCs. RESULTS: In the proposed network, ResNet and DenseNet enables information reusage and new information exploration, respectively, which are both important and compatible for learning good representations. Meanwhile, the SCAM module sequentially infers attention maps from two separate dimensions of space and channel to emphasize important information or suppress unnecessary information, further enhancing the representation power of our model for WBCs to overcome the limitation of sample similarity. Moreover, the data augmentation and transfer learning techniques are used to handle the data of imbalance and insufficiency. In addition, the mixup approach is adopted for modeling the vicinity relation across training samples of different categories to increase the generalizability of the model. By comparing with five representative networks on our developed LDWBC dataset and the publicly available LISC, BCCD, and Raabin WBC datasets, our model achieves the best overall performance. We also implement the occlusion testing by the gradient-weighted class activation mapping (Grad-CAM) algorithm to improve the interpretability of our model. CONCLUSION: The proposed method has great potential for application in intelligent and accurate classification of WBCs.