Nucleus detection in histopathology whole slide images (WSIs) is crucial for a broad spectrum of clinical applications. The gigapixel size of WSIs necessitates the use of sliding window methodology for nucleus detection. However, mainstream methods process each sliding window independently, which overlooks broader contextual information and easily leads to inaccurate predictions. To address this limitation, recent studies additionally crop a large Filed-of-View (LFoV) patch centered on each sliding window to extract contextual features. However, such methods substantially increase whole-slide inference latency. In this work, we propose an effective and efficient context-aware nucleus detection approach. Specifically, instead of using lFoV patches, we aggregate contextual clues from off-the-shelf features of historically visited sliding windows, which greatly enhances the inference efficiency. Moreover, compared to lFoV patches used in previous works, the sliding window patches have higher magnification and provide finer-grained tissue details, thereby enhancing the classification accuracy. To develop the proposed context-aware model, we utilize annotated patches along with their surrounding unlabeled patches for training. Beyond exploiting high-level tissue context from these surrounding regions, we design a post-training strategy that leverages abundant unlabeled nucleus samples within them to enhance the model's context adaptability. Extensive experimental results on three challenging benchmarks demonstrate the superiority of our method.

Tissue segmentation in histopathological images plays a crucial role in computational pathology, owing to its significant potential to indicate the prognosis of cancer patients. Presently, numerous Weakly Supervised Semantic Segmentation (WSSS) methods strive to utilize image-level labels to achieve pixel-level segmentation, aiming to minimize the need for detailed annotations. Most of these methods rely on Class Activation Maps (CAM) extracted from classification models, frequently leading to poor coverage of objects. The major cause is attributed to the strong inductive bias of the classification model, focusing primarily on discriminative feature of objects, rather than non-discriminative features. Inspired by this, we propose a simple yet effective method that introduces a self-supervised task by exploiting both the discriminative and non-discriminative features, and generate Unbiased Activation Maps (UAM) to encompass the whole object. Specifically, our method entails clustering all spatial features of an object class to derive semantic centers. Each center then works as a spatial filter that amplifies similar feature and suppresses dissimilar feature, and extract high-quality pseudo-labels (some noise at object boundaries). Moreover, we further propose a Noise-Reduced (NR) Learning method to train the segmentation network towards credible signals and lessen the impact of false predictions. Comprehensive experimental results on two public histopathology image datasets demonstrate the superior performance of our method over the state-of-the-art weakly supervised segmentation methods.

Tumor cell detection plays a vital role in immunohistochemistry (IHC) quantitative analysis. While recent remarkable developments in fully-supervised deep learning have greatly contributed to the efficiency of this task, the necessity for manually annotating all cells of specific detection types remains impractical. Obviously, if we directly use full supervision to train these datasets, it can cause error in loss calculation due to the misclassification of unannotated cells as background. To address this issue, we observe that although some cells are omitted during the annotation process, these unannotated cells have a significant feature similarity with the annotated ones. Leveraging this characteristic, we propose a novel calibrated loss named Semantic Energy Loss (SEL). Specifically, our SEL automatically adjusts the loss to be lower for unannotated regions with similar semantic to the labeled ones, while penalizing regions with lager semantic difference. Besides, to prevent all regions from having similar semantics during training, we propose Stretched Feature Loss (SFL) that widen the semantic distance. We evaluate our method on two different IHC datasets and achieve significant performance improvements in both sparse and exhaustive annotation scenarios. Furthermore, we also validate that our method holds significant potential for detecting multiple types of cells. Our code is available at here.

Although deep convolution neural networks (DC-NNs) can achieve remarkable success on medical image segmentation, their performance might significantly deteriorate when confronting testing data with the new distribution. Recent studies suggest that one major cause of this issue is the strong inductive bias of DCNNs, which towards image styles (e.g., superficial texture) that are sensitive to change, instead of the invariant content (e.g., object shapes). Inspired by this, we propose a novel method, named Invariant Content Synergistic Learning (ICSL), to improve the generalization ability of DCNNs on unseen data by controlling the inductive bias. Specifically, ICSL first mixes the style of training instances to perturb the training distribution, so that more diverse domains or styles would be made available for training DCNNs. Then, based on the perturbed distribution, we carefully design a dual-branches invariant content synergistic learning strategy to prevent style-biased predictions and maintain the invariant content. Extensive experimental results demonstrate the superior performance of the proposed method over state-of-the-art domain generalization methods on two typical medical segmentation tasks.

Deep convolutional neural networks (DCNNs) significantly improve the performance of medical image segmentation. Nevertheless, medical images frequently experience distribution discrepancies, which fails to maintain their robustness when applying trained models to unseen clinical data. To address this problem, domain generalization methods were proposed to enhance the generalization ability of DCNNs. Feature space-based data augmentation methods have proven their effectiveness to improve domain generalization. However, existing methods still mainly rely on certain prior knowledge or assumption, which has limitations in enriching the diversity of source domain data. In this paper, we propose a random feature augmentation (RFA) method to diversify source domain data at the feature level without prior knowledge. Specifically, we explore the effectiveness of random convolution at the feature level for the first time and prove experimentallyt hat itc an adequately preserve domain-invariant information while perturbing domainspecific information. Furthermore, tocapture the same domain-invariant information from the augmented features of RFA, we present a domain-invariant consistent learning strategy to enable DCNNs to learn a more generalized representation. Our proposed method achieves state-of-the-art performance on two medical image segmentation tasks, including optic cup/disc segmentation on fundus images and prostate segmentation on MRI images.

Diagnostic pathology is the foundation and gold standard for identifying carcinomas, and the accurate quantification of pathological images can provide objective clues for pathologists to make more convincing diagnosis. Recently, the encoder-decoder architectures (EDAs) of convolutional neural networks (CNNs) are widely used in the analysis of pathological images. Despite the rapid innovation of EDAs, we have conducted extensive experiments based on a variety of commonly used EDAs, and found them cannot handle the interference of complex background in pathological images, making the architectures unable to focus on the regions of interest (RoIs), thus making the quantitative results unreliable. Therefore, we proposed a pathway named GLobal Bank (GLB) to guide the encoder and the decoder to extract more features of RoIs rather than the complex background. Sufficient experiments have proved that the architecture remoulded by GLB can achieve significant performance improvement, and the quantitative results are more accurate.

Accurate and automated gland instance segmentation on histology images can assist pathologists to analyze the malignancy degree of adenocarcinoma. Recently, deep-learning-based segmentation networks have been significantly developed to achieve this goal. However, the gland instances are generally proximate to each other and have indiscernible boundaries (i.e., homogeneous intensity values). Most of the existed networks do not define discriminative boundaries representation as context information, resulting in segmenting proximate instances incorrectly. In this paper, to improve the segmentation accuracy between proximate instances, we propose a Boundary Definition Module to boost boundaries feature representation by the guidance of the intra-and-extra glandular features. Moreover, we propose to use the Gumbel-Softmax distribution estimator to clarify the final prediction of boundaries further. Finally, we embed the Boundary Definition Module and Gumbel-Softmax distribution estimator into the gland instance network(FullNet) for performance verification. Experiments on the 2015 MICCAI Gland Segmentation Challenge dataset demonstrate that our proposed method achieves state-of-the-art performance.

Computer-aided diagnosis of pathological images usually requires detection and examination of all positive cells and lesions to make an accurate diagnosis. Therefore, there is an unprecedented demand for effective and reliable methods of training pathological detectors than ever. To train a reliable detector, the training dataset is required to fully annotate all positive instances, such a requirement is challenge and laborious, and is not guaranteed in most cases. However, sparse annotations will limit the training performance of detectors. Here, we propose a novel module named Collaborative Correction Sibling (CCS), which is embedded into the original object detection network to enhance the training performance on sparse annotations in a pioneering way. Specifically, instance-level annotations in the image space can be calibrated by positive instances’ spatial features provided by CCS. Extensive experiments have been conducted on both cellular-and-lesion-level detection tasks, compared with the state of the art methods, our CCS demonstrates the training effectiveness on pathological images.

Although attention mechanisms have been widely used in deep learning for many tasks, they are rarely utilized to solve multiple instance learning (MIL) problems, where only a general category label is given for multiple instances contained in one bag. Additionally, previous deep MIL methods firstly utilize the attention mechanism to learn instance weights and then employ a fully connected layer to predict the bag label, so that the bag prediction is largely determined by the effectiveness of learned instance weights. To alleviate this issue, in this paper, we propose a novel loss based attention mechanism, which simultaneously learns instance weights and predictions, and bag predictions for deep multiple instance learning. Specifically, it calculates instance weights based on the loss function, e.g. softmax+cross-entropy, and shares the parameters with the fully connected layer, which is to predict instance and bag predictions. Additionally, a regularization term consisting of learned weights and cross-entropy functions is utilized to boost the recall of instances, and a consistency cost is used to smooth the training process of neural networks for boosting the model generalization performance. Extensive experiments on multiple types of benchmark databases demonstrate that the proposed attention mechanism is a general, effective and efficient framework, which can achieve superior bag and image classification performance over other state-of-the-art MIL methods, with obtaining higher instance precision and recall than previous attention mechanisms. Source codes are available on https://github.com/xsshi2015/Loss-Attention.

BACKGROUND: Lung cancer is the leading cause of cancer-related deaths in both men and women in the United States, and it has a much lower five-year survival rate than many other cancers. Accurate survival analysis is urgently needed for better disease diagnosis and treatment management. RESULTS: In this work, we propose a survival analysis system that takes advantage of recently emerging deep learning techniques. The proposed system consists of three major components. 1) The first component is an end-to-end cellular feature learning module using a deep neural network with global average pooling. The learned cellular representations encode high-level biologically relevant information without requiring individual cell segmentation, which is aggregated into patient-level feature vectors by using a locality-constrained linear coding (LLC)-based bag of words (BoW) encoding algorithm. 2) The second component is a Cox proportional hazards model with an elastic net penalty for robust feature selection and survival analysis. 3) The third commponent is a biomarker interpretation module that can help localize the image regions that contribute to the survival model's decision. Extensive experiments show that the proposed survival model has excellent predictive power for a public (i.e., The Cancer Genome Atlas) lung cancer dataset in terms of two commonly used metrics: log-rank test (p-value) of the Kaplan-Meier estimate and concordance index (c-index). CONCLUSIONS: In this work, we have proposed a segmentation-free survival analysis system that takes advantage of the recently emerging deep learning framework and well-studied survival analysis methods such as the Cox proportional hazards model. In addition, we provide an approach to visualize the discovered biomarkers, which can serve as concrete evidence supporting the survival model's decision.

The encoder-decoder architecture of convolutional neural networks (CNNs) is widely used in computer vision tasks and various analyses of medical images. However, extracting semantic features from regions of interest (RoIs) in pathological images remains a challenging task because RoIs of different morphologies and scales are embedded in a blurred background. Additionally, it is well known that the classic encoder-decoder architecture is vulnerable to interference from a blurred background and is thus not entirely suitable for precise analysis of pathological images. In this paper, we propose a pathway named global bank (GLB) to guide the encoder and decoder to focus more on the RoIs by providing the decoder with additional effective features of the RoIs. We extend the U-Net and feature pyramid network (FPN) with GLB and evaluate the resulting models on gland segmentation and cancer embolus detection tasks, respectively. Extensive experiments demonstrate that our proposal can significantly improve the performance of the encoder-decoder architecture. The U-Net with GLB achieves the best semantic segmentation performance on the 2015 MICCAI Gland Challenge dataset. Additionally, the FPN with GLB achieves improvements of 2% in average precision and 3.4% in recall on the embolus detection task.

Fully convolutional networks (FCNs) are powerful methods to extract hierarchies of features that have achieved remarkable success in various biomedical image analysis tasks. However, the successful training of FCN requires more than hundreds of pixel-level annotated training samples, which poses a challenge for biomedical image processing tasks. In this paper, we present S3Net, a network that makes more efficient use of available annotated samples on biomedical image segmentation. S3Net is essentially a deeply-supervised encoder-decoder network where the decoder has been redesigned to efficiently restore multi-level encoded feature resolution in a single step. We have conducted extensive experiments on the 2015 MICCAI Gland Challenge dataset. Compared with other methods, S3Net achieves a 0.781 dice-score using 11 images for training, higher than U-Net++ 18.6% and U-Net 12.9%, which verifies the performance of S3Net trained on a small sample. Further, with training on 85 images, S3Net achieves a 0.910 dice-score by using resnet50 as the encoder, which is higher than state-of-the-art semantic segmentation results by 4%.

BACKGROUND: Fast and accurate automatic segmentation of skeletal muscle cell image is crucial for the diagnosis of muscle related diseases, which extremely reduces the labor-intensive manual annotation. Recently, several methods have been presented for automatic muscle cell segmentation. However, most methods exhibit high model complexity and time cost, and they are not adaptive to large-scale images such as whole-slide scanned specimens. METHODS: In this paper, we propose a novel distributed computing approach, which adopts both data and model parallel, for fast muscle cell segmentation. With a master-worker parallelism manner, the image data in the master is distributed onto multiple workers based on the Spark cloud computing platform. On each worker node, we first detect cell contours using a structured random forest (SRF) contour detector with fast parallel prediction and generate region candidates using a superpixel technique. Next, we propose a novel hierarchical tree based region selection algorithm for cell segmentation based on the conditional random field (CRF) algorithm. We divide the region selection algorithm into multiple sub-problems, which can be further parallelized using multi-core programming. RESULTS: We test the performance of the proposed method on a large-scale haematoxylin and eosin (H &E) stained skeletal muscle image dataset. Compared with the standalone implementation, the proposed method achieves more than 10 times speed improvement on very large-scale muscle images containing hundreds to thousands of cells. Meanwhile, our proposed method produces high-quality segmentation results compared with several state-of-the-art methods. CONCLUSIONS: This paper presents a parallel muscle image segmentation method with both data and model parallelism on multiple machines. The parallel strategy exhibits high compatibility to our muscle segmentation framework. The proposed method achieves high-throughput effective cell segmentation on large-scale muscle images.