In pathology, cancer grading is crucial for patient management and treatment. Recent deep learning methods, based upon convolutional neural networks (CNNs), have shown great potential for automated and accurate cancer diagnosis. However, these do not explicitly utilize tissue/cellular composition, and thus difficult to incorporate the existing knowledge of cancer pathology. In this study, we propose a multi-cell type and multi-level graph aggregation network (MMGA-Net) for cancer grading. Given a pathology image, MMGA-Net constructs multiple cell graphs at multiple levels to represent intra- and inter-cell type relationships and to incorporate global and local cell-to-cell interactions. In addition, it extracts tissue contextual information using a CNN. Then, the tissue and cellular information are fused to predict a cancer grade. The experimental results on two types of cancer datasets demonstrate the effectiveness of MMGA-Net, outperforming other competing models. The results also suggest that the information fusion of multiple cell types and multiple levels via graphs is critical for improved pathology image analysis.

Automated nuclei segmentation and classification are the keys to analyze and understand the cellular characteristics and functionality, supporting computer-aided digital pathology in disease diagnosis. However, the task still remains challenging due to the intrinsic variations in size, intensity, and morphology of different types of nuclei. Herein, we propose a self-guided ordinal regression neural network for simultaneous nuclear segmentation and classification that can exploit the intrinsic characteristics of nuclei and focus on highly uncertain areas during training. The proposed network formulates nuclei segmentation as an ordinal regression learning by introducing a distance decreasing discretization strategy, which stratifies nuclei in a way that inner regions forming a regular shape of nuclei are separated from outer regions forming an irregular shape. It also adopts a self-guided training strategy to adaptively adjust the weights associated with nuclear pixels, depending on the difficulty of the pixels that is assessed by the network itself. To evaluate the performance of the proposed network, we employ large-scale multi-tissue datasets with 276349 exhaustively annotated nuclei. We show that the proposed network achieves the state-of-the-art performance in both nuclei segmentation and classification in comparison to several methods that are recently developed for segmentation and/or classification.

Multi-scale approaches have been widely studied in pathology image analysis. These offer an ability to characterize tissues in an image at various scales, in which the tissues may appear differently. Many of such methods have focused on extracting multi-scale hand-crafted features and applied them to various tasks in pathology image analysis. Even, several deep learning methods explicitly adopt the multi-scale approaches. However, most of these methods simply merge the multi-scale features together or adopt the coarse-to-fine/fine-to-coarse strategy, which uses the features one at a time in a sequential manner. Utilizing the multi-scale features in a cooperative and discriminative fashion, the learning capabilities could be further improved. Herein, we propose a multi-scale approach that can identify and leverage the patterns of the multiple scales within a deep neural network and provide the superior capability of cancer classification. The patterns of the features across multiple scales are encoded as a binary pattern code and further converted to a decimal number, which can be easily embedded in the current framework of the deep neural networks. To evaluate the proposed method, multiple sets of pathology images are employed. Under the various experimental settings, the proposed method is systematically assessed and shows an improved classification performance in comparison to other competing methods.