Mainstream deep learning algorithms are designed for typical images sizing from 224 × 224 to 600 × 600 pixels, which are relatively small compared to the whole-slide images (WSI) with billions of pixels, causing existing methods to be inefficient due to computational and memory constraints. Furthermore, most WSIs are annotated with slide-level labels, as patch-level annotations are time-consuming and have to be conducted by experienced pathologists. At the same time, the regions of interest (ROIs), e.g., where cancer, tumor, and bacteria reside, would occupy relatively small areas. Thus, we propose Flexible and Efficient Zoom-In (FEZ) as a method to precisely locate ROIs to accelerate ultra-large image classification. FEZ initially examines a low-resolution WSI with a policy network (PN) to select the potentially critical ROIs and zooms in the selected regions using medium-resolution images to select finer-grain ROIs with another PN. Eventually, FEZ loads the high-resolution patches and classifies them with a value network (VN) to make a prediction. Since FEZ only loads a few selected patches instead of the high-resolution WSI, it dramatically reduces the image loading time and the computation time needed for analyzing the patches. Experimental results show that FEZ is capable of delivering similar accuracy with higher speed than a recently proposed whole-slide training method (WST) and existing sampling methods such as Multiple Instance Learning (MIL) and CLAM. FEZ is 42× and 22× faster than MIL on the Camelyon16 and TCGA Lung Cancer datasets, respectively. With FEZ, we can reduce the classification time for ultra-large images from minutes to seconds to support interactive applications.

The combination of digital pathology and artificial intelligence attracts more and more attention. In order to take into account the global and detailed textural information, recent works proposed to use whole-slide images (WSI) with more than 10 million pixels to train artificial neural networks in pursuit of a high-precision model. However, such an approach faces new technical challenges. When researchers conduct neural network training on a graphics processing unit (GPU), the extremely high spatial resolution of WSI and the large amount of intermediate data generated during the training process exceed the memory capacity of the GPU. Therefore, streaming convolutional neural network (SCNN) was proposed to disassemble a large image into multiple smaller patches that can be forward and backward propagated under the limitation of GPU memory during the training process. However, in a multi-GPU high-performance computing (HPC) system, when the central processing unit (CPU) performs data preprocessing for the distributed training of multiple GPUs, decoding large images and data augmentation consume a lot of memory and CPU time, resulting in a performance bottleneck.This paper first presents a multithreaded image decoder optimized for multi-core CPUs to speedup WSI loading. We also propose a patch-level data augmentation algorithm and implement it on a dedicated GPU to distribute the augmented data to other GPUs for training the SCNN. We use the above method to train the ResNet-50 SCNN model by learning from the TCGA-LUAD and TCGA-LUSC datasets published by The Cancer Genome Atlas. In comparison to training the SCNN on multiple GPUs using the regular distributed data parallelism, our method can save up to 92.8% of memory and increase the training speed by up to 242.5% when training with 8 A100 GPUs on the NVIDIA DGX-A100 system.