Analysis of histopathological images is critical in cancer diagnosis and treatment. Due to the huge size of histopathological images and the varied number of imaging records per patient, many existing works analyze the Whole Slide Image (WSI) as a bag in which its patches are instances. However, these approaches are limited to analyzing the patches in a fixed shape, while the malignant lesions can form varied shapes. To address this challenge, in this article we propose a Multi-Instance Multi-Shape Support Vector Machine (MIMSSVM) to analyze the multiple images (instances) jointly where each instance consists of multiple patches in various shapes. In our approach, we can identify the different morphologic abnormalities of nuclei shapes from the multiple images. In addition to the multi-instance multi-shape learning capability, we derive an efficient solution algorithm to optimize the proposed model that scales well to a large number of features. Our experimental results show our new method outperforms the existing SVMs and deep learning models in histopathological classification. The proposed model also identifies the tissue segments in an image exhibiting an indication of an abnormality which provides utility in the early detection of malignant tumors. All these promising experimental results have demonstrated the effectiveness of our new method. We anticipate that our new method is of interest to biomedical engineering communities beyond WSI research and have open sourced the code of our method online. The implementation of our proposed MIMSSVM model is publicly available at https://github.com/hoonseo0409/MIMSSVM .

MOTIVATION: Breast cancer is a type of cancer that develops in breast tissues, and, after skin cancer, it is the most commonly diagnosed cancer in women in the United States. Given that an early diagnosis is imperative to prevent breast cancer progression, many machine learning models have been developed in recent years to automate the histopathological classification of the different types of carcinomas. However, many of them are not scalable to large-scale datasets. RESULTS: In this study, we propose the novel Primal-Dual Multi-Instance Support Vector Machine to determine which tissue segments in an image exhibit an indication of an abnormality. We derive an efficient optimization algorithm for the proposed objective by bypassing the quadratic programming and least-squares problems, which are commonly employed to optimize Support Vector Machine models. The proposed method is computationally efficient, thereby it is scalable to large-scale datasets. We applied our method to the public BreaKHis dataset and achieved promising prediction performance and scalability for histopathological classification. AVAILABILITY AND IMPLEMENTATION: Software is publicly available at: https://1drv.ms/u/s!AiFpD21bgf2wgRLbQq08ixD0SgRD?e=OpqEmY. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.