• Hierarchical supervision in DINOv2 improves the latent space for WBC classification. • Biologically informed hierarchy reduces error severity and aligns features. • The clustering is improved even for non-leukocytic out-of-domain datasets. • Hierarchy supports labels with different degrees of precision. • Hierarchical supervision in DINOv2 training improves generalizability on white blood cell images. The microscopic observation of blood cells is a crucial step in diagnosing pathologies such as leukemia. DINOv2 models have been employed to extract features from blood cell images, but they do not include biological knowledge, nor do they allow multi-granular labels. To enhance the representation of these cells, we propose leveraging a biologically informed hierarchy of white blood cell types. We train a DINOv2-based foundation model with a semi-supervised framework that uses hierarchical supervision. It enables using datasets with varying levels of label precision within a structure that represents the process of cell differentiation. To support multi-level label precision, we modify the original hierarchical loss function, allowing any hierarchy level to serve as a ground truth class. We evaluate our model on three external datasets, including an out-of-domain set of cervical cells. Our approach improves generalization of the model to new datasets, improving by 1 percentage point the balanced accuracy on the two blood cell external datasets, and by 2.5 percentage point the balanced accuracy on the out-of-domain dataset. In addition the proposed strategy better aligns the model’s latent space with biological properties, leading to more acceptable misclassifications

Building deep learning models that can rapidly segment whole slide images (WSIs) using only a handful of training samples remains an open challenge in computational pathology. The difficulty lies in the histological images themselves: many morphological structures within a slide are closely related and very similar in appearance, making it difficult to distinguish between them. However, a skilled pathologist can quickly identify the relevant phenotypes. Through years of training, they have learned to organize visual features into a hierarchical taxonomy (e.g., identifying carcinoma versus healthy tissue, or distinguishing regions within a tumor as cancer cells, the microenvironment, …). Thus, each region is associated with multiple labels representing different tissue types. Pathologists typically deal with this by analyzing the specimen at multiple scales and comparing visual features between different magnifications. Inspired by this multi-scale diagnostic workflow, we introduce the Navigator, a vision model that navigates through WSIs like a domain expert: it searches for the region of interest at a low scale, zooms in gradually, and localizes ever finer microanatomical classes. As a result, the Navigator can detect coarse-grained patterns at lower resolution and fine-grained features at higher resolution. In addition, to deal with sparsely annotated samples, we train the Navigator with a novel semi-supervised framework called S5CL v2. The proposed model improves the F1 score by up to 8% on various datasets including our challenging new TCGA-COAD-30CLS and Erlangen cohorts.

Spatial transcriptomics enables interrogating the molecular composition of tissue with ever-increasing resolution and sensitivity. However, costs, rapidly evolving technology, and lack of standards have constrained computational methods in ST to narrow tasks and small cohorts. In addition, the underlying tissue morphology, as reflected by H&E-stained whole slide images (WSIs), encodes rich information often overlooked in ST studies. Here, we introduce HEST-1k, a collection of 1,229 spatial transcriptomic profiles, each linked to a WSI and extensive metadata. HEST-1k was assembled from 153 public and internal cohorts encompassing 26 organs, two species (Homo Sapiens and Mus Musculus), and 367 cancer samples from 25 cancer types. HEST-1k processing enabled the identification of 2.1 million expression-morphology pairs and over 76 million nuclei. To support its development, we additionally introduce the HEST-Library, a Python package designed to perform a range of actions with HEST samples. We test HEST-1k and Library on three use cases: (1) benchmarking foundation models for pathology (HEST-Benchmark), (2) biomarker exploration, and (3) multimodal representation learning. HEST-1k, HEST-Library, and HEST-Benchmark can be freely accessed at https://github.com/mahmoodlab/hest.