Medical vision-language learning faces a persistent challenge: limited paired image-text data, especially in expert domains like radiology. To address this, we propose a data-efficient framework that enhances dual-encoder vision-language models through structured comprehensive textual representations. Using GPT-4 and curated clinical knowledge, we generate semantically comprehensive representations for each disease class. These representations capture detailed visual descriptions of the disease, the major causes, and the major symptoms related to chest X-rays. These representations are paired with medical images to pre-train BiomedCLIP using contrastive learning, effectively aligning vision and language modalities without the need for large-scale manual annotations. We then fine-tune the vision encoder for chest X-ray disease classification. Our results show that this approach achieves competitive performance, even with limited supervision. This work highlights the potential of structured generative supervision to scale vision-language learning in data-constrained medical domains.

In Computational Pathology (CPath), the introduction of Vision-Language Models (VLMs) has opened new avenues for research, focusing primarily on aligning image-text pairs at a single magnification level. However, this approach might not be sufficient for tasks like cancer subtype classification, tissue phenotyping, and survival analysis due to the limited level of detail that a single-resolution image can provide. Addressing this, we propose a novel multi-resolution paradigm leveraging Whole Slide Images (WSIs) to extract histology patches at multiple resolutions and generate corresponding textual descriptions through advanced CPath VLM. We introduce visual-textual alignment at multiple resolutions as well as cross-resolution alignment to establish more effective text-guided visual representations. Cross-resolution alignment using a multi-modal encoder enhances the model’s ability to capture context from multiple resolutions in histology images. Our model aims to capture a broader range of information, supported by novel loss functions, enriches feature representation, improves discriminative ability, and enhances generalization across different resolutions. Pre-trained on a comprehensive TCGA dataset with 34 million image-language pairs at various resolutions, our fine-tuned model outperforms State-Of-The-Art (SOTA) counterparts across multiple datasets and tasks, demonstrating its effectiveness in CPath. The code is available on GitHub at: https://github.com/BasitAlawode/MR-PLIP.

Vision-Language models (VLMs) integrate natural language understanding with visual data interpretation, crucial in diverse applications such as medical imaging. However, training VLMs on limited data, especially in radiology, remains a challenge. We propose a strategy to improve dual encoder performance under data constraints. Using contrastive learning to align visual and textual embeddings effectively, we generated a bag of rich textual descriptions using GPT-4 to augment merged information from esteemed medical resources and pre-trained BiomedCLIP. These rich textual descriptions provide in-depth information on disease visual description, major causes, and major symptoms, enhancing the model’s contextual understanding and classification accuracy. Unlike previous methods relying on a single alignment, our multiple alignment strategy associates multiple images with multiple textual descriptions per disease class while capping descriptors to maintain computational efficiency. Adapting the vision encoder for chest X-ray classification, our approach achieves competitive accuracy with fewer training pairs, highlighting its potential for data-limited domains.

Visual analysis of brain organoid imaging is critical for advancing the understanding of in vitro organoid development. However, precise differentiation of mitotic stages remains challenging due to complex cellular morphology and limited imaging data. To address these limitations, we present Organoid-ICLIP, a novel Class Imbalance-Aware Contrastive Language-Image Pretraining model tailored for mitotic nuclei analysis. This approach integrates functional and morphological descriptions of mitotic phases by aligning organoid microscopy images with textual descriptions during visual pattern extraction. The proposed approach introduces a new Mitosis Phase Imbalance-Aware Balancing Loss (MPIA-BL), which dynamically adjusts instance weighting based on mitotic phase distribution to address severe class imbalance. By explicitly addressing both inter-class and intra-class imbalance, Organoid-ICLIP achieves more than 6% improvement over traditional contrastive learning approaches on evaluation metrics. Experimental evaluations on the benchmark Borg brain organoid imaging dataset demonstrate the enhanced performance of the proposed approach in mitotic phase classification compared to SOTA vision models.

Deep learning models have achieved remarkable success in pathology image analysis. However, they still face challenges in effectively modeling fine-grained, object-level features. Topological Data Analysis (TDA) has shown promise for addressing these issues but remains underexplored, particularly for whole-slide pathology applications. Additionally, the effectiveness of TDA has yet to be firmly established, as current studies largely use small-scale datasets. In this work, we address these gaps by introducing Persistent Homology in Multiple Instance Learning (PMIL), the first adaptable TDA-based module within the MIL framework. We validate our approach on a large-scale classification dataset, benchmarking against multiple state-of-the-art methods.

The Coronavirus (COVID-19) outbreak in December 2019 has drastically affected humans worldwide, creating a health crisis that has infected millions of lives and devastated the global economy. COVID-19 is ongoing, with the emergence of many new strains. Deep learning (DL) techniques have proven helpful in efficiently analysing and delineating infectious regions in radiological images. This survey paper draws a taxonomy of deep learning techniques for detecting COVID-19 infection in radiographic imaging modalities Chest X-Ray, and Computer Tomography. DL techniques are broadly categorised into classification, segmentation, and multi-stage approaches for COVID-19 diagnosis at the image and region-level analysis. These techniques are further classified as pre-trained and custom-made Convolutional Neural Network architectures. Furthermore, a discussion is drawn on radiographic datasets, evaluation metrics, and commercial platforms provided for detection. In the end, a brief look is paid to emerging ideas, gaps in existing research, and challenges in developing diagnostic techniques. This survey provides insight into the promising areas of research in DL and is likely to guide the research community on the upcoming development of deep learning techniques for COVID-19. This will pave the way to accelerate the research in designing customised DL-based diagnostic tools for effectively dealing with new variants of COVID-19 and emerging challenges.