The rising prevalence of vision-threatening eye diseases poses a major global health and economic burden, yet timely diagnosis remains limited by workforce shortages, diagnostic delays, and restricted access to specialized care. Artificial intelligence (AI) offers potential solutions. In particular, recent progress in foundation models and large language models-especially multimodal large language models (MLLMs)-has shown promise in medical image interpretation and automated clinical documentation. However, advancing MLLMs for ophthalmology is hindered by the lack of unified, comprehensive benchmark datasets for development and evaluation. Most existing benchmarks were designed for earlier models, which focused on narrow tasks or specific disease conditions. These benchmarks typically provide outputs in the form of disease labels rather than free-text responses. As a result, they are less suitable for assessing emerging generative models. In this work, we present LMOD+, a large-scale multimodal ophthalmology benchmark dataset comprising 32,633 instances with multi-granular annotations across 12 common ophthalmic conditions and 5 imaging modalities. The dataset integrates imaging, anatomical structures, demographics, and free-text annotations. It supports primary ophthalmic applications such as anatomical structure recognition, disease screening, disease staging, and demographic prediction for potential performance bias evaluation. Alongside the dataset, we introduce a systematic and unified data curation pipeline that repurposes existing or new datasets for MLLM development. LMOD+ extends our preliminary LMOD benchmark-the first multimodal ophthalmology benchmark for MLLMs-with three major enhancements. First, we expanded the dataset by nearly 50% (from 21,933 to 32,633 instances). The color fundus photography (CFP) modality, the most accessible imaging modality in ophthalmology, was significantly enlarged to cover a broader range of pathological conditions. Second, we broadened task coverage to include (a) 12 binary disease diagnosis tasks for prevalent conditions such as diabetic retinopathy, age-related macular degeneration, and retinal vein occlusion; (b) multi-class ophthalmic disease diagnosis; (c) disease severity classification, including a diabetic retinopathy staging task, which uses two internationally adopted grading standards: the international clinical diabetic retinopathy classification and the Scottish diabetic retinopathy grading scheme classification; and (d) demographic prediction (age and sex) to assess potential model bias. Third, we systematically evaluated 24 state-of-the-art MLLMs, including recent models from the InternVL, Qwen, and DeepSeek families. Our evaluations highlight both the promise and limitations of current MLLMs in ophthalmology. For example, Qwen-7B and InternVL achieved accuracies of 58.26% and 57.83% in disease screening under a zero-shot setting with a single model-a considerably more challenging paradigm than traditional fine-tuning, where separate models are trained for each specific task. InternVL also demonstrated potential in anatomical recognition. Nonetheless, overall performance remained suboptimal and often close to random baselines for challenging tasks such as disease staging, underscoring the substantial gap between general-domain MLLMs and the specialized requirements of ophthalmology. We publicly release the dataset, curation pipeline, and leaderboard to encourage community-wide development and evaluation of MLLMs, with the goal of advancing ophthalmic applications and ultimately reducing the global burden of vision-threatening diseases through AI. The dataset website, benchmark leaderboard, and download link are available at https://kfzyqin.github.io/lmod_plus.

Glaucoma is a chronic retinal disease that gradually damages the optic nerve. It is a leading cause of irreversible loss of vision. In ophthalmic fundus images, the cup to optic disc ratio measured around the optic nerve is a key measure used to screen for glaucomatous damages. Unfortunately, there is high subjectivity among ophthalmologists in estimating this ratio due to challenges in making reliable disc and cup measurements. To minimize this, we propose an automatic method using deep learning and ensemble method to segment the optic disc and cup. The proposed method comprises two steps. The region of interest (ROI), where optic disc is centered, is detected from a fundus image, following which the optic disc and cup are segmented from the ROI. Mask R-CNN algorithm is used to estimate the ROI, and two ensemble models based on three fully convolutional networks are used for the segmentation of optic disc and cup in parallel. The proposed method is trained and evaluated using the RIGA dataset that contains 750 fundus images and the REFUGE database containing 400 fundus images. The results demonstrate that the proposed method has a better performance compared with the current state-of-the-art algorithms. Our best segmentation results for optic disc shows 0.9303 Jaccard Index (JI) and 0.9635 Dice Coefficient (DC). The best segmentation results for cup shows 0.8096 JI and 0.8915 DC. The average cup to optic disc ratio error shows 0.0429.

Convolutional neural networks (CNNs) have been widely applied to medical images. However, medical images are vulnerable to adversarial attacks by perturbations that are undetectable to human experts. This poses significant security risks and challenges to CNN-based applications in clinic practice. In this work, we quantify the scale of adversarial perturbation imperceptible to clinical practitioners and investigate the cause of the vulnerability in CNNs. Specifically, we discover that noise (i.e., irrelevant or corrupted discriminative information) in medical images might be a key contributor to performance deterioration of CNNs against adversarial perturbations, as noisy features are learned unconsciously by CNNs in feature representations and magnified by adversarial perturbations. In response, we propose a novel defense method by embedding sparsity denoising operators in CNNs for improved robustness. Tested with various state-of-the-art attacking methods on two distinct medical image modalities, we demonstrate that the proposed method can successfully defend against those unnoticeable adversarial attacks by retaining as much as over 90% of its original performance. We believe our findings are critical for improving and deploying CNN-based medical applications in real-world scenarios.

OBJECTIVE: Reticular pseudodrusen (RPD), a key feature of age-related macular degeneration (AMD), are poorly detected by human experts on standard color fundus photography (CFP) and typically require advanced imaging modalities such as fundus autofluorescence (FAF). The objective was to develop and evaluate the performance of a novel multimodal, multitask, multiattention (M3) deep learning framework on RPD detection. MATERIALS AND METHODS: A deep learning framework (M3) was developed to detect RPD presence accurately using CFP alone, FAF alone, or both, employing >8000 CFP-FAF image pairs obtained prospectively (Age-Related Eye Disease Study 2). The M3 framework includes multimodal (detection from single or multiple image modalities), multitask (training different tasks simultaneously to improve generalizability), and multiattention (improving ensembled feature representation) operation. Performance on RPD detection was compared with state-of-the-art deep learning models and 13 ophthalmologists; performance on detection of 2 other AMD features (geographic atrophy and pigmentary abnormalities) was also evaluated. RESULTS: For RPD detection, M3 achieved an area under the receiver-operating characteristic curve (AUROC) of 0.832, 0.931, and 0.933 for CFP alone, FAF alone, and both, respectively. M3 performance on CFP was very substantially superior to human retinal specialists (median F1 score = 0.644 vs 0.350). External validation (the Rotterdam Study) demonstrated high accuracy on CFP alone (AUROC, 0.965). The M3 framework also accurately detected geographic atrophy and pigmentary abnormalities (AUROC, 0.909 and 0.912, respectively), demonstrating its generalizability. CONCLUSIONS: This study demonstrates the successful development, robust evaluation, and external validation of a novel deep learning framework that enables accessible, accurate, and automated AMD diagnosis and prognosis.

Glaucoma is one of the most common eye diseases that can cause irreversible vision loss due to damage to the optic nerve. Ophthalmologists consider a cup to optic disc ratio greater than 0.3 to be suggestive of glaucoma. Unfortunately, there is high variability among ophthalmologists in estimating the ratio since it is not easy to reliably measure optic disc and cup areas in a fundus image. Therefore, this paper proposes automatic methods to segment the optic disc and cup areas. There are two steps to estimate the ratio: region of interest (ROI) area detection (where optic disc is in the center) from a fundus image, followed by optic disc and cup segmentation. This paper focuses on automated methods to segment the optic disc and cup from the ROI. Fully convolutional networks (FCN) with U-Net architectures are used for the segmentation. The RIGA dataset (composed of three different fundus image datasets: MESSIDOR, Bin Rushed, and Magrabi), containing 750 fundus images, is used to train and test the FCNs. Our proposed FCNs show relatively better performance than other existing algorithms. The best segmentation results for optic disc show 0.95 Jaccard index, 0.98 F-measure, and 0.99 accuracy. The best segmentation results for cup show 0.80 Jaccard index, 0.88 F-measure, and 0.99 accuracy.

Ophthalmologists use the optic disc to cup ratio as one of the factors to diagnose glaucoma. The region of interest (ROI) for glaucoma in fundus images is the area that locates optic disc and cup in the center. Therefore, ROI detection is used as a preprocessing step for automatic detection of optic disc and cup areas. This paper proposes an automated method to detect ROI using deep learning. Convolutional Neural Networks (CNNs) are used to classify ROI and non-ROI images. The structure of our CNNs is composed of two convolutional layers, two Max Pooling layers, two fully connected layers, and one output layer. We train two CNNs using fundus images from the MESSIDOR dataset, a public dataset containing 1,200 fundus images. In addition, we estimate blood vessels from the images and use the images embedded with the blood vessels to train two other CNNs. The proposed method moves a window in the horizontal and vertical directions in each fundus image, estimates a probability of each window using the CNNs, and selects the window with the highest probability as ROI. The experimental results are promising. The best-performing CNN from the first CNN group shows over 0.99 accuracy for the MESSIDOR dataset and over 0.93 accuracy for five other public fundus image datasets. The best CNN from the second CNN group shows more robust results: over 0.99 accuracy for the MESSIDOR dataset and over 0.97 accuracy for the five other image datasets.