Current successful choroidal vessel segmentation methods rely on large amounts of voxel-level annotations on the 3D optical coherence tomography images, which are hard and time-consuming. Semi-supervised learning solves this issue by enabling model learning from both unlabeled data and a limited amount of labeled data. A challenge is the defective pseudo labels generated for the unlabeled data. In this work, we propose a model-based label-to-image diffusion (MLD) framework for semi-supervised choroidal vessel segmentation. We first generate pseudo labels from unlabeled images with a coarse correspondence using a model-based strategy. Then, we generate precisely corresponding images of pseudo labels by a hierarchical diffusion probabilistic model. We evaluated our method on myopia data with a new topological connectivity metric. The quantitative and qualitative experimental results indicate the effectiveness of the label-to-image diffusion framework and its benefit for enhancing the existing supervised choroidal segmentation methods. The code is available at: https://github.com/nicetomeetu21/MLD.

Accurate retinal layer segmentation on optical coherence tomography (OCT) images is hampered by the challenges of collecting OCT images with diverse pathological characterization and balanced distribution. Current generative models can produce high-realistic images and corresponding labels without quantitative limitations by fitting distributions of real collected data. Nevertheless, the diversity of their generated data is still limited due to the inherent imbalance of training data. To address these issues, we propose an image-label pair generation framework that generates diverse and balanced potential data from imbalanced real samples. Specifically, the framework first generates diverse layer masks, and then generates plausible OCT images corresponding to these layer masks using two customized diffusion probabilistic models respectively. To learn from imbalanced data and facilitate balanced generation, we introduce pathological-related conditions to guide the generation processes. To enhance the diversity of the generated image-label pairs, we propose a potential structure modeling technique that transfers the knowledge of diverse sub-structures from lowly- or non-pathological samples to highly pathological samples. We conducted extensive experiments on two public datasets for retinal layer segmentation. Firstly, our method generates OCT images with higher image quality and diversity compared to other generative methods. Furthermore, based on the extensive training with the generated OCT images, downstream retinal layer segmentation tasks demonstrate improved results. The code is publicly available at: https://github.com/nicetomeetu21/GenPSM.

Large volume of labeled data is a cornerstone for deep learning (DL) based segmentation methods. Medical images require domain experts to annotate, and full segmentation annotations of large volumes of medical data are difficult, if not impossible, to acquire in practice. Compared with full annotations, image-level labels are multiple orders of magnitude faster and easier to obtain. Image-level labels contain rich information that correlates with the underlying segmentation tasks and should be utilized in modeling segmentation problems. In this article, we aim to build a robust DL-based lesion segmentation model using only image-level labels (normal v.s. abnormal). Our method consists of three main steps: (1) training an image classifier with image-level labels; (2) utilizing a model visualization tool to generate an object heat map for each training sample according to the trained classifier; (3) based on the generated heat maps (as pseudo-annotations) and an adversarial learning framework, we construct and train an image generator for Edema Area Segmentation (EAS). We name the proposed method Lesion-Aware Generative Adversarial Networks (LAGAN) as it combines the merits of supervised learning (being lesion-aware) and adversarial training (for image generation). Additional technical treatments, such as the design of a multi-scale patch-based discriminator, further enhance the effectiveness of our proposed method. We validate the superior performance of LAGAN via comprehensive experiments on two publicly available datasets (i.e., AI Challenger and RETOUCH).

Automated layer segmentation plays an important role for retinal disease diagnosis in optical coherence tomography (OCT) images. However, the severe retinal diseases result in the performance degeneration of automated layer segmentation approaches. In this paper, we present a robust semi-supervised layer segmentation network to relieve the model failures on abnormal retinas. We obtain the lesion features from the labeled images with disease-balanced distribution, and utilize the unlabeled images to supplement the layer structure information. Specifically, in our method, the cross-consistency training is utilized over the predictions of different decoders, and we enforce a consistency between different decoder predictions to improve the encoder's representation. Then, we propose a sequence prediction branch based on self-supervised manner, which is designed to predict the position of each jigsaw puzzle to obtain sensory perception of the retinal layer structure. To this task, a layer spatial pyramid pooling (LSPP) module is designed to extract multi-scale layer spatial features. Furthermore, we use the optical coherence tomography angiography (OCTA) to supplement the information damaged by diseases. The experimental results illustrate that our method achieves more robust results compared with current supervised segmentation methods. Meanwhile, advanced segmentation performance can be obtained compared with state-of-the-art semi-supervised segmentation methods.

Retinal related diseases are the leading cause of vision loss, and severe retinal lesion causes irreversible damage to vision. Therefore, the automatic methods for retinal diseases detection based on medical images is essential for timely treatment. Considering that manual diagnosis and analysis of medical images require a large number of qualified experts, deep learning can effectively diagnosis and locate critical biomarkers. In this paper, we present a novel model by jointly optimize the cycle generative adversarial network (CycleGAN) and the convolutional neural network (CNN) to detect retinal diseases and localize lesion areas with limited training data. The CycleGAN with cycle consistency can generate more realistic and reliable images. The discriminator and the generator achieve a local optimal solution in an adversarial manner, and the generator and the classifier are in a cooperative manner to distinguish the domain of input images. A novel res-guided sampling block is proposed by combining learnable residual features and pixel-adaptive convolutions. A res-guided U-Net is constructed as the generator by substituting the traditional convolution with the res-guided sampling blocks. Our model achieve superior classification and localization performance on LAG, Ichallenge-PM and Ichallenge-AMD datasets. With clear localization for lesion areas, the competitive results reveal great potentials of the joint optimization network. The source code is available at https://github.com/jizexuan/JointOptmization.

Choroidal neovascularization (CNV) volume prediction has an important clinical significance to predict the therapeutic effect and schedule the follow-up. In this paper, we propose a Lesion Attention Maps-Guided Network (LamNet) to automatically predict the CNV volume of next follow-up visit after therapy based on 3-dimentional spectral-domain optical coherence tomography (SD-OCT) images. In particular, the backbone of LamNet is a 3D convolutional neural network (3D-CNN). In order to guide the network to focus on the local CNV lesion regions, we use CNV attention maps generated by an attention map generator to produce the multi-scale local context features. Then, the multi-scale of both local and global feature maps are fused to achieve the high-precision CNV volume prediction. In addition, we also design a synergistic multi-task predictor, in which a trend-consistent loss ensures that the change trend of the predicted CNV volume is consistent with the real change trend of the CNV volume. The experiments include a total of 541 SD-OCT cubes from 68 patients with two types of CNV captured by two different SD-OCT devices. The results demonstrate that LamNet can provide the reliable and accurate CNV volume prediction, which would further assist the clinical diagnosis and design the treatment options.

One major cause of performance degradation in predictive models is that the test samples are not well covered by the training data. Such not well-represented samples are called OoD samples. In this article, we propose OoDAnalyzer, a visual analysis approach for interactively identifying OoD samples and explaining them in context. Our approach integrates an ensemble OoD detection method and a grid-based visualization. The detection method is improved from deep ensembles by combining more features with algorithms in the same family. To better analyze and understand the OoD samples in context, we have developed a novelkNN-based grid layout algorithm motivated by Hall's theorem. The algorithm approximates the optimal layout and has O(kN2)O(kN2) time complexity, faster than the grid layout algorithm with overall best performance but O(N3)O(N3) time complexity. Quantitative evaluation and case studies were performed on several datasets to demonstrate the effectiveness and usefulness of OoDAnalyzer.

The presence of hyperreflective foci (HFs) is related to retinal disease progression, and the quantity has proven to be a prognostic factor of visual and anatomical outcome in various retinal diseases. However, lack of efficient quantitative tools for evaluating the HFs has deprived ophthalmologist of assessing the volume of HFs. For this reason, we propose an automated quantification algorithm to segment and quantify HFs in spectral domain optical coherence tomography (SD-OCT). The proposed algorithm consists of two parallel processes namely: region of interest (ROI) generation and HFs estimation. To generate the ROI, we use morphological reconstruction to obtain the reconstructed image and histogram constructed for data distributions and clustering. In parallel, we estimate the HFs by extracting the extremal regions from the connected regions obtained from a component tree. Finally, both the ROI and the HFs estimation process are merged to obtain the segmented HFs. The proposed algorithm was tested on 40 3D SD-OCT volumes from 40 patients diagnosed with non-proliferative diabetic retinopathy (NPDR), proliferative diabetic retinopathy (PDR), and diabetic macular edema (DME). The average dice similarity coefficient (DSC) and correlation coefficient (r) are 69.70%, 0.99 for NPDR, 70.31%, 0.99 for PDR, and 71.30%, 0.99 for DME, respectively. The proposed algorithm can provide ophthalmologist with good HFs quantitative information, such as volume, size, and location of the HFs.

We present an image projection network (IPN), which is a novel end-to-end architecture and can achieve 3D-to-2D image segmentation in optical coherence tomography angiography (OCTA) images. Our key insight is to build a projection learning module (PLM) which uses a unidirectional pooling layer to conduct effective features selection and dimension reduction concurrently. By combining multiple PLMs, the proposed network can input 3D OCTA data, and output 2D segmentation results such as retinal vessel segmentation. It provides a new idea for the quantification of retinal indicators: without retinal layer segmentation and without projection maps. We tested the performance of our network for two crucial retinal image segmentation issues: retinal vessel (RV) segmentation and foveal avascular zone (FAZ) segmentation. The experimental results on 316 OCTA volumes demonstrate that the IPN is an effective implementation of 3D-to-2D segmentation networks, and the uses of multi-modality information and volumetric information make IPN perform better than the baseline methods.

In this paper, an automatic method is reported for simultaneously segmenting layers and fluid in 3-D OCT retinal images of subjects suffering from central serous retinopathy. To enhance contrast between adjacent layers, multiscale bright and dark layer detection filters are proposed. Due to appearance of serous fluid or pigment epithelial detachment caused fluid, contrast between adjacent layers is often reduced, and also large morphological changes are caused. In addition, 24 features are designed for random forest classifiers. Then, 8 coarse surfaces are obtained based on the trained random forest classifiers. Finally, a hypergraph is constructed based on the smoothed image and the layer structure detection responses. A modified live wire algorithm is proposed to accurately detect surfaces between retinal layers, even though OCT images with fluids are of low contrast and layers are largely deformed. The proposed method was evaluated on 48 spectral domain OCT images with central serous retinopathy. The experimental results showed that the proposed method outperformed the state-of-art methods with regard to layers and fluid segmentation.