Large-scale datasets are essential for training deep learning models in medical imaging. However, many of these datasets contain poor-quality images that can compromise model performance and clinical reliability. In this study, we propose a framework to detect non-compliant images, such as corrupted scans, incomplete thorax X-rays, and images of non-thoracic body parts, by leveraging contrastive learning for feature extraction and parametric or non-parametric scoring methods for out-of-distribution ranking. Our approach was developed and tested on the CheXpert dataset, achieving an AUC of 0.75 in a manually labeled subset of 1,000 images, and further qualitatively and visually validated on the external PadChest dataset, where it also performed effectively. Our results demonstrate the potential of contrastive learning to detect non-compliant images in large-scale medical datasets, laying the foundation for future work on reducing dataset pollution and improving the robustness of deep learning models in clinical practice.

Machine Learning models need large amounts of annotated data for training. In the field of medical imaging, labeled data is especially difficult to obtain because the annotations have to be performed by qualified physicians. Natural Language Processing (NLP) tools can be applied to radiology reports to extract labels for medical images automatically. Compared to manual labeling, this approach requires smaller annotation efforts and can therefore facilitate the creation of labeled medical image data sets. In this article, we summarize the literature on this topic spanning from 2013 to 2023, starting with a meta-analysis of the included articles, followed by a qualitative and quantitative systematization of the results. Overall, we found four types of studies on the extraction of labels from radiology reports: those describing systems based on symbolic NLP, statistical NLP, neural NLP, and those describing systems combining or comparing two or more of the latter. Despite the large variety of existing approaches, there is still room for further improvement. This work can contribute to the development of new techniques or the improvement of existing ones.

The integration of explanation techniques promotes the comprehension of a model’s output and contributes to its interpretation e.g. by generating heat maps highlighting the most decisive regions for that prediction. However, there are several drawbacks to the current heat map-generating methods. Probability by itself is not indicative of the model’s conviction in a prediction, as it is influenced by multiple factors, such as class imbalance. Consequently, it is possible that a model yields two true positive predictions - one with an accurate explanation map, and the other with an inaccurate one. Current state-of-the-art explanations are not able to distinguish both scenarios and alert the user to dubious explanations. The goal of this work is to represent these maps more intuitively based on how confident the model is regarding the diagnosis, by adding an extra validation step over the state-of-the-art results that indicates whether the user should trust the initial explanation or not. The proposed method, Confident-CAM, facilitates the interpretation of the results by measuring the distance between the output probability and the corresponding class threshold, using a confidence score to generate nearly null maps when the initial explanations are most likely incorrect. This study implements and validates the proposed algorithm on a multi-label chest X-ray classification exercise, targeting 14 radiological findings in the ChestX-Ray14 dataset with significant class imbalance. Results indicate that confidence scores can distinguish likely accurate and inaccurate explanations. Code available via GitHub.

The importance of X-Ray imaging analysis is paramount for health care institutions since it is the main imaging modality for patient diagnosis, and deep learning can be used to aid clinicians in image diagnosis or structure segmentation. In recent years, several articles demonstrate the capability that deep learning models have in classifying and segmenting chest x-ray images if trained in an annotated dataset. Unfortunately, for segmentation tasks, only a few relatively small datasets have annotations, which poses a problem for the training of robust deep learning strategies. In this work, a semi-supervised approach is developed which consists of using available information regarding other anatomical structures to guide the segmentation when the groundtruth segmentation for a given structure is not available. This semi-supervised is compared with a fully- supervised approach for the tasks of lung segmentation and for multi-structure segmentation (lungs, heart and clavicles) in chest x-ray images. The semi-supervised lung predictions are evaluated visually and show relevant improvements, therefore this approach could be used to improve performance in external datasets with missing groundtruth. The multi-structure predictions show an improvement in mean absolute and Hausdorff distances when compared to a fully supervised approach and visual analysis of the segmentations shows that false positive predictions are removed. In conclusion, the developed method results in a new strategy that can help solve the problem of missing annotations and increase the quality of predictions in new datasets.

The development of accurate methods for OCT image analysis is highly dependent on the availability of large annotated datasets. As such datasets are usually expensive and hard to obtain, novel approaches based on deep generative models have been proposed for data augmentation. In this work, a flow-based network (SRFlow) and a generative adversarial network (ESRGAN) are used for synthesizing high-resolution OCT B-scans from low-resolution versions of real OCT images. The quality of the images generated by the two models is assessed using two standard fidelity-oriented metrics and a learned perceptual quality metric. The performance of two classification models trained on real and synthetic images is also evaluated. The obtained results show that the images generated by SRFlow preserve higher fidelity to the ground truth, while the outputs of ESRGAN present, on average, better perceptual quality. Independently of the architecture of the network chosen to classify the OCT B-scans, the model's performance always improves when images generated by SRFlow are included in the training set.

Chest radiography is one of the most ubiquitous medical imaging modalities. Nevertheless, the interpretation of chest radiography images is time-consuming, complex and subject to observer variability. As such, automated diagnosis systems for pathology detection have been proposed, aiming to reduce the burden on radiologists. The advent of deep learning has fostered the development of solutions for both abnormality detection with promising results. However, these tools suffer from poor explainability as the reasons that led to a decision cannot be easily understood, representing a major hurdle for their adoption in clinical practice. In order to overcome this issue, a method for chest radiography abnormality detection is presented which relies on an object detection framework to detect individual findings and thus separate normal and abnormal CXRs. It is shown that this framework is capable of an excellent performance in abnormality detection (AUC: 0.993), outperforming other state- of-the-art classification methodologies (AUC: 0.976 using the same classes). Furthermore, validation on external datasets shows that the proposed framework has a smaller drop in performance when applied to previously unseen data (21.9 % vs 23.4 % on average). Several approaches for object detection are compared and it is shown that merging pathology classes to minimize radiologist variability improves the localization of abnormal regions (0.529 vs 0.491 APF when using all pathology classes), resulting in a network which is more explainable and thus more suitable for integration in clinical practice.

Backed by more powerful computational resources and optimized training routines, deep learning models have attained unprecedented performance in extracting information from chest X-ray data. Preceding other tasks, an automated abnormality detection stage can be useful to prioritize certain exams and enable a more efficient clinical workflow. However, the presence of image artifacts such as lettering often generates a harmful bias in the classifier, leading to an increase of false positive results. Consequently, health care would benefit from a system that selects the thoracic region of interest prior to deciding whether an image is possibly pathologic. The current work tack-les this binary classification exercise using an attention-driven and spatially unsupervised Spatial Transformer Network (STN). The results indicate that the STN achieves similar results to using YOLO-cropped images, with fewer computational expenses and without the need for localization labels. More specifically, the system is able to distinguish between normal and abnormal CheXpert Images with a mean AUC of 84.22%.

Deep Learning (DL) algorithms allow fast results with high accuracy in medical imaging analysis solutions. However, to achieve a desirable performance, they require large amounts of high quality data. Active Learning (AL) is a subfield of DL that aims for more efficient models requiring ideally fewer data, by selecting the most relevant information for training. CheXpert is a Chest X-Ray (CXR) dataset, containing labels for different pathologic findings, alongside a “Support Devices” (SD) label. The latter contains several misannotations, which may impact the performance of a pathology detection model. The aim of this work is the detection of SDs in CheXpert CXR images and the comparison of the resulting predictions with the original CheXpert SD annotations, using AL approaches. A subset of 10,220 images was selected, manually annotated for SDs and used in the experimentations. In the first experiment, an initial model was trained on the seed dataset (6,200 images from this subset). The second and third approaches consisted in AL random sampling and least confidence techniques. In both of these, the seed dataset was used initially, and more images were iteratively employed. Finally, in the fourth experiment, a model was trained on the full annotated set. The AL least confidence experiment outperformed the remaining approaches, presenting an AUC of 71.10% and showing that training a model with representative information is favorable over training with all labeled data. This model was used to obtain predictions, which can be useful to limit the use of SD mislabelled images in future models.

Chest radiography is one of the most ubiquitous imaging modalities, playing an essential role in screening, diagnosis and disease management. However, chest radiography interpretation is a time-consuming and complex task, requiring the availability of experienced radiologists. As such, automated diagnosis systems for pathology detection have been proposed aiming to reduce the burden on radiologists and reduce variability in image interpretation. While promising results have been obtained, particularly since the advent of deep learning, there are significant limitations in the developed solutions, namely the lack of representative data for less frequent pathologies and the learning of biases from the training data, such as patient position, medical devices and other markers as proxies for certain pathologies. The lack of explainability is also a challenge for the adoption of these solutions in clinical practice.Generative adversarial networks could play a significant role as a solution for these challenges as they allow to artificially create new realistic images. This way, new synthetic chest radiography images could be used to increase the prevalence of less represented pathology classes and decrease model biases as well as improving the explainability of automatic decisions by generating samples that serve as examples or counter-examples to the image being analysed, ensuring patient privacy.In this study, a few-shot generative adversarial network is used to generate synthetic chest radiography images. A minimum Fréchet Inception Distance score of 17.83 was obtained, allowing to generate convincing synthetic images. Perceptual validation was then performed by asking multiple readers to classify a mixed set of synthetic and real images. An average accuracy of 83.5% was obtained but a strong dependency on reader experience level was observed. While synthetic images showed structural irregularities, the overall image sharpness was a major factor in the decision of readers. The synthetic images were then validated using a MobileNet abnormality classifier and it was shown that over 99% of images were classified correctly, indicating that the generated images were correctly interpreted by the classifier. Finally, the use of the synthetic images during training of a YOLOv5 pathology detector showed that the addition of the synthetic images led to an improvement of mean average precision of 0.05 across 14 pathologies.In conclusion, the usage of few-shot generative adversarial networks for chest radiography image generation was shown and tested in multiple scenarios, establishing a baseline for future experiments to increase the applicability of generative models in clinical scenarios of automatic CXR screening and diagnosis tools.

Diabetic retinopathy (DR) grading is crucial in determining the adequate treatment and follow up of patient, but the screening process can be tiresome and prone to errors. Deep learning approaches have shown promising performance as computer-aided diagnosis (CAD) systems, but their black-box behaviour hinders clinical application. We propose DR|GRADUATE, a novel deep learning-based DR grading CAD system that supports its decision by providing a medically interpretable explanation and an estimation of how uncertain that prediction is, allowing the ophthalmologist to measure how much that decision should be trusted. We designed DR|GRADUATE taking into account the ordinal nature of the DR grading problem. A novel Gaussian-sampling approach built upon a Multiple Instance Learning framework allow DR|GRADUATE to infer an image grade associated with an explanation map and a prediction uncertainty while being trained only with image-wise labels. DR|GRADUATE was trained on the Kaggle DR detection training set and evaluated across multiple datasets. In DR grading, a quadratic-weighted Cohen's kappa (κ) between 0.71 and 0.84 was achieved in five different datasets. We show that high κ values occur for images with low prediction uncertainty, thus indicating that this uncertainty is a valid measure of the predictions' quality. Further, bad quality images are generally associated with higher uncertainties, showing that images not suitable for diagnosis indeed lead to less trustworthy predictions. Additionally, tests on unfamiliar medical image data types suggest that DR|GRADUATE allows outlier detection. The attention maps generally highlight regions of interest for diagnosis. These results show the great potential of DR|GRADUATE as a second-opinion system in DR severity grading.

22nd International Conference on Knowledge-Based and Intelligent Information and Engineering Systems, KES 2018, Belgrade, 3 September 2018 - 5 September 2018

In medical image analysis applications, the availability of the large amounts of annotated data is becoming increasingly critical. However, annotated medical data is often scarce and costly to obtain. In this paper, we address the problem of synthesizing retinal color images by applying recent techniques based on adversarial learning. In this setting, a generative model is trained to maximize a loss function provided by a second model attempting to classify its output into real or synthetic. In particular, we propose to implement an adversarial autoencoder for the task of retinal vessel network synthesis. We use the generated vessel trees as an intermediate stage for the generation of color retinal images, which is accomplished with a generative adversarial network. Both models require the optimization of almost everywhere differentiable loss functions, which allows us to train them jointly. The resulting model offers an end-to-end retinal image synthesis system capable of generating as many retinal images as the user requires, with their corresponding vessel networks, by sampling from a simple probability distribution that we impose to the associated latent space. We show that the learned latent space contains a well-defined semantic structure, implying that we can perform calculations in the space of retinal images, e.g., smoothly interpolating new data points between two retinal images. Visual and quantitative results demonstrate that the synthesized images are substantially different from those in the training set, while being also anatomically consistent and displaying a reasonable visual quality.

Image quality assessment has been a topic of intense research over the last decades. Although its application to other disciplines is growing tremendously, its use in retinal imaging is still immature and some fundamental challenges remain unsolved. Thus, we present a research methodology for the objective assessment of the quality in retinal images. The methodology can be used as a preliminary step in any computer-aided system, and is composed of four main steps: the location of the region-of-interest, the extraction of relevant image properties and their analysis by feature selection, and the final binary classification into two classes (good and poor quality). The experimental results demonstrate the adequacy of the proposed methodology in this context, being able to objectively assess the quality of retinal images with an accuracy over 99%.

The classification of retinal vessels into artery/vein (A/V) is an important phase for automating the detection of vascular changes, and for the calculation of characteristic signs associated with several systemic diseases such as diabetes, hypertension, and other cardiovascular conditions. This paper presents an automatic approach for A/V classification based on the analysis of a graph extracted from the retinal vasculature. The proposed method classifies the entire vascular tree deciding on the type of each intersection point (graph nodes) and assigning one of two labels to each vessel segment (graph links). Final classification of a vessel segment as A/V is performed through the combination of the graph-based labeling results with a set of intensity features. The results of this proposed method are compared with manual labeling for three public databases. Accuracy values of 88.3%, 87.4%, and 89.8% are obtained for the images of the INSPIRE-AVR, DRIVE, and VICAVR databases, respectively. These results demonstrate that our method outperforms recent approaches for A/V classification.

This paper presents an automatic approach for the estimation of Arteriolar-to-Venular Ratio (AVR) in retinal images. The method was assessed using the images of the INSPIRE-AVR database. A mean error of 0.05 was obtained when the method's results were compared with reference AVR values provided with this dataset, thus demonstrating the adequacy of the proposed solution for AVR estimation.

Plant development is orchestrated by transcription factors whose expression has become observable in living plants through the use of fluorescence microscopy. However, the exact quantification of expression levels is still not solved and most analysis is only performed through visual inspection. With the objective of automating the quantification of cell nuclei fluorescence we present a new approach to detect cell nuclei in 3D fluorescence confocal microscopy, based on the use of the sliding band convergence filter (SBF). The SBF filter detects cell nuclei and estimate their shape with high accuracy in each 2D image plane. For 3D detection, individual 2D shapes are joined into 3D estimates and then corrected based on the analysis of the fluorescence profile. The final nuclei detection's precision/recall are of 0.779/0.803 respectively, and the average Dice's coefficient of 0.773.

Microscopy cell image analysis is a fundamental tool for biological research. In particular, multivariate fluorescence microscopy is used to observe different aspects of cells in cultures. It is still common practice to perform analysis tasks by visual inspection of individual cells which is time consuming, exhausting and prone to induce subjective bias. This makes automatic cell image analysis essential for large scale, objective studies of cell cultures. Traditionally the task of automatic cell analysis is approached through the use of image segmentation methods for extraction of cells' locations and shapes. Image segmentation, although fundamental, is neither an easy task in computer vision nor is it robust to image quality changes. This makes image segmentation for cell detection semi-automated requiring frequent tuning of parameters. We introduce a new approach for cell detection and shape estimation in multivariate images based on the sliding band filter (SBF). This filter's design makes it adequate to detect overall convex shapes and as such it performs well for cell detection. Furthermore, the parameters involved are intuitive as they are directly related to the expected cell size. Using the SBF filter we detect cells' nucleus and cytoplasm location and shapes. Based on the assumption that each cell has the same approximate shape center in both nuclei and cytoplasm fluorescence channels, we guide cytoplasm shape estimation by the nuclear detections improving performance and reducing errors. Then we validate cell detection by gathering evidence from nuclei and cytoplasm channels. Additionally, we include overlap correction and shape regularization steps which further improve the estimated cell shapes. The approach is evaluated using two datasets with different types of data: a 20 images benchmark set of simulated cell culture images, containing 1000 simulated cells; a 16 images Drosophila melanogaster Kc167 dataset containing 1255 cells, stained for DNA and actin. Both image datasets present a difficult problem due to the high variability of cell shapes and frequent cluster overlap between cells. On the Drosophila dataset our approach achieved a precision/recall of 95%/69% and 82%/90% for nuclei and cytoplasm detection respectively and an overall accuracy of 76%.

The study of cancer cell invasion under the effect of different conditions is fundamental for the understanding of the cancer invasion mechanism and to test possible therapies for its regulation. To simulate invasion across tissue basement membrane, biologists established in vitro assays with cancer cells invading extracellular matrix components. However, analysis of such assays is manual, being timeconsuming and error-prone, which motivates an objective and automated analysis tool. Towards automating such analysis we present a methodology to detect cells in 3D matrix cell assays and correctly estimate their invasion, measured by the depth of the penetration in the gel. Detection is based on the sliding band filter, by evaluating the gradient convergence and not intensity. As such it can detect low contrast cells which otherwise would be lost. For cell depth estimation we present a focus estimator based on the convergence gradients magnitude. The final cell detections precision and recall are of 0.896 and 0.910 respectively, and the average error in the cells position estimate is of 0.41µm, 0.37µm and 3.7µm in the x, y and z directions, respectively.

In vivo observation of cells in the Arabidopsis thaliana root, by time-lapse confocal microscopy, is central to biology research. The research herein described is based on large amount of image data, which must be analyzed to determine the location and state of individual cells. Automating the process of cell tracking is an important step to create tools which will facilitate the analysis of cellspsila evolution through time. Here we introduce a confocal tracking system designed in two stages. At the image acquisition stage, we track the area under analysis based on point-to-point correspondences and motion estimation. After image acquisition, we compute cell-to-cell correspondences through time. The final result is a temporal structured information about each cell.

This paper presents an automated method for the segmentation of the vascular network in retinal images. The algorithm starts with the extraction of vessel centerlines, which are used as guidelines for the subsequent vessel filling phase. For this purpose, the outputs of four directional differential operators are processed in order to select connected sets of candidate points to be further classified as centerline pixels using vessel derived features. The final segmentation is obtained using an iterative region growing method that integrates the contents of several binary images resulting from vessel width dependent morphological filters. Our approach was tested on two publicly available databases and its results are compared with recently published methods. The results demonstrate that our algorithm outperforms other solutions and approximates the average accuracy of a human observer without a significant degradation of sensitivity and specificity.

Registration methods invariably demand the evaluation of similarity between image areas using criteria based on the maintenance of pixel relative intensities. Because of the severe brightness changes in retinal images, intensity dependent similarity measures are often unable to produce useful registration results. The main purpose of this paper is the presentation of a new similarity evaluation criterion, prepared to overcome the problems raised by important brightness modifications in the images to register. The new criterion is based on the detection of edge point localizations, which are used to assess the correspondence of the compared areas. Some attention was also dedicated to the measure calculation process, and, as an ultimate result, a two-stage implementation was adopted. The values generated by the similarity measure determination step are later used by two registration algorithms prepared to compensate for distinct geometric misalignments between the images.>