Deep learning models are powerful tools for addressing challenging medical imaging problems. However, for an ever-growing range of applications, interpreting a model’s prediction remains non-trivial. Understanding decisions made by black-box algorithms is critical, and assessing their fairness and susceptibility to bias is a key step towards healthcare deployment. In this paper, we propose DBCE (Deformation Based Counterfactual Explainability). We optimise a diffeomorphic transformation that deforms a given input image to change the prediction of the model. This provides anatomically meaningful saliency maps indicating tissue atrophy and expansion, which can be easily interpreted by clinicians. In our test case, DBCE replicates the transition of a patient from healthy control (HC) to Alzheimer’s disease (AD). We benchmark DBCE against three commonly used saliency methods. We show that it provides more meaningful saliency maps when applied to one subject and disease-consistent atrophy patterns when used over a larger cohort. In addition, our method fulfils a recent sanity check and is repeatable for different model initialisations in contrast to classical sensitivity-based methods.

Cortical thickness (CTh) is routinely used to quantify grey matter atrophy as it is a significant biomarker in studying neurodegenerative and neurological conditions. Clinical studies commonly employ one of several available CTh estimation software tools to estimate CTh from brain MRI scans. In recent years, machine learning-based methods emerged as a faster alternative to the main-stream CTh estimation methods (e.g. FreeSurfer). Evaluation and comparison of CTh estimation methods often include various metrics and downstream tasks, but none fully covers the sensitivity to sub-voxel atrophy characteristic of neurodegeneration. In addition, current evaluation methods do not provide a framework for the intra-method region-wise evaluation of CTh estimation methods. Therefore, we propose a method for brain MRI synthesis capable of generating a range of sub-voxel atrophy levels (global and local) with quantifiable changes from the baseline scan. We further create a synthetic test set and evaluate four different CTh estimation methods: FreeSurfer (cross-sectional), FreeSurfer (longitudinal), DL+DiReCT and HerstonNet. DL+DiReCT showed superior sensitivity to sub-voxel atrophy over other methods in our testing framework. The obtained results indicate that our synthetic test set is suitable for benchmarking CTh estimation methods on both global and local scales as well as regional inter-and intra-method performance comparison.

In this paper, we introduce CorticalFlow, a new geometric deep-learning model that, given a 3-dimensional image, learns to deform a reference template towards a targeted object. To conserve the template mesh’s topological properties, we train our model over a set of diffeomorphic transformations. This new implementation of a flow Ordinary Differential Equation (ODE) framework benefits from a small GPU memory footprint, allowing the generation of surfaces with several hundred thousand vertices. To reduce topological errors introduced by its discrete resolution, we derive numeric conditions which improve the manifoldness of the predicted triangle mesh. To exhibit the utility of CorticalFlow, we demonstrate its performance for the challenging task of brain cortical surface reconstruction. In contrast to the current state-of-the-art, CorticalFlow produces superior surfaces while reducing the computation time from nine and a half minutes to one second. More significantly, CorticalFlow enforces the generation of anatomically plausible surfaces; the absence of which has been a major impediment restricting the clinical relevance of such surface reconstruction methods.

The study of neurodegenerative diseases relies on the reconstruction and analysis of the brain cortex from magnetic resonance imaging (MRI). Traditional frameworks for this task like FreeSurfer demand lengthy runtimes, while its accelerated variant FastSurfer still relies on a voxel-wise segmentation which is limited by its resolution to capture narrow continuous objects as cortical surfaces. Having these limitations in mind, we propose DeepCSR, a 3D deep learning framework for cortical surface reconstruction from MRI. Towards this end, we train a neural network model with hypercolumn features to predict implicit surface representations for points in a brain template space. After training, the cortical surface at a desired level of detail is obtained by evaluating surface representations at specific coordinates, and subsequently applying a topology correction algorithm and an isosurface extraction method. Thanks to the continuous nature of this approach and the efficacy of its hypercolumn features scheme, DeepCSR efficiently reconstructs cortical surfaces at high resolution capturing fine details in the cortical folding. Moreover, DeepCSR is as accurate, more precise, and faster than the widely used FreeSurfer toolbox and its deep learning powered variant FastSurfer on reconstructing cortical surfaces from MRI which should facilitate large-scale medical studies and new healthcare applications.

In clinics, the information about the appearance and location of brain tumors is essential to assist doctors in diagnosis and treatment. Automatic brain tumor segmentation on the images acquired by magnetic resonance imaging (MRI) is a common way to attain this information. However, MR images are not quantitative and can exhibit significant variation in signal depending on a range of factors, which increases the difficulty of training an automatic segmentation network and applying it to new MR images. To deal with this issue, this paper proposes to learn a sample-adaptive intensity lookup table (LuT) that dynamically transforms the intensity contrast of each input MR image to adapt to the following segmentation task. Specifically, the proposed deep SA-LuT-Net framework consists of a LuT module and a segmentation module, trained in an end-to-end manner: the LuT module learns a sample-specific nonlinear intensity mapping function through communication with the segmentation module, aiming at improving the final segmentation performance. In order to make the LuT learning sample-adaptive, we parameterize the intensity mapping function by exploring two families of non-linear functions (i.e., piece-wise linear and power functions) and predict the function parameters for each given sample. These sample-specific parameters make the intensity mapping adaptive to samples. We develop our SA-LuT-Nets separately based on two backbone networks for segmentation, i.e., DMFNet and the modified 3D Unet, and validate them on BRATS2018 and BRATS2019 datasets for brain tumor segmentation. Our experimental results clearly demonstrate the superior performance of the proposed SA-LuT-Nets using either single or multiple MR modalities. It not only significantly improves the two baselines (DMFNet and the modified 3D Unet), but also wins a set of state-of-the-art segmentation methods. Moreover, we show that, the LuTs learnt using one segmentation model could also be applied to improving the performance of another segmentation model, indicating the general segmentation information captured by LuTs.

Generative adversarial network (GAN) has been widely explored for cross-modality medical image synthesis. The existing GAN models usually adversarially learn a global sample space mapping from the source-modality to the target-modality and then indiscriminately apply this mapping to all samples in the whole space for prediction. However, due to the scarcity of training samples in contrast to the complicated nature of medical image synthesis, learning a single global sample space mapping that is "optimal" to all samples is very challenging, if not intractable. To address this issue, this paper proposes sample-adaptive GAN models, which not only cater for the global sample space mapping between the source- and the target-modalities but also explore the local space around each given sample to extract its unique characteristic. Specifically, the proposed sample-adaptive GANs decompose the entire learning model into two cooperative paths. The baseline path learns a common GAN model by fitting all the training samples as usual for the global sample space mapping. The new sample-adaptive path additionally models each sample by learning its relationship with its neighboring training samples and using the target-modality features of these training samples as auxiliary information for synthesis. Enhanced by this sample-adaptive path, the proposed sample-adaptive GANs are able to flexibly adjust themselves to different samples, and therefore optimize the synthesis performance. Our models have been verified on three cross-modality MR image synthesis tasks from two public datasets, and they significantly outperform the state-of-the-art methods in comparison. Moreover, the experiment also indicates that our sample-adaptive strategy could be utilized to improve various backbone GAN models. It complements the existing GANs models and can be readily integrated when needed.

Magnetic resonance (MR) imaging is a widely used medical imaging protocol that can be configured to provide different contrasts between the tissues in human body. By setting different scanning parameters, each MR imaging modality reflects the unique visual characteristic of scanned body part, benefiting the subsequent analysis from multiple perspectives. To utilize the complementary information from multiple imaging modalities, cross-modality MR image synthesis has aroused increasing research interest recently. However, most existing methods only focus on minimizing pixel/voxel-wise intensity difference but ignore the textural details of image content structure, which affects the quality of synthesized images. In this paper, we propose edge-aware generative adversarial networks (Ea-GANs) for cross-modality MR image synthesis. Specifically, we integrate edge information, which reflects the textural structure of image content and depicts the boundaries of different objects in images, to reduce this gap. Corresponding to different learning strategies, two frameworks are proposed, i.e., a generator-induced Ea-GAN (gEa-GAN) and a discriminator-induced Ea-GAN (dEa-GAN). The gEa-GAN incorporates the edge information via its generator, while the dEa-GAN further does this from both the generator and the discriminator so that the edge similarity is also adversarially learned. In addition, the proposed Ea-GANs are 3D-based and utilize hierarchical features to capture contextual information. The experimental results demonstrate that the proposed Ea-GANs, especially the dEa-GAN, outperform multiple state-of-the-art methods for cross-modality MR image synthesis in both qualitative and quantitative measures. Moreover, the dEa-GAN also shows excellent generality to generic image synthesis tasks on benchmark datasets about facades, maps, and cityscapes.

Automated detection of small, low level shapes such as circular/spherical objects in images is a challenging computer vision problem. For many applications, especially microbleed detection in Alzheimer's disease, an automatic pre-screening scheme is required to identify potential seeds with high sensitivity and reasonable specificity. A new method is proposed to detect spherical objects in 3D medical images within the multi-scale Laplacian of Gaussian framework. The major contributions are(1)breaking down 3D sphere detection into 1D line profile detection along each coordinate dimension, (2) identifying center of structures bynormalizing the line response profile and (3) employing eigenvalues of the Hessian matrix at optimum scale for the center points to determine spherical objects. The method is validated both on simulated data and susceptibility weighted MRI images with ground truth provided by a medical expert. Validation results demonstrate that the current approach has higher performance in terms of sensitivity and specificity and is effective in detecting adjacent microbleeds, with invariance to intensity, orientation, translation and object scale.

OBJECTIVE: As large-scale medical imaging studies are becoming more common, there is an increasing reliance on automated software to extract quantitative information from these images. As the size of the cohorts keeps increasing with large studies, there is a also a need for tools that allow results from automated image processing and analysis to be presented in a way that enables fast and efficient quality checking, tagging and reporting on cases in which automatic processing failed or was problematic. MATERIALS AND METHODS: MilxXplore is an open source visualization platform, which provides an interface to navigate and explore imaging data in a web browser, giving the end user the opportunity to perform quality control and reporting in a user friendly, collaborative and efficient way. DISCUSSION: Compared to existing software solutions that often provide an overview of the results at the subject's level, MilxXplore pools the results of individual subjects and time points together, allowing easy and efficient navigation and browsing through the different acquisitions of a subject over time, and comparing the results against the rest of the population. CONCLUSIONS: MilxXplore is fast, flexible and allows remote quality checks of processed imaging data, facilitating data sharing and collaboration across multiple locations, and can be easily integrated into a cloud computing pipeline. With the growing trend of open data and open science, such a tool will become increasingly important to share and publish results of imaging analysis.

We report in this communication a new formulation for the cost function of the well-known fuzzy C-means classification technique whereby we introduce weights. We derive the equations of this new weighted fuzzy C-means algorithm (WFCM) in the presence of additive and multiplicative bias field. We show that the weights can be designed in the same manner as prior probabilities commonly used in maximum a posteriori classifier (MAP) to introduce prior knowledge (e.g. using atlas), and increase robustness to noise (e.g. using Markov random field). Using prior probabilities of three popular MAP algorithms, we compare the performances of our proposed WFCM scheme using the simulated MRI T1W BrainWeb datasets, as well as five T1W MR patient scans. Our results show that WFCM achieves superior performances for low SNR conditions, whereas a Gaussian mixture model is desirable for high noise levels. WFCM allows rigorous comparison of fuzzy and probabilistic classifiers, and offers a framework where improvements can be shared between those two types of classifier.

In the last decade, the accessibility of inexpensive and powerful computers has allowed true digital holography to be used for industrial inspection. This technique allows capturing a complex image of a scene (i.e. containing magnitude and phase), and reconstructing the phase and magnitude information. Digital holograms give a new dimension to texture analysis since the topology information can be used as an additional way to extract features. This new technique can be used to extend previous work on image segmentation of patterned wafers for defect detection. This paper presents a combination of features obtained from Gabor filters on different complex images. The combination enables to cope with the intensity variations occurring during the holography and provides final results which are independent from the selected training samples.

The accessibility of inexpensive and powerful computers has allowed true digital holography to be used for industrial inspection using microscopy. This technique allows the capture of a complex image (i.e., one containing magnitude and phase), and the reconstruction of the phase and magnitude information. Digital holograms give a new dimension to texture analysis, since the topology information can be used as an additional way to extract features. This new technique can be used to extend previous work on the image segmentation of patterned wafers for defect detection. The paper presents a comparison between the features obtained using Gabor filtering on complex images under illumination and focus variations.