The joint modeling of genetic data and brain imaging information allows for determining the pathophysiological pathways of neurodegenerative diseases such as Alzheimer's disease (AD). This task has typically been approached using mass-univariate methods that rely on a complete set of Single Nucleotide Polymorphisms (SNPs) to assess their association with selected image-derived phenotypes (IDPs). However, such methods are prone to multiple comparisons bias and, most importantly, fail to account for potential cross-feature interactions, resulting in insufficient detection of significant associations. Ways to overcome these limitations while reducing the number of traits aim at conveying genetic information at the gene level and capturing the integrated genetic effects of a set of genetic variants, rather than looking at each SNP individually. Their associations with brain IDPs are still largely unexplored in the current literature, though they can uncover new potential genetic determinants for brain modulations in the AD continuum. In this work, we explored an explainable multivariate model to analyze the genetic basis of the grey matter modulations, relying on the AD Neuroimaging Initiative (ADNI) phase 3 dataset. Cortical thicknesses and subcortical volumes derived from T1-weighted Magnetic Resonance were considered to describe the imaging phenotypes. At the same time the genetic counterpart was represented by gene variant scores extracted by the Sequence Kernel Association Test (SKAT) filtering model. Moreover, transcriptomic analysis was carried on to assess the expression of the resulting genes in the main brain structures as a form of validation. Results highlighted meaningful genotype-phenotype interactionsas defined by three latent components showing a significant difference in the projection scores between patients and controls. Among the significant associations, the model highlighted EPHX1 and BCAS1 gene variant scores involved in neurodegenerative and myelination processes, hence relevant for AD. In particular, the first was associated with decreased subcortical volumes and the second with decreasedtemporal lobe thickness. Noteworthy, BCAS1 is particularly expressed in the dentate gyrus. Overall, the proposed approach allowed capturing genotype-phenotype interactions in a restricted study cohort that was confirmed by transcriptomic analysis, offering insights into the underlying mechanisms of neurodegeneration in AD in line with previous findings and suggesting new potential disease biomarkers.

Explainable Artificial Intelligence (XAI) provides tools to help understanding how AI models work and reach a particular decision or outcome. It helps to increase the interpretability of models and makes them more trustworthy and transparent. In this context, many XAI methods have been proposed to make black-box and complex models more digestible from a human perspective. However, one of the main issues that XAI methods have to face especially when dealing with a high number of features is the presence of multicollinearity, which casts shadows on the robustness of the XAI outcomes, such as the ranking of informative features. Most of the current XAI methods either do not consider the collinearity or assume the features are independent which, in general, is not necessarily true. Here, we propose a simple, yet useful, proxy that modifies the outcome of any XAI feature ranking method allowing to account for the dependency among the features, and to reveal their impact on the outcome. The proposed method was applied to SHAP, as an example of XAI method which assume that the features are independent. For this purpose, several models were exploited for a well-known classification task (males versus females) using nine cardiac phenotypes extracted from cardiac magnetic resonance imaging as features. Principal component analysis and biological plausibility were employed to validate the proposed method. Our results showed that the proposed proxy could lead to a more robust list of informative features compared to the original SHAP in presence of collinearity.

While stroke is one of the leading causes of disability, the prediction of upper limb (UL) functional recovery following rehabilitation is still unsatisfactory, hampered by the clinical complexity of post-stroke impairment. Predictive models leading to accurate estimates while revealing which features contribute most to the predictions are the key to unveil the mechanisms subserving the post-intervention recovery, prompting a new focus on individualized treatments and precision medicine in stroke. Machine learning (ML) and explainable artificial intelligence (XAI) are emerging as the enabling technology in different fields, being promising tools also in clinics. In this study, we had the twofold goal of evaluating whether ML can allow deriving accurate predictions of UL recovery in sub-acute patients, and disentangling the contribution of the variables shaping the outcomes. To do so, Random Forest equipped with four XAI methods was applied to interpret the results and assess the feature relevance and their consensus. Our results revealed increased performance when using ML compared to conventional statistical approaches. Moreover, the features deemed as the most relevant were concordant across the XAI methods, suggesting good stability of the results. In particular, the baseline motor impairment as measured by simple clinical scales had the largest impact, as expected. Our findings highlight the core role of ML not only for accurately predicting the individual outcome scores after rehabilitation, but also for making ML results interpretable when associated to XAI methods. This provides clinicians with robust predictions and reliable explanations that are key factors in therapeutic planning/monitoring of stroke patients.

Alzheimer's disease (AD) is a neurodegenerative process characterized by the accumulation of amyloid-beta plaques and neurofibrillary tangles and is the most common cause of dementia. Studies have been striving to analyze the disease using available physiological and behavioral data. Functional/structural neuroimaging and genomics are complementary modalities for exploring the mechanisms subserving the development of AD. In this paper, we present a deep multimodal generative data fusion framework for integrating these sources in a classification task involving AD patients and healthy controls from the ADNI database. Biological data fusion has the potential to improve the final prediction, but at the same time, it is particularly challenging due to the unavailability of all sources of input for the entire cohort of subjects. Our proposed model allows us to perform prediction even if individuals are missing certain modalities. Our method addresses the missing modalities problem via knowledge transfer from two generative adversarial networks. The model exhibits superior performance for predicting AD versus healthy control, even with missing modalities. This could have an important impact from the patient point of view since certain clinical tests may not be necessary or available to a given individual.

Artificial intelligence has become pervasive across disciplines and fields, and biomedical image and signal processing is no exception. The growing and widespread interest on the topic has triggered a vast research activity that is reflected in an exponential research effort. Through study of massive and diverse biomedical data, machine and deep learning models have revolutionized various tasks such as modeling, segmentation, registration, classification and synthesis, outperforming traditional techniques. However, the difficulty in translating the results into biologically/clinically interpretable information is preventing their full exploitation in the field. Explainable AI (XAI) attempts to fill this translational gap by providing means to make the models interpretable and providing explanations. Different solutions have been proposed so far and are gaining increasing interest from the community. This paper aims at providing an overview on XAI in biomedical data processing and points to an upcoming Special Issue on Deep Learning in Biomedical Image and Signal Processing of the IEEE Signal Processing Magazine that is going to appear in March 2022.

Deep learning methods show great promise in a range of settings including the biomedical field. Explainability of these models is important in these fields for building end-user trust and to facilitate their confident deployment. Although several Machine Learning Interpretability tools have been proposed so far, there is currently no recognized evaluation standard to transfer the explainability results into a quantitative score. Several measures have been proposed as proxies for quantitative assessment of explainability methods. However, the robustness of the list of significant features provided by the explainability methods has not been addressed. In this work, we propose a new proxy for assessing the robustness of the list of significant features provided by two explainability methods. Our validation is defined at functionality-grounded level based on the ranked correlation statistical index and demonstrates its successful application in the framework of brain aging estimation. We assessed our proxy to estimate brain age using neuroscience data. Our results indicate small variability and high robustness in the considered explainability methods using this new proxy.

This review article discusses the definition and implementation of brain–computer interface (BCI) system relying on brain connectivity (BC) and machine learning/deep learning (DL) for motor imagery (MI)-based applications. During the past few years, many approaches have been explored in terms of types of neurological sources of information, feature extraction, and intention prediction for BCI applications. Two novel aspects are becoming increasingly interesting for the BCI community: BC modeling and DL. The former aims at describing the interactions among different brain regions as connectivity patterns that reflect the dynamics of information flow either at rest or when performing a task. The latter is becoming pervasive for its capability of modeling and predicting complex data, where a huge amount of information is involved. In this scenario, we conducted a systematic literature review on BCI studies that led to the selection of 34 articles meeting all the required criteria. This provides evidence of the rapid growth of the topic over the past few years, though being still in its infancy. The last part of this article is dedicated to this new frontier of BCI that we call MI BC-based computer interfaces highlighting the potential of BC features. This, jointly with DL as enabling technology, has the potential of improving the performance of electroencephalography-based systems.

Graph-based network modeling is becoming increasingly pervasive touching very different fields. Among these are social networks analysis and brain connectivity modeling. Though apparently very far apart, these two domains share the same questions about how the underlying network is structured and h ow this can be measured. This determines an a-priori unexpected convergence of the research efforts of two different communities, that is neurosciences and information technology. In this work, we put forth some basic issues emerging from the overlaps of the two domains and propose a first simple measure allowing to capture one among the features of interest: the transtopic closeness centrality. To this end, the related concepts are briefly recalled and two case studies are considered. Then, relying on social network analysis principles, the transposition to functional brain networks is proposed highlighting and discussing some of the inherent critical issues.

Diffusion magnetic resonance imaging (dMRI) yields parameters sensitive to brain tissue microstructure. A structurally important aspect of this microstructure is the myelin wrapping around the axons. This paper investigated the forward problem concerning whether water exchange via the spiraling structure of the myelin can meaningfully contribute to the signal in dMRI. Monte Carlo simulations were performed in a system with intra-axonal, myelin, and extra-axonal compartments. Diffusion in the myelin was simulated as a spiral wrapping the axon, with a custom number of wraps. Exchange (or intra-axonal residence) times were analyzed for various number of wraps and axon diameters. Pulsed gradient sequences were employed to simulate the dMRI signal, which was analyzed using different methods. Diffusional kurtosis imaging analysis yielded the radial diffusivity (RD) and radial kurtosis (RK), while the two-compartment Kärger model yielded estimates the intra-axonal volume fraction (νic) and exchange time (τ). Results showed that τ was on the sub-second level for geometries with axon diameters below 1.0 μm and less than eight wraps. Otherwise, exchange was negligible compared to typical experimental durations, with τ of seconds or longer. In situations where exchange influenced the signal, estimates of RK and νicincreased with the number of wraps, while RD decreased. τ estimates from simulated signals were in agreement with predicted ones. In conclusion, exchange through spiraling myelin permits sub-second τfor small diameters and low number of wraps. Such conditions may arise in the developing brain or in neurodegenerative disease, and thus the results could aid the interpretation of dMRI studies.

The application of time-varying measures of causality between source time series can be very informative to elucidate the direction of communication among the regions of an epileptic brain. The aim of the study was to identify the dynamic patterns of epileptic networks in focal epilepsy by applying multivariate adaptive directed transfer function (ADTF) analysis and graph theory to high-density electroencephalographic recordings. The cortical network was modeled after source reconstruction and topology modulations were detected during interictal spikes. First a distributed linear inverse solution, constrained to the individual grey matter, was applied to the averaged spikes and the mean source activity over 112 regions, as identified by the Harvard-Oxford Atlas, was calculated. Then, the ADTF, a dynamic measure of causality, was used to quantify the connectivity strength between pairs of regions acting as nodes in the graph, and the measure of node centrality was derived. The proposed analysis was effective in detecting the focal regions as well as in characterizing the dynamics of the spike propagation, providing evidence of the fact that the node centrality is a reliable feature for the identification of the epileptogenic zones. Validation was performed by multimodal analysis as well as from surgical outcomes. In conclusion, the time-variant connectivity analysis applied to the epileptic patients can distinguish the generator of the abnormal activity from the propagation spread and identify the connectivity pattern over time.

Early detection of epileptic seizures is still a challenge in the state-of-the-art. The proposed method exploits multiresolution sample entropy for both seizure detection and fingerprinting. First, a SVM classifier is used to detect the seizures' onset with high temporal accuracy, then the seizures fingerprints across the subband structure are derived exploiting sample entropy non stationarity. Over 8 hours of EEG data recordings from patients suffering from temporal lobe epilepsy were used for training and testing the system, and validation was performed based on annotation by one expert neurophysiologist. All the seizures were successfully detected and provides an effective time-scale fingerprinting of their evolution. A prominent impact in high (γ) frequency band was observed whose neurophysiological ground is currently under investigation.

It has been shown that the combination of multi-modal MRI images can improve the discrimination of diseased tissue. The fusion of dissimilar imaging data for classification and segmentation purposes however, is not a trivial task, as there is an inherent difference in information domains, dimensionality and scales. This work proposes a multi-view consensus clustering methodology for the integration of multi-modal MR images into a unified segmentation of tumoral lesions for heterogeneity assessment. Using a variety of metrics and distance functions this multi-view imaging approach calculates multiple vectorial dissimilarity-spaces for each MRI modality and makes use of cluster ensembles to combine a set of un-supervised base segmentations into an unified partition of the voxel-based data. The methodology is demonstrated in application to DCE-MRI and DTI-MR, for which a manifold learning step is implemented in order to account for the geometric constrains of the high dimensional diffusion information.

Validation is arguably the bottleneck in the diffusion magnetic resonance imaging (MRI) community. This paper evaluates and compares 20 algorithms for recovering the local intra-voxel fiber structure from diffusion MRI data and is based on the results of the "HARDI reconstruction challenge" organized in the context of the "ISBI 2012" conference. Evaluated methods encompass a mixture of classical techniques well known in the literature such as diffusion tensor, Q-Ball and diffusion spectrum imaging, algorithms inspired by the recent theory of compressed sensing and also brand new approaches proposed for the first time at this contest. To quantitatively compare the methods under controlled conditions, two datasets with known ground-truth were synthetically generated and two main criteria were used to evaluate the quality of the reconstructions in every voxel: correct assessment of the number of fiber populations and angular accuracy in their orientation. This comparative study investigates the behavior of every algorithm with varying experimental conditions and highlights strengths and weaknesses of each approach. This information can be useful not only for enhancing current algorithms and develop the next generation of reconstruction methods, but also to assist physicians in the choice of the most adequate technique for their studies.

Abstract In human behaviour analysis, the visual focus of attention (VFOA) of a person is a very important cue. VFOA detection is difficult, though, especially in a unconstrained and crowded environment, typical of video surveillance scenarios. In this paper, we estimate the VFOA by defining the Subjective View Frustum, which approximates the visual field of a person in a three‐dimensional representation of the scene. This opens up to several intriguing behavioural investigations. In particular, we propose the Inter‐Relation Pattern Matrix, which suggests possible social interactions between the people present in a scene. Theoretical justifications and experimental results substantiate the validity and the goodness of the analysis performed.

We present a novel approach for detecting social interactions in a crowded scene by employing solely visual cues. The detection of social interactions in unconstrained scenarios is a valuable and important task, especially for surveillance purposes. Our proposal is inspired by the social signaling literature, and in particular it considers the sociological notion of F-formation. An F-formation is a set of possible configurations in space that people may assume while participating in a social interaction. Our system takes as input the positions of the people in a scene and their (head) orientations; then, employing a voting strategy based on the Hough transform, it recognizes F-formations and the individuals associated with them. Experiments on simulations and real data promote our idea.

Validation is the main bottleneck preventing the adoption of many medical image processing algorithms in the clinical practice. In the classical approach, a-posteriori analysis is performed based on some objective metrics. In this work, a different approach based on Petri Nets (PN) is proposed. The basic idea consists in predicting the accuracy that will result from a given processing based on the characterization of the sources of inaccuracy of the system. Here we propose a proof of concept in the scenario of a diffusion imaging analysis pipeline. A PN is built after the detection of the possible sources of inaccuracy. By integrating the first qualitative insights based on the PN with quantitative measures, it is possible to optimize the PN itself, to predict the inaccuracy of the system in a different setting. Results show that the proposed model provides a good prediction performance and suggests the optimal processing approach.

We propose a pilot study in view of the definition of a semantic based absolute metric for automatic image quality evaluation. The basic idea is that the behavioral relevance of the objects present in the scene determines the extent of the impact of a given amount of (measurable) degradation on the perceived quality. Such an assumption is investigated following the top-down approach. The images are considered as collections of semantic units holding different relevance in the scene interpretation process. The test set is built such that some pre-defined categories of objects are present and the goal is to determine the impact of the selective degradation of instances of objects pertaining to the different categories on the perceived quality of the corresponding images. In particular, we focus on human faces. Under the assumption that human faces are salient cues from a perceptual point of view in an image quality evaluation task, we investigate the impact of the selective degradation of face/not-face regions of a pre-defined set of natural images through a subjective test performed by human observers. Our hypothesis is that the perceived quality is ruled by the nesses evaluated on human faces.

We propose a fully three-dimensional (3-D) wavelet-based coding system featuring 3-D encoding/two-dimensional (2-D) decoding functionalities. A fully 3-D transform is combined with context adaptive arithmetic coding; 2-D decoding is enabled by encoding every 2-D subband image independently. The system allows a finely graded up to lossless quality scalability on any 2-D image of the dataset. Fast access to 2-D images is obtained by decoding only the corresponding information thus avoiding the reconstruction of the entire volume. The performance has been evaluated on a set of volumetric data and compared to that provided by other 3-D as well as 2-D coding systems. Results show a substantial improvement in coding efficiency (up to 33%) on volumes featuring good correlation properties along the z axis. Even though we did not address the complexity issue, we expect a decoding time of the order of one second/image after optimization. In summary, the proposed 3-D/2-D multidimensional layered zero coding system provides the improvement in compression efficiency attainable with 3-D systems without sacrificing the effectiveness in accessing the single images characteristic of 2-D ones.

We propose a coding system featuring 3D encoding/2D decoding object-based functionalities. Any object of any 2D image of the dataset can be recovered at a finely graded up to lossless quality. Compression is improved by exploiting the full correlation among data samples by means of 3D DWT. A swift access to 2D images is obtained by enabling 2D decoding. Given the index of the image of interest along the z axis, only the concerned portion of the bitstream is decoded, at the desired quality. The selective access to data can be improved by splitting the image in regions corresponding to the different objects. Then, a suitable ordering of the encoded information within the bitstream enables random access to any object at the desired rate. This enables a pseudo-lossless regime, where the diagnostically relevant parts of the image are represented without loss, while a lower quality is assumed to be acceptable for the others. Results show that the proposed system is a good compromise between the gain in compression efficiency provided by 3D systems and the fast access to the data of 2D ones.

We propose a fully three-dimensional (3-D) object-based coding system exploiting the diagnostic relevance of the different regions of the volumetric data for rate allocation. The data are first decorrelated via a 3-D discrete wavelet transform. The implementation via the lifting steps scheme allows to map integer-to-integer values, enabling lossless coding, and facilitates the definition of the object-based inverse transform. The coding process assigns disjoint segments of the bitstream to the different objects, which can be independently accessed and reconstructed at any up-to-lossless quality. Two fully 3-D coding strategies are considered: embedded zerotree coding (EZW-3D) and multidimensional layered zero coding (MLZC), both generalized for region of interest (ROI)-based processing. In order to avoid artifacts along region boundaries, some extra coefficients must be encoded for each object. This gives rise to an overheading of the bitstream with respect to the case where the volume is encoded as a whole. The amount of such extra information depends on both the filter length and the decomposition depth. The system is characterized on a set of head magnetic resonance images. Results show that MLZC and EZW-3D have competitive performances. In particular, the best MLZC mode outperforms the others state-of-the-art techniques on one of the datasets for which results are available in the literature.

We propose a non-parametric texture modeling and synthesis technique based on the integer version of the discrete wavelet transform (DWT). The successive levels of the DWT pyramid of the input texture are progressively sampled starting from the signal approximation to generate the analogous wavelet pyramid for the synthetic texture. An underlying statistical model is assumed, where the appearance of wavelet coefficients at each scale is conditioned by the appearance of the corresponding ancestors at coarser scales. A non-parametric Parzen estimator is used for sampling. The integer DWT is obtained by the lifting steps implementation. The proposed method provides results comparable to the other state-of-the-art techniques for random (unstructured) textures, but at a very low computational complexity. For structured textures, performance depends on the specific orientation features and structure size.

The use of the discrete wavelet transform (DWT) for embedded lossy image compression is now well established. One of the possible implementations of the DWT is the lifting scheme (LS). Because perfect reconstruction is granted by the structure of the LS, nonlinear transforms can be used, allowing efficient lossless compression as well. The integer wavelet transform (IWT) is one of them. This is an interesting alternative to the DWT because its rate-distortion performance is similar and the differences can be predicted. This topic is investigated in a theoretical framework. A model of the degradations caused by the use of the IWT instead of the DWT for lossy compression is presented. The rounding operations are modeled as additive noise. The noise are then propagated through the LS structure to measure their impact on the reconstructed pixels. This methodology is verified using simulations with random noise as input. It predicts accurately the results obtained using images compressed by the well-known EZW algorithm. Experiment are also performed to measure the difference in terms of bit rate and visual quality. This allows to a better understanding of the impact of the IWT when applied to lossy image compression.

The last generation medical imaging equipment produce multidimensional (3D or 3D+time) data distributions. On a coding perspective, it is reasonable to expect that the exploitation of the full dimensional correlation among data samples would lead to a sensible improvement in compression performances, especially for isotropic datasets. We propose a fully three-dimensional wavelet-based coding system providing a finely-graded up to lossless data representation in a single bistream. The data are first decorrelated by a 3D discrete wavelet transform, performed by the non-linear lifting scheme mapping integers to integers. This enables the lossless mode and permits the in-place implementation of the transform at a reduced computational complexity. The coding scheme is inspired to the layered-zero coding proposed by Taubman and Zakhor (1994), extended to handle fully 3D subband structures. Performances are characterized with respect to both the 2D version of the same algorithm and the JPEG standard. The rate-saving is strongly influenced by the amount of the data correlation in the z dimension, ranging between 16.5% and 5.5% for the considered datasets.

In this paper, we present a parametric hybrid model used in the framework of multidimensional object representation, for applications to both object visualization and object-based data compression. Our model is defined as a set of hybrid ellipsoids suitable for both globally and locally deforming the reconstructed shape. Its new parameterization, as compared to classical techniques, allows us to preserve its analytical representation during the fitting process. It is fitted to the object contours by means of a genetic algorithm minimizing a mean-square error criterion. Several criteria are proposed and discussed according to the stability of the optimization process, as well as the ability to efficiently initialize the model parameters. Finally, fitting results are presented for 2D and 3D data and different applications are proposed.