Deep Learning (DL) models excel at automatically learning intricate patterns within complex data, but their black box nature undermines human trust. To address this, current validation strategies typically focus on the model itself, modifying its architecture to assess the role and importance of the components. However, this model-centric view overlooks the critical learning substrate, which is represented by the data, implicitly assuming that it accurately represents the target phenomenon. This implicit trust in data means that evaluation may fail to detect whether high performance stems from exploiting biases or data quirks rather than learning relevant patterns. We present a novel data-related ablation as a complement to the traditional architectural ablation. Using this framework for Electroencephalography (EEG) signals of Emotional Recognition (ER) and Motor Execution (ME) as a case study, we show that seemingly high-accuracy models often rely heavily on process-irrelevant features, maintaining performance even when key information is eliminated. This shows that a standard, data-independent evaluation can be misleading about whether a model truly captured the intended process; the proposed approach helps distinguish robust learning from leaning on incidental characteristics. Therefore, incorporating data-related ablation is essential for developing reliable and generalizable DL models in fields that rely on data derived from complex and often not completely known phenomena.

• The role of each imaging modality is evaluated in the identification and segmentation of the lesions. • Changes in the importance of imaging modalities are evaluated for different segmentation tasks. • The impact and challenges of using AI-generated data instead of real counterpart scans are evaluated to check how the network can explain potential differences. Brain studies require the use of several complementary imaging modalities. When some modality is unavailable, Artificial Intelligence (AI) has recently provided ways to estimate them. Radiologists modulate the use of the available modalities depending on the task they have to perform. We aim to trace artificially the radiological process through a multibranch neural network architecture, the StarNet. The goal is to explain how and where different imaging modalities, either really collected or artificially reconstructed, are used in different radiological tasks by reading inside the structure of the network. To do that, StarNet includes several satellite networks, one per source modality, connected at each layer by a central unit. This design enables us to assess the contribution of each imaging modality, identifying where the contribution occurs, and to quantify the variations if certain modalities are substituted with AI-generated counterparts. The ultimate goal is to enable data-related and task-related ablation studies through the complete explainability of StarNet, thus offering radiologists clear guidance on which imaging sequences contribute to the task, to what extent, and at which stages of the process. As an example, we applied the proposed architecture to the 2D slices extracted from 3D volumes acquired with multimodal magnetic resonance imaging (MRI), to assess: 1. The role of the used imaging modalities; 2. The change in role when the radiological task changes; 3. The effects of synthetic data on the process. The results are presented and discussed.

Despite several automated strategies for identification/segmentation of Multiple Sclerosis (MS) lesions in Magnetic Resonance Imaging (MRI) being developed, they consistently fall short when compared to the performance of human experts. This emphasizes the unique skills and expertise of human professionals in dealing with the uncertainty resulting from the vagueness and variability of MS, the lack of specificity of MRI concerning MS, and the inherent instabilities of MRI. Physicians manage this uncertainty in part by relying on their radiological, clinical, and anatomical experience. We have developed an automated framework for identifying and segmenting MS lesions in MRI scans by introducing a novel approach to replicating human diagnosis, a significant advancement in the field. This framework has the potential to revolutionize the way MS lesions are identified and segmented, being based on three main concepts: (1) Modeling the uncertainty; (2) Use of separately trained Convolutional Neural Networks (CNNs) optimized for detecting lesions, also considering their context in the brain, and to ensure spatial continuity; (3) Implementing an ensemble classifier to combine information from these CNNs. The proposed framework has been trained, validated, and tested on a single MRI modality, the FLuid-Attenuated Inversion Recovery (FLAIR) of the MSSEG benchmark public data set containing annotated data from seven expert radiologists and one ground truth. The comparison with the ground truth and each of the seven human raters demonstrates that it operates similarly to human raters. At the same time, the proposed model demonstrates more stability, effectiveness and robustness to biases than any other state-of-the-art model though using just the FLAIR modality.

One of the studies of the active brain-computer interface (BCI) focuses on identifying movements from human neurophysiological signals to control external devices such as robotic arms. In the literature, EEG-based BCI is utilized to decode user’s information to perform action or fill the gap from the brain to the arms in the case of illness. The purpose of this work is to understand, among the scientific literature, what the best Deep Learning (DL) architecture is for motor execution (ME) classification. Data from 105 people from the Physionet dataset and 15 subjects from the Upper Limb dataset were used. EEGnetv4, Deep4Net, and EEGITnet were used to classify EEG signals under ME for real-time BCI. The best results were achieved from the EEGNET trained without Common Spatial Pattern transformation, for both datasets.

Digital Twin technology in healthcare offers personalized care through advanced analytics, real-time data, and virtual models. In this paper we propose the adoption of the Digital Twin approach as a means for modeling hands, both injured and healthy, in a Double Digital Twin (DDT) using the Virtual Glove. The VG acts as a supportive rehabilitation device, capable of collecting data and reconstructing models for both the impaired and healthy hands. This study emphasizes the calibration process, which adjusts the model of the healthy hand to match the injured hand. Additionally, the transformed healthy hand model is used to guide the task with the injured system, ensuring synchronization and repeatability of exercises. Furthermore, the framework associated with DDT facilitates the analysis and quantification of the mobility of the impaired hand in comparison to the healthy one. Finally, a preliminary experiment is presented. Index Terms—Virtual Glove, Tele-Medicine, Double Digital Twin, Hand Rehabiliation, Virtual Reality

Magnetic Resonance Imaging (MRI) stands as a noninvasive tool for diagnosing and monitoring various diseases. The flexibility of MRI configuration parameters allows for adaptable imaging sequences, and at the same time poses challenges in terms of reproducibility, as variability in imaging sequences leads to significant differences in image contrast. This is one of the major causes that compromise the reliability of deep learning methods. Since the majority of the literature is focused on documenting the effects of this issue rather than delving into its underlying causes, this work follows a different approach. A Siamese Neural Network (SNN) has been trained to identify the scanner that acquired the input image. Experimental results include the use of Euclidean Distance (ED) and machine learning algorithms trained and tested using the feature vectors generated with the SNN. The results have shown that the proposed method is capable of distinguishing the scanner used for the acquisition with high accuracy. For a comprehensive interpretation of the results, the feature vectors have been dimensionality reduced and visualized with a 3D plot. Finally, the proposed method is sensitive to MR image contrast variability and could be used to detect data-related inconsistencies and provide a mechanism to make users aware of potential issues.

We propose a new approach for Alzheimer's disease (AD) detection using diffusion tensor imaging, specifically fractional anisotropy (FA) images, based on a combination of unsupervised and supervised deep learning techniques. Our method involves training a 3D convolutional autoencoder to learn low-dimensional representations of FA images in an unsupervised manner and using the learned representations to pre-train a supervised 3D convolutional classifier to predict the presence or absence of AD. Unsupervised pre-training can improve the classifier's performance, especially when difficult-to-collect labeled data are limited. We evaluate our approach on the OASIS-3 dataset and demonstrate promising performance.

Emotion recognition is useful in several fields, starting from medical diagnosis to driving a Brain-Computer Interface (BCI) or helping people with disabilities. During the last decades, many researchers applied automatic strategies to identify emotional states based on data acquired by electroen-cephalography (EEG). However, the task is very hard and results have been often ambiguous. This work aims to perform brain connectivity studies of EEG data of four self-stimulated emotional classes (“relax”, “anger”, “happiness”, “sadness”) using a graph model of the Phase Lag Index (PLI), being PLI a measurement of connection insensitive to volume conduction effect. Qualitative results show that, for the analyzed emotions, connectivity analysis indicates some relevant differences both in the active brain regions and in the bandwidths involved in the activation. This method for connectome generation and analysis shows that useful information can be derived and used for contributing to disambiguating the problem of automatic emotion recognition.

Magnetic resonance imaging (MRI) is an effective imaging tool that, due to its non-invasiveness and multiple-parameter nature, is frequently used in medicine. In particular, the MRI's inherent flexibility deriving from the usage of multiple parameters allows to obtain images of variable contrast and quality. However, intrinsic MRI contrast variability often comes with drawbacks in terms of differences in different scanners, thus resulting in the impossibility of standardizing the image contrast. In particular, this variability could negatively affect the automatic analysis of Deep Learning (DL) methods, both in the training phase and in the test phase. In this work, we present several results on how images collected from different MRI scanners are handled by DL methods. To this end, we trained a Siamese network (SNN), based on the EfficientNet-B0 Convolutional Neural Network (EN-CNN), to learn how to recognize the scanner that has generated a given image. The output encoding features of the SNN have been projected into a 2D space with Uniform Manifold Approximation and Projection (UMAP) and have been discussed. Regarding the training phase, the UMAP projects show that the network is capable of separating MR images encoded features from different MRI scanners. Moreover, even if the MR images of different subjects are acquired with the same scanner, the results suggest that there are considerable differences in how the SNN encoded those features. The test phase confirmed that the SNN architecture is capable of recognizing images from different MRI scanners.

In this paper we analyze different Mixed Integer Linear Programming (MILP) models in order to produce 1D and 3D Spread-Out Bragg peaks (SOBP) for protons in water. Our techniques do not use much computational resources; in particular, all our experiments have been performed by a standard personal computer. As main result we give the proof of concept that the techniques that we use to create parameterized uniform SOBP can be fruitfully used in Treatment Planning Systems (TPS) for Intensity Modulated Proton Therapy (IMPT). As technical result we show, for the first time to our best knowledge, that there is a trade-off between the minimum number of energies (or layers) to be used to have a SOBP peak within a uniformity tolerance parameter Dtand the same parameter Dt. Minimizing the number of energies also has the advantage of reducing the delivery time using the facilities in operation nowadays.

Telerehabilitation has many proven advantages to present for hand mobility restoration. To produce a way for the patient to benefit from it without losing the continuous support of the therapist that traditional rehabilitation offers, we built a complete system of controlled rehabilitation based on touchless hand tracking and developed the framework for its use from therapists and patients. In this paper, we present the devices it uses together with user-friendly applications developed keeping in mind the patients and therapists. Furthermore, we present the methods that can be used for analyzing hand tracking data in order to acquire information for joint-wise mobility improvement.

Hand tracking and gesture recognition are fundamental in a multitude of applications. Various sensors have been used for this purpose, however, all monocular vision systems face limitations caused by occlusions. Wearable equipment overcome said limitations, although deemed impractical in some cases. Using more than one sensor provides a way to overcome this problem, but necessitates more complicated designs. In this work, we aim to highlight contemporary methods used for hand tracking and gesture recognition by collecting publications of systems developed in the last decade, that employ contactless devices as RGB cameras, IR, and depth sensors, along with some preceding pillar works. Additionally, we briefly present common steps, techniques, and basic algorithms used during the process of developing modern hand tracking and gesture recognition systems and, finally, we derive the trend for the next future.

In medical imaging, images represent the quantification of the interaction between electromagnetic waves and our body and are represented in grey-scale. In addition, medical imaging often produces multimodal images. However, the analysis and interpretation of these images mostly occur in sequence or, as in the case of automatic tools, they are simply concatenated as independent sources of information. In both cases, color perception and color contrast are not exploited. Color perception and color contrast play a crucial role in human vision to recognize objects effectively and efficiently, and this can in principle extend to automatic systems. In this paper we show how color coding, particularly using color opponent models, can become an effective tool for preliminary color-based segmentation. Tests have been conducted on multimodal Magnetic Resonance Imaging (MRI) of the brain collected in a public database and the results obtained show the importance of color coding in medical imaging analysis.

Proton therapy is a cancer therapy that is more expensive than classical radiotherapy but that is considered the gold standard in several situations. Since there is also a limited amount of delivering facilities for this techniques, it is fundamental to increase the number of treated patients over time. The objective of this work is to offer an insight on the problem of the optimization of the part of the delivery time of a treatment plan that relates to the movements of the system. We denote it as the Nozzle Travel Time Problem (NTTP), in analogy with the Leaf Travel Time Problem (LTTP) in classical radiotherapy. In particular this work: (i) describes a mathematical model for the delivery system and formalize the optimization problem for finding the optimal sequence of movements of the system (nozzle and bed) that satisfies the covering of the prescribed irradiation directions; (ii) provides an optimization pipeline that solves the problem for instances with an amount of irradiation directions much greater than those usually employed in the clinical practice; (iii) reports preliminary results about the effects of employing two different resolution strategies within the aforementioned pipeline, that rely on an exact Traveling Salesman Problem (TSP) solver, Concorde, and an efficient Vehicle Routing Problem (VRP) heuristic, VROOM.

Proton therapy is nowadays a major clinical modality in the fight against cancer due to the advantages offered by its peculiar depth dose profile, that allows to improve its efficacy on tumors while reducing damages to healthy tissues. The number of worldwide facilities and of treated patients is increasing every year. A challenge of proton therapy is that treatment planning systems require accurate dose computation, but the golden standard in accuracy are Monte Carlo algorithms that are slow. For this reason, accurate and faster dose calculation algorithms are needed. In this paper we use Deep Learning to achieve both speed and accuracy in dose calculation for proton therapy. The results positively compare with previous existing literature in the thorax cases, that are usually the most difficult to calculate by fast algorithms.

Abstract Virtual Glove (VG) is a low-cost computer vision system that utilizes two orthogonal LEAP motion sensors to provide detailed 4D hand tracking in real–time. VG can find many applications in the field of human-system interaction, such as remote control of machines or tele-rehabilitation. An innovative and efficient data-integration strategy, based on the velocity calculation, for selecting data from one of the LEAPs at each time, is proposed for VG. The position of each joint of the hand model, when obscured to a LEAP, is guessed and tends to flicker. Since VG uses two LEAP sensors, two spatial representations are available each moment for each joint: the method consists of the selection of the one with the lower velocity at each time instant. Choosing the smoother trajectory leads to VG stabilization and precision optimization, reduces occlusions (parts of the hand or handling objects obscuring other hand parts) and/or, when both sensors are seeing the same joint, reduces the number of outliers produced by hardware instabilities. The strategy is experimentally evaluated, in terms of reduction of outliers with respect to a previously used data selection strategy on VG, and results are reported and discussed. In the future, an objective test set has to be imagined, designed, and realized, also with the help of an external precise positioning equipment, to allow also quantitative and objective evaluation of the gain in precision and, maybe, of the intrinsic limitations of the proposed strategy. Moreover, advanced Artificial Intelligence-based (AI-based) real-time data integration strategies, specific for VG, will be designed and tested on the resulting dataset.

Many critical aspects affect the correct operation of a Brain Computer Interface. The term ‘BCI-illiteracy’ describes the impossibility of using a BCI paradigm. At present, a universal solution does not exist and seeking innovative protocols to drive a BCI is mandatory. This work presents a meta-analytic review on recent advances in emotions recognition with the perspective of using emotions as voluntary, stimulus-independent, commands for BCIs. 60 papers, based on electroencephalography measurements, were selected to evaluate what emotions have been most recognised and what brain regions were activated by them. It was found that happiness, sadness, anger and calm were the most recognised emotions. Relevant discriminant locations for emotions recognition and for the particular case of discrete emotions recognition were identified in the temporal, frontal and parietal areas. The meta-analysis was mainly performed on stimulus-elicited emotions, due to the limited amount of literature about self-induced emotions. The obtained results represent a good starting point for the development of BCI driven by emotions and allow to: (1) ascertain that emotions are measurable and recognisable one from another (2) select a subset of most recognisable emotions and the corresponding active brain regions.

Hand rehabilitation therapy is fundamental in the recovery process for patients suffering from post-stroke or post-surgery impairments.Traditional approaches require the presence of therapist during the sessions, involving high costs and subjective measurements of the patients' abilities and progresses.Recently, several alternative approaches have been proposed.Mechanical devices are often expensive, cumbersome and patient specific, while virtual devices are not subject to this limitations, but, especially if based on a single sensor, could suffer from occlusions.In this paper a novel multi-sensor approach, based on the simultaneous use of two LEAP motion controllers, is proposed.The hardware and software design is illustrated and the measurements error induced by the mutual infrared interference is discussed.Finally, a calibration procedure, a tracking model prototype based on the sensors turnover and preliminary experimental results are presented.

The aim of this paper is to propose a real-time classification algorithm for the low-amplitude electroencephalography (EEG) signals, such as those produced by remembering an unpleasant odor, to drive a brain-computer interface. The peculiarity of these EEG signals is that they require ad hoc signals preprocessing by wavelet decomposition, and the definition of a set of features able to characterize the signals and to discriminate among different conditions. The proposed method is completely parameterized, aiming at a multiclass classification and it might be considered in the framework of machine learning. It is a two stages algorithm. The first stage is offline and it is devoted to the determination of a suitable set of features and to the training of a classifier. The second stage, the real-time one, is to test the proposed method on new data. In order to avoid redundancy in the set of features, the principal components analysis is adapted to the specific EEG signal characteristics and it is applied; the classification is performed through the support vector machine. Experimental tests on ten subjects, demonstrating the good performance of the algorithm in terms of both accuracy and efficiency, are also reported and discussed.