Accurate compensation of brain deformation is critical for reliable image-guided neurosurgery. Surgical manipulation and tumor resection induce tissue motion, causing preoperative planning images to become misaligned with the intraoperative anatomy. In this review, we examine data-driven methods developed between 2020 and 2025 for brain deformation registration and modeling, with a particular focus on learning-based approaches. A comprehensive literature search was conducted in PubMed, IEEE Xplore, Scopus, and Web of Science using predefined inclusion and exclusion criteria for computational methods addressing brain deformation in neurosurgical imaging, resulting in 46 eligible studies. We provide a unified analysis of methodological strategies, including deep learning-based image registration, direct deformation field regression, synthesis-driven multimodal alignment, resection-aware architectures for handling missing correspondences, and hybrid models integrating biomechanical priors. We also examine dataset utilization, evaluation metrics, validation protocols, and the assessment of uncertainty and generalization across studies. While learning-based methods demonstrate promising accuracy and computational efficiency, current approaches remain limited by out-of-distribution robustness, standardized benchmarking, interpretability, and readiness for clinical deployment. Our review highlights these gaps and outlines future directions toward more robust, generalizable, and clinically translatable solutions for neurosurgical guidance. By organizing recent advances and critically assessing evaluation practices, this work provides a comprehensive reference for researchers and clinicians working on data-driven registration and modeling of brain deformation.

Routine clinical assessments for Parkinson's disease are essential instruments in both clinical practice and research, often used to identify disease sub-types and monitor the progression of disease severity. However, each clinic has limited access to information and the quality of these assessments is often degraded by the amount of missing information recorded at the time of each visit. The main objective of this study is to evaluate the performance of Federated Learning (FL) algorithms for imputing missing clinical data, enhancing the quality of decentralized Parkinson's disease assessments while maintaining data privacy. Specifically, we explore the impact of various aggregation strategies on the imputation of clinical data from 1,370 patients in the Parkinson Progression Marker Initiative (PPMI). Notably, the Cyclic Weight Transfer (CWT) algorithm stands out for its lower imputation errors. To validate this study, we conducted a downstream analysis using imputed data to predict symptoms progression. We observed that a FL-based approach yields superior model performance based on imputation errors, when compared to traditional learning strategies. These improvements can achieve 37.7% and 31.5% lower mean imputation errors with low and moderate degree of missing scores in the training data, respectively. In addition, we achieved better classification scores with Random Forest models trained with imputed data from FL-based approaches, compared to traditional statistical methods, with improvements of 0.5% in PR-AUC, 0.6% in ROC-AUC, and 1.3% in F-1 score. These results highlight FL as a robust and secure solution for decentralized clinical data management, offering improved performance while preserving patient privacy.

Hand visualization significantly impacts user performance in virtual reality (VR), particularly in tasks requiring precise finger movements. Common hand avatar visualizations, such as opaque or transparent, often occlude critical elements or distract users, potentially reducing accuracy. To address this issue, we investigated adaptive hand visibility techniques, which vary the hand avatar visibility based on the current movement sub-task aiming to enhance user performance. We evaluated these techniques on a VR-based Pedicle Screw Placement task with 15 participants. Each participant inserted virtual screws into a spine using five hand visualization conditions: opaque, transparent, invisible, speed-based visibility (hand visibility changes with hand speed), and position-based visibility (hand visibility adapts to the proximity of critical elements). Results showed that speed-based and position-based visualizations significantly improved accuracy, usability, and overall performance compared to traditional methods. The widely adopted opaque visualization yielded the lowest accuracy and usability. Our findings emphasize the benefits of adaptive hand visualization based on sub-tasks, recommending its implementation in VR applications to increase usability and user accuracy.

Magnetic Resonance Imaging (MRI) examinations are frequently associated with significant anxiety and phobias in patients, negatively impacting imaging quality and patient compliance. In this study, we explore the use of Virtual Reality Exposure Therapy (VRET) to reduce MRI-related anxiety by examining physiological and subjective responses across three virtual scenarios: a 2D video, a 360° video, and a fully immersive VR environment. The study aimed to determine how different levels of immersion and the order in which scenarios are experienced impact anxiety. Thirteen participants engaged in all three scenarios, with heart rate (HR), skin temperature (SKT), and electrodermal activity (EDA) monitored, and self-reported anxiety assessments collected before, during, and after the study. Results showed no significant differences in average or maximum heart rates between the three scenarios. However, the fully immersive VR environment generally elicited higher HR peaks, higher EDA, and lower SKT, suggesting stronger physiological responses in some participants. Self-reported anxiety decreased after the VR experience, particularly for participants with moderate to high anxiety levels prior to the sessions, independent of the scenario order. These findings suggest that individual responses to VRET vary, emphasizing the need for personalized approaches rather than a one-size-fits-all solution. While larger studies are necessary to validate these outcomes, the results suggest that incorporating real-time biofeedback monitoring in VRET could allow for dynamic adjustments to exposure levels based on participants’ physiological responses, creating a more adaptive and therapeutic environment.

ABSTRACT Federated learning (FL) using the federated averaging (FedAvg) algorithm has shown great advantages for large‐scale applications that rely on collaborative learning, especially when the training data is either unbalanced or inaccessible due to privacy constraints. We hypothesize that FedAvg underestimates the full extent of heterogeneity of data when the aggregation is performed. We propose Precision‐Weighted Federated Learning (PW) a novel algorithm that takes into account the second raw moment (uncentered variance) of the stochastic gradient when computing the weighted average of the parameters of independent models trained in a FL setting. With PW, we address the communication and statistical challenges for the training of distributed models with private data and provide an alternate averaging scheme that leverages the heterogeneity of the data when it has a large diversity of features in its composition. Our method was evaluated using three standard image classification datasets (MNIST, Fashion‐MNIST, and CIFAR) under two different data partitioning strategies: independent and identically distributed (IID), and nonidentical and nonindependent (non‐IID). These experiments were designed to measure the performance and efficiency of our method in resource‐constrained environments, such as mobile and IoT devices. The experimental results demonstrate that we can obtain a good balance between computational efficiency and convergence rates with PW. Our performance evaluations show better predictions with MNIST, with Fashion‐MNIST, and with CIFAR‐10 in the non‐IID setting. Further reliability evaluations ratify the stability in our method by reaching a 99% reliability index with IID partitions and 96% with non‐IID partitions. In addition, we obtained a speedup on Fashion‐MNIST with only 10 clients and up to with 100 clients participating in the aggregation concurrently per communication round. Overall, PW demonstrates improved stability and accuracy with increasing batch sizes, and it benefits significantly from lower learning rates and longer local training, compared to FedAvg and FedProx. The results indicate that PW is an effective and faster alternative approach for aggregating model updates derived from private data, especially in domains where data is highly heterogeneous.

In this paper, we present a new volumetric ambient occlusion algorithm called Contextual Ambient Occlusion (CAO) that supports real-time clipping. The algorithm produces ambient occlusion images of exactly the same quality as Local Ambient Occlusion (LAO) while enabling real-time modification to the shape used to clip the volume. The main idea of the algorithm is that clipping only affects the ambient value of a small number of voxels, so by identifying these voxels and recalculating the ambient factor only for them, it is possible to significantly increase the rendering performance (by 2-5x) without decreasing the quality of the rendered image. Due to its fast performance, the algorithm is suitable for interactive environments where clipping changes could occur every frame. Additionally, the algorithm doesn’t have any stereoscopic inconsistency, which makes it suitable for mixed reality environments. This paper is an extended version of the “Contextual Ambient Occlusion” article presented during the 2023 Graphics Interface conference, and includes, among other additions, the source code of the algorithm.

The virtual hand interaction technique is one of the most common interaction techniques used in virtual reality (VR) systems. A VR application can be designed with different hand visualization styles, which might impact motor dexterity. In this paper, we aim to investigate the effects of three different hand visualization styles — transparent, opaque, and invisible — on participants’ performance through a VR-based Purdue Pegboard Test (PPT). A total of 24 participants were recruited and instructed to place pegs on the board as quickly and accurately as possible. The results indicated that using the invisible hand visualization significantly increased the number of task repetitions completed compared to the opaque hand visualization. However, no significant difference was observed in participants’ preference for the hand visualization styles. These findings suggest that an invisible hand visualization may enhance performance in the VR-based PPT, potentially indicating the advantages of a less obstructive hand visualization style. We hope our results can guide developers, researchers, and practitioners when designing novel virtual hand interaction techniques.

The evolution of digitally connected devices and artificial intelligence has opened the door for novel health and fitness applications that can be used by individuals at a time and in an environment convenient to them. The purpose of our research was to develop a platform that requires no additional hardware to provide an online balance training program. Balance exercises are often prescribed for healthy aging to keep the body active, improve balance and coordination, and prevent falls and injuries, as well as, for those doing rehabilitation after injuries or diseases such as stroke. We developed a simple web application (BaART: Balance Augmented Reality Trainer) that uses PoseNet to determine a user's location and pose to count the number of repetitions that were done successfully. Furthermore, we looked at how augmented reality, and specifically adding a virtual chair, might impact a user's sense of balance. In a study of 20 participants with and without balance disorders, we found that the developed system was easy to use and many would consider using such a system, particularly our older participants who spend more time at home. However, we also found that the virtual object (i.e. chair) was not used by most people. Furthermore, those with balance issues felt they required a real chair for balance and some even felt that the virtual object was distracting from the exercise. In the future, we plan to explore other uses of augmented reality, such as feedback on exercise quality, gaming features, and a virtual avatar trainer.

Eye fatigue is a common challenge in eye tracking applications caused by physical and/or mental triggers. Its impact should be analyzed in eye tracking applications, especially for the dwell-time method. As emerging interaction techniques become more sophisticated, their impacts should be analyzed based on various aspects. We propose a novel compound measure for gaze-based interaction techniques that integrates subjective NASA TLX scores with objective measurements of eye movement fixation points. The measure includes two variations depending on the importance of (a) performance, and (b) accuracy, for measuring potential eye fatigue for eye tracking interactions. These variations enable researchers to compare eye tracking techniques on different criteria. We evaluated our measure in two user studies with 33 participants and report on the results of comparing dwell-time and gaze-based selection using voice recognition techniques.

User interaction has the potential to greatly facilitate the exploration and understanding of 3D medical images for diagnosis and treatment. However, in certain specialized environments such as in an operating room (OR), technical and physical constraints such as the need to enforce strict sterility rules, make interaction challenging. In this paper, we propose to facilitate the intraoperative exploration of angiographic volumes by leveraging the motion of a tracked surgical pointer, a tool that is already manipulated by the surgeon when using a navigation system in the OR. We designed and implemented three interactive rendering techniques based on this principle. The benefit of each of these techniques is compared to its non-interactive counterpart in a psychophysics experiment where 20 medical imaging experts were asked to perform a reaching/targeting task while visualizing a 3D volume of angiographic data. The study showed a significant improvement of the appreciation of local vascular structure when using dynamic techniques, while not having a negative impact on the appreciation of the global structure and only a marginal impact on the execution speed. A qualitative evaluation of the different techniques showed a preference for dynamic chroma-depth in accordance with the objective metrics but a discrepancy between objective and subjective measures for dynamic aerial perspective and shading.

Cerebral vascular images obtained through angiography are used by neurosurgeons for diagnosis, surgical planning, and intraoperative guidance. The intricate branching of the vessels and furcations, however, make the task of understanding the spatial three-dimensional layout of these images challenging. In this paper, we present empirical studies on the effect of different perceptual cues (fog, pseudo-chromadepth, kinetic depth, and depicting edges) both individually and in combination on the depth perception of cerebral vascular volumes and compare these to the cue of stereopsis. Two experiments with novices and one experiment with experts were performed. The results with novices showed that the pseudo-chromadepth and fog cues were stronger cues than that of stereopsis. Furthermore, the addition of the stereopsis cue to the other cues did not improve relative depth perception in cerebral vascular volumes. In contrast to novices, the experts also performed well with the edge cue. In terms of both novice and expert subjects, pseudo-chromadepth and fog allow for the best relative depth perception. By using such cues to improve depth perception of cerebral vasculature, we may improve diagnosis, surgical planning, and intraoperative guidance.

Mixed reality visualizations are increasingly studied for use in image guided surgery (IGS) systems, yet few mixed reality systems have been introduced for daily use into the operating room (OR). This may be the result of several factors: the systems are developed from a technical perspective, are rarely evaluated in the field, and/or lack consideration of the end user and the constraints of the OR. We introduce the Data, Visualization processing, View (DVV) taxonomy which defines each of the major components required to implement a mixed reality IGS system. We propose that these components be considered and used as validation criteria for introducing a mixed reality IGS system into the OR. A taxonomy of IGS visualization systems is a step toward developing a common language that will help developers and end users discuss and understand the constituents of a mixed reality visualization system, facilitating a greater presence of future systems in the OR. We evaluate the DVV taxonomy based on its goodness of fit and completeness. We demonstrate the utility of the DVV taxonomy by classifying 17 state-of-the-art research papers in the domain of mixed reality visualization IGS systems. Our classification shows that few IGS visualization systems' components have been validated and even fewer are evaluated.

We present empirical studies that consider the effects of stereopsis and simulated aerial perspective on depth perception in translucent volumes. We consider a purely absorptive lighting model, in which light is not scattered or reflected, but is simply absorbed as it passes through the volume. A purely absorptive lighting model is used, for example, when rendering digitally reconstructed radiographs (DRRs), which are synthetic X-ray images reconstructed from CT volumes. Surgeons make use of DRRs in planning and performing operations, so an improvement of depth perception in DRRs may help diagnosis and surgical planning.