Video super-resolution (VSR) aims to enhance low-resolution videos by leveraging both spatial and temporal information. While deep learning has led to impressive progress, it typically requires centralized data, which raises privacy concerns. Federated learning (FL) offers a privacy-friendly solution, but general FL frameworks often struggle with low-level vision tasks, resulting in blurry, low-quality outputs. To address this, we introduce FedVSR, the first FL framework specifically designed for VSR. It is architecture-agnostic and stateless, and introduces a lightweight loss function based on the Discrete Wavelet Transform (DWT) to better preserve high-frequency details during local training. Additionally, a loss-aware aggregation strategy combines both DWT-based and task-specific losses to guide global updates effectively. Extensive experiments across multiple VSR models and datasets show that FedVSR not only improves perceptual video quality (up to +0.89 dB PSNR, +0.0370 SSIM, -0.0347 LPIPS and 4.98 VMAF) but also achieves these gains with close to zero computation and communication overhead compared to its rivals. These results demonstrate Fed-VSR's potential to bridge the gap between privacy, efficiency, and perceptual quality, setting a new benchmark for federated learning in low-level vision tasks. Please refer to this link for the code. https://github.com/alimd94/FedVSR

Ultra-high-definition (UHD) video datasets have played a critical role in advancing video quality assessment (VQA), compression, and general computer vision (CV) research. Despite recent progress in the availability of 8K datasets, existing resources remain limited in their coverage of real-world urban environments, where complex motion, uncontrolled lighting, crowds, traffic, and nighttime conditions pose challenges that are not adequately represented in cu-rated or semi-controlled recordings. To address this gap, we present VIDEA-Dublin, an additive urban extension to the VIDEA-8K-60FPS dataset, captured entirely in real-world public environments across Dublin, Ireland.

Multimedia streaming has become essential in various applications, such as security, healthcare, and education. However, it is an operation that demands a high amount of resources from CPU, GPU, memory, and network. This creates the need to develop solutions to predict the amount of resources required to provision services to aid in better decision-making for functionalities such as adaptive video quality control and load balancing, thus ensuring the optimal quality of service (QoS) possible to users given any condition. The existing resource utilization prediction solutions in the literature tend to focus on applications related to cloud computing. However, serverbased architectures play a significant role in use cases where data privacy issues require data to remain on-premise such as security camera feeds. Due to the constrained resources within server-based architectures, accurately predicting resource utilization becomes imperative for optimal system performance. In this paper, we present a dataset of the utilization of resources including hardware, network, and power consumption of serverbased architectures in multimedia streaming. The dataset consists of$37^{\prime} 800$different multimedia streaming use cases covering a variety of video resolutions, client numbers, and stream qualities. We detail the process used to collect the dataset and the parameters obtained from the involved systems, analyzing information that can be extracted from the data. Moreover, we establish a benchmark test by evaluating the performance of a plethora of regression models in predicting the resource utilization required from servers for multimedia streaming on a per-resource level pre-service provisioning, where we show that standard regression models can predict the utilization of resources with a root mean squared error of 2.08%, and that power utilization can also be predicted for both CPU and GPU with an error of$<2$Watts. Finally, we evaluate multivariate systems' ability to predict the values of various parameters together, and show that with a deep learning model we can predict resource utilization and power consumption with an accuracy based on the mean absolute error of 94.42% for utilization and$<2$Watts of power consumption error. The data collected on resource utilization can be found on the GitHub repo: https://github.com/talshoura/Resource-Utilization-of-Multimedia

In this paper, we introduce DiQP; a novel Transformer-Diffusion model for restoring 8K video quality degraded by codec compression. To the best of our knowledge, our model is the first to consider restoring the artifacts introduced by various codecs (AV1, HEVC) by Denoising Diffusion without considering additional noise. This approach allows us to model the complex, non-Gaussian nature of compression artifacts, effectively learning to reverse the degradation. Our architecture combines the power of Transformers to capture long-range dependencies with an enhanced windowed mechanism that preserves spatiotemporal context within groups of pixels across frames. To further enhance restoration, the model incorporates auxiliary “Look Ahead” and “Look Around” modules, providing both future and surrounding frame information to aid in reconstructing fine details and enhancing overall visual quality. Extensive experiments on different datasets demonstrate that our model outperforms state-of-the-art methods, particularly for high-resolution videos such as 4K and 8K, showcasing its effectiveness in restoring perceptually pleasing videos from highly compressed sources.11https://github.com/alimd94/DiQP.

This paper provides an overview of our open (Software Engineering Practice and Education) SEPE 8K dataset which is made of 40 different 8K (8192 x 4320) video sequences and 40 variant 8K (8192 x 5464) images. The video sequences were captured at a framerate of 29.97 frames per second (FPS) and had been encoded into videos using AVC/H.264, HEVC/H.265, and AV1 codecs at resolutions from 8K to 480p. The images, video sequences, encoded videos, and various other statistics related to the media that make the dataset are stored online, published, and maintained on the repo on GitHub for non-commercial use. In this paper, the dataset components are described and analyzed using various methods. The proposed dataset is - as far as we know - the first to publish true 8K natural sequences; thus, it is important for the next level of applications dealing with multimedia such as video quality assessment, super-resolution, video coding, video compression, and many more.

Quantifying uncertainty of predictions has been identified as one way to develop more trustworthy artificial intelligence (AI) models beyond conventional reporting of performance metrics. When considering their role in a clinical decision support setting, AI classification models should ideally avoid confident wrong predictions and maximise the confidence of correct predictions. Models that do this are said to be well calibrated with regard to confidence. However, relatively little attention has been paid to how to improve calibration when training these models, i.e. to make the training strategy uncertainty-aware. In this work we: (i) evaluate three novel uncertainty-aware training strategies with regard to a range of accuracy and calibration performance measures, comparing against two state-of-the-art approaches, (ii) quantify the data (aleatoric) and model (epistemic) uncertainty of all models and (iii) evaluate the impact of using a model calibration measure for model selection in uncertainty-aware training, in contrast to the normal accuracy-based measures. We perform our analysis using two different clinical applications: cardiac resynchronisation therapy (CRT) response prediction and coronary artery disease (CAD) diagnosis from cardiac magnetic resonance (CMR) images. The best-performing model in terms of both classification accuracy and the most common calibration measure, expected calibration error (ECE) was the Confidence Weight method, a novel approach that weights the loss of samples to explicitly penalise confident incorrect predictions. The method reduced the ECE by 17% for CRT response prediction and by 22% for CAD diagnosis when compared to a baseline classifier in which no uncertainty-aware strategy was included. In both applications, as well as reducing the ECE there was a slight increase in accuracy from 69% to 70% and 70% to 72% for CRT response prediction and CAD diagnosis respectively. However, our analysis showed a lack of consistency in terms of optimal models when using different calibration measures. This indicates the need for careful consideration of performance metrics when training and selecting models for complex high risk applications in healthcare.

Catheter ablation is currently the only curative treatment for scar-related ventricular tachycardias (VTs). However, not only are ablation procedures long, with relatively high risk, but success rates are punitively low, with frequent VT recurrence. Personalized in-silico approaches have the opportunity to address these limitations. However, state-of-the-art reaction diffusion (R-D) simulations of VT induction and subsequent circuits used for in-silico ablation target identification require long execution times, along with vast computational resources, which are incompatible with the clinical workflow. Here, we present the Virtual Induction and Treatment of Arrhythmias (VITA), a novel, rapid and fully automated computational approach that uses reaction-Eikonal methodology to induce VT and identify subsequent ablation targets. The rationale for VITA is based on finding isosurfaces associated with an activation wavefront that splits in the ventricles due to the presence of an isolated isthmus of conduction within the scar; once identified, each isthmus may be assessed for their vulnerability to sustain a reentrant circuit, and the corresponding exit site automatically identified for potential ablation targeting. VITA was tested on a virtual cohort of 7 post-infarcted porcine hearts and the results compared to R-D simulations. Using only a standard desktop machine, VITA could detect all scar-related VTs, simulating activation time maps and ECGs (for clinical comparison) as well as computing ablation targets in 48 minutes. The comparable VTs probed by the R-D simulations took 68.5 hours on 256 cores of high-performance computing infrastructure. The set of lesions computed by VITA was shown to render the ventricular model VT-free. VITA could be used in near real-time as a complementary modality aiding in clinical decision-making in the treatment of post-infarction VTs.

Structural changes to the wall of the left atrium are known to occur with conditions that predispose to Atrial fibrillation. Imaging studies have demonstrated that these changes may be detected non-invasively. An important indicator of this structural change is the wall's thickness. Present studies have commonly measured the wall thickness at few discrete locations. Dense measurements with computer algorithms may be possible on cardiac scans of Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). The task is challenging as the atrial wall is a thin tissue and the imaging resolution is a limiting factor. It is unclear how accurate algorithms may get and how they compare in this new emerging area. We approached this problem of comparability with the Segmentation of Left Atrial Wall for Thickness (SLAWT) challenge organised in conjunction with MICCAI 2016 conference. This manuscript presents the algorithms that had participated and evaluation strategies for comparing them on the challenge image database that is now open-source. The image database consisted of cardiac CT (n=10) and MRI (n=10) of healthy and diseased subjects. A total of 6 algorithms were evaluated with different metrics, with 3 algorithms in each modality. Segmentation of the wall with algorithms was found to be feasible in both modalities. There was generally a lack of accuracy in the algorithms and inter-rater differences showed that algorithms could do better. Benchmarks were determined and algorithms were ranked to allow future algorithms to be ranked alongside the state-of-the-art techniques presented in this work. A mean atlas was also constructed from both modalities to illustrate the variation in thickness within this small cohort.

Studies have demonstrated the feasibility of late Gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) imaging for guiding the management of patients with sequelae to myocardial infarction, such as ventricular tachycardia and heart failure. Clinical implementation of these developments necessitates a reproducible and reliable segmentation of the infarcted regions. It is challenging to compare new algorithms for infarct segmentation in the left ventricle (LV) with existing algorithms. Benchmarking datasets with evaluation strategies are much needed to facilitate comparison. This manuscript presents a benchmarking evaluation framework for future algorithms that segment infarct from LGE CMR of the LV. The image database consists of 30 LGE CMR images of both humans and pigs that were acquired from two separate imaging centres. A consensus ground truth was obtained for all data using maximum likelihood estimation. Six widely-used fixed-thresholding methods and five recently developed algorithms are tested on the benchmarking framework. Results demonstrate that the algorithms have better overlap with the consensus ground truth than most of the n-SD fixed-thresholding methods, with the exception of the Full-Width-at-Half-Maximum (FWHM) fixed-thresholding method. Some of the pitfalls of fixed thresholding methods are demonstrated in this work. The benchmarking evaluation framework, which is a contribution of this work, can be used to test and benchmark future algorithms that detect and quantify infarct in LGE CMR images of the LV. The datasets, ground truth and evaluation code have been made publicly available through the website: https://www.cardiacatlas.org/web/guest/challenges.

Knowledge of left atrial (LA) anatomy is important for atrial fibrillation ablation guidance, fibrosis quantification and biophysical modelling. Segmentation of the LA from Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) images is a complex problem. This manuscript presents a benchmark to evaluate algorithms that address LA segmentation. The datasets, ground truth and evaluation code have been made publicly available through the http://www.cardiacatlas.org website. This manuscript also reports the results of the Left Atrial Segmentation Challenge (LASC) carried out at the STACOM'13 workshop, in conjunction with MICCAI'13. Thirty CT and 30 MRI datasets were provided to participants for segmentation. Each participant segmented the LA including a short part of the LA appendage trunk and proximal sections of the pulmonary veins (PVs). We present results for nine algorithms for CT and eight algorithms for MRI. Results showed that methodologies combining statistical models with region growing approaches were the most appropriate to handle the proposed task. The ground truth and automatic segmentations were standardised to reduce the influence of inconsistently defined regions (e.g., mitral plane, PVs end points, LA appendage). This standardisation framework, which is a contribution of this work, can be used to label and further analyse anatomical regions of the LA. By performing the standardisation directly on the left atrial surface, we can process multiple input data, including meshes exported from different electroanatomical mapping systems.

An unresolved issue in patients with diastolic dysfunction is that the estimation of myocardial stiffness cannot be decoupled from diastolic residual active tension (AT) because of the impaired ventricular relaxation during diastole. To address this problem, this paper presents a method for estimating diastolic mechanical parameters of the left ventricle (LV) from cine and tagged MRI measurements and LV cavity pressure recordings, separating the passive myocardial constitutive properties and diastolic residual AT. Dynamic C1-continuous meshes are automatically built from the anatomy and deformation captured from dynamic MRI sequences. Diastolic deformation is simulated using a mechanical model that combines passive and active material properties. The problem of non-uniqueness of constitutive parameter estimation using the well known Guccione law is characterized by reformulation of this law. Using this reformulated form, and by constraining the constitutive parameters to be constant across time points during diastole, we separate the effects of passive constitutive properties and the residual AT during diastolic relaxation. Finally, the method is applied to two clinical cases and one control, demonstrating that increased residual AT during diastole provides a potential novel index for delineating healthy and pathological cases.

Multiscale cardiac modeling has made great advances over the last decade. Highly detailed atrial models were created and used for the investigation of initiation and perpetuation of atrial fibrillation. The next challenge is the use of personalized atrial models in clinical practice. In this study, a framework of simple and robust tools is presented, which enables the generation and validation of patient-specific anatomical and electrophysiological atrial models. Introduction of rule-based atrial fiber orientation produced a realistic excitation sequence and a better correlation to the measured electrocardiograms. Personalization of the global conduction velocity lead to a precise match of the measured P-wave duration. The use of a virtual cohort of nine patient and volunteer models averaged out possible model-specific errors. Intra-atrial excitation conduction was personalized manually from left atrial local activation time maps. Inclusion of LE-MRI data into the simulations revealed possible gaps in ablation lesions. A fast marching level set approach to compute atrial depolarization was extended to incorporate anisotropy and conduction velocity heterogeneities and reproduced the monodomain solution. The presented chain of tools is an important step towards the use of atrial models for the patient-specific AF diagnosis and ablation therapy planing.

In this paper, we present a novel technique based on nonrigid image registration for myocardial motion estimation using both untagged and 3-D tagged MR images. The novel aspect of our technique is its simultaneous usage of complementary information from both untagged and 3-D tagged MR images. To estimate the motion within the myocardium, we register a sequence of tagged and untagged MR images during the cardiac cycle to a set of reference tagged and untagged MR images at end-diastole. The similarity measure is spatially weighted to maximize the utility of information from both images. In addition, the proposed approach integrates a valve plane tracker and adaptive incompressibility into the framework. We have evaluated the proposed approach on 12 subjects. Our results show a clear improvement in terms of accuracy compared to approaches that use either 3-D tagged or untagged MR image information alone. The relative error compared to manually tracked landmarks is less than 15% throughout the cardiac cycle. Finally, we demonstrate the automatic analysis of cardiac function from the myocardial deformation fields.

This paper presents the prototype design and development of a miniature MR-compatible fiber optic force sensor suitable for the detection of force during MR-guided cardiac catheterization. The working principle is based on light intensity modulation where a fiber optic cable interrogates a reflective surface at a predefined distance inside a catheter shaft. When a force is applied to the tip of the catheter, a force sensitive structure varies the distance and the orientation of the reflective surface with reference to the optical fiber. The visual feedback from the MRI scanner can be used to determine whether or not the catheter tip is normal or tangential to the tissue surface. In both cases the light is modulated accordingly and the axial or lateral force can be estimated. The sensor exhibits adequate linear response, having a good working range, very good resolution and good sensitivity in both axial and lateral force directions. In addition, the use of low-cost and MR-compatible materials for its development makes the sensor safe for use inside MRI environments.

For many image-guided interventions there exists a need to compute the registration between preprocedure image(s) and the physical space of the intervention. Real-time intraprocedure imaging such as ultrasound (US) can be used to image the region of interest directly and provide valuable anatomical information for computing this registration. Unfortunately, real-time US images often have poor signal-to-noise ratio and suffer from imaging artefacts. Therefore, registration using US images can be challenging and significant preprocessing is often required to make the registrations robust. In this paper we present a novel technique for computing the image-to-physical registration for minimally invasive cardiac interventions using 3-D US. Our technique uses knowledge of the physics of the US imaging process to reduce the amount of preprocessing required on the 3-D US images. To account for the fact that clinical US images normally undergo significant image processing before being exported from the US machine our optimization scheme allows the parameters of the US imaging model to vary. We validated our technique by computing rigid registrations for 12 cardiac US/magnetic resonance imaging (MRI) datasets acquired from six volunteers and two patients. The technique had mean registration errors of 2.1-4.4 mm, and 75% capture ranges of 5-30 mm. We also demonstrate how the same approach can be used for respiratory motion correction: on 15 datasets acquired from five volunteers the registration errors due to respiratory motion were reduced by 45%-92%.

Magnetic resonance (MR) imaging has become a routine modality for the determination of patient cardiac morphology. The extraction of this information can be important for the development of new clinical applications as well as the planning and guidance of cardiac interventional procedures. To avoid inter- and intra-observer variability of manual delineation, it is highly desirable to develop an automatic technique for whole heart segmentation of cardiac magnetic resonance images. However, automating this process is complicated by the limited quality of acquired images and large shape variation of the heart between subjects. In this paper, we propose a fully automatic whole heart segmentation framework based on two new image registration algorithms: the locally affine registration method (LARM) and the free-form deformations with adaptive control point status (ACPS FFDs). LARM provides the correspondence of anatomical substructures such as the four chambers and great vessels of the heart, while the registration using ACPS FFDs refines the local details using a constrained optimization scheme. We validated our proposed segmentation framework on 37 cardiac MR volumes on the end-diastolic phase, displaying a wide diversity of morphology and pathology, and achieved a mean accuracy of 2.14 +/- 0.63 mm (rms surface distance) and a maximal error of 4.31 mm.

This paper describes a predictive and adaptive single parameter motion model for updating roadmaps to correct for respiratory motion in image-guided interventions. The model can adapt its motion estimates to respond to changes in breathing pattern, such as deep or fast breathing, which normally would result in a decrease in the accuracy of the motion estimates. The adaptation is made possible by interpolating between the motion estimates of multiple submodels, each of which describes the motion of the target organ during cycles of different amplitudes. We describe a predictive technique which can predict the amplitude of a breathing cycle before it has finished. The predicted amplitude is used to interpolate between the motion estimates of the submodels to tune the adaptive model to the current breathing pattern. The proposed technique is validated on affine motion models formed from cardiac magnetic resonance imaging (MRI) datasets acquired from seven volunteers and one patient. The amplitude prediction technique showed errors of 1.9-6.5 mm. The combined predictive and adaptive technique showed 3-D motion prediction errors of 1.0-2.8 mm, which represents an improvement in modelling performance of up to 40% over a standard nonadaptive single parameter motion model. We also applied the combined technique in a clinical setting to test the feasibility of using it for respiratory motion correction of roadmaps in image-guided cardiac catheterisations. In this clinical case we show that 2-D registration errors due to respiratory motion are reduced from 7.7 to 2.8 mm using the proposed technique.

We present an adaptive algorithm which uses a fast electrophysiological (EP) model to estimate apparent electrical conductivity and local conduction velocity from noncontact mapping of the endocardial surface potential. Development of such functional imaging revealing hidden parameters of the heart can be instrumental for improved diagnosis and planning of therapy for cardiac arrhythmia and heart failure, for example during procedures such as radio-frequency ablation and cardiac resynchronisation therapy. The proposed model is validated on synthetic data and applied to clinical data derived using hybrid X-ray/magnetic resonance imaging. We demonstrate a qualitative match between the estimated conductivity parameter and pathology locations in the human left ventricle. We also present a proof of concept for an electrophysiological model which utilizes the estimated apparent conductivity parameter to simulate the effect of pacing different ventricular sites. This approach opens up possibilities to directly integrate modelling in the cardiac EP laboratory.

The hybrid magnetic resonance (MR)/X-ray suite (XMR) is a recently introduced imaging solution that provides new possibilities for guidance of cardiovascular catheterization procedures. We have previously described and validated a technique based on optical tracking to register MR and X-ray images obtained from the sliding table XMR configuration. The aim of our recent work was to extend our technique by providing an improved calibration stage, real-time guidance during cardiovascular catheterization procedures, and further off-line analysis for mapping cardiac electrical data to patient anatomy. Specially designed optical trackers and a dedicated calibration object have resulted in a single calibration step that can be efficiently checked and updated before each procedure. An X-ray distortion model has been implemented that allows for distortion correction for arbitrary c-arm orientations. During procedures, the guidance system provides a real-time combined MR/X-ray image display consisting of live X-ray images with registered recently acquired MR derived anatomy. It is also possible to reconstruct the location of catheters seen during X-ray imaging in the MR derived patient anatomy. We have applied our registration technique to 13 cardiovascular catheterization procedures. Our system has been used for the real-time guidance of ten radiofrequency ablations and one aortic stent implantation. We demonstrate the real-time guidance using two exemplar cases. In a further two cases we show how off-line analysis of registered image data, acquired during electrophysiology study procedures, has been used to map cardiac electrical measurements to patient anatomy for two different types of mapping catheters. The cardiologists that have used the guidance system suggest that real-time XMR guidance could have substantial value in difficult interventional and electrophysiological procedures, potentially reducing procedure time and delivered radiation dose. Also, the ability to map measured electrical data to patient specific anatomy provides improved visualization and a path to investigation of cardiac electromechanical models.

We describe a registration and tracking technique to integrate cardiac X-ray images and cardiac magnetic resonance (MR) images acquired from a combined X-ray and MR interventional suite (XMR). Optical tracking is used to determine the transformation matrices relating MR image coordinates and X-ray image coordinates. Calibration of X-ray projection geometry and tracking of the X-ray C-arm and table enable three-dimensional (3-D) reconstruction of vessel centerlines and catheters from bi-plane X-ray views. We can, therefore, combine single X-ray projection images with registered projection MR images from a volume acquisition, and we can also display 3-D reconstructions of catheters within a 3-D or multi-slice MR volume. Registration errors were assessed using phantom experiments. Errors in the combined projection images (two-dimensional target registration error--TRE) were found to be 2.4 to 4.2 mm, and the errors in the integrated volume representation (3-D TRE) were found to be 4.6 to 5.1 mm. These errors are clinically acceptable for alignment of images of the great vessels and the chambers of the heart. Results are shown for two patients. The first involves overlay of a catheter used for invasive pressure measurements on an MR volume that provides anatomical context. The second involves overlay of invasive electrode catheters (including a basket catheter) on a tagged MR volume in order to relate electrophysiology to myocardial motion in a patient with an arrhythmia. Visual assessment of these results suggests the errors were of a similar magnitude to those obtained in the phantom measurements.

This paper describes a quantitative assessment of respiratory motion of the heart and the construction of a model of respiratory motion. Three-dimensional magnetic resonance scans were acquired on eight normal volunteers and ten patients. The volunteers were imaged at multiple positions in the breathing cycle between full exhalation and full inhalation while holding their breath. The exhalation volume was segmented and used as a template to which the other volumes were registered using an intensity-based rigid registration algorithm followed by nonrigid registration. The patients were imaged at inhale and exhale only. The registration results were validated by visual assessment and consistency measurements indicating subvoxel registration accuracy. For all subjects, we assessed the nonrigid motion of the heart at the right coronary artery, right atrium, and left ventricle. We show that the rigid-body motion of the heart is primarily in the craniocaudal direction with smaller displacements in the right-left and anterior-posterior directions; this is in agreement with previous studies. Deformation was greatest for the free wall of the right atrium and the left ventricle; typical deformations were 3-4 mm with deformations of up to 7 mm observed in some subjects. Using the registration results, landmarks on the template surface were mapped to their correct positions through the breathing cycle. Principal component analysis produced a statistical model of the motion and deformation of the heart. We discuss how this model could be used to assist motion correction.