Cardiovascular disease (CVD) is one of the leading causes of death and illness across the world. Especially, early prediction of heart failure (HF) is complicated due to the heterogeneity of its clinical presentations and symptoms. These challenges underscore the need for a multidisciplinary approach for comprehensive evaluation of cardiac state. To this end, we specifically select electrocardiogram (ECG) and 3D cardiac anatomy for their complementary coverage of cardiac electrical activities and fine-grained structural modeling. Building upon this, we present a novel pre-training framework, named Anatomy-Electrocardiogram Model (AEM), to explore their complex interactions. AEM adopts a multi-task self-supervised scheme that combines a masked reconstruction objective with a cardiac measurement (CM) regression branch to embed cardiac functional priors and structural details. Unlike image-domain models that typically localize the whole heart within the image, our 3D anatomy is background-free and continuous in 3D space. Hence, the model can naturally concentrate on finer structures at the patch level. The further integration with ECG captures functional dynamics through electrical conduction, encapsulating holistic cardiac representations. Extensive experiments are conducted on the multi-modal datasets collected from the UK Biobank, which contain paired biventricular point cloud anatomy and 12-lead ECG data. Our proposed AEM achieves an area under the receiver operating characteristic curve of 0.8192 for incident HF prediction and a concordance index of 0.6976 for survival prediction under linear evaluation, outperforming the state-of-the-art multi-modal methods. Additionally, we study the interpretability of the disease prediction by observing that our model effectively recognizes clinically plausible patterns and exhibits a high association with clinical features.

Atrial fibrillation (AF), the most common cardiac arrhythmia, affects one in three adults over 45 years of age. Improving its treatment requires a better understanding of bi-atrial anatomy. Existing benchmarks have focused on the left atrial (LA) cavity, overlooking the fundamental challenges posed by bi-atrial anatomy, most notably the thin atrial walls, which are critical for substrate-guided ablation planning in patients with atrial fibrillation. To address these limitations, the Multi-class Bi-Atrial Segmentation 2024 Challenge (MBAS2024) introduced the first large-scale, multi-class benchmark for simultaneous segmentation of the LA cavity, right atrial (RA) cavity, and bi-atrial walls from late gadolinium-enhanced (LGE) MRI. We systematically evaluated 13 state-of-the-art methods on the world's largest curated bi-atrial dataset, comprising 175 3D multi-center scans with expert-validated annotations, providing a comprehensive assessment of current methodological capabilities and limitations. Key findings include: segmentation of the LA and RA cavities is generally robust to image quality, whereas atrial wall delineation is highly sensitive to image degradation. Performance varies across centers, indicating limited generalization of atrial wall segmentation across different acquisition protocols. Model architecture, rather than hyperparameter tuning, is the primary driver of performance, with U-Net-based models and emerging state-space models (e.g., UMambaBot) achieving higher accuracy at modest computational cost. Segmentation accuracy also varies along the slice dimension, with central slices segmented more reliably. Finally, hybrid labeling strategies-separating LA and RA cavities while merging bi-atrial walls into a single class-consistently improve performance. The MBAS2024 challenge establishes a foundational benchmark for bi-atrial segmentation, providing validated baselines and actionable insights to guide the development of clinically relevant, efficient, and anatomically aware segmentation algorithms to improve targeted ablation in patients with AF.

Cardiovascular diseases (CVDs) are the most common cause of death worldwide. Invasive x-ray coronary angiography (ICA) is one of the most important imaging modalities for the diagnosis of CVDs. ICA typically acquires only two 2D projections, which makes the 3D geometry of coronary vessels difficult to interpret, thus requiring 3D coronary artery tree reconstruction from two projections. State-of-the-art approaches require significant manual interactions and cannot correct the non-rigid cardiac and respiratory motions between non-simultaneous projections. In this study, we propose a novel deep learning pipeline named DeepCA. We leverage the Wasserstein conditional generative adversarial network with gradient penalty, latent convolutional transformer layers, and a dynamic snake convolutional critic to implicitly compensate for the non-rigid motion and provide 3D coronary artery tree reconstruction. Through simulating projections from coronary computed tomography angiography (CCTA), we achieve the generalisation of 3D coronary tree reconstruction on real non-simultaneous ICA projections. We incorporate an application-specific evaluation metric to validate our proposed model on both a CCTA dataset and a real ICA dataset, together with Chamfer £2 distance. The results demonstrate promising performance of our DeepCA model in vessel topology preservation, recovery of missing features, and generalisation ability to real ICA data. To the best of our knowledge, this is the first study that leverages deep learning to achieve 3D coronary tree reconstruction from two real non-simultaneous x-ray angiographic projections. The implementation of this work is available at: https//github.com/WangStephen/DeepCA.

Invasive coronary angiography (ICA) is the gold standard imaging modality during cardiac interventions. Accurate segmentation of coronary vessels in ICA is required for aiding diagnosis and creating treatment plans. Current automated algorithms for vessel segmentation face task-specific challenges, including motion artifacts and unevenly distributed contrast, as well as the general challenge inherent to X-ray imaging, which is the presence of shadows from overlapping organs in the background. To address these issues, we present Temporal Vessel Segmentation Network (TVS-Net) model that fuses sequential ICA information into a novel densely connected 3D encoder-2D decoder structure with a loss function based on elastic interaction. We develop our model using an ICA dataset comprising 323 samples, split into 173 for training, 82 for validation, and 68 for testing, with a relatively relaxed annotation protocol that produced coarse-grained samples, and achieve 83.4% Dice and 84.3% recall on the test dataset. We additionally perform an external evaluation over 60 images from a local hospital, achieving 78.5% Dice and 82.4% recall and outperforming the state-of-the-art approaches. We also conduct a detailed manual re-segmentation for evaluation only on a subset of the first dataset under strict annotation protocol, achieving a Dice score of 86.2% and recall of 86.3% and surpassing even the coarse-grained gold standard used in training. The results indicate our TVS-Net is effective for multi-frame ICA segmentation, highlights the network's generalizability and robustness across diverse settings, and showcases the feasibility of weak supervision in ICA segmentation.

Cardiac digital twins (CDTs) offer personalized in-silico cardiac representations for the inference of multi-scale properties tied to cardiac mechanisms. The creation of CDTs requires precise information about the electrode position on the torso, especially for the personalized electrocardiogram (ECG) calibration. However, current studies commonly rely on additional acquisition of torso imaging and manual/semi-automatic methods for ECG electrode localization. In this study, we propose a novel and efficient topology-informed model to fully automatically extract personalized ECG standard electrode locations from 2D clinically standard cardiac MRIs. Specifically, we obtain the sparse torso contours from the cardiac MRIs and then localize the standard electrodes of 12-lead ECG from the contours. Cardiac MRIs aim at imaging of the heart instead of the torso, leading to incomplete torso geometry within the imaging. To tackle the missing topology, we incorporate the electrodes as a subset of the keypoints, which can be explicitly aligned with the 3D torso topology. The experimental results demonstrate that the proposed model outperforms the time-consuming conventional model projection-based method in terms of accuracy (Euclidean distance: 1.24±0.293 cm vs. 1.48±0.362 cm) and efficiency (2 s vs. 30-35 min). We further demonstrate the effectiveness of using the detected electrodes for in-silico ECG simulation, highlighting their potential for creating accurate and efficient CDT models. The code is available at https://github.com/lileitech/12lead_ECG_electrode_localizer.

Heart failure (HF) poses a significant public health challenge, with a rising global mortality rate. Early detection and prevention of HF could significantly reduce its impact. We introduce a novel methodology for predicting HF risk using 12-lead electrocardiograms (ECGs). We present a novel, lightweight dual attention ECG network designed to capture complex ECG features essential for early HF risk prediction, despite the notable imbalance between low and high-risk groups. This network incorporates a cross-lead attention module and 12 lead-specific temporal attention modules, focusing on cross-lead interactions and each lead's local dynamics. To further alleviate model overfitting, we leverage a large language model (LLM) with a public ECG-Report dataset for pretraining on an ECG-Report alignment task. The network is then fine-tuned for HF risk prediction using two specific cohorts from the UK Biobank study, focusing on patients with hypertension (UKB-HYP) and those who have had a myocardial infarction (UKB-MI). The results reveal that LLM-informed pre-training substantially enhances HF risk prediction in these cohorts. The dual attention design not only improves interpretability but also predictive accuracy, outperforming existing competitive methods with C-index scores of 0.6349 for UKB-HYP and 0.5805 for UKB-MI. This demonstrates our method's potential in advancing HF risk assessment with clinical complex ECG data.

Hypertension is a leading risk factor for a number of diseases and can cause severe damage to the vital organs such as the brain and heart. However, the level of hypertension itself does not necessarily reflect the full extent of underlying end-organ changes, which may hinder the development of effective treatment strategies. While recent research has demonstrated that these end-organ changes can be measured with deep phenotyping, its clinical translation may not be feasible. In this study, we propose a state-of-art deep learning approach that can quantify multi-organ (e.g., heart, brain, vasculature) phenotypical changes due to persistent hypertension from simple and popular clinical measures such as electrocardiogram (ECG), routinely acquired clinical data (age, BMI, diastolic and systolic blood pressures), and cardiac short axis (SAX) images from the UK Biobank, one of the largest open-access biomedical databases. Our proposed approach captures the intricate patterns of hypertensive disease state without resorting to the complex measures, which is hard to obtain in practical settings. It generates a numeric score between 0 and 1 of multi-organ damage, as well as provides an estimate of the overall uncertainty. The performance of our models is evaluated in different experimental settings and compared against the reference model. The results consistently demonstrate that the proposed approach can effectively model the multi-organ phenotypical changes from simple clinical measures with high performance (best-performing model MAE=0.108, MSE=0.019, variance=0.0005), and underscores its feasibility for potential clinical use.

• This paper investigates the capacity of AI models for airway modelling on national datasets with paired clinical metadata. • We evaluated AI models against unharmonised, noisy, and out-of-distribution data, as well as the prognostication for FLD. • We found a new biomarker for mortality prediction, outperforming existing clinical measurements (FVC% and fibrosis scores). • In-depth analysis of AI models on airway modelling and prognosis, highlighting challenges and future research directions. Airway-related quantitative imaging biomarkers are crucial for examination, diagnosis, and prognosis in pulmonary diseases. However, the manual delineation of airway structures remains prohibitively time-consuming. While significant efforts have been made towards enhancing automatic airway modelling, current public-available datasets predominantly concentrate on lung diseases with moderate morphological variations. The intricate honeycombing patterns present in the lung tissues of fibrotic lung disease patients exacerbate the challenges, often leading to various prediction errors. To address this issue, the 'Airway-Informed Quantitative CT Imaging Biomarker for Fibrotic Lung Disease 2023′ (AIIB23) competition was organized in conjunction with the official 2023 International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI). The airway structures were meticulously annotated by three experienced radiologists. Competitors were encouraged to develop automatic airway segmentation models with high robustness and generalization abilities, followed by exploring the most correlated QIB of mortality prediction. A training set of 120 high-resolution computerised tomography (HRCT) scans were publicly released with expert annotations and mortality status. The online validation set incorporated 52 HRCT scans from patients with fibrotic lung disease and the offline test set included 140 cases from fibrosis and COVID-19 patients. The results have shown that the capacity of extracting airway trees from patients with fibrotic lung disease could be enhanced by introducing voxel-wise weighted general union loss and continuity loss. In addition to the competitive image biomarkers for mortality prediction, a strong airway-derived biomarker (Hazard ratio>1.5, p < 0.0001) was revealed for survival prognostication compared with existing clinical measurements, clinician assessment and AI-based biomarkers.

Abstract The development and application of artificial intelligence-based computer vision systems in medicine, environment, and industry are playing an increasingly prominent role. Hence, the need for optimal and efficient hyperparameter tuning strategies is more than crucial to deliver the highest performance of the deep learning networks in large and demanding datasets. In our study, we have developed and evaluated a new training methodology named deep multi-metric training (DMMT) for enhanced training performance. The DMMT delivers a state of robust learning for deep networks using a new important criterion of multi-metric performance evaluation. We have tested the DMMT methodology in multi-class (three, four, and ten), multi-vendors (different X-ray imaging devices), and multi-size (large, medium, and small) datasets. The validity of the DMMT methodology has been tested in three different classification problems: (i) medical disease classification, (ii) environmental classification, and (iii) ecological classification. For disease classification, we have used two large COVID-19 chest X-rays datasets, namely the BIMCV COVID-19+ and Sheffield hospital datasets. The environmental application is related to the classification of weather images in cloudy, rainy, shine or sunrise conditions. The ecological classification task involves a classification of three animal species (cat, dog, wild) and a classification of ten animals and transportation vehicles categories (CIFAR-10). We have used state-of-the-art networks of DenseNet-121, ResNet-50, VGG-16, VGG-19, and DenResCov-19 (DenRes-131) to verify that our novel methodology is applicable in a variety of different deep learning networks. To the best of our knowledge, this is the first work that proposes a training methodology to deliver robust learning, over a variety of deep learning networks and multi-field classification problems.

Global single-valued biomarkers, such as ejection fraction, are widely used in clinical practice to assess cardiac function. However, they only approximate the heart's true 3D deformation process, thus limiting diagnostic accuracy and the understanding of cardiac mechanics. Metrics based on 3D shape have been proposed to alleviate these shortcomings. In this work, we present the Point Cloud Deformation Network (PCD-Net) as a novel geometric deep learning approach for direct modeling of 3D cardiac mechanics of the biventricular anatomy between the extreme ends of the cardiac cycle. Its encoder-decoder architecture combines a low-dimensional latent space with recent advances in point cloud deep learning for effective multi-scale feature learning directly on flexible and memory-efficient point cloud representations of the cardiac anatomy. We first evaluate the PCD-Net's predictive capability for both cardiac contraction and relaxation on a large UK Biobank dataset of over 10,000 subjects and find average Chamfer distances between the predicted and ground truth anatomies below the pixel resolution of the underlying image acquisition. We then show the PCD-Net's ability to capture subpopulation-specific differences in 3D cardiac mechanics between normal and myocardial infarction (MI) subjects and visualize abnormal phenotypes between predicted normal 3D shapes and corresponding observed ones. Finally, we demonstrate that the PCD-Net's learned 3D deformation encodings outperform multiple clinical and machine learning benchmarks by 11% in terms of area under the receiver operating characteristic curve for the tasks of prevalent MI detection and incident MI prediction and by 7% in terms of Harrell's concordance index for MI survival analysis.

Cardiac digital twins (CDTs) have the potential to offer individualized evaluation of cardiac function in a non-invasive manner, making them a promising approach for personalized diagnosis and treatment planning of myocardial infarction (MI). The inference of accurate myocardial tissue properties is crucial in creating a reliable CDT of MI. In this work, we investigate the feasibility of inferring myocardial tissue properties from the electrocardiogram (ECG) within a CDT platform. The platform integrates multi-modal data, such as cardiac MRI and ECG, to enhance the accuracy and reliability of the inferred tissue properties. We perform a sensitivity analysis based on computer simulations, systematically exploring the effects of infarct location, size, degree of transmurality, and electrical activity alteration on the simulated QRS complex of ECG, to establish the limits of the approach. We subsequently present a novel deep computational model, comprising a dual-branch variational autoencoder and an inference model, to infer infarct location and distribution from the simulated QRS. The proposed model achieves mean Dice scores of 0.457 ±0.317 and 0.302 ±0.273 for the inference of left ventricle scars and border zone, respectively. The sensitivity analysis enhances our understanding of the complex relationship between infarct characteristics and electrophysiological features. The in silico experimental results show that the model can effectively capture the relationship for the inverse inference, with promising potential for clinical application in the future. The code is available at https://github.com/lileitech/MI_inverse_inference.

Hypertension is a serious medical condition that affects over a billion people worldwide. The proper management of disease progression requires an extended knowledge of the overall functional and structural changes in the whole body in response to the hypertension. Here, we propose HyperScore, an integrative and unified measure of hypertension progression relative to multi-organ and multi-modality clinical measurements and based on a semi-supervised machine learning (ML) approach. We developed the measure based on a large participating cohort from the UK Biobank database (n=27,099) with over 500 imaging and clinical variables from multiple modalities. The semi-supervised approach was developed based on the contrastive trajectory inference mechanism to provide a score that reflects the proximity of a participant to the disease state (range: 0–1). Modelling revealed that majority of hypertensive participants had scores above 0.25, whereas normotensives had scores below this threshold. The sensitivity and specificity were above 89%, with an area under the receiver operating characteristics of 96.4%. The modelling showed a stable performance when evaluating hidden testing sets on a 10-fold cross-validation scheme with nearly 0.1 error. There was a strong association (r2>0.6) between HyperScore and organs’ phenotypic patterns, especially for variables such as white matter hyperintensity and body mass index. This study is the first to potentiate ML-based modelling of hypertension progression from a multi-organ perspective, which could significantly aid in clinical decision making to save lives.

Cine magnetic resonance imaging (MRI) is the current gold standard for the assessment of cardiac anatomy and function. However, it typically only acquires a set of two-dimensional (2D) slices of the underlying three-dimensional (3D) anatomy of the heart, thus limiting the understanding and analysis of both healthy and pathological cardiac morphology and physiology. In this paper, we propose a novel fully automatic surface reconstruction pipeline capable of reconstructing multi-class 3D cardiac anatomy meshes from raw cine MRI acquisitions. Its key component is a multi-class point cloud completion network (PCCN) capable of correcting both the sparsity and misalignment issues of the 3D reconstruction task in a unified model. We first evaluate the PCCN on a large synthetic dataset of biventricular anatomies and observe Chamfer distances between reconstructed and gold standard anatomies below or similar to the underlying image resolution for multiple levels of slice misalignment. Furthermore, we find a reduction in reconstruction error compared to a benchmark 3D U-Net by 32% and 24% in terms of Hausdorff distance and mean surface distance, respectively. We then apply the PCCN as part of our automated reconstruction pipeline to 1000 subjects from the UK Biobank study in a cross-domain transfer setting and demonstrate its ability to reconstruct accurate and topologically plausible biventricular heart meshes with clinical metrics comparable to the previous literature. Finally, we investigate the robustness of our proposed approach and observe its capacity to successfully handle multiple common outlier conditions.

In general, the hidden Markov random field (HMRF) represents the class label distribution of an image in probabilistic model based segmentation. The class label distributions provided by existing HMRF models consider either the number of neighboring pixels with similar class labels or the spatial distance of neighboring pixels with dissimilar class labels. Also, this spatial information is only considered for estimation of class labels of the image pixels, while its contribution in parameter estimation is completely ignored. This, in turn, deteriorates the parameter estimation, resulting in sub-optimal segmentation performance. Moreover, the existing models assign equal weightage to the spatial information for class label estimation of all pixels throughout the image, which, create significant misclassification for the pixels in boundary region of image classes. In this regard, the paper develops a new clique potential function and a new class label distribution, incorporating the information of image class parameters. Unlike existing HMRF model based segmentation techniques, the proposed framework introduces a new scaling parameter that adaptively measures the contribution of spatial information for class label estimation of image pixels. The importance of the proposed framework is depicted by modifying the HMRF based segmentation methods. The advantage of proposed class label distribution is also demonstrated irrespective of the underlying intensity distributions. The comparative performance of the proposed and existing class label distributions in HMRF model is demonstrated both qualitatively and quantitatively for brain MR image segmentation, HEp-2 cell delineation, natural image and object segmentation.

X-ray angiography is the most commonly used imaging modality for the detection of coronary stenoses due to its high spatial and temporal resolution of lumen contour and its utility to guide coronary interventions in real time. However, the high inter- and intra-observer variability in interpreting the geometry of 3D vascular structure based on multiple 2D image projections is a limitation in the accurate determination of lesion severity. This could be addressed by the 3D reconstruction of the coronary arterial (CA) tree. The automated reconstruction of 3D CA tree from 2D projections is challenging due to the existence of several imaging artifacts, such as vessel overlap, foreshortening, and most importantly respiratory and cardiac motion. Along with these artifacts, the acquisition geometry introduces the possibility of generating false vessel segments in the reconstruction. Our approach aims to reduce the motion artifacts in angiographic projections by developing a new method for rigid and non-rigid motion correction. A novel point-cloud based approach is subsequently introduced for reconstruction of 3D vessel centerlines by iteratively minimizing the reconstruction error. The performance of the proposed 3D reconstruction is evaluated using angiographic projections from 45 patients, producing average reprojection errors of 0.092 ±0.055 mm and 0.910 ±0.352 mm for 3D centerlines reconstruction, when co-registered with the parent vessels on projection planes that were/were not used to derive the 3D reconstruction, respectively. A comparison of the reconstructed 3D lumen surface with optical coherence tomography (OCT) measurements has been performed, showing no statistically significant difference in the luminal cross-sections reconstructed with our method, compared to OCT.

The segmentation of brain MR images into different tissue classes is an important task for automatic image analysis technique, particularly due to the presence of intensity inhomogeneity artifact in MR images. In this regard, this paper presents a novel approach for simultaneous segmentation and bias field correction in brain MR images. It integrates judiciously the concept of rough sets and the merit of a novel probability distribution, called stomped normal (SN) distribution. The intensity distribution of a tissue class is represented by SN distribution, where each tissue class consists of a crisp lower approximation and a probabilistic boundary region. The intensity distribution of brain MR image is modeled as a mixture of finite number of SN distributions and one uniform distribution. The proposed method incorporates both the expectation-maximization and hidden Markov random field frameworks to provide an accurate and robust segmentation. The performance of the proposed approach, along with a comparison with related methods, is demonstrated on a set of synthetic and real brain MR images for different bias fields and noise levels.

One of the challenging tasks for magnetic resonance (MR) image analysis is to remove the intensity inhomogeneity artifact present in MR images, which often degrades the performance of an automatic image analysis technique. In this regard, the paper presents a novel approach for bias field correction in MR images. It judiciously integrates the merits of rough sets and contraharmonic mean. While the contraharmonic mean is used in low-pass averaging filter to estimate the bias field in multiplicative model, the concept of lower approximation and boundary region of rough sets deals with vagueness and incompleteness in filter structure definition. A theoretical analysis is presented to justify the use of both rough sets and contraharmonic mean for bias field estimation. The integration enables the algorithm to estimate optimum or near optimum bias field. Some new quantitative indexes are introduced to measure intensity inhomogeneity artifact present in a MR image. The performance of the proposed approach, along with a comparison with other approaches, is demonstrated on both simulated and real MR images for different bias fields and noise levels.