Denoising Diffusion Models (DDMs) are widely used for high-quality image generation and medical image segmentation but often rely on Unet-based architectures, leading to high computational overhead, especially with high-resolution images. This work proposes three NCA-based improvements for diffusion-based medical image segmentation. First, CBAMMedSegDiffNCA incorporates channel and spatial attention for improved segmentation. Second, Multi-MedSegDiffNCA uses a multilevel NCA framework to refine rough noise estimates generated by lower-level NCA models. Third, MultiCBAMMedSegDiffNCA combines these methods with a new RGB channel loss for semantic guidance. Evaluations on Lesion segmentation show that MultiCBAM-MedSegDiffNCA matches Unet-based model performance with a dice score of 87.84% while using 60-110 times fewer parameters and 5 times faster training, offering an efficient solution for low-resource medical settings.

Augmentation by generative modelling yields a promising alternative to the accumulation of surgical data, where ethical, organisational and regulatory aspects must be considered. Yet, the joint synthesis of (image, mask) pairs for segmentation, a major application in surgery, is rather unexplored. We propose to learn semantically comprehensive yet compact latent representations of the (image, mask) space, which we jointly model with a Latent Diffusion Model. We show that our approach can effectively synthesise unseen high-quality paired segmentation data of remarkable semantic coherence. Generative augmentation is typically applied pre-training by synthesising a fixed number of additional training samples to improve downstream task models. To enhance this approach, we further propose Generative Adaptive Uncertainty-guided Diffusionbased Augmentation (GAUDA), leveraging the epistemic uncertainty of a Bayesian downstream model for targeted online synthesis. We condition the generative model on classes with high estimated uncertainty during training to produce additional unseen samples for these classes. By adaptively utilising the generative model online, we can minimise the number of additional training samples and centre them around the currently most uncertain parts of the data distribution. GAUDA effectively improves downstream segmentation results over comparable methods by an average absolute IoU of 1.6% on CaDISv2 and 1.5% on CholecSeg8k, two prominent surgical datasets for semantic segmentation.

Continual learning (CL) in medical imaging presents a unique challenge, requiring models to adapt to new domains while retaining previously acquired knowledge. We introduce NCAdapt, a Neural Cellular Automata (NCA) based method designed to address this challenge. NCAdapt features a domain-specific multi-head structure, integrating adaptable convolutional layers into the NCA backbone for each new domain encountered. After initial training, the NCA backbone is frozen, and only the newly added adaptable convolutional layers, consisting of 384 parameters, are trained along with domain-specific NCA convolutions. We evaluate NCAdapt on hippocampus segmentation tasks, benchmarking its performance against Lifelong nnU-Net and U-Net models with state-of-the-art (SOTA) CL methods. Our lightweight approach achieves SOTA performance, underscoring its effectiveness in addressing CL challenges in medical imaging. Upon acceptance, we will make our code base publicly accessible to support reproducibility and foster further advancements in medical CL.

Intracranial Hemorrhage is a potentially lethal condition whose manifestation is vastly diverse and shifts across clinical centers worldwide. Deep-learning-based solutions are starting to model complex relations between brain structures, but still struggle to generalize. While gathering more diverse data is the most natural approach, privacy regulations often limit the sharing of medical data. We propose the first application of Federated Scene Graph Generation. We show that our models can leverage the increased training data diversity. For Scene Graph Generation, they can recall up to 20% more clinically relevant relations across datasets compared to models trained on a single centralized dataset. Learning structured data representation in a federated setting can open the way to the development of new methods that can leverage this finer information to regularize across clients more effectively.

The reliance on computationally intensive U-Net and Transformer architectures significantly limits their accessibility in low-resource environments, creating a technological divide that hinders global healthcare equity, especially in medical diagnostics and treatment planning. This divide is most pronounced in low- and middle-income countries, primary care facilities, and conflict zones. We introduced MED-NCA, Neural Cellular Automata (NCA) based segmentation models characterized by their low parameter count, robust performance, and inherent quality control mechanisms. These features drastically lower the barriers to high-quality medical image analysis in resource-constrained settings, allowing the models to run efficiently on hardware as minimal as a Raspberry Pi or a smartphone. Building upon the foundation laid by MED-NCA, this paper extends its validation across eight distinct anatomies, including the hippocampus and prostate (MRI, 3D), liver and spleen (CT, 3D), heart and lung (X-ray, 2D), breast tumor (Ultrasound, 2D), and skin lesion (Image, 2D). Our comprehensive evaluation demonstrates the broad applicability and effectiveness of MED-NCA in various medical imaging contexts, matching the performance of two magnitudes larger UNet models. Additionally, we introduce NCA-VIS, a visualization tool that gives insight into the inference process of MED-NCA and allows users to test its robustness by applying various artifacts. This combination of efficiency, broad applicability, and enhanced interpretability makes MED-NCA a transformative solution for medical image analysis, fostering greater global healthcare equity by making advanced diagnostics accessible in even the most resource-limited environments. • Introducing bio-inspired emergent systems for resilient medical image segmentation. • MED-NCA needs only 10k–70k parameters for high-quality medical image segmentation. • MED-NCA matches the average Dice accuracy of UNet models 2–3 magnitudes larger. • NCAs enable unique insight into the inference process via their one-cell architecture. • NCA-VIS visualizes inference and allows robustness testing with various artifacts.

Continual learning (CL) methods designed for natural image classification often fail to reach basic quality standards for medical image segmentation. Atlas-based segmentation, a well-established approach in medical imaging, incorporates domain knowledge on the region of interest, leading to semantically coherent predictions. This is especially promising for CL, as it allows us to leverage structural information and strike an optimal balance between model rigidity and plasticity over time. When combined with privacy-preserving prototypes, this process offers the advantages of rehearsal-based CL without compromising patient privacy. We propose Atlas Replay, an atlas-based segmentation approach that uses prototypes to generate high-quality segmentation masks through image registration that maintain consistency even as the training distribution changes. We explore how our proposed method performs compared to state-of-the-art CL methods in terms of knowledge transferability across seven publicly available prostate segmentation datasets. Prostate segmentation plays a vital role in diagnosing prostate cancer, however, it poses challenges due to substantial anatomical variations, benign structural differences in older age groups, and fluctuating acquisition parameters. Our results show that Atlas Replay is both robust and generalizes well to yet-unseen domains while being able to maintain knowledge, unlike end-to-end segmentation methods. Our code base is available under https://github.com/MECLabTUDA/Atlas-Replay.

Histological images are frequently impaired by local artifacts from scanner malfunctions or iatrogenic processes - caused by preparation - impacting the performance of Deep Learning models. Models often struggle with the slightest out-of-distribution shifts, resulting in compromised performance. Detecting artifacts and failure modes of the models is crucial to ensure open-world applicability to whole slide images for tasks like segmentation or diagnosis. We introduce a novel technique for out-of-distribution detection within whole slide images, compatible with any segmentation or classification model. Our approach tiles multi-layer features into sliding window patches and leverages optimal transport to align them with recognized in-distribution samples. We average the optimal transport costs over tiles and layers to detect out-of-distribution samples. Notably, our method excels in identifying failure modes that would harm downstream performance, surpassing contemporary out-of-distribution detection techniques. We evaluate our method for both natural and synthetic artifacts, considering distribution shifts of various sizes and types. The results confirm that our technique outperforms alternative methods for artifact detection. We assess our method components and the ability to negate the impact of artifacts on the downstream tasks. Finally, we demonstrate that our method can mitigate the risk of performance drops in downstream tasks, enhancing reliability by up to 77%. In testing 7 annotated whole slide images with natural artifacts, our method boosted the Dice score by 68%, highlighting its real open-world utility.

Statistical shape modeling is a powerful tool for visualizing and quantifying geometric and functional patterns of the heart. After myocardial infarction (MI), the left ventricle typically remodels in response to physiological challenges. Several methods have been proposed in the literature to describe statistical shape changes. Which method best characterizes left ventricular remodeling after MI is an open research question. A better descriptor of remodeling is expected to provide a more accurate evaluation of disease status in MI patients. We therefore designed a challenge to test shape characterization in MI given a set of three-dimensional left ventricular surface points. The training set comprised 100 MI patients, and 100 asymptomatic volunteers (AV). The challenge was initiated in 2015 at the Statistical Atlases and Computational Models of the Heart workshop, in conjunction with the MICCAI conference. The training set with labels was provided to participants, who were asked to submit the likelihood of MI from a different (validation) set of 200 cases (100 AV and 100 MI). Sensitivity, specificity, accuracy and area under the receiver operating characteristic curve were used as the outcome measures. The goals of this challenge were to (1) establish a common dataset for evaluating statistical shape modeling algorithms in MI, and (2) test whether statistical shape modeling provides additional information characterizing MI patients over standard clinical measures. Eleven groups with a wide variety of classification and feature extraction approaches participated in this challenge. All methods achieved excellent classification results with accuracy ranges from 0.83 to 0.98. The areas under the receiver operating characteristic curves were all above 0.90. Four methods showed significantly higher performance than standard clinical measures. The dataset and software for evaluation are available from the Cardiac Atlas Project website1.

A fully automated 2-D+time myocardial segmentation framework is proposed for cardiac magnetic resonance (CMR) blood-oxygen-level-dependent (BOLD) data sets. Ischemia detection with CINE BOLD CMR relies on spatio-temporal patterns in myocardial intensity, but these patterns also trouble supervised segmentation methods, the de facto standard for myocardial segmentation in cine MRI. Segmentation errors severely undermine the accurate extraction of these patterns. In this paper, we build a joint motion and appearance method that relies on dictionary learning to find a suitable subspace. Our method is based on variational pre-processing and spatial regularization using Markov random fields, to further improve performance. The superiority of the proposed segmentation technique is demonstrated on a data set containing cardiac phase-resolved BOLD MR and standard CINE MR image sequences acquired in baseline and ischemic condition across ten canine subjects. Our unsupervised approach outperforms even supervised state-of-the-art segmentation techniques by at least 10% when using Dice to measure accuracy on BOLD data and performs at par for standard CINE MR. Furthermore, a novel segmental analysis method attuned for BOLD time series is utilized to demonstrate the effectiveness of the proposed method in preserving key BOLD patterns.

A comprehensive framework for detection and characterization of partial intrinsic symmetry over 3D shapes is proposed. To identify prominent symmetric regions which overlap in space and vary in form, the proposed framework is decoupled into a Correspondence Space Voting (CSV) procedure followed by a Transformation Space Mapping (TSM) procedure. In the CSV procedure, significant symmetries are first detected by identifying surface point pairs on the input shape that exhibit local similarity in terms of their intrinsic geometry while simultaneously maintaining an intrinsic distance structure at a global level. To allow detection of potentially overlapping symmetric shape regions, a global intrinsic distance-based voting scheme is employed to ensure the inclusion of only those point pairs that exhibit significant intrinsic symmetry. In the TSM procedure, the Functional Map framework is employed to generate the final map of symmetries between point pairs. The TSM procedure ensures the retrieval of the underlying dense correspondence map throughout the 3D shape that follows a particular symmetry. The TSM procedure is also shown to result in the formulation of a metric symmetry space where each point in the space represents a specific symmetry transformation and the distance between points represents the complexity between the corresponding transformations. Experimental results show that the proposed framework can successfully analyze complex 3D shapes that possess rich symmetries.

A novel multi-criteria optimization framework for matching of partially visible shapes in multiple images using joint geometric graph embedding is proposed. The proposed framework achieves matching of partial shapes in images that exhibit extreme variations in scale, orientation, viewpoint and illumination and also instances of occlusion; conditions which render impractical the use of global contour-based descriptors or local pixel-level features for shape matching. The proposed technique is based on optimization of the embedding distances of geometric features obtained from the eigenspectrum of the joint image graph, coupled with regularization over values of the mean pixel intensity or histogram of oriented gradients. It is shown to obtain successfully the correspondences denoting partial shape similarities as well as correspondences between feature points in the images. A new benchmark dataset is proposed which contains disparate image pairs with extremely challenging variations in viewing conditions when compared to an existing dataset [18]. The proposed technique is shown to significantly outperform several state-of-the-art partial shape matching techniques on both datasets.

The limitations of conventional imaging techniques have hitherto precluded a thorough and formal investigation of the complex morphology of the left ventricular (LV) endocardial surface and its relation to the severity of coronary artery disease (CAD). However, recent developments in high-resolution multirow-detector computed tomography (MDCT) scanner technology have enabled the imaging of the complex LV endocardial surface morphology in a single heartbeat. Analysis of high-resolution computed tomography images from a 320-MDCT scanner allows for the noninvasive study of the relationship between the percent diameter stenosis (DS) values of the major coronary arteries and localization of the cardiac segments affected by coronary arterial stenosis. In this paper, a novel approach for the analysis of the nonrigid LV endocardial surface from MDCT images, using a combination of rigid body transformation-invariant shape descriptors and a more generalized isometry-invariant Bag-of-Features descriptor, is proposed and implemented. The proposed approach is shown to be successful in identifying, localizing, and quantifying the incidence and extent of CAD and, thus, is seen to have a potentially significant clinical impact. Specifically, the association between the incidence and extent of CAD, determined via the percent DS measurements of the major coronary arteries, and the alterations in the endocardial surface morphology is formally quantified. The results of the proposed approach on 16 normal datasets and 16 abnormal datasets exhibiting CAD with varying levels of severity are presented. A multivariable regression test is employed to test the effectiveness of the proposed morphological analysis approach. Experiments performed on a strictly leave-one-out basis are shown to exhibit a distinct and interesting pattern in terms of the correlation coefficient values within the cardiac segments, where the incidence of coronary arterial stenosis is localized.

A novel intrinsic geometric scale space formulation for 3D deformable surfaces termed as the Biharmonic Density Estimate (BDE) is proposed. The proposed BDE signature allows for multiscale surface feature-based representation of deformable 3D shapes for subsequent image and scene analysis. It is shown to provide an underlying theoretical framework for the concept of intrinsic geometric scale space, resulting in a highly descriptive characterization of both, the local surface structure and the global metric of the 3D shape. The compactness and robustness of the proposed BDE signature are demonstrated via a series of experiments and a key components detection application.