Deep neural networks exploit shortcuts—spurious correlations like laterality markers (spatial) or scanner-specific noise (spectral)—that severely compromise generalization. Many healthcare applications face multiple concurrent shortcuts that are both spatial and spectral, which existing methods struggle to handle. We present SilverLining, an attention-based preprocessing framework that simultaneously identifies and mitigates both spatial and spectral shortcuts without introducing new spurious correlations. Our key insight is that naive removal of shortcut features can itself create new shortcuts, where models learn to exploit the removal patterns as new spurious correlations. We address this through a principled confounder-free correction strategy that maintains consistent preprocessing patterns across all classes in both spatial and frequency domains, preventing new confounders. Extensive experiments demonstrate SilverLining’s effectiveness: achieving 0.87 AUC on controlled vision tasks and 0.94 AUC on counter-shortcut medical imaging evaluation where shortcuts are reversed; improving cross-institutional chest X-ray classification from 0.72 to 0.77 AUC; and 0.54 mAP on polyp detection despite natural spurious correlations from surgical overlays. Our data-centric approach provides an effective solution for reducing multiple types of data shortcuts without architectural modifications, creating preprocessed datasets that improve model robustness across both classification and detection tasks. Our codebase is available at https://github.com/theidentity/SilverLining_WACV2026/.

Radiation therapy is an integral part of cancer treatment, but to date it remains highly manual. Plans are created through optimization of dose volume objectives that specify intent to minimize, maximize, or achieve a prescribed dose level to clinical targets and organs. Optimization is NP-hard, requiring highly iterative and manual initialization procedures. We present a proof-of-concept for a method to automatically infer the radiation dose directly from the patient's treatment planning image based on a database of previous patients with corresponding clinical treatment plans. Our method uses regression forests augmented with density estimation over the most informative features to learn an automatic atlas-selection metric that is tailored to dose prediction. We validate our approach on 276 patients from 3 clinical treatment plan sites (whole breast, breast cavity, and prostate), with an overall dose prediction accuracies of 78.68%, 64.76%, 86.83% under the Gamma metric.

Radiation therapy is used to treat cancer patients around the world. High quality treatment plans maximally radiate the targets while minimally radiating healthy organs at risk. In order to judge plan quality and safety, segmentations of the targets and organs at risk are created, and the amount of radiation that will be delivered to each structure is estimated prior to treatment. If the targets or organs at risk are mislabelled, or the segmentations are of poor quality, the safety of the radiation doses will be erroneously reviewed and an unsafe plan could proceed. We propose a technique to automatically label groups of segmentations of different structures from a radiation therapy plan for the joint purposes of providing quality assurance and data mining. Given one or more segmentations and an associated image we seek to assign medically meaningful labels to each segmentation and report the confidence of that label. Our method uses random forests to learn joint distributions over the training features, and then exploits a set of learned potential group configurations to build a conditional random field (CRF) that ensures the assignment of labels is consistent across the group of segmentations. The CRF is then solved via a constrained assignment problem. We validate our method on 1574 plans, consisting of 17[Formula: see text] 579 segmentations, demonstrating an overall classification accuracy of 91.58%. Our results also demonstrate the stability of RF with respect to tree depth and the number of splitting variables in large data sets.

We explore the application of genetic algorithms (GA) to deformable models through the proposition of a novel method for medical image segmentation that combines GA with nonconvex, localized, medial-based shape statistics. We replace the more typical gradient descent optimizer used in deformable models with GA, and the convex, implicit, global shape statistics with nonconvex, explicit, localized ones. Specifically, we propose GA to reduce typical deformable model weaknesses pertaining to model initialization, pose estimation and local minima, through the simultaneous evolution of a large number of models. Furthermore, we constrain the evolution, and thus reduce the size of the search-space, by using statistically-based deformable models whose deformations are intuitive (stretch, bulge, bend) and are driven in terms of localized principal modes of variation, instead of modes of variation across the entire shape that often fail to capture localized shape changes. Although GA are not guaranteed to achieve the global optima, our method compares favorably to the prevalent optimization techniques, convex/nonconvex gradient-based optimizers and to globally optimal graph-theoretic combinatorial optimization techniques, when applied to the task of corpus callosum segmentation in 50 mid-sagittal brain magnetic resonance images.

Image segmentation is often performed via the minimization of an energy function over a domain of possible segmentations. The effectiveness and applicability of such methods depends greatly on the properties of the energy function and its domain, and on what information can be encoded by it. Here we propose an energy function that achieves several important goals. Specifically, our energy function is convex and incorporates shape prior information while simultaneously generating a probabilistic segmentation for multiple regions. Our energy function represents multi-region probabilistic segmentations as elements of a vector space using the isometric log-ratio (ILR) transformation. To our knowledge, these four goals (convex, with shape priors, multi-region, and probabilistic) do not exist together in any other method, and this is the first time ILR is used in an image segmentation method. We provide examples demonstrating the usefulness of these features.

A method for visualizing manifold-valued medical image data is proposed. The method operates on images in which each pixel is assumed to be sampled from an underlying manifold. For example, each pixel may contain a high dimensional vector, such as the time activity curve (TAC) in a dynamic positron emission tomography (dPET) or a dynamic single photon emission computed tomography (dSPECT) image, or the positive semi-definite tensor in a diffusion tensor magnetic resonance image (DTMRI). A nonlinear mapping reduces the dimensionality of the pixel data to achieve two goals: distance preservation and embedding into a perceptual color space. We use multidimensional scaling distance-preserving mapping to render similar pixels (e.g., DT or TAC pixels) with perceptually similar colors. The 3D CIELAB perceptual color space is adopted as the range of the distance preserving mapping, with a final similarity transform mapping colors to a maximum gamut size. Similarity between pixels is either determined analytically as geodesics on the manifold of pixels or is approximated using manifold learning techniques. In particular, dissimilarity between DTMRI pixels is evaluated via a Log-Euclidean Riemannian metric respecting the manifold of the rank 3, second-order positive semi-definite DTs, whereas the dissimilarity between TACs is approximated via ISOMAP. We demonstrate our approach via artificial high-dimensional, manifold-valued data, as well as case studies of normal and pathological clinical brain and heart DTMRI, dPET, and dSPECT images. Our results demonstrate the effectiveness of our approach in capturing, in a perceptually meaningful way, important features in the data.

We present a novel approach to the segmentation and analysis of vasculature from volumetric medical image data. Our method is an adoption and significant extension of deformable organisms, an artificial life framework for medical image analysis that complements classical deformable models with high-level, anatomically-driven control mechanisms. We extend deformable organisms to 3D, model their bodies as tubular spring-mass systems, and equip them with a new repertoire of sensory modules, behavioral routines, and decision making strategies. The result is a new breed of robust deformable organisms, vessel crawlers, that crawl along vasculature in 3D images, accurately segmenting vessel boundaries, detecting and exploring bifurcations, and providing sophisticated, clinically-relevant structural analysis. We validate our method through the segmentation and analysis of vascular structures in both noisy synthetic and real medical image data.