Accurate prediction of breast cancer recurrence after treatment is essential for improving long-term outcomes. However, existing models are limited by three key challenges: (1) they typically rely on single-modal data, missing cross-modal interactions; (2) they analyze static snapshots, failing to capture disease progression over time; and (3) they often perform coarse feature fusion, lacking semantic disentanglement and interpretability. To address these issues, we propose LUMIN (Longitudinal Multi-modal Knowledge Decomposition Network), a novel framework that integrates longitudinal mammograms and electronic health records (EHRs) for recurrence prediction. LUMIN leverages a vision-language contrastive pretraining backbone to align multi-modal representations and introduces two knowledge extraction modules: (1) a Cross-Modal Disentangled Knowledge Extractor (CM-DKE) that separates shared, complementary, and modality-specific information across imaging and text; and (2) a Temporal Evolution Disentangled Knowledge Extractor (TE-DKE) that captures time-invariant, time-varying, and time-specific features to model disease dynamics. Experiments on a large-scale dataset of 3,924 patients and 19,684 exams show that LUMIN significantly outperforms state-of-the-art baselines, demonstrating its effectiveness in capturing both multi-modal semantics and temporal heterogeneity for recurrence prediction.

Longitudinal medical imaging is crucial for monitoring neoadjuvant therapy (NAT) response in clinical practice. However, mainstream artificial intelligence (AI) methods for disease monitoring commonly rely on extensive segmentation labels to evaluate lesion progression. While self-supervised vision-language (VL) learning efficiently captures medical knowledge from radiology reports, existing methods focus on single time points, missing opportunities to leverage temporal self-supervision for disease progression tracking. In addition, extracting dynamic progression from longitudinal unannotated images with corresponding textual data poses challenges. In this work, we explicitly account for longitudinal NAT examinations and accompanying reports, encompassing scans before NAT and follow-up scans during mid-/post-NAT. We introduce the multi-modal longitudinal representation learning pipeline (MLRL), a temporal foundation model, that employs multi-scale self-supervision scheme, including single-time scale vision-text alignment (VTA) learning and multi-time scale visual/textual progress (TVP/TTP) learning to extract temporal representations from each modality, thereby facilitates the downstream evaluation of tumor progress. Our method is evaluated against several state-of-the-art self-supervised longitudinal learning and multi-modal VL methods. Results from internal and external datasets demonstrate that our approach not only enhances label efficiency across the zero-, few- and full-shot regime experiments but also significantly improves tumor response prediction in diverse treatment scenarios. Furthermore, MLRL enables interpretable visual tracking of progressive areas in temporal examinations, offering insights into longitudinal VL foundation tools and potentially facilitating the temporal clinical decision-making process.

Lesion volume is an important predictor for prognosis in breast cancer. However, it is currently impossible to compute lesion volumes accurately from digital mammography data, which is the most popular and readily available imaging modality for breast cancer. We make a step towards a more accurate lesion volume measurement on digital mammograms by developing a model that allows to estimate lesion volumes on processed mammogram. Processed mammograms are the images routinely used by radiologists in clinical practice as well as in breast cancer screening and are available in medical centers. Processed mammograms are obtained from raw mammograms, which are the X-ray data coming directly from the scanner, by applying certain vendor-specific non-linear transformations. At the core of our volume estimation method is a physics-based algorithm for measuring lesion volumes on raw mammograms. We subsequently extend this algorithm to processed mammograms via a deep learning image-to-image translation model that produces synthetic raw mammograms from processed mammograms in a multi-vendor setting. We assess the reliability and validity of our method using a dataset of 1778 mammograms with an annotated mass. Firstly, we investigate the correlations between lesion volumes computed from mediolateral oblique and craniocaudal views, with a resulting Pearson correlation of 0.93 [95% confidence interval (CI) 0.92 - 0.93]. Secondly, we compare the resulting lesion volumes from true and synthetic raw data, with a resulting Pearson correlation of 0.998 [95%CI 0.998 - 0.998] . Finally, for a subset of 100 mammograms with a malignant mass and concurrent MRI examination available, we analyze the agreement between lesion volume on mammography and MRI, resulting in an intraclass correlation coefficient of 0.81 [95%CI 0.73 - 0.87] for consistency and 0.78 [95%CI 0.66 - 0.86] for absolute agreement. In conclusion, we developed an algorithm to measure mammographic lesion volume that reached excellent reliability and good validity, when using MRI as ground truth. The algorithm may play a role in lesion characterization and breast cancer prognostication on mammograms.

Breast cancer is one of the most diagnosed types of cancer worldwide. Volumetric ultrasound breast imaging, combined with MRI can improve lesion detection rate, reduce examination time, and improve lesion diagnosis. However, to our knowledge, there are no 3D US breast imaging systems available that facilitate 3D US - MRI image fusion. In this paper, a novel Automated Cone-based Breast Ultrasound System (ACBUS) is introduced. The system facilitates volumetric ultrasound acquisition of the breast in a prone position without deforming it by the US transducer. Quality of ACBUS images for reconstructions at different voxel sizes (0.25 and 0.50 mm isotropic) was compared to quality of the Automated Breast Volumetric Scanner (ABVS) (Siemens Ultrasound, Issaquah, WA, USA) in terms of signal-to-noise ratio (SNR), contrast-to-noise ratio (CNR), and resolution using a custom made phantom. The ACBUS image data were registered to MRI image data utilizing surface matching and the registration accuracy was quantified using an internal marker. The technology was also evaluated in vivo. The phantom-based quantitative analysis demonstrated that ACBUS can deliver volumetric breast images with an image quality similar to the images delivered by a currently commercially available Siemens ABVS. We demonstrate on the phantom and in vivo that ACBUS enables adequate MRI-3D US fusion. To our conclusion, ACBUS might be a suitable candidate for a second-look breast US exam, patient follow-up, and US guided biopsy planning.

Breast density measurement is an important aspect in breast cancer diagnosis as dense tissue has been related to the risk of breast cancer development. The purpose of this study is to develop a method to automatically compute breast density in breast MRI. The framework is a combination of image processing techniques to segment breast and fibroglandular tissue. Intra- and interpatient signal intensity variability is initially corrected. The breast is segmented by automatically detecting body-breast and air-breast surfaces. Subsequently, fibroglandular tissue is segmented in the breast area using expectation-maximization. A dataset of 50 cases with manual segmentations was used for evaluation. Dice similarity coefficient (DSC), total overlap, false negative fraction (FNF), and false positive fraction (FPF) are used to report similarity between automatic and manual segmentations. For breast segmentation, the proposed approach obtained DSC, total overlap, FNF, and FPF values of 0.94, 0.96, 0.04, and 0.07, respectively. For fibroglandular tissue segmentation, we obtained DSC, total overlap, FNF, and FPF values of 0.80, 0.85, 0.15, and 0.22, respectively. The method is relevant for researchers investigating breast density as a risk factor for breast cancer and all the described steps can be also applied in computer aided diagnosis systems.

Determining corresponding regions between an MRI and an X-ray mammogram is a clinically useful task that is challenging for radiologists due to the large deformation that the breast undergoes between the two image acquisitions. In this work we propose an intensity-based image registration framework, where the biomechanical transformation model parameters and the rigid-body transformation parameters are optimised simultaneously. Patient-specific biomechanical modelling of the breast derived from diagnostic, prone MRI has been previously used for this task. However, the high computational time associated with breast compression simulation using commercial packages, did not allow the optimisation of both pose and FEM parameters in the same framework. We use a fast explicit Finite Element (FE) solver that runs on a graphics card, enabling the FEM-based transformation model to be fully integrated into the optimisation scheme. The transformation model has seven degrees of freedom, which include parameters for both the initial rigid-body pose of the breast prior to mammographic compression, and those of the biomechanical model. The framework was tested on ten clinical cases and the results were compared against an affine transformation model, previously proposed for the same task. The mean registration error was 11.6±3.8mm for the CC and 11±5.4mm for the MLO view registrations, indicating that this could be a useful clinical tool.

Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) of the breast has become an invaluable tool in the clinical work-up of patients suspected of having breast carcinoma. The purpose of this study is to introduce novel features extracted from the kinetics of contrast agent uptake imaged by a short (100 s) view-sharing MRI protocol, and to investigate how these features measure up to commonly used features for regular DCE-MRI of the breast. Performance is measured with a computer aided diagnosis (CADx) system aimed at distinguishing benign from malignant lesions. A bi-temporal breast MRI protocol was used. This protocol produces five regular, high spatial-resolution T1-weighted acquisitions interleaved with a series of 20 ultrafast view-sharing acquisitions during contrast agent uptake. We measure and compare the performances of morphological and kinetic features derived from both the regular DCE-MRI sequence and the ultrafast view-sharing sequence with four different classifiers. The classification performance of kinetics derived from the short (100 s) ultrafast acquisition starting with contrast agent administration, is significantly higher than the performance of kinetics derived from a much lengthier (510 s), commonly used 3-D gradient echo acquisition. When combined with morphology information all classifiers show a higher performance for the ultrafast acquisition (two out of four results are significantly better).

Digital breast tomosynthesis (DBT) is a promising 3-D modality that may replace mammography in the future. However, lesion search is likely to require more time in DBT volumes, while comparisons between views from different projections and prior exams might be harder to make. This may make screening with DBT cumbersome. A solution may be provided by synthesizing 2-D mammograms from DBT, which may then be used to guide the search for abnormalities. In this work we focus on synthesizing mammograms in which masses and architectural distortions are optimally visualized. Our approach first determines relevant points in a DBT volume with a computer-aided detection system and then renders a mammogram from the intersection of a surface fitted through these points and the DBT volume. The method was evaluated in a pilot observer study where three readers reported mass findings in 87 patients (25 malignant, 62 normal) for which both DBT and digital mammograms were available. We found that on average, diagnostic accuracy in the synthetic mammograms was higher (Az=0.85) than in conventional mammograms (Az=0.81), although the difference was not statistically significant. Preliminary results suggest that the synthesized mammograms are an acceptable alternative for real mammograms regarding the detection of mass lesions.

Automated 3-D breast ultrasound (ABUS) has gained a lot of interest and may become widely used in screening of dense breasts, where sensitivity of mammography is poor. However, reading ABUS images is time consuming, and subtle abnormalities may be missed. Therefore, we are developing a computer aided detection (CAD) system to help reduce reading time and prevent errors. In the multi-stage system we propose, segmentations of the breast, the nipple and the chestwall are performed, providing landmarks for the detection algorithm. Subsequently, voxel features characterizing coronal spiculation patterns, blobness, contrast, and depth are extracted. Using an ensemble of neural-network classifiers, a likelihood map indicating potential abnormality is computed. Local maxima in the likelihood map are determined and form a set of candidates in each image. These candidates are further processed in a second detection stage, which includes region segmentation, feature extraction and a final classification. On region level, classification experiments were performed using different classifiers including an ensemble of neural networks, a support vector machine, a k-nearest neighbors, a linear discriminant, and a gentle boost classifier. Performance was determined using a dataset of 238 patients with 348 images (views), including 169 malignant and 154 benign lesions. Using free response receiver operating characteristic (FROC) analysis, the system obtains a view-based sensitivity of 64% at 1 false positives per image using an ensemble of neural-network classifiers.