Visual retrieval from brain signals is a key challenge in Brain-Computer Interfaces (BCIs). Existing methods directly map noisy neural signals into a unified feature space, ignoring the dual-stream mechanism of neural decoding: brain signals are generated through the interaction of bottom-up perception (processing visual saliency) and top-down cognition (interpreting semantic concepts based on prior knowledge). In Rapid Serial Visual Presentation paradigm, standard encoders prioritize high-level semantic abstractions over actual salient features encoded in human vision, lacking the semantic guidance needed to stabilize inherently noisy signals. This mismatch causes perceptual misalignment and semantic inconsistency. To resolve this, we propose NeuroAlign, a dynamic dual-stream framework that explicitly separates and coordinates these two processes. Specifically, we utilize Visual Saliency Extraction (VSE) to capture salient features and Semantic Guidance Alignment (SGA) to provide top-down semantic guidance. These processes are coordinated by the Dynamic Loss Adjustment (DLA) mechanism, which adaptively prioritizes structural alignment before shifting to semantic refinement. Extensive experiments on THINGS-EEG2 and THINGS-MEG datasets demonstrate that NeuroAlign achieves SOTA performance, reaching 48.1% (\(\uparrow 2.4\%\)) and 14.5% (\(\uparrow 2.6\%\)) Top-1 accuracy in intra- and inter-subject retrieval, respectively.

Long-axial PET/CT systems have enabled ultrahigh sensitivity and a longer axial field of view for clinical imaging and diagnosis. However, radiation risks from radiotracers and CT scans have remained a persistent concern within total-body PET/CT systems. Conventional approaches focus mainly on PET radiotracer-based dose reduction, ignoring the substantial radiation burden inherent in CT acquisition. Therefore, we proposed a hybrid ultra-low-dose imaging framework (Mamba-SUM) for total-body PET/CT systems to restore high-quality PET images from ultra-low-dose PET (ULD PET) and ultra-low-dose CT (ULD CT) images. Our method innovatively integrates the Mamba architecture with wavelet transformation, enabling effective modeling of long-range dependencies while reducing computational overhead. Specifically, ULD PET and ULD CT images are first subjected to domain decomposition. Afterward, a custom-designed Low-Frequency Enhancement Module and a High-Frequency Denoising Module work in concert to leverage cross-domain and multimodal information, enhancing structural details and suppressing noise across different frequency subbands. Finally, a Mamba-based decoder progressively reconstructs the refined features to produce high-quality PET images with improved fidelity and diagnostic value. Experimental results have illustrated that our method achieved superior performance (PSNR: 28.88 dB $\pm ~3.26$ , SSIM: $0.92~\pm ~0.16$ , p< 0.05) compared with other models (VMamba, Mamba-Swin, SwinTransformer, CycleGAN and UNet). Moreover, the statistical analysis also revealed that the data distribution of our generated PET images was consistent with that of the ground truth (Pearson Correlation Coefficient> 0.95, p< 0.05). Our Mamba-SUM has provided a computationally effective approach for total-body ultra-low-dose PET/CT imaging. The code is available at https://github.com/LEE12365/Mamba-SUM.

Clinical imaging protocols designed to accelerate acquisition or reduce radiation dose often lead to degraded image quality, compromising diagnostic confidence. The heterogeneity in degradation types and severities across imaging modalities further challenges the development of generalized restoration solutions. In this work, we introduce a unified framework that formulates medical image restoration as posterior sampling from self-supervised Latent Diffusion Models (LDMs), pretrained on multi-modal high-quality images. At the core of our method is an Estimation Posterior Sampling (EPS) strategy, which enhances both data fidelity and anatomical detail retention. EPS incorporates two key components: (i) estimated diffusion initialization to constrain sampling within the measurement-consistent solution space, and (ii) gradient-balanced optimization to adaptively trade off denoising strength and detail preservation throughout the diffusion trajectory. Unlike traditional task-specific models, our approach enables Plug-and-Play (PnP) deployment, supporting diverse degradations without retraining. Extensive experiments conducted on deterministic degradations (e.g., under-sampled MRI, sparse-view CT) and blind degradations (e.g., low-dose PET) across multiple degradation levels demonstrate superior quantitative and qualitative performance compared to both supervised baselines and state-of-the-art posterior sampling methods. Notably, our method achieves PSNR improvements of up to +2.9 dB (MRI), +1.1 dB (CT), and +0.9 dB (PET) in PnP mode. These results highlight the robustness and broad applicability of our framework for clinical deployment.

Deep learning has demonstrated remarkable efficacy in reconstructing low-count PET (Positron EmissionTomography) images, attracting considerable attention in the medical imaging community. However, most existing deep learning approaches have not fully exploited the unique physical characteristics of PET imaging in the design of fidelity and prior regularization terms, resulting in constrained model performance and interpretability. In light of these considerations, we introduce an unrolled deep network based on maximum likelihood estimation for the Poisson distribution and a Generalized domain transformation for Sparsity learning, dubbed GS-Net. To address this complex optimization challenge, we employ the Alternating Direction Method of Multipliers (ADMM) framework, integrating a modified Expectation Maximization (EM) approach to address the primary objective and utilize the shrinkage thresholding approach to optimize the L1 norm term. Additionally, within this unrolled deep network, all hyperparameters are adaptively adjusted through end-to-end learning to eliminate the need for manual parameter tuning. Through extensive experiments on simulated patient brain datasets and real patient whole-body clinical datasets with multiple count levels, our method has demonstrated advanced performance compared to traditional non-iterative and iterative reconstruction, deep learning-based direct reconstruction, and hybrid unrolled methods, as demonstrated by qualitative and quantitative evaluations.

In total-body positron emission tomography and computed tomography (PET/CT) imaging, reducing the radiation dose of diagnostic CT scans is essential for minimizing overall radiation exposure, particularly in pediatric patients. Although deep learning-based denoising methods have shown promise in restoring low-dose CT (LDCT) to normal-dose CT (NDCT) quality, most approaches rely on structurally aligned paired data, which are difficult to acquire in clinical practice. Models trained on synthetic pairs often exhibit limited generalizability to real LDCT data. Unconditional diffusion models demonstrate outstanding generalizability, but fail to preserve structural fidelity. To address these challenges, we propose an enhanced diffusion posterior sampling (E-DPS) framework that combines a one-step denoiser U-Net with an unconditional diffusion model. Specifically, the U-Net estimator, trained on simulated LDCT-NDCT pairs, provides preliminary denoised outputs as structural constraints, whereas the diffusion model captures the prior distribution of NDCT images to enhance realism and generalizability. During inference, the U-Net predictions are integrated as constraints with tunable weights, thereby guiding diffusion posterior sampling. In addition, an intermediate-stage initialization strategy is introduced, significantly reducing the number of required sampling steps. Extensive experiments on simulated LDCT datasets across three dose levels demonstrate the superiority of our method, yielding average PSNR gains of +5.2% and +4.3% at unseen dose levels compared with state-of-the-art approaches. Moreover, on real LDCT images, E-DPS exhibits strong zero-shot generalizability, achieving better noise suppression while preserving anatomical detail. These results highlight the robustness and clinical potential of E-DPS for LDCT denoising.

Positron Emission Tomography (PET) images reflect the metabolic rate of tracers in different tissues of the human body, crucial for early cancer diagnosis and treatment. Accurate tumor segmentation is essential to aid clinicians in determining drug dosages. Due to the low resolution of PET images, prior information (such as CT, MRI or distance information) are often incorporated to assist PET segmentation. In this paper, we propose an automatic 3D PET tumor segmentation framework assisted by geodesic sequences. Specifically, considering the intrinsic characteristics of PET images, we first construct geodesic prior, which effectively enhances the contrast between the tumor and background while suppressing noise and the influence of other tissues. To address the need for seed points in the geodesic prior, an automatic marking strategy is designed that identifies all suspected lesion regions and uses their central points as a series of seeds to generate the corresponding geodesic sequences. Subsequently, we develop a three-branch network architecture to simultaneously process PET images, geodesic sequences, and background geodesic information. To enhance image features, a distance attention mechanism is introduced at the end of the network encoder to effectively measure the similarity between different geodesic features, refining the image features. Finally, the network incorporates spatial regularization and local PET intensity information into the activation function via the Soft Threshold Dynamics with Local Intensity Fitting (STDLIF) module, further improving segmentation accuracy. Experimental results demonstrate that, compared to existing state-of-the-art algorithms, the proposed method shows better segmentation performance on both clinical and public datasets.

Magnetic resonance diffusion tensor imaging (DTI) is a unique non-invasive technique for measuring in vivo water molecule diffusion, reflecting tissue microstructure. However, acquiring high-quality DTI typically requires numerous diffusion-weighted images (DWIs) in multiple directions, resulting in long scan times that restrict its use in clinical and research settings. To address this limitation, we propose Diff-DTI, a fast DTI processing framework based on a feature-enhanced joint diffusion model, to reduce the number of DWIs needed for tensor fitting. Diff-DTI models the joint probability distribution of DWIs and DTI maps, supporting guided generation during inference. The incorporated feature enhancement fusion module further enhances image precision and details generated by the diffusion model. Experiments were performed on three public DWI datasets. Results demonstrate that Diff-DTI achieves up to 10-fold acceleration (using 6 DWIs) while maintaining relatively low normalized mean square error (NMSE) for DTI maps (2.89% for FA, 0.89% for MD, 0.95% for AD, and 0.98% for RD). Even using Diff-DTI with only 3 DWIs, the NMSEs of the generated DTI maps showed a gradual decrease, with 3.51% for FA, 0.89% for MD, 1.13% for AD, and 1.10% for RD. We conclude that Diff-DTI can significantly reduce the number of acquired DWIs and the scan time, without compromising image quality too much.

The T1ρ mapping technique necessitates acquiring multiple T1ρ-weighted images at various spin-lock times (TSL), which results in a lengthy scan time and significantly limits its widespread clinical use. Undersampling is a significant strategy to accelerate T1ρ imaging, where it is crucial to model and utilize the joint spatiotemporal correlations priors among different TSL multi-contrast images for high-quality reconstruction. However, current methods that use simplified physical relaxation correlations or non-interpretable deep neural networks to define joint correlations often yield inaccurate results. From a Bayesian framework, the joint distribution provides a powerful capability to represent the joint correlations among multi-contrast T1ρ images. Therefore, a new method is introduced to accelerate T1ρ parameter imaging, leveraging accurate joint distribution modeling and the captured joint distribution to guide the reconstruction. Specifically, a joint diffusion model is proposed to approximate the joint distribution of multi-contrast T1ρ images exploiting the score-matching method. Subsequently, the ill-posed problem caused by the reconstruction of undersampled multi-contrast T1ρ images is addressed through the learned joint distribution by employing a constructed joint reverse denoising diffusion model. The superior performance of the proposed method and the capability to accurately characterize the joint distribution was further verified by performing various in vivo experiments. The patient image reconstruction also verifies the feasibility and superiority of the proposed method.

Motion estimation of left ventricle myocardium on the cardiac image sequence is crucial for assessing cardiac function. However, the intensity variation of cardiac image sequences brings the challenge of uncertain interference to myocardial motion estimation. Such imaging-related uncertain interference appears in different cardiac imaging modalities. We propose adaptive sequential Bayesian iterative learning to overcome the challenge. Specifically, our method applies the adaptive structural inference to state transition and observation to cope with a complex myocardial motion under uncertain setting. In state transition, adaptive structural inference establishes a hierarchical structure recurrence to obtain the complex latent representation of cardiac image sequences. In state observation, the adaptive structural inference forms a chain structure mapping to correlate the latent representation of the cardiac image sequence with that of the motion. Extensive experiments on US, CMR, and TMR datasets concerning 1270 patients (650 patients for CMR, 500 patients for US and 120 patients for TMR) have shown the effectiveness of our method, as well as the superiority to eight state-of-the-art motion estimation methods.

Dynamic medical imaging sequences, such as cardiac cine MRI and dynamic CT, are vital for capturing organ motion and enabling functional assessment. However, these modalities face an inherent spatiotemporal trade-off: achieving high spatial resolution often necessitates prolonged acquisition times, which compromises temporal resolution and obscures critical motion details. To address this challenge, we introduce DIFA, a novel framework for arbitrary-time frame interpolation in dynamic medical imaging sequences. DIFA integrates two synergistic components: (1) A Bilateral Flow Estimation (BFE) module incorporates a deformable inter-frame attention mechanism, which explicitly models non-rigid pixel trajectories under the guidance of gradient priors, thereby ensuring the accurate capture of large organ motions. (2) A Bilateral Flow Fusion (BFF) module employs a multi-scale pyramid architecture with cross-attention warping to achieve precise integration of motionwarped features across temporal scales. This framework enables the generation of anatomically coherent intermediate frames at any user-specified time point. Extensive experiments on cardiac cine MRI and dynamic lung CT datasets demonstrate that DIFA surpasses state-of-the-art frame interpolation methods in both quantitative metrics (e.g., PSNR and LPIPS) and qualitative visual assessments, offering a robust solution for enhancing temporal resolution in dynamic medical imaging.

Deep unrolling methods have achieved notable success in accelerated cardiac cine MRI reconstruction. However, their effectiveness remains limited by constrained receptive fields and rigid convolutional sampling, which hinder scalability in high-resolution reconstruction tasks. To address these challenges, we propose an Equivariant Deformable Convolutional Unrolling Network (EDCU-Net) that integrates a Spatiotemporal Deformable Module (STDM) and a Rotation Equivariant Module (REM). EDCU-Net effectively enlarges the receptive field while maintaining low computational cost and high parameter efficiency, enabling more effective suppression of large-scale aliasing artifacts. Specifically, STDM adaptively adjusts sampling locations to better capture complex anatomical structures and dynamic cardiac motion, while reducing interpolation overhead. Furthermore, REM embeds deformable convolutions within an equivariant framework, allowing shared parameters across orientations and further promoting parameter efficiency and generalization capability. Extensive experiments on cardiac cine MRI data demonstrate that EDCU-Net consistently outperforms state-of-the-art methods in both reconstruction accuracy and visual quality.

In image generation, Schrödinger Bridge (SB)-based methods theoretically enhance the efficiency and quality compared to the diffusion models by finding the least costly path between two distributions. However, they are computationally expensive and time-consuming when applied to complex image data. The reason is that they focus on fitting globally optimal paths in high-dimensional spaces, directly generating images as next step on the path using complex networks through self-supervised training, which typically results in a gap with the global optimum. Meanwhile, most diffusion models are in the same path subspace generated by weights fA(t) and fB(t), as they follow the paradigm (xt= fA(t)xImg+ fB(t)ϵ). To address the limitations of SB-based methods, this paper proposes for the first time to find local Diffusion Schrödinger Bridges (LDSB) in the diffusion path subspace, which strengthens the connection between the SB problem and diffusion models. Specifically, our method optimizes the diffusion paths using Kolmogorov-Arnold Network (KAN), which has the advantage of resistance to forgetting and continuous output. The experiment shows that our LDSB significantly improves the quality and efficiency of image generation using the same pretrained denoising network and the KAN for optimising is only less than 0.1MB. The FID metric is reduced by more than 15%, especially with a reduction of 48.50% when NFE of DDIM is 5 for the CelebA dataset. Code is available at https://github.com/PerceptionComputingLab/LDSB.

Reconstructing visual information from brain activity via computer vision technology provides an intuitive understanding of visual neural mechanisms. Despite progress in decoding fMRI data with generative models, achieving accurate cross‑subject reconstruction of visual stimuli remains challenging and computationally demanding. This difficulty arises from inter‑subject variability in neural representations and the brain’s abstract encoding of core semantic features in complex visual inputs.

Pretrained vision-language models (VLMs), such as CLIP, have shown promising zero-shot out-of-distribution (OOD) detection capabilities by leveraging semantic similarities between input images and textual labels. However, most existing approaches focus solely on expanding the label space in the text domain, ignoring complementary visual cues that can further enhance discriminative power. In this paper, we introduce DualCnst, a novel framework that integrates text-image dual consistency for improved zero-shot OOD detection. Specifically, we generate synthetic images from both ID and mined OOD textual labels using a text-to-image generative model, and jointly evaluate each test image based on (i) its semantic similarity to class labels and (ii) its visual similarity to the synthesized images. The resulting unified score function effectively combines multimodal information without requiring access to in-distribution images or additional training. We further provide theoretical analysis showing that incorporating multimodal negative labels reduces score variance and improves OOD separability. Extensive experiments across diverse OOD benchmarks demonstrate that DualCnst achieves state-of-the-art performance while remaining scalable, data-agnostic, and fully compatible with prior text-only VLM-based methods.

Bias fields inevitably degrade MRI that seriously interferes the diagnosis of physicians for accurate analysis, and removing it is a crucial image analysis task. Generative models (such as GANs) are used for bias field correction, and outperform traditional methods, however are hindered by the high cost of data annotation and instability during training. Recently, the diffusion-based methods have excelled over GANs in many applications, and they are powerful in removing noise from images, while the bias field can be regarded as a smooth noise. However, it is a challenge to directly apply to 3D bias field correction due to sampling inefficiency, the heavy computational demand, and implicit correction process. We propose a Memory-Efficient Multivariate Gaussian Bias Diffusion Model (MeMGB-Diff) that is an explicit, sampling, and memory both efficient diffusion model for 3D bias field correction without using clinical labels. MeMGB-Diff extends the diffusion models to multivariate Gaussian and models the bias field as a multivariate Gaussian variable, allowing direct diffusion and removal of the 3D bias fields without Gaussian noise. For memory efficiency, MeMGB-Diff performs diffusion model in smaller readable image domain at the expense of a negligible accuracy loss, based on the strong correlation among adjacent voxels of bias field. We also propose a loss function to mainly learn the intensity trend, which mainly causes the inhomogeneity of MRI, and effectively increases the correction accuracy. For comprehensive performance comparison, we propose a synthetic method for generating more varied bias fields during testing. Both quantitative and qualitative assessments on synthetic and clinical data confirm the high fidelity and uniform intensity of our results. MeMGB-Diff reduces data size by 64 times to use less memory, improves sampling efficiency by more than 10 times compared to other diffusion-based methods, and achieves optimal metrics, including SSIM, PSNR, COCO, and CV for various tissues. Hence, our MeMGB-Diff is a state-of-the-art (SOTA) method for 3D bias field correction.

The field of accelerated magnetic resonance imaging (AMRI) has garnered significant attention, focusing on reconstructing target image from compressively sampled k-space measurements to address an ill-posed linear inverse problem. In this study, we exploit the multiparameterization of MRI to propose a new plug-and-play prior (P3) for enhancing reconstruction quality. We begin by introducing a mutual-structure guided P3 (MS-GP3) framework, based on jointly penalized least-squares regression (JPLSR), to selectively transfer common priors from a reference image to the target one, thereby minimizing errors caused by indiscriminate pattern transfer. Furthermore, we establish a self-sharpening weighting (SSW) scheme that effectively differentiates between sharp and smooth image components, contributing to a boosted variant of MS-GP3 (BMS-GP3) for further improvement in artifact suppression and detail restoration. Finally, embedding BMS-GP3 in half-quadratic splitting (HQS) iterations yields an advanced AMRI algorithm, dubbed BMS-GP3 -HQS, which not only outperforms state-ofthe-art (SOTA) methods but also provides robust theoretical guarantees, including ensured convergence and resilience to noise.

Fast PET imaging is clinically important for reducing motion artifacts and improving patient comfort. While recent diffusion-based deep learning methods have shown promise, they often fail to capture the true PET degradation process, suffer from accumulated inference errors, introduce artifacts, and require extensive reconstruction iterations. To address these challenges, we propose a novel multistage diffusion framework tailored for fast PET imaging. At the coarse level, we design a multistage structure to approximate the temporal non-linear PET degradation process in a data-driven manner, using paired PET images collected under different acquisition duration. A Phase Error Correction Network (PECNet) ensures consistency across stages by correcting accumulated deviations. At the fine level, we introduce a deterministic cold diffusion mechanism, which simulates intra-stage degradation through interpolation between known acquisition durations—significantly reducing reconstruction iterations to as few as 10. Evaluations on [68Ga]FAPI and [18F]FDG PET datasets demonstrate the superiority of our approach, achieving peak PSNRs of 36.2 dB and 39.0 dB, respectively, with average SSIMs over 0.97. Our framework offers high-fidelity PET imaging with fewer iterations, making it practical for accelerated clinical imaging.

Physiological and external motion cause inter-frame misalignment in chemical exchange saturation transfer magnetic resonance imaging (CEST-MRI), thereby compromising quantitative accuracy. In CEST-MRI, saturation effects induce intensity variations, resulting in motion-intensity coupling that makes registration particularly challenging. To address this issue, we extend the finite element digital image correlation (FE-DIC) framework by introducing an alternating correction strategy that iteratively refines both motion and intensity estimation. Unlike conventional FE-DIC approaches that assume intensity constancy, the proposed method incorporates mechanical regularization to suppress non-physical deformations, alongside intensity correction to compensate for reference-target contrast discrepancies. This mutual reinforcement enables progressively improved registration across the CEST sequence. The robustness and effectiveness of the method were evaluated on three datasets. In simulated liver data, it maintains RMSE within 0.4 pixels, reducing error by 0.5 pixels compared to RPCA & PCA (a PCA-based synthetic reference generation method for CEST registration). On clinical brain and pig cardiac data, it achieves average SSIM of 0.83, outperforming RPCA & PCA by 0.03 and surpassing CNN-based registration (e.g., AirLab) by 0.10. The consistent results across datasets highlight its generalizability, making it a promising tool for metabolic quantification in clinical and research settings.

The precise segmentation of different brain regions and tissues is usually a prerequisite for the detection and diagnosis of various neurological disorders in neuroscience. Considering the abundance of functional and structural dual-modality information for positron emission tomography/magnetic resonance (PET/MR) images, we propose a novel 3D whole-brain segmentation network with a cross-fusion mechanism introduced to obtain 45 brain regions. Specifically, the network processes PET and MR images simultaneously, employing UX-Net and a cross-fusion block for feature extraction and fusion in the encoder. We test our method by comparing it with other deep learning-based methods, including 3DUXNET, SwinUNETR, UNETR, nnFormer, UNet3D, NestedUNet, ResUNet, and VNet. The experimental results demonstrate that the proposed method achieves better segmentation performance in terms of both visual and quantitative evaluation metrics and achieves more precise segmentation in three views while preserving fine details. In particular, the proposed method achieves superior quantitative results, with a Dice coefficient of 85.73% 0.01%, a Jaccard index of 76.68% 0.02%, a sensitivity of 85.00% 0.01%, a precision of 83.26% 0.03% and a Hausdorff distance (HD) of 4.4885 14.85%. Moreover, the distribution and correlation of the SUV in the volume of interest (VOI) are also evaluated (PCC > 0.9), indicating consistency with the ground truth and the superiority of the proposed method. In future work, we will utilize our whole-brain segmentation method in clinical practice to assist doctors in accurately diagnosing and treating brain diseases.

Supervised deep learning (SDL) methodology holds promise for accelerated magnetic resonance imaging (AMRI) but is hampered by the reliance on extensive training data. Some self-supervised frameworks, such as deep image prior (DIP), have emerged, eliminating the explicit training procedure but often struggling to remove noise and artifacts under significant degradation. This work introduces a novel self-supervised accelerated parallel MRI approach called PEARL, leveraging a multiple-stream joint deep decoder with two cross-fusion schemes to accurately reconstruct one or more target images from compressively sampled k-space. Each stream comprises cascaded cross-fusion sub-block networks (SBNs) that sequentially perform combined upsampling, 2D convolution, joint attention, ReLU activation and batch normalization (BN). Among them, combined upsampling and joint attention facilitate mutual learning between multiple-stream networks by integrating multi-parameter priors in both additive and multiplicative manners. Long-range unified skip connections within SBNs ensure effective information propagation between distant cross-fusion layers. Additionally, incorporating dual-normalized edge-orientation similarity regularization into the training loss enhances detail reconstruction and prevents overfitting. Experimental results consistently demonstrate that PEARL outperforms the existing state-of-the-art (SOTA) self-supervised AMRI technologies in various MRI cases. Notably, 5-fold$\sim$6-fold accelerated acquisition yields a 1$\%$ $\sim$ 2$\%$ improvement in SSIM$_{\mathsf{ROI}}$ and a 3$\%$ $\sim$ 6$\%$ improvement in PSNR$_{\mathsf{ROI}}$, along with a significant 15$\%$ $\sim$ 20$\%$ reduction in RLNE$_{\mathsf{ROI}}$.

Diffusion models have emerged as a leading methodology for image generation and have proven successful in the realm of magnetic resonance imaging (MRI) reconstruction. However, existing reconstruction methods based on diffusion models are primarily formulated in the image domain, making the reconstruction quality susceptible to inaccuracies in coil sensitivity maps (CSMs). k-space interpolation methods can effectively address this issue but conventional diffusion models are not readily applicable in k-space interpolation. To overcome this challenge, we introduce a novel approach called SPIRiT-Diffusion, which is a diffusion model for k-space interpolation inspired by the iterative self-consistent SPIRiT method. Specifically, we utilize the iterative solver of the self-consistent term (i.e., k-space physical prior) in SPIRiT to formulate a novel stochastic differential equation (SDE) governing the diffusion process. Subsequently, k-space data can be interpolated by executing the diffusion process. This innovative approach highlights the optimization model's role in designing the SDE in diffusion models, enabling the diffusion process to align closely with the physics inherent in the optimization model-a concept referred to as model-driven diffusion. We evaluated the proposed SPIRiT-Diffusion method using a 3D joint intracranial and carotid vessel wall imaging dataset. The results convincingly demonstrate its superiority over image-domain reconstruction methods, achieving high reconstruction quality even at a substantial acceleration rate of 10. Our code are available at https://github.com/zhyjSIAT/SPIRiT-Diffusion.

The limited availability of labeled data has driven advancements in semi-supervised learning for medical image segmentation. Modern large-scale models tailored for general segmentation, such as the Segment Anything Model (SAM), have revealed robust generalization capabilities. However, applying these models directly to medical image segmentation still exposes performance degradation. In this paper, we propose a learnable prompting SAM-induced Knowledge distillation framework (KnowSAM) for semi-supervised medical image segmentation. Firstly, we propose a Multi-view Co-training (MC) strategy that employs two distinct sub-networks to employ a co-teaching paradigm, resulting in more robust outcomes. Secondly, we present a Learnable Prompt Strategy (LPS) to dynamically produce dense prompts and integrate an adapter to fine-tune SAM specifically for medical image segmentation tasks. Moreover, we propose SAM-induced Knowledge Distillation (SKD) to transfer useful knowledge from SAM to two sub-networks, enabling them to learn from SAM's predictions and alleviate the effects of incorrect pseudo-labels during training. Notably, the predictions generated by our subnets are used to produce mask prompts for SAM, facilitating effective inter-module information exchange. Extensive experimental results on various medical segmentation tasks demonstrate that our model outperforms the state-of-the-art semi-supervised segmentation approaches. Crucially, our SAM distillation framework can be seamlessly integrated into other semi-supervised segmentation methods to enhance performance. The code will be released upon acceptance of this manuscript at https://github.com/taozh2017/KnowSAM.

Long scan time significantly hinders the widespread applications of three-dimensional multi-contrast cardiac magnetic resonance (3D-MC-CMR) imaging. This study aims to accelerate 3D-MC-CMR acquisition by a novel method based on score-based diffusion models with self-supervised learning. Specifically, we first establish a mapping between the undersampled k-space measurements and the MR images, utilizing a self-supervised Bayesian reconstruction network. Secondly, we develop a joint score-based diffusion model on 3D-MC-CMR images to capture their inherent distribution. The 3D-MC-CMR images are finally reconstructed using the conditioned Langenvin Markov chain Monte Carlo sampling. This approach enables accurate reconstruction without fully sampled training data. Its performance was tested on the dataset acquired by a 3D joint myocardial $ \text {T}_{{1}}$ and $ \text {T}_{{1}\rho }$ mapping sequence. The $ \text {T}_{{1}}$ and $ \text {T}_{{1}\rho }$ maps were estimated via a dictionary matching method from the reconstructed images. Experimental results show that the proposed method outperforms traditional compressed sensing and existing self-supervised deep learning MRI reconstruction methods. It also achieves high quality $ \text {T}_{{1}}$ and $ \text {T}_{{1}\rho }$ parametric maps close to the reference maps, even at a high acceleration rate of 14.

Low-count Positron Emission Tomography reconstruction is critical for maintaining high imaging quality while minimizing tracer doses and radiation exposure. Although integrating structural information from CT and MR data has been shown to enhance PET reconstruction, this typically requires simultaneous PET and CT/MRI scans, complicating workflows and increasing radiation exposure. Recent advancements in foundation models offer a promising alternative to in-person CT/MRI imaging, potentially overcoming these limitations. However, the use of foundation models' segmentation masks as semantic guides has been observed to introduce erroneous structures in low-count PET reconstructions. To address this challenge, this work introduces an innovative prompting agent-based framework that dynamically interacts with the foundation model to retrieve and refine priors, minimizing undue influence on the reconstruction process. Specifically, a box agent is designed for single-instance local area information retrieval, while a point agent is introduced to progressively prompt broader semantic structures globally, utilizing history point prompts. Additionally, an MDP paradigm has been developed to address the challenges of utilizing historical point prompts while maintaining the independence required by MDPs. Evaluated on both simulated and real datasets, the proposed method demonstrates superior qualitative and quantitative performance compared to state-of-the-art methods, even those leveraging in-person CT/MRI priors.

Positron emission tomography/magnetic resonance imaging (PET/MRI) systems can provide precise anatomical and functional information with exceptional sensitivity and accuracy for neurological disorder detection. Nevertheless, the radiation exposure risks and economic costs of radiopharmaceuticals may pose significant burdens on patients. To mitigate image quality degradation during low-dose PET imaging, we proposed a novel 3D network equipped with a spatial brain transform (SBF) module for low-dose whole-brain PET and MR images to synthesize high-quality PET images. The FreeSurfer toolkit was applied to derive the spatial brain anatomical alignment information, which was then fused with low-dose PET and MR features through the SBF module. Moreover, several deep learning methods were employed as comparison measures to evaluate the model performance, with the peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and Pearson correlation coefficient (PCC) serving as quantitative metrics. Both the visual results and quantitative results illustrated the effectiveness of our approach. The obtained PSNR and SSIM were 41.96 ± 4.91 dB (p < 0.01) and 0.9654 ± 0.0215 (p < 0.01), which achieved a 19% and 20% improvement, respectively, compared to the original low-dose brain PET images. The volume of interest (VOI) analysis of brain regions such as the left thalamus (PCC = 0.959) also showed that the proposed method could achieve a more accurate standardized uptake value (SUV) distribution while preserving the details of brain structures. In future works, we hope to apply our method to other multimodal systems, such as PET/CT, to assist clinical brain disease diagnosis and treatment.

Accuratedetection and segmentation of brain tumors is critical for medical diagnosis. However, current supervised learning methods require extensively annotated images and the state-of-the-art generative models used in unsupervised methods often have limitations in covering the whole data distribution. In this paper, we propose a novel framework Two-Stage Generative Model (TSGM) that combines Cycle Generative Adversarial Network (CycleGAN) and Variance Exploding stochastic differential equation using joint probability (VE-JP) to improve brain tumor detection and segmentation. The CycleGAN is trained on unpaired data to generate abnormal images from healthy images as data prior. Then VE-JP is implemented to reconstruct healthy images using synthetic paired abnormal images as a guide, which alters only pathological regions but not regions of healthy. Notably, our method directly learned the joint probability distribution for conditional generation. The residual between input and reconstructed images suggests the abnormalities and a thresholding method is subsequently applied to obtain segmentation results. Furthermore, the multimodal results are weighted with different weights to improve the segmentation accuracy further. We validated our method on three datasets, and compared with other unsupervised methods for anomaly detection and segmentation. The DSC score of 0.8590 in BraTs2020 dataset, 0.6226 in ITCS dataset and 0.7403 in In-house dataset show that our method achieves better segmentation performance and has better generalization.

Nasopharyngeal carcinoma (NPC) is a malignant tumor primarily treated by radiotherapy. Accurate delineation of the target tumor is essential for improving the effectiveness of radiotherapy. However, the segmentation performance of current models is unsatisfactory due to poor boundaries, large-scale tumor volume variation, and the labor-intensive nature of manual delineation for radiotherapy. In this paper, MMCA-Net, a novel segmentation network for NPC using PET/CT images that incorporates an innovative multimodal cross attention transformer (MCA-Transformer) and a modified U-Net architecture, is introduced to enhance modal fusion by leveraging cross-attention mechanisms between CT and PET data. Our method, tested against ten algorithms via fivefold cross-validation on samples from Sun Yat-sen University Cancer Center and the public HECKTOR dataset, consistently topped all four evaluation metrics with average Dice similarity coefficients of 0.815 and 0.7944, respectively. Furthermore, ablation experiments were conducted to demonstrate the superiority of our method over multiple baseline and variant techniques. The proposed method has promising potential for application in other tasks.

Diffusion models with continuous stochastic differential equations (SDEs) have shown superior performances in image generation. It can serve as a deep generative prior to solving the inverse problem in magnetic resonance (MR) reconstruction. However, low-frequency regions of k -space data are typically fully sampled in fast MR imaging, while existing diffusion models are performed throughout the entire image or k -space, inevitably introducing uncertainty in the reconstruction of low-frequency regions. Additionally, existing diffusion models often demand substantial iterations to converge, resulting in time-consuming reconstructions. To address these challenges, we propose a novel SDE tailored specifically for MR reconstruction with the diffusion process in high-frequency space (referred to as HFS-SDE). This approach ensures determinism in the fully sampled low-frequency regions and accelerates the sampling procedure of reverse diffusion. Experiments conducted on the publicly available fastMRI dataset demonstrate that the proposed HFS-SDE method outperforms traditional parallel imaging methods, supervised deep learning, and existing diffusion models in terms of reconstruction accuracy and stability. The fast convergence properties are also confirmed through theoretical and experimental validation. Our code and weights are available at https://github.com/Aboriginer/HFS-SDE.

Recently, diffusion models have shown considerable promise for MRI reconstruction. However, extensive experimentation has revealed that these models are prone to generating artifacts due to the inherent randomness involved in generating images from pure noise. To achieve more controlled image reconstruction, we reexamine the concept of interpolatable physical priors in k-space data, focusing specifically on the interpolation of high-frequency (HF) k-space data from low-frequency (LF) k-space data. Broadly, this insight drives a shift in the generation paradigm from random noise to a more deterministic approach grounded in the existing LF k-space data. Building on this, we first establish a relationship between the interpolation of HF k-space data from LF k-space data and the reverse heat diffusion process, providing a fundamental framework for designing diffusion models that generate missing HF data. To further improve reconstruction accuracy, we integrate a traditional physics-informed k-space interpolation model into our diffusion framework as a data fidelity term. Experimental validation using publicly available datasets demonstrates that our approach significantly surpasses traditional k-space interpolation methods, deep learning-based k-space interpolation techniques, and conventional diffusion models, particularly in HF regions. Finally, we assess the generalization performance of our model across various out-of-distribution datasets. Our code are available at https://github.com/ZhuoxuCui/Heat-Diffusion.

The short frames of low-count positron emission tomography (PET) images generally cause high levels of statistical noise. Thus, improving the quality of low-count images by using image postprocessing algorithms to achieve better clinical diagnoses has attracted widespread attention in the medical imaging community. Most existing deep learning-based low-count PET image enhancement methods have achieved satisfying results, however, few of them focus on denoising low-count PET images with the magnetic resonance (MR) image modality as guidance. The prior context features contained in MR images can provide abundant and complementary information for single low-count PET image denoising, especially in ultralow-count (2.5%) cases. To this end, we propose a novel two-stream dual PET/MR cross-modal interactive fusion network with an optical flow pre-alignment module, namely, OIF-Net. Specifically, the learnable optical flow registration module enables the spatial manipulation of MR imaging inputs within the network without any extra training supervision. Registered MR images fundamentally solve the problem of feature misalignment in the multimodal fusion stage, which greatly benefits the subsequent denoising process. In addition, we design a spatial-channel feature enhancement module (SC-FEM) that considers the interactive impacts of multiple modalities and provides additional information flexibility in both the spatial and channel dimensions. Furthermore, instead of simply concatenating two extracted features from these two modalities as an intermediate fusion method, the proposed cross-modal feature fusion module (CM-FFM) adopts cross-attention at multiple feature levels and greatly improves the two modalities' feature fusion procedure. Extensive experimental assessments conducted on real clinical datasets, as well as an independent clinical testing dataset, demonstrate that the proposed OIF-Net outperforms the state-of-the-art methods.

Deep learning methods have been successfully used in various computer vision tasks. Inspired by that success, deep learning has been explored in magnetic resonance imaging (MRI) reconstruction. In particular, integrating deep learning and model-based optimization methods has shown considerable advantages. However, a large amount of labeled training data is typically needed for high reconstruction quality, which is challenging for some MRI applications. In this paper, we propose a novel reconstruction method, named DURED-Net, that enables interpretable self-supervised learning for MR image reconstruction by combining a self-supervised denoising network and a plug-and-play method. We aim to boost the reconstruction performance of Noise2Noise in MR reconstruction by adding an explicit prior that utilizes imaging physics. Specifically, the leverage of a denoising network for MRI reconstruction is achieved using Regularization by Denoising (RED). Experiment results demonstrate that the proposed method requires a reduced amount of training data to achieve high reconstruction quality among the state-of-the-art approaches utilizing Noise2Noise.

Typical tomographic image reconstruction methods require that the imaged object is static and stationary during the time window to acquire a minimally complete data set. The violation of this requirement leads to temporal-averaging errors in the reconstructed images. For a fixed gantry rotation speed, to reduce the errors, it is desired to reconstruct images using data acquired over a narrower angular range, i.e., with a higher temporal resolution. However, image reconstruction with a narrower angular range violates the data sufficiency condition, resulting in severe data-insufficiency-induced errors. The purpose of this work is to decouple the trade-off between these two types of errors in contrast-enhanced computed tomography (CT) imaging. We demonstrated that using the developed data consistency constrained deep temporal extrapolation method (AIRPORT), the entire time-varying imaged object can be accurately reconstructed with 40 frames-per-second temporal resolution, the time window needed to acquire a single projection view data using a typical C-arm cone-beam CT system. AIRPORT is applicable to general non-sparse imaging tasks using a single short-scan data acquisition.

The prevailing understanding in the field of machine learning and deep learning (ML/DL) is that, given a highquality dataset, one can effectively learn data-related priors through supervised learning. However, in medical imaging, this assumption faces two critical challenges: 1) high-quality training data are often scarce and 2) data are highly heterogeneous, stemming from different imaging scanners, protocols, or populations at various institutions. This diversity makes it impractical to represent the data with a single, universal prior using traditional methods, leading to limited generalizability in medical imaging tasks.

In flat-panel detector (FPD) based cone-beam computed tomography (CBCT) imaging, the native receptor array is usually binned into a smaller matrix size. By doing so, the signal readout speed could be increased by 4-9 times at the expense of a spatial resolution loss of 50%-67%. Clearly, such manipulation poses a key bottleneck in generating high spatial and high temporal resolution CBCT images at the same time. In addition, the conventional FPD is also difficult in generating dual-energy CBCT images. In this paper, we propose an innovative super resolution dual-energy CBCT imaging method, named as suRi, based on dual-layer FPD (DL-FPD) to overcome these aforementioned difficulties at once. With suRi, specifically, a 1D or 2D sub-pixel (half pixel in this study) shifted binning is applied instead of the conventionally aligned binning to double the spatial sampling rate during the dual-energy data acquisition. As a result, the suRi approach provides a new strategy to enable high spatial resolution CBCT imaging while at high readout speed. Moreover, a penalized likelihood material decomposition algorithm is developed to directly reconstruct the high resolution bases from these dual-energy CBCT projections containing sub-pixel shifts. Numerical and physical experiments are performed to validate this newly developed suRi method with phantoms and biological specimen. Results demonstrate that suRi can significantly improve the spatial resolution of the CBCT image. We believe this developed suRi method would greatly enhance the imaging performance of the DL-FPD based dual-energy CBCT systems in future.

Full quantification of brain PET requires the blood input function (IF), which is traditionally achieved through an invasive and time-consuming arterial catheter procedure, making it unfeasible for clinical routine. This study presents a deep learning based method to estimate the input function (DLIF) for a dynamic brain FDG scan. A long short-term memory combined with a fully connected network was used. The dataset for training was generated from 85 total-body dynamic scans obtained on a uEXPLORER scanner. Time-activity curves from 8 brain regions and the carotid served as the input of the model, and labelled IF was generated from the ascending aorta defined on CT image. We emphasize the goodness-of-fitting of kinetic modeling as an additional physical loss to reduce the bias and the need for large training samples. DLIF was evaluated together with existing methods in terms of RMSE, area under the curve, regional and parametric image quantifications. The results revealed that the proposed model can generate IFs that closer to the reference ones in terms of shape and amplitude compared with the IFs generated using existing methods. All regional kinetic parameters calculated using DLIF agreed with reference values, with the correlation coefficient being 0.961 (0.913) and relative bias being 1.68±8.74% (0.37±4.93%) for [Formula: see text] ( [Formula: see text]. In terms of the visual appearance and quantification, parametric images were also highly identical to the reference images. In conclusion, our experiments indicate that a trained model can infer an image-derived IF from dynamic brain PET data, which enables subsequent reliable kinetic modeling.

Low-count positron emission tomography (PET) imaging is challenging because of the ill-posedness of this inverse problem. Previous studies have demonstrated that deep learning (DL) holds promise for achieving improved low-count PET image quality. However, almost all data-driven DL methods suffer from fine structure degradation and blurring effects after denoising. Incorporating DL into the traditional iterative optimization model can effectively improve its image quality and recover fine structures, but little research has considered the full relaxation of the model, resulting in the performance of this hybrid model not being sufficiently exploited. In this paper, we propose a learning framework that deeply integrates DL and an alternating direction of multipliers method (ADMM)-based iterative optimization model. The innovative feature of this method is that we break the inherent forms of the fidelity operators and use neural networks to process them. The regularization term is deeply generalized. The proposed method is evaluated on simulated data and real data. Both the qualitative and quantitative results show that our proposed neural network method can outperform partial operator expansion-based neural network methods, neural network denoising methods and traditional methods.

In this paper, a dynamic dual-graph fusion convolutional network is proposed to improve Alzheimer’s disease (AD) diagnosis performance. The following are the paper’s main contributions: (a) propose a novel dynamic Graph Convolutional Network (GCN) architecture, which is an end-to-end pipeline for diagnosis of the AD task; (b) the proposed architecture can dynamically adjust the graph structure for GCN to produce better diagnosis outcomes by learning the optimal underlying latent graph; (c) incorporate feature graph learning and dynamic graph learning, giving those useful features of subjects more weight while decreasing the weights of other noise features. Experiments indicate that our model provides flexibility and stability while achieving excellent classification results in AD diagnosis.

Computed tomography (CT) is a widely-used imaging technology that assists clinical decision-making with high-quality human body representations. To reduce the radiation dose posed by CT, sparse-view (SV) CT is developed with preserved image quality. However, these methods are still stuck with a fixed uniform SV (USV) sampling strategy, which inhibits the possibility of acquiring a better image with an even reduced dose. In this paper, we explore this possibility via learning an active SV (ASV) sampling policy that optimizes the sampling positions for regions of interest (RoI)-specific, high-quality reconstruction. To this end, we design an sampling agent for the recommendation of ASV sampling positions based on on-the-fly reconstruction with obtained sinograms in a progressive fashion. With such a design, we achieve better performances on the NIH-AAPM dataset over popular USV sampling, especially when the number of views is small. Finally, such a design enables the RoI-aware reconstruction with improved local quality within the RoI that are clinically important. Experiments on the VerSe dataset demonstrate the ability of the proposed sampling policy, which is difficult to achieve with USV sampling.

Abstract Magnetic resonance (MR) parametric imaging can help the assessment of some certain diseases with its various contrast mechanisms. However, the main issue of MR parametric imaging is the long acquisition time, introducing many problems such as uncomfortable experiences for patients and motion artefacts. With the deep learning methods developing, some unrolling ones have been introduced to MR imaging as solutions. The purpose of this study is to improve both the quality and the speed of parametric imaging. The proposed method, named as LSA‐Net, introduced a learned low rank plus sparsity model with attention mechanism to reconstruct highly under‐sampled data for MR parametric imaging. The L + S model was used to represent the shared image structure among the parameter‐weighted images as low‐rank part and the difference among images and the ideal model as sparse part. Besides, the attention block was introduced to effectively improved quality for the region of interest, which was most concerned in clinical applications. Then the parameter map was generated from the reconstructed parameter‐weighted images by its exponential model. The LSA‐Net was evaluated on the in‐vivo 3D mapping, showing better performance on both image quality and time consumed than the comparing methods including LLR, SCOPE, L + S , and L + S ‐Net.

The axial field of view (FOV) is a key factor that affects the quality of PET images. Due to hardware FOV restrictions, conventional short-axis PET scanners with FOVs of 20 to 35 cm can acquire only low-quality PET (LQ-PET) images in fast scanning times (2-3 minutes). To overcome hardware restrictions and improve PET image quality for better clinical diagnoses, several deep learning-based algorithms have been proposed. However, these approaches use simple convolution layers with residual learning and local attention, which insufficiently extract and fuse long-range contextual information. To this end, we propose a novel two-branch network architecture with swin transformer units and graph convolution operation, namely SW-GCN. The proposed SW-GCN provides additional spatial- and channel-wise flexibility to handle different types of input information flow. Specifically, considering the high computational cost of calculating self-attention weights in full-size PET images, in our designed spatial adaptive branch, we take the self-attention mechanism within each local partition window and introduce global information interactions between nonoverlapping windows by shifting operations to prevent the aforementioned problem. In addition, the convolutional network structure considers the information in each channel equally during the feature extraction process. In our designed channel adaptive branch, we use a Watts Strogatz topology structure to connect each feature map to only its most relevant features in each graph convolutional layer, substantially reducing information redundancy. Moreover, ensemble learning is adopted in our SW-GCN for mapping distinct features from the two well-designed branches to the enhanced PET images. We carried out extensive experiments on three single-bed position scans for 386 patients. The test results demonstrate that our proposed SW-GCN approach outperforms state-of-the-art methods in both quantitative and qualitative evaluations.

In recent times, model-driven deep learning has evolved an iterative algorithm into a cascade network by replacing the regularizer's first-order information, such as the (sub)gradient or proximal operator, with a network module. This approach offers greater explainability and predictability compared to typical data-driven networks. However, in theory, there is no assurance that a functional regularizer exists whose first-order information matches the substituted network module. This implies that the unrolled network output may not align with the regularization models. Furthermore, there are few established theories that guarantee global convergence and robustness (regularity) of unrolled networks under practical assumptions. To address this gap, we propose a safeguarded methodology for network unrolling. Specifically, for parallel MR imaging, we unroll a zeroth-order algorithm, where the network module serves as a regularizer itself, allowing the network output to be covered by a regularization model. Additionally, inspired by deep equilibrium models, we conduct the unrolled network before backpropagation to converge to a fixed point and then demonstrate that it can tightly approximate the actual MR image. We also prove that the proposed network is robust against noisy interferences if the measurement data contain noise. Finally, numerical experiments indicate that the proposed network consistently outperforms state-of-the-art MRI reconstruction methods, including traditional regularization and unrolled deep learning techniques.

Quantitative magnetic resonance (MR) [Formula: see text] mapping is a promising approach for characterizing intrinsic tissue-dependent information. However, long scan time significantly hinders its widespread applications. Recently, low-rank tensor models have been employed and demonstrated exemplary performance in accelerating MR [Formula: see text] mapping. This study proposes a novel method that uses spatial patch-based and parametric group-based low-rank tensors simultaneously (SMART) to reconstruct images from highly undersampled k-space data. The spatial patch-based low-rank tensor exploits the high local and nonlocal redundancies and similarities between the contrast images in [Formula: see text] mapping. The parametric group-based low-rank tensor, which integrates similar exponential behavior of the image signals, is jointly used to enforce multidimensional low-rankness in the reconstruction process. In vivo brain datasets were used to demonstrate the validity of the proposed method. Experimental results demonstrated that the proposed method achieves 11.7-fold and 13.21-fold accelerations in two-dimensional and three-dimensional acquisitions, respectively, with more accurate reconstructed images and maps than several state-of-the-art methods. Prospective reconstruction results further demonstrate the capability of the SMART method in accelerating MR [Formula: see text] imaging.

Magnetic resonance imaging serves as an essential tool for clinical diagnosis. However, it suffers from a long acquisition time. The utilization of deep learning, especially the deep generative models, offers aggressive acceleration and better reconstruction in magnetic resonance imaging. Nevertheless, learning the data distribution as prior knowledge and reconstructing the image from limited data remains challenging. In this work, we propose a novel Hankel-k-space generative model (HKGM), which can generate samples from a training set of as little as one k-space. At the prior learning stage, we first construct a large Hankel matrix from k-space data, then extract multiple structured k-space patches from the Hankel matrix to capture the internal distribution among different patches. Extracting patches from a Hankel matrix enables the generative model to be learned from the redundant and low-rank data space. At the iterative reconstruction stage, the desired solution obeys the learned prior knowledge. The intermediate reconstruction solution is updated by taking it as the input of the generative model. The updated result is then alternatively operated by imposing low-rank penalty on its Hankel matrix and data consistency constraint on the measurement data. Experimental results confirmed that the internal statistics of patches within single k-space data carry enough information for learning a powerful generative model and providing state-of-the-art reconstruction.

In order to test the performance of online reconstruction of deep low-rank pulse sparse network (L+S-Net) for fast dynamic MR imaging. The L+S-Net was implemented on Gadgetron platform for online reconstruction of the scanner. Although L+S-net has a good image reconstruction performance., it takes a long time to estimate the coil sensitivity using ESPIRiT method. In this study, SigPy's signal processing software package was adopted to accelerate the calculation of coil sensitivity to speed up the online reconstruction. The results of experiments showed that compared with the CPU based method., the time of the coil sensitivity estimation could be shortened more than 100 times by using the gridding reconstruction method based on SigPy GPU. The reconstruction performance is stable and can realize online fast dynamic MR imaging reconstruction within 10 seconds.

Convolutional neural network (CNN) models have achieved great success in many fields. With the advent of ResNet, networks used in practice are getting deeper and wider. However, is each layer non-trivial in networks? To answer this question, we trained a network on the training set, then we replace the network convolution kernels with zeros and test the result models on the test set. We compared experimental results with baseline and showed that we can reach similar or even the same performances. Although convolution kernels are the cores of networks, we demonstrate that some of them are trivial and regular in ResNet.

Dose reduction in computed tomography (CT) has gained considerable attention in clinical applications because it decreases radiation risks. However, a lower dose generates noise in low-dose computed tomography (LDCT) images. Previous deep learning (DL)-based works have investigated ways to improve diagnostic performance to address this ill-posed problem. However, most of them disregard the anatomical differences among different human body sites in constructing the mapping function between LDCT images and their high-resolution normal-dose CT (NDCT) counterparts. In this article, we propose a novel deep convolutional neural network (CNN) denoising approach by introducing information of the anatomical prior. Instead of designing multiple networks for each independent human body anatomical site, a unified network framework is employed to process anatomical information. The anatomical prior is represented as a pattern of weights of the features extracted from the corresponding LDCT image in an anatomical prior fusion module. To promote diversity in the contextual information, a spatial attention fusion mechanism is introduced to capture many local regions of interest in the attention fusion module. Although many network parameters are saved, the experimental results demonstrate that our method, which incorporates anatomical prior information, is effective in denoising LDCT images. Furthermore, the anatomical prior fusion module could be conveniently integrated into other DL-based methods and avails the performance improvement on multiple anatomical data.

Magnetic resonance (MR) image reconstruction from undersampled k-space data can be formulated as a minimization problem involving data consistency and image prior. Existing deep learning (DL)-based methods for MR reconstruction employ deep networks to exploit the prior information and integrate the prior knowledge into the reconstruction under the explicit constraint of data consistency, without considering the real distribution of the noise. In this work, we propose a new DL-based approach termed Learned DC that implicitly learns the data consistency with deep networks, corresponding to the actual probability distribution of system noise. The data consistency term and the prior knowledge are both embedded in the weights of the networks, which provides an utterly implicit manner of learning reconstruction model. We evaluated the proposed approach with highly undersampled dynamic data, including the dynamic cardiac cine data with up to 24-fold acceleration and dynamic rectum data with the acceleration factor equal to the number of phases. Experimental results demonstrate the superior performance of the Learned DC both quantitatively and qualitatively than the state-of-the-art.

Deep learning methods have achieved attractive performance in dynamic MR cine imaging. However, most of these methods are driven only by the sparse prior of MR images, while the important low-rank (LR) prior of dynamic MR cine images is not explored, which may limit further improvements in dynamic MR reconstruction. In this paper, a learned singular value thresholding (Learned-SVT) operator is proposed to explore low-rank priors in dynamic MR imaging to obtain improved reconstruction results. In particular, we put forward a model-based unrolling sparse and low-rank network for dynamic MR imaging, dubbed as SLR-Net. SLR-Net is defined over a deep network flow graph, which is unrolled from the iterative procedures in the iterative shrinkage-thresholding algorithm (ISTA) for optimizing a sparse and LR-based dynamic MRI model. Experimental results on a single-coil scenario show that the proposed SLR-Net can further improve the state-of-the-art compressed sensing (CS) methods and sparsity-driven deep learning-based methods with strong robustness to different undersampling patterns, both qualitatively and quantitatively. Besides, SLR-Net has been extended to a multi-coil scenario, and achieved excellent reconstruction results compared with a sparsity-driven multi-coil deep learning-based method under a high acceleration. Prospective reconstruction results on an open real-time dataset further demonstrate the capability and flexibility of the proposed method on real-time scenarios.

Deep learning, particularly the generative model, has demonstrated tremendous potential to significantly speed up image reconstruction with reduced measurements recently. Rather than the existing generative models that often optimize the density priors, in this work, by taking advantage of the denoising score matching, homotopic gradients of generative density priors (HGGDP) are exploited for magnetic resonance imaging (MRI) reconstruction. More precisely, to tackle the low-dimensional manifold and low data density region issues in generative density prior, we estimate the target gradients in higher-dimensional space. We train a more powerful noise conditional score network by forming high-dimensional tensor as the network input at the training phase. More artificial noise is also injected in the embedding space. At the reconstruction stage, a homotopy method is employed to pursue the density prior, such as to boost the reconstruction performance. Experiment results implied the remarkable performance of HGGDP in terms of high reconstruction accuracy. Only 10% of the k-space data can still generate image of high quality as effectively as standard MRI reconstructions with the fully sampled data.

The Wave Controlled Aliasing In Parallel Imaging (Wave-CAIPI) technique manifests great potential to highly accelerate three-dimensional (3D) balanced steady-state free precession (bSSFP) through substantially reducing the geometric factor (g-factor) and aliasing artifacts of image reconstruction. However, severe banding artifacts appear in bSSFP imaging due to unbalanced gradients with nonzero 0thmoment applied by the conventional Wave-CAIPI technique. In this study, we propose a 3D Wave-bSSFP scheme that adopts truncated wave gradients with zero 0thmoment to avoid introducing additional banding artifacts and to maintain the advantages of wave encoding. The simulation results indicate that the number of wave cycles that are truncated and different options of applying wave gradients affect both the g-factor reduction and image quality, but the influence is limited. In phantom experiments, the proposed technique shows similar acceleration performance as the conventional Wave-CAIPI technique and effectively eliminates its introduced banding artifacts. Additionally, Wave-bSSFP obtains up to $12\times $ retrospective acceleration at 0.8 mm isotropic resolution in in vivo 3D brain experiments and is superior to the state-of-the-art Controlled Aliasing In Parallel Imaging Results IN Higher Acceleration (CAIPIRINHA) technique, according to both visual validation and quantitative analysis. Moreover, in vivo 3D spine and abdomen imaging demonstrate the potential clinical applications of Wave-bSSFP with fast acquisition speed, improved isotropic resolution and fine image quality.

Image restoration (IR) is a long-standing challenging problem in low-level image processing. It is of utmost importance to learn good image priors for pursuing visually pleasing results. In this paper, we develop a multi-channel and multi-model-based denoising autoencoder network as image prior for solving IR problem. Specifically, the network that trained on RGB-channel images is used to construct a prior at first, and then the learned prior is incorporated into single-channel grayscale IR tasks. To achieve the goal, we employ the auxiliary variable technique to integrate the higher-dimensional network-driven prior information into the iterative restoration procedure. In addition, according to the weighted aggregation idea, a multi-model strategy is put forward to enhance the network stability that favors to avoid getting trapped in local optima. Extensive experiments on image deblurring and deblocking tasks show that the proposed algorithm is efficient, robust, and yields state-of-the-art restoration quality on grayscale images.

The segmentation of brain tissue in MRI is valuable for extracting brain structure to aid diagnosis, treatment and tracking the progression of different neurologic diseases. Medical image data are volumetric and some neural network models for medical image segmentation have addressed this using a 3D convolutional architecture. However, this volumetric spatial information has not been fully exploited to enhance the representative ability of deep networks, and these networks have not fully addressed the practical issues facing the analysis of multimodal MRI data. In this paper, we propose a spatially-weighted 3D network (SW-3D-UNet) for brain tissue segmentation of single-modality MRI, and extend it using multimodality MRI data. We validate our model on the MRBrainS13 and MALC12 datasets. This unpublished model ranked first on the leaderboard of the MRBrainS13 Challenge.

The conventional calibration-based parallel imaging method assumes a linear relationship between the acquired multi-channel k-space data and the unacquired missing data, where the linear coefficients are estimated using some auto-calibration data. In this paper, we first analyze the model errors in the conventional calibration-based methods and demonstrate the nonlinear relationship. Then, a much more general nonlinear framework is proposed for auto-calibrated parallel imaging. In this framework, kernel tricks are employed to represent the general nonlinear relationship between acquired and unacquired k-space data without increasing the computational complexity. Identification of the nonlinear relationship is still performed by solving linear equations. Experimental results demonstrate that the proposed method can achieve reconstruction quality superior to GRAPPA and NL-GRAPPA at high net reduction factors.

Due to a lack of fetal imaging coils, the standard commercial abdominal coil is often used for fetal imaging, the performance of which is limited by its insufficient coverage, element number, and Signal-to-noise ratio (SNR). In this paper, a dedicated 36-channel coil array, of which size can best fit the body sizes of pregnancy gestation from 20 to 37+ weeks, was designed for fetal imaging at 3T. SNR with full phase encoding and G-factor denoted as noise amplification for parallel imaging were quantitatively evaluated by phantom studies. Compared with a commercial abdominal coil array, the proposed 36-channel fetal array provides not only SNR improvements in full phase encoding (with 10% in the region where the whole fetal body was located, and up to 40% in the edge region where the fetal brain and heart may appear) but also an augmented parallel imaging capability and remarkable SNR improvements at high acceleration factors.

The motion of the common carotid artery (CCA) wall has been established to be useful in early diagnosis of atherosclerotic disease. However, tracking the CCA wall motion from ultrasound images remains a challenging task. In this paper, a nonlinear state-space approach has been developed to track CCA wall motion from ultrasound sequences. In this approach, a nonlinear state-space equation with a time-variant control signal was constructed from a mathematical model of the dynamics of the CCA wall. Then, the unscented Kalman filter (UKF) was adopted to solve the nonlinear state transfer function in order to evolve the state of the target tissue, which involves estimation of the motion trajectory of the CCA wall from noisy ultrasound images. The performance of this approach has been validated on 30 simulated ultrasound sequences and a real ultrasound dataset of 103 subjects by comparing the motion tracking results obtained in this study to those of three state-of-the-art methods and of the manual tracing method performed by two experienced ultrasound physicians. The experimental results demonstrated that the proposed approach is highly correlated with (intra-class correlation coefficient ≥ 0.9948 for the longitudinal motion and ≥ 0.9966 for the radial motion) and well agrees (the 95% confidence interval width is 0.8871 mm for the longitudinal motion and 0.4159 mm for the radial motion) with the manual tracing method on real data and also exhibits high accuracy on simulated data (0.1161 ~ 0.1260 mm). These results appear to demonstrate the effectiveness of the proposed approach for motion tracking of the CCA wall.

The integration of compressed sensing and parallel imaging (CS-PI) has shown an increased popularity in recent years to accelerate magnetic resonance (MR) imaging. Among them, calibration-free techniques have presented encouraging performances due to its capability in robustly handling the sensitivity information. Unfortunately, existing calibration-free methods have only explored joint-sparsity with direct analysis transform projections. To further exploit joint-sparsity and improve reconstruction accuracy, this paper proposes to Learn joINt-sparse coDes for caliBration-free parallEl mR imaGing (LINDBERG) by modeling the parallel MR imaging problem as an - - minimization objective with an norm constraining data fidelity, Frobenius norm enforcing sparse representation error and the mixed norm triggering joint sparsity across multichannels. A corresponding algorithm has been developed to alternatively update the sparse representation, sensitivity encoded images and K-space data. Then, the final image is produced as the square root of sum of squares of all channel images. Experimental results on both physical phantom and in vivo data sets show that the proposed method is comparable and even superior to state-of-the-art CS-PI reconstruction approaches. Specifically, LINDBERG has presented strong capability in suppressing noise and artifacts while reconstructing MR images from highly undersampled multichannel measurements.

Most low-rank tensor approximations are NP-hard problems. In this paper, we introduce a novel concept: field-of-experts (FoE) filters guided tensor completion, which aims to integrate the strengths of the emerging tensor completion method and the conventional FoE filters. Specifically, the target image is convolved by FoE filters to produce multiview features as a high-order tensor, which captures complementary information from multiple views. In order to impose the concept, we employ two strategies to model the new tensor, one is called FoE filters guided low-rank tensor completion, and another is called FoE filters guided simultaneous tensor decomposition and completion (FoE-STDC). The resulting objectives are solved efficiently by alternating minimization. Extensive experimental results validate the superior performance and robustness of the proposed methods over their corresponding counterparts in all cases. Particularly, the proposed FoE-STDC is superior to the state-of-the-art tensor completion methods.

While many low rank and sparsity-based approaches have been developed for accelerated dynamic magnetic resonance imaging (dMRI), they all use low rankness or sparsity in input space, overlooking the intrinsic nonlinear correlation in most dMRI data. In this paper, we propose a kernel-based framework to allow nonlinear manifold models in reconstruction from sub-Nyquist data. Within this framework, many existing algorithms can be extended to kernel framework with nonlinear models. In particular, we have developed a novel algorithm with a kernel-based low-rank model generalizing the conventional low rank formulation. The algorithm consists of manifold learning using kernel, low rank enforcement in feature space, and preimaging with data consistency. Extensive simulation and experiment results show that the proposed method surpasses the conventional low-rank-modeled approaches for dMRI.

Foreground detection is fundamental in surveillance video analysis and meaningful toward object tracking and higher level tasks, such as anomaly detection and activity analysis. Nevertheless, existing methods are still limited in accurately detecting the foreground due to the complex scene settings. To robustly handle the diverse background variations and foreground challenges, this paper proposes a Background REpresentation approach With Dictionary Learning and Historical Pixel Maintenance (BREW-DLHPM). Specifically, a dictionary learning problem is formulated at the frame level to adaptively represent the background signals with the varied structure information captured, while a pixel-level maintenance is exploited to grasp the dynamic nature of historical information under the help of the learned background. The simultaneous utilization of dictionary learning and historical pixel maintenance facilitates the accurate description of the background and thus guides a wise foreground detection decision. The proposed BREW-DLHPM has been evaluated on the prestigious change detection challenge data set against 11 state-of-the-art foreground detection approaches and encouraging performances have been achieved by our method.

The problem of recovering an image from limited or sparsely sampled Fourier measurements occurs in the application of magnetic resonance imaging. To address this problem, we propose a novel MR image reconstruction method with convolutional characteristic constraints. We first estimate the convolutional characteristics using standard compressed sensing method in a parallel fashion. Then we use the recovered image characteristics to constrain the target image function. The image characteristics should either be sparser or of higher SNR than the original image to enable superior performance. In this work, we studied using thirteen kernels and experiments based on a brain data set were conducted. It is demonstrated that the proposed method outperforms the existing methods in terms of high quality imaging due to multiple characteristic constraints and the robustness to measurement noise.

This paper presents a novel gradient correlation similarity (Gcs) measure-based decolorization model for faithfully preserving the appearance of the original color image. Contrary to the conventional data-fidelity term consisting of gradient error-norm-based measures, the newly defined Gcs measure calculates the summation of the gradient correlation between each channel of the color image and the transformed grayscale image. Two efficient algorithms are developed to solve the proposed model. On one hand, due to the highly nonlinear nature of Gcs measure, a solver consisting of the augmented Lagrangian and alternating direction method is adopted to deal with its approximated linear parametric model. The presented algorithm exhibits excellent iterative convergence and attains superior performance. On the other hand, a discrete searching solver is proposed by determining the solution with the minimum function value from the linear parametric model-induced candidate images. The non-iterative solver has advantages in simplicity and speed with only several simple arithmetic operations, leading to real-time computational speed. In addition, it is very robust with respect to the parameter and candidates. Extensive experiments under a variety of test images and a comprehensive evaluation against existing state-of-the-art methods consistently demonstrate the potential of the proposed model and algorithms.

The combination of sparse coding and manifold learning has received much attention recently. However, the computational complexity of the resulting optimization problem hinders its practical application. In this paper, an augmented Lagrangian method is proposed to address this issue, which first transforms the unconstrained problem to an equivalent constrained problem and then an alternating direction method is used to iteratively solve the subproblems. Experimental results validate the effectiveness of the propose algorithm.

This paper presents a novel structure gradient and texture decor relating regularization (SGTD) for image decomposition. The motivation of the idea is under the assumption that the structure gradient and texture components should be properly decor related for a successful decomposition. The proposed model consists of the data fidelity term, total variation regularization and the SGTD regularization. An augmented Lagrangian method is proposed to address this optimization issue, by first transforming the unconstrained problem to an equivalent constrained problem and then applying an alternating direction method to iteratively solve the sub problems. Experimental results demonstrate that the proposed method presents better or comparable performance as state-of-the-art methods do.

Initialization sensitivity usually occurs in dictionary learning algorithm for image decomposition. In this paper, we propose an adaptive dictionary learning algorithm by promoting structural incoherence at the stage of dictionary updating. The structural incoherence based dictionary learning (SIDL) method guides the cartoon and texture parts to be more properly represented by two incoherent dictionaries. The resulting minimization is approximately addressed by majorization-minimization (MM) technique. Experimental results demonstrate that the dictionaries generated by SIDL can better describe different morphological contents and subsequently the cartoon and texture components are better separated, in terms of visual comparisons and quantitative measures.

This paper presents an efficient alternating direction method with patch-based dictionary updating, ADMDU-DEB, for sparse representation regularization framework of image deblurring. The main idea of the proposed method is to reformulate the variational problem as a linear equality constrained problem and then minimize its augmented Lagrangian function. The alternating direction method decouples the minimization by alternately iterating the pixel-based regularization and the patch-based sparse representation. Typically, accelerated sparse coding and simple dictionary updating applied in the sparse representation stage enable the whole algorithm to converge at a relatively small number of iterations. Additionally, the approach is readily extended to solve the same kind of variational problem with a nonnegativity constraint. Experimental results on benchmark test images consistently validate the superiority of the proposed approach and demonstrate that it achieves very competitive deblurring performance, compared with state-of-the-art deconvolution algorithms.

Image recovery from undersampled data has always been challenging due to its implicit ill-posed nature but becomes fascinating with the emerging compressed sensing (CS) theory. This paper proposes a novel gradient based dictionary learning method for image recovery, which effectively integrates the popular total variation (TV) and dictionary learning technique into the same framework. Specifically, we first train dictionaries from the horizontal and vertical gradients of the image and then reconstruct the desired image using the sparse representations of both derivatives. The proposed method enables local features in the gradient images to be captured effectively, and can be viewed as an adaptive extension of the TV regularization. The results of various experiments on MR images consistently demonstrate that the proposed algorithm efficiently recovers images and presents advantages over the current leading CS reconstruction approaches.

In recent years Bregman iterative method (or related augmented Lagrangian method) has shown to be an efficient optimization technique for various inverse problems. In this paper, we propose a two-level Bregman Method with dictionary updating for highly undersampled magnetic resonance (MR) image reconstruction. The outer-level Bregman iterative procedure enforces the sampled k-space data constraints, while the inner-level Bregman method devotes to updating dictionary and sparse representation of small overlapping image patches, emphasizing local structure adaptively. Modified sparse coding stage and simple dictionary updating stage applied in the inner minimization make the whole algorithm converge in a relatively small number of iterations, and enable accurate MR image reconstruction from highly undersampled k-space data. Experimental results on both simulated MR images and real MR data consistently demonstrate that the proposed algorithm can efficiently reconstruct MR images and present advantages over the current state-of-the-art reconstruction approach.