Abstract Automated floorplan generation aims to improve design quality, architectural efficiency, and sustainability by jointly modeling global spatial organization and precise geometric detail. However, existing approaches operate in raster space and rely on post hoc vectorization, which introduces structural inconsistencies and hinders end‐to‐end learning. Motivated by compositional spatial reasoning, we propose TLC‐Plan, a hierarchical generative model that directly synthesizes vector floorplans from input boundaries, aligning with human architectural workflows based on modular and reusable patterns. TLC‐Plan employs a two‐level VQ‐VAE to encode global layouts as semantically labeled room bounding boxes and to refine local geometries using polygon‐level codes. This hierarchy is unified in a CodeTree representation, while an autoregressive transformer samples codes conditioned on the boundary to generate diverse and topologically valid designs, without requiring explicit room topology or dimensional priors. Extensive experiments show state‐of‐the‐art performance on RPLAN dataset (FID = 1.84, MSE = 2.06) and leading results on LIFULL dataset. The proposed framework advances constraint‐aware and scalable vector floorplan generation for real‐world architectural applications. Source code and trained models are released at https://github.com/rosolose/TLC‐PLAN .

Sparse-view sampling in dual-energy computed tomography (DECT) significantly reduces radiation dose and increases imaging speed, yet is highly prone to artifacts. Although diffusion models have demonstrated potential in effectively handling incomplete data, most existing methods in this field focus on the image domain and lack global constraints, which consequently leads to insufficient reconstruction quality. In this study, we propose a dual-domain virtual-mask informed diffusion model for sparse-view reconstruction by leveraging the high inter-channel correlation in DECT. Specifically, the study designs a virtual mask and applies it to the high-energy and low-energy data to perform perturbation operations, thus constructing high-dimensional tensors that serve as the prior information of the diffusion model. In addition, a dual-domain collaboration strategy is adopted to integrate the information of the randomly selected high-frequency components in the wavelet domain with the information in the projection domain, for the purpose of optimizing the global structures and local details. The experimental results show that the method exhibits excellent performance on multiple datasets. Under 30-view sparse sampling conditions, VIP-DECT improves PSNR by at least 1.02 dB and enhances SSIM by 1.91% .

MRI has become essential in clinical diagnosis due to its high resolution and multiple contrast mechanisms. However, the relatively long acquisition time limits its broader application. To address this issue, this study presents an innovative conditional guided diffusion model, named TC-KANRecon, which incorporates the Multi-Free U-KAN module and a dynamic clipping strategy. TC-KANRecon model aims to accelerate the MRI reconstruction process through deep learning methods while maintaining the reconstruction quality. The MF-UKAN module can effectively balance the tradeoff between image denoising and structure preservation. Specifically, it presents the multi-head attention mechanisms and scalar modulation factors, which significantly enhance the model's robustness and structure preservation capabilities in complex noise environments. Moreover, the dynamic clipping strategy in TC-KANRecon adjusts the cropping interval according to the sampling steps, thereby mitigating image detail loss while preserving the visual features of the images. Furthermore, the Conditional Guidance Model incorporates full-sampling k-space information, realizing efficient fusion of conditional information, enhancing the model's ability to process complex data, and improving the realism and detail richness of reconstructed images. Experimental results demonstrate that the proposed method outperforms other MRI reconstruction methods in both qualitative and quantitative evaluations. Notably, TC-KANRecon method exhibits excellent reconstruction results when processing high-noise, low-sampling-rate MRI data.

Point cloud place recognition (PCPR) determines the geo-location within a prebuilt map and plays a crucial role in photogrammetry and robotics applications such as autonomous driving, intelligent transportation, and augmented reality. In real-world large-scale deployments of a geographic positioning system, PCPR models must continuously acquire, update, and accumulate knowledge to adapt to diverse and dynamic environments, i.e., the ability known as continual learning (CL). However, existing PCPR models often suffer from catastrophic forgetting, leading to significant performance degradation in previously learned scenes when adapting to new environments or sensor types. This results in poor model scalability, increased maintenance costs, and system deployment difficulties, undermining the practicality of PCPR. To address these issues, we propose LifelongPR, a novel continual learning framework for PCPR, which effectively extracts and fuses knowledge from sequential point cloud data. First, to alleviate the knowledge loss, we propose a replay sample selection method that dynamically allocates sample sizes according to each dataset’s information quantity and selects spatially diverse samples for maximal representativeness. Second, to handle domain shifts, we design a prompt learning-based CL framework with a lightweight continuous prompt module and a two-stage training strategy, enabling domain-specific feature adaptation while minimizing forgetting. Comprehensive experiments on large-scale public and self-collected datasets are conducted to validate the effectiveness of the proposed method. Compared with the state-of-the-art (SOTA) method, our method achieves 6.50% improvement in$mIR\text{@}1$, 7.96% improvement in$mR\text{@}1$, and an 8.95% reduction in$F$. The code and pre-trained models are publicly available athttps://zouxianghong.github.io/LifelongPR

Generative diffusion models have received increasing attention in medical imaging, particularly in limited-angle computed tomography (LACT). Standard diffusion models achieve high-quality image reconstruction but require a large number of sampling steps during inference, imposing a heavy computational burden. Although skip-sampling strategies have been proposed to improve efficiency, they often lead to loss of fine structural details. To address this issue, we propose a Prior-aware Wavelet Diffusion for Efficient Dental Limited-angle CT Reconstruction (PWD). The PWD enables efficient sampling while preserving reconstruction fidelity in LACT, and effectively mitigates the degradation typically introduced by skip-sampling. Specifically, during the training phase, PWD maps the distribution of LACT images to that of fully sampled target images, enabling the model to learn structural correspondences between them. During inference, the LACT image serves as an prior-aware to guide the sampling trajectory, allowing for high-quality reconstruction with significantly fewer steps. In addition, PWD performs multi-scale feature fusion in the wavelet domain, effectively enhancing the reconstruction of fine details by leveraging both low-frequency and high-frequency information. Quantitative and qualitative evaluations on clinical dental arch CBCT and periapical datasets demonstrate that PWD outperforms existing methods under the same sampling condition. Using only 50 sampling steps, PWD achieves at least 1.7 dB improvement in PSNR and 10% gain in SSIM.

Corneal fluorescein staining (CFS) imaging, while crucial for diagnosing corneal diseases, faces limitations in clinical applications due to operational complexity and patient discomfort. We present EyeTransGAN, a novel framework for unsupervised translation of anterior segment photographs to CFS images with semantic consistency. Our approach addresses color dispersion and semantic inconsistency challenges through a dual consistency constraint mechanism, combining contrastive learning and disease-aware semantic guidance. The framework introduces Mobile inverted Bottleneck Convolution (MBConv)-based classifiers for precise semantic guidance via class activation maps. Validated across 11 corneal conditions, EyeTransGAN demonstrates superior performance in both image quality metrics and diagnostic capability. This work offers a promising solution for corneal disease assessment, potentially reducing the need for frequent fluorescein staining, thus minimizing patient discomfort and clinical resources.

To address multi-user interference and codebook design challenges in sparse code multiple access (SCMA) systems, we propose a Wasserstein generative adversarial network (WGAN) integrated convolutional residual neural network (CR-Net) for end-to-end joint optimization of hybrid autoencoder-channel components. The proposed architecture applies convolutional neural networks (CNNs) for multidimensional codebooks encoding and residual neural networks (ResNet) for interferenc-resistant decoding, forming a hybrid autoencoder structure. In practical wireless channels with variable timing, encoder-generated codebooks do not dynamically adapt to channel variations, resulting in a degraded bit error rate (BER) performance. We employ WGAN to model channel effects: 1) the generator models channel distortion to drive encoder-generated channel-adaptive codebooks; 2) the discriminator uses Wasserstein distance-based feature extraction to co-optimize decoder interference suppression via backpropagation. Simulation results demonstrate BER performance and computational complexity that surpasses deep learning (DL)-based benchmarks.

Based on the hybrid-index model, this article investigates the asymptotic feedback set stabilization of Boolean control networks (BCNs) with random impulsive disturbances. In this model, it is assumed that the sequence of intervals between adjacent impulsive instants is independent and identically distributed. This assumption ensures that the subsequence of solutions sampled at impulsive moments is a Markov chain. Based on this assumption and the semi-tensor product (STP), random impulsive BCNs (RI-BCNs) can be converted into impulsive-interval driven probabilistic BCNs (ID-PBCNs), and the input-state transition probability matrix (IS-TPM) is constructed, the calculations of convergent target set in the hybrid domain and the time domain are discussed, and the necessary and sufficient conditions for asymptotic feedback set stabilizability are obtained. On this basis, we propose a design algorithm of state feedback controllers to stabilize RI-BCNs asymptotically with respect to a target set by using state-space partition, which enables the system to converge to a given set with the least number of impulsive intervals. Finally, the effectiveness of the obtained results is verified by simulations.

Seismic full-waveform inversion (FWI) is a powerful technique for high-resolution imaging of subsurface physical properties. However, it suffers from the possibility of falling into local minimum due to the inherent nonlinearity and ill-posedness. Recently, the data-driven deep learning approach has been used to solve ill-posed inverse problems, with the limitations of high data collection costs and poor model generalization capabilities. To alleviate these difficulties, we unroll the iterative optimization algorithm based on the augmented Lagrangian method into a layer-wise network architecture to solve seismic FWI, called ALFWI-Net. The ALFWI-Net decomposes the constrained optimization problem into three unconstrained subproblems. Correspondingly, the velocity model is updated by three alternating iterative formulations implemented by four modules in the network. One of the modules uses the Lagrange multiplier method to solve for the gradient of FWI, while the others correspond to the solution of the three subproblems. A soft-threshold-based convolutional neural network is used to learn the proximal operator in the implicit regularized optimization subproblem. Therefore, ALFWI-Net alleviates limitations in applicability since both the regularization function and parameters, step size and thresholds are automatically learned in the training process. Experiments were conducted on two geologic models, with the SEG salt model posing a challenge due to its limited data samples. Nevertheless, we successfully derived improved velocity models from seismic full-waveform recordings. Numerical experiments on two kinds of geological models demonstrated that ALFWI-Net outperforms classical FWI and data-driven methods in reconstruction accuracy, convergence speed, and out-of-distribution generalization.

Recently, diffusion models have shown considerable promise for MRI reconstruction. However, extensive experimentation has revealed that these models are prone to generating artifacts due to the inherent randomness involved in generating images from pure noise. To achieve more controlled image reconstruction, we reexamine the concept of interpolatable physical priors in k-space data, focusing specifically on the interpolation of high-frequency (HF) k-space data from low-frequency (LF) k-space data. Broadly, this insight drives a shift in the generation paradigm from random noise to a more deterministic approach grounded in the existing LF k-space data. Building on this, we first establish a relationship between the interpolation of HF k-space data from LF k-space data and the reverse heat diffusion process, providing a fundamental framework for designing diffusion models that generate missing HF data. To further improve reconstruction accuracy, we integrate a traditional physics-informed k-space interpolation model into our diffusion framework as a data fidelity term. Experimental validation using publicly available datasets demonstrates that our approach significantly surpasses traditional k-space interpolation methods, deep learning-based k-space interpolation techniques, and conventional diffusion models, particularly in HF regions. Finally, we assess the generalization performance of our model across various out-of-distribution datasets. Our code are available at https://github.com/ZhuoxuCui/Heat-Diffusion.

Given the obstacle in accentuating the reconstruction accuracy for diagnostically significant tissues, most existing MRI reconstruction methods perform targeted reconstruction of the entire MR image without considering fine details, especially when dealing with highly under-sampled images. Therefore, a considerable volume of efforts has been directed towards surmounting this challenge, as evidenced by the emergence of numerous methods dedicated to preserving high-frequency content as well as fine textural details in the reconstructed image. In this case, exploring the merits associated with each method of mining high-frequency information and formulating a reasonable principle to maximize the joint utilization of these approaches will be a more effective solution to achieve accurate reconstruction. Specifically, this work constructs an innovative principle named Correlated and Multi-frequency Diffusion Model (CM-DM) for highly under-sampled MRI reconstruction. In essence, the rationale underlying the establishment of such principle lies not in assembling arbitrary models, but in pursuing the effective combinations and replacement of components. It also means that the novel principle focuses on forming a correlated and multi-frequency prior through different high-frequency operators in the diffusion process. Moreover, multi-frequency prior further constraints the noise term closer to the target distribution in the frequency domain, thereby making the diffusion process converge faster. Experimental results verify that the proposed method achieved superior reconstruction accuracy, with a notable enhancement of approximately 2dB in PSNR compared to state-of-the-art methods.

The multi-source stationary CT, where both the detector and X-ray source are fixed, represents a novel imaging system with high temporal resolution that has garnered significant interest. Limited space within the system restricts the number of X-ray sources, leading to sparse-view CT imaging challenges. Recent diffusion models for reconstructing sparse-view CT have generally focused separately on sinogram or image domains. Sinogram-centric models effectively estimate missing projections but may introduce artifacts, lacking mechanisms to ensure image correctness. Conversely, image-domain models, while capturing detailed image features, often struggle with complex data distribution, leading to inaccuracies in projections. Addressing these issues, the Dual-domain Collaborative Diffusion Sampling (DCDS) model integrates sinogram and image domain diffusion processes for enhanced sparse-view reconstruction. This model combines the strengths of both domains in an optimized mathematical framework. A collaborative diffusion mechanism underpins this model, improving sinogram recovery and image generative capabilities. This mechanism facilitates feedback-driven image generation from the sinogram domain and uses image domain results to complete missing projections. Optimization of the DCDS model is further achieved through the alternative direction iteration method, focusing on data consistency updates. Extensive testing, including numerical simulations, real phantoms, and clinical cardiac datasets, demonstrates the DCDS model's effectiveness. It consistently outperforms various state-of-the-art benchmarks, delivering exceptional reconstruction quality and precise sinogram.

The score-based generative model (SGM) has demonstrated remarkable performance in addressing challenging under-determined inverse problems in medical imaging. However, acquiring high-quality training datasets for these models remains a formidable task, especially in medical image reconstructions. Prevalent noise perturbations or artifacts in low-dose Computed Tomography (CT) or under-sampled Magnetic Resonance Imaging (MRI) hinder the accurate estimation of data distribution gradients, thereby compromising the overall performance of SGMs when trained with these data. To alleviate this issue, we propose a wavelet-improved denoising technique to cooperate with the SGMs, ensuring effective and stable training. Specifically, the proposed method integrates a wavelet sub-network and the standard SGM sub-network into a unified framework, effectively alleviating inaccurate distribution of the data distribution gradient and enhancing the overall stability. The mutual feedback mechanism between the wavelet sub-network and the SGM sub-network empowers the neural network to learn accurate scores even when handling noisy samples. This combination results in a framework that exhibits superior stability during the learning process, leading to the generation of more precise and reliable reconstructed images. During the reconstruction process, we further enhance the robustness and quality of the reconstructed images by incorporating regularization constraint. Our experiments, which encompass various scenarios of low-dose and sparse-view CT, as well as MRI with varying under-sampling rates and masks, demonstrate the effectiveness of the proposed method by significantly enhanced the quality of the reconstructed images. Especially, our method with noisy training samples achieves comparable results to those obtained using clean data. Our code at https://zenodo.org/record/8266123.

Diffusion model has emerged as a potential tool to tackle the challenge of sparse-view CT reconstruction, displaying superior performance compared to conventional methods. Nevertheless, these prevailing diffusion models predominantly focus on the sinogram or image domains, which can lead to instability during model training, potentially culminating in convergence towards local minimal solutions. The wavelet transform serves to disentangle image contents and features into distinct frequency-component bands at varying scales, adeptly capturing diverse directional structures. Employing the wavelet transform as a guiding sparsity prior significantly enhances the robustness of diffusion models. In this study, we present an innovative approach named the Stage-by-stage Wavelet Optimization Refinement Diffusion (SWORD) model for sparse-view CT reconstruction. Specifically, we establish a unified mathematical model integrating low-frequency and high-frequency generative models, achieving the solution with an optimization procedure. Furthermore, we perform the low-frequency and high-frequency generative models on wavelet's decomposed components rather than the original sinogram, ensuring the stability of model training. Our method is rooted in established optimization theory, comprising three distinct stages, including low-frequency generation, high-frequency refinement and domain transform. The experimental results demonstrated that the proposed method outperformed existing state-of-the-art methods both quantitatively and qualitatively.

With the successful application of deep learning to magnetic resonance (MR) imaging, parallel imaging techniques based on neural networks have attracted wide attention. However, in the absence of high-quality, fully sampled datasets for training, the performance of these methods is limited. And the interpretability of models is not strong enough. To tackle this issue, this paper proposes a Physics-bAsed unsupeRvised Contrastive rEpresentation Learning (PARCEL) method to speed up parallel MR imaging. Specifically, PARCEL has a parallel framework to contrastively learn two branches of model-based unrolling networks from augmented undersampled multi-coil k-space data. A sophisticated co-training loss with three essential components has been designed to guide the two networks in capturing the inherent features and representations for MR images. And the final MR image is reconstructed with the trained contrastive networks. PARCEL was evaluated on two vivo datasets and compared to five state-of-the-art methods. The results show that PARCEL is able to learn essential representations for accurate MR reconstruction without relying on fully sampled datasets. The code will be made available at https://github.com/ternencewu123/PARCEL.

In this paper, two novel methods for SAR image super-resolution (SR) are proposed. The main challenge for SAR image SR reconstruction is speckle noise. For this reason, a novel algorithm termed the importance sampling cubature Kalman filter (ISCKF) is proposed to reconstruct a high resolution (HR) image from a series of low-resolution (LR) images. However, as the reconstructed image usually embraces residual noise visually, we establish a nonlinear low-rank optimization model in order to further reduce the speckle noise drastically. Correspondingly, an alternating direction method of multipliers based on the low-rank model (ADMM-LR) algorithm is proposed to solve it, which yields the other novel method termed ISCKF+ADMM-LR for SAR image SR. In addition, we establish the computational complexity of the proposed algorithms. The experimental results of both the simulated images and real SAR images demonstrate that the performance of the proposed methods are superior to some state-of-art methods in both SR and despeckle.

As an effective way to integrate the information contained in multiple medical images under different modalities, medical image synthesis and fusion have emerged in various clinical applications such as disease diagnosis and treatment planning. In this paper, an invertible and variable augmented network (iVAN) is proposed for medical image synthesis and fusion. In iVAN, the channel number of the network input and output is the same through variable augmentation technology, and data relevance is enhanced, which is conducive to the generation of characterization information. Meanwhile, the invertible network is used to achieve the bidirectional inference processes. Empowered by the invertible and variable augmentation schemes, iVAN not only be applied to the mappings of multi-input to one-output and multi-input to multi-output, but also to the case of one-input to multi-output. Experimental results demonstrated superior performance and potential task flexibility of the proposed method, compared with existing synthesis and fusion methods.

Magnetic resonance imaging serves as an essential tool for clinical diagnosis. However, it suffers from a long acquisition time. The utilization of deep learning, especially the deep generative models, offers aggressive acceleration and better reconstruction in magnetic resonance imaging. Nevertheless, learning the data distribution as prior knowledge and reconstructing the image from limited data remains challenging. In this work, we propose a novel Hankel-k-space generative model (HKGM), which can generate samples from a training set of as little as one k-space. At the prior learning stage, we first construct a large Hankel matrix from k-space data, then extract multiple structured k-space patches from the Hankel matrix to capture the internal distribution among different patches. Extracting patches from a Hankel matrix enables the generative model to be learned from the redundant and low-rank data space. At the iterative reconstruction stage, the desired solution obeys the learned prior knowledge. The intermediate reconstruction solution is updated by taking it as the input of the generative model. The updated result is then alternatively operated by imposing low-rank penalty on its Hankel matrix and data consistency constraint on the measurement data. Experimental results confirmed that the internal statistics of patches within single k-space data carry enough information for learning a powerful generative model and providing state-of-the-art reconstruction.

Unsupervised deep learning has recently demonstrated the promise of producing high-quality samples. While it has tremendous potential to promote the image colorization task, the performance is limited owing to the high-dimension of data manifold and model capability. This study presents a novel scheme that exploits the score-based generative model in wavelet domain to address the issues. By taking advantage of the multi-scale and multi-channel representation via wavelet transform, the proposed model learns the richer priors from stacked coarse and detailed wavelet coefficient components jointly and effectively. This strategy also reduces the dimension of the original manifold and alleviates the curse of dimensionality, which is beneficial for estimation and sampling. Moreover, dual consistency terms in the wavelet domain, namely data-consistency and structure-consistency are devised to leverage colorization task better. Specifically, in the training phase, a set of multi-channel tensors consisting of wavelet coefficients is used as the input to train the network with denoising score matching. In the inference phase, samples are iteratively generated via annealed Langevin dynamics with data and structure consistencies. Experiments demonstrated remarkable improvements of the proposed method on both generation and colorization quality, particularly in colorization robustness and diversity.

Clutter rejection is a key technique for high-frequency passive radar (HFPR). To solve this problem, the traditional signal processing methods have been used, which mainly depend on prior information of the feature differences between target and clutter in time, space, or frequency domain. As a new attempt to deep-mine the clutter feature automatically and reject it by only data-driven processing, a novel clutter rejection method based on graph-relational mapping using a deep learning network is proposed in this letter. In this method, the clutter rejection problem is turned into an image-to-image translation problem between the range-Doppler (RD) spectrograms before and after clutter rejection. A deep-learning-enabled image translation network (CycleGAN) is exploited to learn from training data of RD spectrograms and to establish the mapping relationship. When processing clutter rejection tasks, the trained network can automatically extract clutter features without prior information and save manpower. The performance evaluations of the novel clutter rejection method are also investigated, and the experimental results confirm that the proposed method can effectively reject clutter in HFPR.

The clutter encountered in the ground-penetrating radar (GPR) system severely obscures the visibility of subsurface objects, especially in the case of overlapping target responses and clutter. In this letter, a novel self-supervised learning strategy with dual-network architecture and pseudolabels is proposed. First, the dual-network consists of two subnetworks: one is to simulate the low-rank part, and another simulates the sparse part. Second, the raw GPR data are decomposed as the sum of low-rank and sparse matrices by robust nonnegative matrix factorization (RNMF), termed two pseudolabels. Then, these pseudolabels guide the two subnetworks to accurately reconstruct the target response and clutter trace by trace, respectively. Results based on simulated data by gprMax and real datasets demonstrate that the proposed method is effective in separating target response from clutter and can achieve a similar effect as RNMF in less time without any prior information.

Breast density is widely adopted to reflect the likelihood of early breast cancer development. Existing methods of mammographic density classification either require steps of manual operations or achieve only moderate classification accuracy due to the limited model capacity. In this study, we present a radiomics approach based on dilated and attention-guided residual learning for the task of mammographic density classification. The proposed method was instantiated with two datasets, one clinical dataset and one publicly available dataset, and classification accuracies of 88.7 and 70.0 percent were obtained, respectively. Although the classification accuracy of the public dataset was lower than the clinical dataset, which was very likely related to the dataset size, our proposed model still achieved a better performance than the naive residual networks and several recently published deep learning-based approaches. Furthermore, we designed a multi-stream network architecture specifically targeting at analyzing the multi-view mammograms. Utilizing the clinical dataset, we validated that multi-view inputs were beneficial to the breast density classification task with an increase of at least 2.0 percent in accuracy and the different views lead to different model classification capacities. Our method has a great potential to be further developed and applied in computer-aided diagnosis systems. Our code is available at https://github.com/lich0031/Mammographic_Density_Classification.

Deep learning, particularly the generative model, has demonstrated tremendous potential to significantly speed up image reconstruction with reduced measurements recently. Rather than the existing generative models that often optimize the density priors, in this work, by taking advantage of the denoising score matching, homotopic gradients of generative density priors (HGGDP) are exploited for magnetic resonance imaging (MRI) reconstruction. More precisely, to tackle the low-dimensional manifold and low data density region issues in generative density prior, we estimate the target gradients in higher-dimensional space. We train a more powerful noise conditional score network by forming high-dimensional tensor as the network input at the training phase. More artificial noise is also injected in the embedding space. At the reconstruction stage, a homotopy method is employed to pursue the density prior, such as to boost the reconstruction performance. Experiment results implied the remarkable performance of HGGDP in terms of high reconstruction accuracy. Only 10% of the k-space data can still generate image of high quality as effectively as standard MRI reconstructions with the fully sampled data.

X-ray computed tomography (CT) is of great clinical significance in medical practice because it can provide anatomical information about the human body without invasion, while its radiation risk has continued to attract public concerns. Reducing the radiation dose may induce noise and artifacts to the reconstructed images, which will interfere with the judgments of radiologists. Previous studies have confirmed that deep learning (DL) is promising for improving low-dose CT imaging. However, almost all the DL-based methods suffer from subtle structure degeneration and blurring effect after aggressive denoising, which has become the general challenging issue. This paper develops the Comprehensive Learning Enabled Adversarial Reconstruction (CLEAR) method to tackle the above problems. CLEAR achieves subtle structure enhanced low-dose CT imaging through a progressive improvement strategy. First, the generator established on the comprehensive domain can extract more features than the one built on degraded CT images and directly map raw projections to high-quality CT images, which is significantly different from the routine GAN practice. Second, a multi-level loss is assigned to the generator to push all the network components to be updated towards high-quality reconstruction, preserving the consistency between generated images and gold-standard images. Finally, following the WGAN-GP modality, CLEAR can migrate the real statistical properties to the generated images to alleviate over-smoothing. Qualitative and quantitative analyses have demonstrated the competitive performance of CLEAR in terms of noise suppression, structural fidelity and visual perception improvement.

Colorization is the process of coloring monochrome images. It has been widely used in photo processing and scientific illustration. However, colorizing grayscale images is an intrinsic ill-posed and ambiguous problem, with multiple plausible solutions. To address this issue, we develop a novel progressive automatic colorization via iterative generative models (iGM) that can produce satisfactory colorization in an unsupervised manner. In particular, the generative model is exploited in multi-color spaces (e.g., RGB, YCbCr) jointly and enforced with linearly autocorrelative constraint. This is regarded as the key prior information to pave the way for producing the most probable colorization in high-dimensional space. Experiments on indoor and outdoor scenes reveal that iGM produces more realistic and finer results, compared to state-of-the-arts.

Image restoration (IR) is a long-standing challenging problem in low-level image processing. It is of utmost importance to learn good image priors for pursuing visually pleasing results. In this paper, we develop a multi-channel and multi-model-based denoising autoencoder network as image prior for solving IR problem. Specifically, the network that trained on RGB-channel images is used to construct a prior at first, and then the learned prior is incorporated into single-channel grayscale IR tasks. To achieve the goal, we employ the auxiliary variable technique to integrate the higher-dimensional network-driven prior information into the iterative restoration procedure. In addition, according to the weighted aggregation idea, a multi-model strategy is put forward to enhance the network stability that favors to avoid getting trapped in local optima. Extensive experiments on image deblurring and deblocking tasks show that the proposed algorithm is efficient, robust, and yields state-of-the-art restoration quality on grayscale images.

Convolutional neural networks (CNNs) have presented their potential in filling large missing areas with plausible contents. To address the blurriness issue commonly existing in the CNN-based inpainting, a typical approach is to conduct texture refinement on the initially completed images by replacing the neural patch in the predicted region using the closest one in the known region. However, such a processing might introduce undesired content change in the predicted region, especially when the desired content does not exist in the known region. To avoid generating such incorrect content, in this paper, we propose a content inference and style imitation network (CISI-net), which explicitly separate the image data into content code and style code. The content inference is realized by performing inference in the latent space to infer the content code of the corrupted images similar to the one from the original images. It can produce more detailed content than a similar inference procedure in the pixel domain, due to the dimensional distribution of content being lower than that of the entire image. On the other hand, the style code is used to represent the rendering of content, which will be consistent over the entire image. The style code is then integrated with the inferred content code to generate the complete image. Experiments on multiple datasets including structural and natural images demonstrate that our proposed approach out-performs the existing ones in terms of content accuracy as well as texture details.

This work exploits the basic denoising autoencoding (DAE) as enhanced priori for color image restoration (IR). The proposed method consists of two steps: enhanced DAE network learning and iterative restoration. To be special, at the training phase, a denoising network taking 6-dimensional variable as input is trained. Then, the network-driven high-dimensional prior information embedded DAE priori is utilized in the iterative restoration procedure. We first map the intermediate color image to be 6-dimensional and employ the higher-dimensional network to handle its corrupted version. The average operator is used to turn it back to the 3-channel image. The higher-dimensional prior alleviates the issue of the basic DAE that getting trapped in local optimal solution and effectively overcomes the instability. Experimental results on single image super-restoration (SISR) and deblurring demonstrate that the proposed algorithm can achieve good performance and prime visual inspection.

Image textures, as a kind of local variations, provide important information for the human visual system. Many image textures, especially the small-scale or stochastic textures, are rich in high-frequency variations, and are difficult to be preserved. Current state-of-the-art denoising algorithms typically adopt a nonlocal approach consisting of image patch grouping and group-wise denoising filtering. To achieve a better image denoising while preserving the variations in texture, we first adaptively group high correlated image patches with the same kinds of texture elements (texels) via an adaptive clustering method. This adaptive clustering method is applied in an over-clustering-and-iterative-merging approach, where its noise robustness is improved with a custom merging threshold relating to the noise level and cluster size. For texture-preserving denoising of each cluster, considering that the variations in texture are captured and wrapped in not only the between-dimension energy variations but also the within-dimension variations of PCA transform coefficients, we further propose a PCA-transform-domain variation adaptive filtering method to preserve the local variations in textures. Experiments on natural images show the superiority of the proposed transform-domain variation adaptive filtering to traditional PCA-based hard or soft threshold filtering. As a whole, the proposed denoising method achieves a favorable texture-preserving performance both quantitatively and visually, especially for irregular textures, which is further verified in camera raw image denoising.

The integration of compressed sensing and parallel imaging (CS-PI) has shown an increased popularity in recent years to accelerate magnetic resonance (MR) imaging. Among them, calibration-free techniques have presented encouraging performances due to its capability in robustly handling the sensitivity information. Unfortunately, existing calibration-free methods have only explored joint-sparsity with direct analysis transform projections. To further exploit joint-sparsity and improve reconstruction accuracy, this paper proposes to Learn joINt-sparse coDes for caliBration-free parallEl mR imaGing (LINDBERG) by modeling the parallel MR imaging problem as an - - minimization objective with an norm constraining data fidelity, Frobenius norm enforcing sparse representation error and the mixed norm triggering joint sparsity across multichannels. A corresponding algorithm has been developed to alternatively update the sparse representation, sensitivity encoded images and K-space data. Then, the final image is produced as the square root of sum of squares of all channel images. Experimental results on both physical phantom and in vivo data sets show that the proposed method is comparable and even superior to state-of-the-art CS-PI reconstruction approaches. Specifically, LINDBERG has presented strong capability in suppressing noise and artifacts while reconstructing MR images from highly undersampled multichannel measurements.

Most low-rank tensor approximations are NP-hard problems. In this paper, we introduce a novel concept: field-of-experts (FoE) filters guided tensor completion, which aims to integrate the strengths of the emerging tensor completion method and the conventional FoE filters. Specifically, the target image is convolved by FoE filters to produce multiview features as a high-order tensor, which captures complementary information from multiple views. In order to impose the concept, we employ two strategies to model the new tensor, one is called FoE filters guided low-rank tensor completion, and another is called FoE filters guided simultaneous tensor decomposition and completion (FoE-STDC). The resulting objectives are solved efficiently by alternating minimization. Extensive experimental results validate the superior performance and robustness of the proposed methods over their corresponding counterparts in all cases. Particularly, the proposed FoE-STDC is superior to the state-of-the-art tensor completion methods.

Registration of images from different modalities in the presence of intra-image fluctuation and noise contamination is a challenging task. The accuracy and robustness of the deformable registration largely depend on the definition of appropriate objective function, measuring the similarity between the images. Among them the multi-dimensional modality independent neighbourhood descriptor (MIND) is a promising method, yet its ability is limited by non-uniform bias fields and image noise, etc. Motivated by the fact that Log-Euclidean metric has promising invariance properties such as inversion invariant and similarity invariant, this paper introduces an objective function that embeds Log-Euclidean similarity metric between patches to form a multi-dimensional descriptor. The Gaussian-like penalty function consisting of the log-Euclidean metric between images to be registered is incorporated to better reflect the degree of preserving feature discriminability and structure ordering. Experimental results show the advantages of the proposed method over state-of-the-art techniques both quantitatively and qualitatively.

Since the significant intensity variations existed between different modal images, the deformable registration is still very challenging. In this paper, in order to alleviate the variations deficiency and attain robust alignment, we propose a multi-dimensional tensor based modality independent neighbourhood descriptor (tMIND) to measure the similarity between the images. The tMIND compares the neighboring tensors which consisting of multi-filters induced features. In this work we learn these filters via PCA network (PCANet). We additionally describe the scheme of incorporating these filters into the tMIND. Experimental evaluations demonstrate its promise and effectiveness over the current state-of-the-art approaches.

Synthesis learning and analysis learning, with sparse coding (SC) and Markov random fields (MRFs) as two representative types of models, are two complementary tools to describe the image manifolds. SC has strengths in representing the regular features/explicit visual manifolds while its effectiveness depends on the training dataset. While MRFs have great potentials to characterize the stochastic textures/implicit visual manifolds but at the cost of high training complexity. In this paper, by means of the convolutional operator, a unified synthesis and analysis deconvolutional network (SADN) is presented. It not only requires the generative coding coefficients to be sparse, but also enforces the convolution between the filter and trained images to be sparse. The proposed model incorporates the strengths of both SC and MRFs, which enables it to represent general images with both generative and discriminative abilities. The resulting minimization is tackled by the combination of alternating optimization and Iterative Re-weighted Least Square (IRLS). Experiments conducted on compressed sensing (CS) application show its great potentials both quantitatively and qualitatively.

Image prior information is a determinative factor to tackle with the ill-posed problem. In this paper, we present a multi-filters guided low-rank tensor coding (MF-LRTC) model for image restoration. The appeal of constructing a low-rank tensor is obvious in many cases for data that naturally comes from different scales and directions. The MF-LRTC takes advantages of the low-rank tensor coding to capture the sparse convolutional features generated by multi-filters representation. Using such a low-rank tensor coding would reduce the redundancy between feature vectors at neighboring locations and improve the efficiency of the overall sparse representation. In this work, we are committed to achieving this goal by convoluting the target image with filters to formulate multi-features images. Then similarity-grouped cube set extracted from the multi-features images is regarded as a low-rank tensor. The potential effectiveness of this tensor construction strategy is demonstrated in image restoration including image deblurring and compressed sensing (CS) applications.

Most of low-rank tensor approximation problems are NP-hard. Hence a great number of synthesis tensor decomposition approximation have been proposed. In this paper, we instead present an analysis-operator guided tensor decomposition. The proposed method first employs the classical Field-of-Experts (FoE) filters to produce multi-view features such that forming a higher-order tensor, and then do simultaneous tensor decomposition and completion (STDC). The multi-view features are obtained by convolving the target image with high-frequency FoE filters along different directions and scales. The proposed method is solved efficiently by alternating direction of multipliers method (ADMM). Experiments are conducted to demonstrate the superior performance of our method to state-of-the-art tensor completion methods.

The RGB2GRAY conversion model is the most popular and classical tool for image decolorization. A recent study showed that adapting the three weighting parameters in this first-order linear model with a discrete searching solver has a great potential in its conversion ability. In this paper, we present a two-step strategy to efficiently extend the parameter searching solver to a two-order multivariance polynomial model, as a sum of three subspaces. We show that the first subspace in the two-order model is the most important and the second one can be seen as a refinement. In the first stage of our model, the gradient correlation similarity (Gcs) measure is used on the first subspace to obtain an immediate grayed image. Then, Gcs is applied again to select the optimal result from the immediate grayed image plus the second subspace-induced candidate images. Experimental results show the advantages of the proposed approach in terms of quantitative evaluation, qualitative evaluation, and algorithm complexity.

While the RGB2GRAY conversion with fixed parameters is a classical and widely used tool for image decolorization, recent studies showed that adapting weighting parameters in a two-order multivariance polynomial model has great potential to improve the conversion ability. In this paper, by viewing the two-order model as the sum of three subspaces, it is observed that the first subspace in the two-order model has the dominating importance and the second and the third subspace can be seen as refinement. Therefore, we present a semiparametric strategy to take advantage of both the RGB2GRAY and the two-order models. In the proposed method, the RGB2GRAY result on the first subspace is treated as an immediate grayed image, and then the parameters in the second and the third subspace are optimized. Experimental results show that the proposed approach is comparable to other state-of-the-art algorithms in both quantitative evaluation and visual quality, especially for images with abundant colors and patterns. This algorithm also exhibits good resistance to noise. In addition, instead of the color contrast preserving ratio using the first-order gradient for decolorization quality metric, the color contrast correlation preserving ratio utilizing the second-order gradient is calculated as a new perceptual quality metric.

This paper presents a novel Log-Euclidean metric inspired color-to-gray conversion model for faithfully preserving the contrast details of color image, which differs from the traditional Euclidean metric approaches. In the proposed model, motivated by the fact that Log-Euclidean metric has promising invariance properties such as inversion invariant and similarity invariant, we present a Log-Euclidean metric-based maximum function to model the decolorization procedure. The Gaussian-like penalty function consisting of the Log-Euclidean metric between gradients of the input color image and transformed grayscale image is incorporated to better reflect the degree of preserving feature discriminability and color ordering in color-to-gray conversion. A discrete searching algorithm is employed to solve the proposed model with linear parametric and non-negative constraints. Extensive evaluation experiments show that the proposed method outperforms the state-of-the-art methods both quantitatively and qualitatively.

This study considers the phase noise filtering problem for interferometric phase image using sparse optimisation technique. Since the original model can be formulated as a rank minimisation problem, it is difficult to solve. One appealing approach is to use a nuclear norm (NN) regularisation to relax the rank regulariser. However, the performance of such approach is not satisfying. In this study, the authors propose to use reweighted NN regularisation to approximate the rank regulariser, which leads to the low‐rank reformulation. Though this reformulation is non‐convex, a new algorithm termed as spatially adaptive iterative weighted singular‐value thresholding algorithm is proposed to effectively solve it. Specifically, the weight and image variables are updated alternatively by block coordinate descent iteration scheme. In addition, the corresponding computational complexity of the algorithm has been established. Simulation results based on simulated and measured data show that this new phase noise reduction method has much better performance than several existing phase filtering methods.

High-dimensional MR imaging often requires long data acquisition time, thereby limiting its practical applications. This paper presents a low-rank tensor based method for accelerated high-dimensional MR imaging using sparse sampling. This method represents high-dimensional images as low-rank tensors (or partially separable functions) and uses this mathematical structure for sparse sampling of the data space and for image reconstruction from highly undersampled data. More specifically, the proposed method acquires two datasets with complementary sampling patterns, one for subspace estimation and the other for image reconstruction; image reconstruction from highly undersampled data is accomplished by fitting the measured data with a sparsity constraint on the core tensor and a group sparsity constraint on the spatial coefficients jointly using the alternating direction method of multipliers. The usefulness of the proposed method is demonstrated in MRI applications; it may also have applications beyond MRI.

This paper presents a novel gradient correlation similarity (Gcs) measure-based decolorization model for faithfully preserving the appearance of the original color image. Contrary to the conventional data-fidelity term consisting of gradient error-norm-based measures, the newly defined Gcs measure calculates the summation of the gradient correlation between each channel of the color image and the transformed grayscale image. Two efficient algorithms are developed to solve the proposed model. On one hand, due to the highly nonlinear nature of Gcs measure, a solver consisting of the augmented Lagrangian and alternating direction method is adopted to deal with its approximated linear parametric model. The presented algorithm exhibits excellent iterative convergence and attains superior performance. On the other hand, a discrete searching solver is proposed by determining the solution with the minimum function value from the linear parametric model-induced candidate images. The non-iterative solver has advantages in simplicity and speed with only several simple arithmetic operations, leading to real-time computational speed. In addition, it is very robust with respect to the parameter and candidates. Extensive experiments under a variety of test images and a comprehensive evaluation against existing state-of-the-art methods consistently demonstrate the potential of the proposed model and algorithms.

The combination of sparse coding and manifold learning has received much attention recently. However, the computational complexity of the resulting optimization problem hinders its practical application. In this paper, an augmented Lagrangian method is proposed to address this issue, which first transforms the unconstrained problem to an equivalent constrained problem and then an alternating direction method is used to iteratively solve the subproblems. Experimental results validate the effectiveness of the propose algorithm.

This paper presents a novel structure gradient and texture decor relating regularization (SGTD) for image decomposition. The motivation of the idea is under the assumption that the structure gradient and texture components should be properly decor related for a successful decomposition. The proposed model consists of the data fidelity term, total variation regularization and the SGTD regularization. An augmented Lagrangian method is proposed to address this optimization issue, by first transforming the unconstrained problem to an equivalent constrained problem and then applying an alternating direction method to iteratively solve the sub problems. Experimental results demonstrate that the proposed method presents better or comparable performance as state-of-the-art methods do.

Initialization sensitivity usually occurs in dictionary learning algorithm for image decomposition. In this paper, we propose an adaptive dictionary learning algorithm by promoting structural incoherence at the stage of dictionary updating. The structural incoherence based dictionary learning (SIDL) method guides the cartoon and texture parts to be more properly represented by two incoherent dictionaries. The resulting minimization is approximately addressed by majorization-minimization (MM) technique. Experimental results demonstrate that the dictionaries generated by SIDL can better describe different morphological contents and subsequently the cartoon and texture components are better separated, in terms of visual comparisons and quantitative measures.

This paper presents an efficient alternating direction method with patch-based dictionary updating, ADMDU-DEB, for sparse representation regularization framework of image deblurring. The main idea of the proposed method is to reformulate the variational problem as a linear equality constrained problem and then minimize its augmented Lagrangian function. The alternating direction method decouples the minimization by alternately iterating the pixel-based regularization and the patch-based sparse representation. Typically, accelerated sparse coding and simple dictionary updating applied in the sparse representation stage enable the whole algorithm to converge at a relatively small number of iterations. Additionally, the approach is readily extended to solve the same kind of variational problem with a nonnegativity constraint. Experimental results on benchmark test images consistently validate the superiority of the proposed approach and demonstrate that it achieves very competitive deblurring performance, compared with state-of-the-art deconvolution algorithms.

Image recovery from undersampled data has always been challenging due to its implicit ill-posed nature but becomes fascinating with the emerging compressed sensing (CS) theory. This paper proposes a novel gradient based dictionary learning method for image recovery, which effectively integrates the popular total variation (TV) and dictionary learning technique into the same framework. Specifically, we first train dictionaries from the horizontal and vertical gradients of the image and then reconstruct the desired image using the sparse representations of both derivatives. The proposed method enables local features in the gradient images to be captured effectively, and can be viewed as an adaptive extension of the TV regularization. The results of various experiments on MR images consistently demonstrate that the proposed algorithm efficiently recovers images and presents advantages over the current leading CS reconstruction approaches.

Dictionary learning based sparse modeling has been increasingly recognized as providing high performance in the restoration of noisy images. Although a number of dictionary learning algorithms have been developed, most of them attack this learning problem in its primal form, with little effort being devoted to exploring the advantage of solving this problem in a dual space. In this paper, a novel Fenchel duality based dictionary learning (FD-DL) algorithm has been proposed for the restoration of noise-corrupted images. With the restricted attention to the additive white Gaussian noise, the sparse image representation is formulated as an 2-1 minimization problem, whose dual formulation is constructed using a generalization of Fenchel’s duality theorem and solved under the augmented Lagrangian framework. The proposed algorithm has been compared with four state-of-the-art algorithms, including the local pixel grouping-principal component analysis, method of optimal directions, K-singular value decomposition, and beta process factor analysis, on grayscale natural images. Our results demonstrate that the FD-DL algorithm can effectively improve the image quality and its noisy image restoration ability is comparable or even superior to the abilities of the other four widely-used algorithms.

In recent years Bregman iterative method (or related augmented Lagrangian method) has shown to be an efficient optimization technique for various inverse problems. In this paper, we propose a two-level Bregman Method with dictionary updating for highly undersampled magnetic resonance (MR) image reconstruction. The outer-level Bregman iterative procedure enforces the sampled k-space data constraints, while the inner-level Bregman method devotes to updating dictionary and sparse representation of small overlapping image patches, emphasizing local structure adaptively. Modified sparse coding stage and simple dictionary updating stage applied in the inner minimization make the whole algorithm converge in a relatively small number of iterations, and enable accurate MR image reconstruction from highly undersampled k-space data. Experimental results on both simulated MR images and real MR data consistently demonstrate that the proposed algorithm can efficiently reconstruct MR images and present advantages over the current state-of-the-art reconstruction approach.