Recently, incorporating Retinex theory with unfolding networks has attracted increasing attention in the low-light image enhancement field. However, existing methods have two limitations, i.e., ignoring the modeling of the physical prior of Retinex theory and relying on a large amount of paired data. To advance this field, we propose a novel self-supervised unfolding network, named S2UNet, for the LIE task. Specifically, we formulate a novel optimization model based on the principle that content-consistent images under different illumination should share the same reflectance. The model simultaneously decomposes two illumination-different images into a shared reflectance component and two independent illumination components. Due to the absence of the normal-light image, we process the low-light image with gamma correction to create the illumination-different image pair. Then, we translate this model into a multi-stage unfolding network, in which each stage alternately optimizes the shared reflectance component and the respective illumination components of the two images. During progressive multi-stage optimization, the network inherently encodes the reflectance consistency prior by jointly estimating an optimal reflectance across varying illumination conditions. Finally, considering the presence of noise in low-light images and to suppress noise amplification, we propose a self-supervised denoising mechanism. Extensive experiments on nine benchmark datasets demonstrate that our proposed S2UNet outperforms state-of-the-art unsupervised methods in terms of both quantitative metrics and visual quality, while achieving competitive performance compared to supervised methods. The source code will be available at https://github.com/J-Liu-DL/S2UNet.

In the field of compressed sensing magnetic resonance imaging (CS-MRI), deep unfolding networks (DUNs) achieve high interpretability and superior performance. However, existing DUN-based methods often treat different components of the MR image uniformly without considering their respective unique characteristics, leading to insufficient detail capture and suboptimal performance. To address this issue, we propose a Dual-BrancH Deep Unfolding Network (DBH-Net), which employs parallel under-complete (UC) and over-complete (OC) branches to alternately reconstruct different components from the under-sampled MR image. The UC branch focuses on extracting low-frequency features by expanding the receptive field, while the OC branch emphasizes high-frequency features by restricting the receptive field. Besides the independent descriptive abilities of dual-branch, the unique characteristics of DUN facilitate a tighter integration between the two branches. Additionally, we introduce an Auxiliary Information Fusion Block (AIFB) to transfer multi-channel auxiliary information between stages, effectively reducing information loss. Extensive experiments on three datasets demonstrate that our proposed DBH-Net outperforms existing state-of-the-art methods.

Despite the effectiveness of single-photon emission computed tomography (SPECT) imaging in clinics, the ionizing radiation induced by its radiotracer poses a potential hazard to human health. Clinically, a lower radiation dose can be achieved by reducing the activity of administered radiotracer, which inevitably leads to increased Poisson noise, severe artifacts and degraded spatial resolution in the sinogram domain. Although existing sinogram restoration methods for the low-dose scenario have made significant progress in noise suppression, they still fail to effectively recover the detailed sinusoidal features and intrinsic contrast within the sinogram. In addition, existing methods seldom explore the sinogram innerstructure, which may hinder further improvement on reconstructed image quality. To address these issues, we propose a residual framework based on the diffusion model that leverages the frequency characteristics of sinograms. Indeed, the proposed framework consists of two stages. The first stage employs the Residual Denoising Diffusion Model (RDDM) to denoise the low-dose sinogram, thereby producing a noise-suppressed coarse output. In the second stage, we develop a novel Wavelet-based Sinogram Structure Interaction network (WSSI-net) to explicitly and selectively process high- and low-frequency features of the coarse output. In particular, we propose a High-Frequency Restoration Module (HFRM) to further enhance high-frequency features, as well as a low-frequency Graph Convolution block (GC block) to effectively exploit the inherent inner-structure within low-frequency components. Moreover, we further introduce a Cross-Frequency Interaction Module (CFIM) to achieve correlation learning between high- and low-frequency features. Extensive experiments demonstrate that the proposed framework achieves superior reconstruction performance compared to the state-of-the-art methods.

Low-light image enhancement (LIE) aims to restore images taken under poor lighting conditions, thereby extracting more information and details to robustly support subsequent visual tasks. While past deep learning (DL)-based techniques have achieved certain restoration effects, these existing methods treat all samples equally, ignoring the fact that difficult samples may be detrimental to the network's convergence at the initial training stages of network training. In this paper, we introduce a self-paced learning (SPL)-based LIE method named SPNet, which consists of three key components: the feature extraction module (FEM), the low-light image decomposition module (LIDM), and a pre-trained denoise module. Specifically, for a given low-light image, we first input the image, its pseudo-reference image, and its histogram-equalized version into the FEM to obtain preliminary features. Second, to avoid ambiguities during the early stages of training, these features are then adaptively fused via an SPL strategy and processed for retinex decomposition via LIDM. Third, we enhance the network performance by constraining the gradient prior relationship between the illumination components of the images. Finally, a pre-trained denoise module reduces noise inherent in LIE. Extensive experiments on nine public datasets reveal that the proposed SPNet outperforms eight state-of-the-art DL-based methods in both qualitative and quantitative evaluations and outperforms three conventional methods in quantitative assessments.

Camouflaged object detection (COD) aims to segment targeted objects that have similar colors, textures, or shapes to their background environment. Due to the limited ability in distinguishing highly similar patterns, existing COD methods usually produce inaccurate predictions, especially around the boundary areas, when coping with complex scenes. This paper proposes a Progressive Region-to-Boundary Exploration Network (PRBE-Net) to accurately detect camouflaged objects. PRBE-Net follows an encoder-decoder framework and includes three key modules. Specifically, firstly, both high-level and low-level features of the encoder are integrated by a region and boundary exploration module to explore their complementary information for extracting the object's coarse region and fine boundary cues simultaneously. Secondly, taking the region cues as the guidance information, a Region Enhancement (RE) module is used to adaptively localize and enhance the region information at each layer of the encoder. Subsequently, considering that camouflaged objects usually have blurry boundaries, a Boundary Refinement (BR) decoder is used after the RE module to better detect the boundary areas with the assistance of boundary cues. Through top-down deep supervision, PRBE-Net can progressively refine the prediction. Extensive experiments on four datasets indicate that our PRBE-Net achieves superior results over 21 state-of-the-art COD methods. Additionally, it also shows good results on polyp segmentation, a COD-related task in the medical field.

Low-light image enhancement (LIE) is important for many high-level vision tasks as the poor visibility of underexposed images can severely degrade the performance of the subsequent image recognition, analysis, etc. Although recent deep-learning-based LIE methods exhibit promising performance, most of them require a large number of paired training images, thereby limiting the practicability to real scenarios. In this paper, we propose a pseudo-supervised LIE method with the integration of mutual learning. Specifically, for the given low-light image, we first use a quadratic curve to generate a pseudo-clear image, which is served as the auxiliary ground truth for supervision, then the pseudo-paired images are simultaneously input to two parallel homogeneous branches to learn the expected enhanced result through the knowledge distillation of two branches via mutual learning. As both the generated image and the input low-light image underlies the desired solution, the mutual learning strategy enables the two branches learn from each other and produce the final results. Extensive experiments demonstrate that the proposed method outperforms most existing unsupervised LIE methods in terms of both qualitative and quantitative evaluations, and also achieves competitive performance against many supervised and semi-supervised methods.

Data-driven machine learning is increasingly involved in human life and industrial development due to its large-scale testing and low time cost. However, existing learning algorithms are not suitable for real-world applications with data dilemmas, such as extremely high-dimension-low-sample-size problems, non-Gaussian noise, and uncertainty. In this article, we propose a novel fuzzy multikernel subspace learning (FMKSL) to address these problems, which provides a robust multikernel representation with a fuzzy constraint and sparse coding. We then develop an adaptive learner chain optimization method based on the iterative process of FMKSL to speed up learning and achieve the best performance. Different from previous methods, we also design a flexible data augmentation method, namely generalized correntropy-based adaptive data augmentation (GC-ADA), to effectively use the$\alpha$-order statistics between samples to transform the exact value prediction task into a simpler classification one. It is important that our general framework only needs an extremely small dataset to predict the related ranking of the sample since the exact label value measured by different institutions in reality varies largely. A typical scenario is the drug screening task, i.e., the inhibitory potency prediction of the nicotinamide phosphoribosyltransferase inhibitors. Extensive experiments on nine real-world datasets (four tasks) show that our framework outperforms state-of-the-art methods in prioritizing candidate samples and chemicals for experimental research and analysis via a data-driven computational approach.

Perceptual image quality is related to content and distortion. Distortion classification is a common way to learn distortion information. How to extract distortion information consistent with human perception is a problem to be solved. Besides, the joint effect on image quality caused by the interplay of content and distortion has not been fully studied. In this paper, a novel Content Distortion Interaction Network (CDINet) is proposed for blind image quality assessment. Distortion representation are guided by content representation to learn quality-aware representation. CDINet consists of four components: a Distortion-Aware Module (DAM), a Content-Aware Module (CAM), an Asymmetric Content-Distortion Interaction (ACDI) module, and a quality regression module. The content representation and distortion representation are extracted respectively and fused interactively in CDINet. Specifically, with the assistance of image restoration, distortion representation consistent with human perception is learned. To further improve the ability in distortion representation, the DAM is used to construct the differences between the distorted image and its reference image. The proposed ACDI module enables the interaction of content and distortion representations to occur at different levels with less computational cost. Since the proposed CDINet considers the joint impact on image quality caused by the interplay of content and distortion, the predicted image qualities highly align with human perception. Comprehensive experiments on 8 benchmark datasets demonstrate that the proposed CDINet effectively extracts quality-aware representation, achieving state-of-the-art performance in evaluating both synthetically and authentically distorted images.

For human beings, there is a natural preference for judging the relative quality rather than directly predicting the quality score of an image. Based on this view, we propose an image quality difference learning network (IQDLNet) for evaluating image quality in a no-reference manner. Specifically, the proposed IQDLNet consists of a quality difference-aware network (QDAN) and a quality assessment network (QAN). The QDAN aims to predict the score difference between two randomly matched images and the QAN aims to predict the quality score of these two images. To further enhance the mutual understanding of image semantics, a semantic interaction module (SIM) is proposed with a dual regressor set up to carry out competitive learning in combination with the quality difference-aware feature. Experimental results on five IQA datasets demonstrate the superior performance of the proposed method over eight state-of-the-arts.

Extensive traffic flow data are received from the loop detector networks every second, which requires us to develop an effective and efficient algorithm to predict future traffic flow. However, dynamic traffic conditions on a road are not just influenced by sequential patterns in the temporal dimension, but also by other roadways in the spatial dimension. Although many successful models have been developed in previous studies to forecast future traffic flows, most of them have shortcomings in modeling spatial and temporal dependencies. In this article, we focus on spatial-temporal factors and propose a new adaptive spatial-temporal transformer graph network (ASTTGN) to improve the accuracy of traffic forecasting by jointly modeling the spatial-temporal information of road networks. Specifically, we propose an adaptive spatial-temporal transformer module, which contains two developed adaptive transformer modules for capturing dynamic spatial dependence and temporal dependence across multiple time steps, respectively. Finally, feature fusion is performed through a gated feature aggregation layer to simulate the effect of complex spatial-temporal factors on traffic conditions. In particular, the multihead attention mechanism employed by the transformer can effectively explore the potential spatial-temporal dependence patterns in different subspaces. Experimental results on two real-world traffic data sets demonstrate the superiority of the proposed model compared to existing techniques.

Multi-contrast magnetic resonance imaging (MRI) is widely used in clinical diagnosis. However, it is time-consuming to obtain MR data of multi-contrasts and the long scanning time may bring unexpected physiological motion artifacts. To obtain MR images of higher quality within limited acquisition time, we propose an effective model to reconstruct images from under-sampled k-space data of one contrast by utilizing another fully-sampled contrast of the same anatomy. Specifically, multiple contrasts from the same anatomical section exhibit similar structures. Enlightened by the fact that co-support of an image provides an appropriate characterization of morphological structures, we develop a similarity regularization of the co-supports across multi-contrasts. In this case, the guided MRI reconstruction problem is naturally formulated as a mixed integer optimization model consisting of three terms, the data fidelity of k-space, smoothness-enforcing regularization, and co-support regularization. An effective algorithm is developed to solve this minimization model alternatively. In the numerical experiments, T2-weighted images are used as the guidance to reconstruct T1-weighted/T2-weighted-Fluid-Attenuated Inversion Recovery (T2-FLAIR) images and PD-weighted images are used as the guidance to reconstruct PDFS-weighted images, respectively, from their under-sampled k-space data. The experimental results demonstrate that the proposed model outperforms other state-of-the-art multi-contrast MRI reconstruction methods in terms of both quantitative metrics and visual performance at various sampling ratios.

Owing to the rapid development of deep networks, single image deraining tasks have achieved significant progress. Various architectures have been designed to recursively or directly remove rain, and most rain streaks can be removed by existing deraining methods. However, many of them cause a loss of details during deraining, resulting in visual artifacts. To resolve the detail-losing issue, we propose a novel unrolling rain-guided detail recovery network (URDRN) for single image deraining based on the observation that the most degraded areas of the background image tend to be the most rain-corrupted regions. Furthermore, to address the problem that most existing deep-learning-based methods trivialize the observation model and simply learn an end-to-end mapping, the proposed URDRN unrolls the single image deraining task into two subproblems: rain extraction and detail recovery. Specifically, first, a context aggregation attention network is introduced to effectively extract rain streaks, and then, a rain attention map is generated as an indicator to guide the detail-recovery process. For a detail-recovery sub-network, with the guidance of the rain attention map, a simple encoder–decoder model is sufficient to recover the lost details. Experiments on several well-known benchmark datasets show that the proposed approach can achieve a competitive performance in comparison with other state-of-the-art methods.

In this paper, we propose a cross-channel, cross-scale, and cross-stage network (C3Net) for single image super-resolution, which effectively shares the features learned from multiple channels, multiple scales, and multiple stages. Multi-scale spatial features are extracted in each stage in an encoder-decoder fashion. The channel attention is performed after each encoder to exploit the inter-channel dependencies. After that, we design a cross-stage and cross-scale feature sharing module to accelerate the feature sharing across different scales and different stages. The whole network is optimized by multiple similar stages to reduce the number of parameters. Finally, super-resolution images of multiple resolutions are reconstructed simultaneously. We evaluate the proposed network on four benchmark datasets by comparing it with eleven state-of-the-art methods. Comprehensive experiments show the proposed network outperforms state-of-the-art methods by fewer parameters. The source code is available at https://github.com/thinkerww/SR_Version.

Abstract Compared with video de‐raining, single image de‐raining is more technically difficult due to the lack of temporally redundant information. This paper proposes a new two‐stage method for single image de‐raining. In the first stage, the authors develop an effective two‐step model to detect the rain streaks by taking pixel intensity, and direction of rain streaks as priors. In the second stage, the rain repair process is performed at the patch level. The authors first define a way to search for similar patches of each patch, and then group the similar patches together to form a matrix. Finally, a low‐rank matrix completion technique is utilized to recover the rain‐stained pixels based on the rain map obtained from the first stage. Compared with several state‐of‐the‐art methods, authors' proposed method is competitive in terms of the abilities of removing rain streaks, and preserving image details.

Abstract In this paper, a novel method with good numerical stability is proposed from the perspective of energy preserving to alleviate the numerical dissipations in the advection step of Eulerian fluid simulation. The main idea is to measure the vorticity loss during advection, calculate the lost angular kinetic energy with a proposed scheme, and then synthesize a high‐frequency incompressible details field to compensate the lost energy in a way that is consistent with Kolmogorov's theory, which prevents the synthetic details from interfering with the existing fluid flow. The method works independently of the advection scheme and can be easily combined with other advection schemes to enhance the effect. It adds only 5% to 10% of the computational overhead while producing convincing fluid details without changing the overall behavior of the original flow.

Visual distortions on images caused by bad weather conditions can have a negative impact on the performance of many outdoor vision systems. One often seen bad weather is rain which causes significant yet complex local intensity fluctuations in images. The paper aims at developing an effective algorithm to remove visual effects of rain from a single rainy image, i.e. separate the rain layer and the de-rained image layer from an rainy image. Built upon a non-linear generative model of rainy image, namely screen blend mode, we proposed a dictionary learning based algorithm for single image de-raining. The basic idea is to sparsely approximate the patches of two layers by very high discriminative codes over a learned dictionary with strong mutual exclusivity property. Such discriminative sparse codes lead to accurate separation of two layers from their non-linear composite. The experiments showed that the proposed method outperformed the existing single image de-raining methods on tested rain images.

Based on the concept of lacunarity in fractal geometry, we developed a statistical approach to texture description, which yields highly discriminative feature with strong robustness to a wide range of transformations, including pho- tometric changes and geometric changes. The texture feature is constructed by concatenating the lacunarity-related parameters estimated from the multi-scale local binary patterns of image. Benefiting from the ability of lacunarity analysis to distinguish spatial patterns, our method is able to characterize the spatial distribution of local image structures from multiple scales. The proposed feature was applied to texture classification and has demonstrated excellent performance in comparison with several state-of-the- art approaches on four benchmark datasets.