Vision Transformers have achieved impressive performance in the field of low-light image enhancement. Some Transformer-based methods acquire attention maps within channel dimension, whereas the spatial resolutions of queries and keys involved in matrix multiplication are much larger than the dimensions of channels. During the key-query dot-product interaction to generate attention maps, massive information redundancy and expensive computational costs are incurred. Simultaneously, most previous feed-forward networks in Transformers do not model the multi-range information that plays an important role for feature reconstruction. Based on the above observations, we propose an effective Multi-Scale Downsampling-Based Transformer (MDFormer) for low-light image enhancement, which consists of multi-scale downsampling-based self-attention (MDSA) and multi-range gated extraction block (MGEB). MDSA employs downsampling with two different factors for queries and keys to save the computational cost when implementing self-attention operations within channel dimension. Furthermore, we introduce learnable parameters for the two generated attention maps to adjust the weights for fusion, which allows MDSA to adaptively retain the most significant attention scores from attention maps. The proposed MGEB captures multi-range information by virtue of the multi-scale depth-wise convolutions and dilated convolutions, to enhance modeling capabilities. Extensive experiments on four challenging low-light image enhancement datasets demonstrate that our method outperforms the state-of-the-art.

Object detection approaches are expanding by leaps and bounds with recent progress in deep learning. However, there is a considerable amount of environments hampering and challenging generic detectors in open-world scenarios, which received quite limited attention. In this paper, we focus on three specific challenging conditions: 1) targets presented with low lightness, 2) camouflaged objects merged in backgrounds, 3) complex acquisition scenarios, and present a novel end-to-end detector accordingly, termed Context-awareness Network (CANet). Specifically, we propose Global Context Encoder and Context Feature Fusion module to model the context-awareness (CA) mechanism that plays a crucial role in the human visual system (HVS) in an explicit way, which integrates both latent global and local context information to make each region of interest (RoI) more informative, and thus more discriminative. To our knowledge, such high-level mechanisms are under-explored for object detection in the literature. In addition, Global Semantic Awareness module is designed to regress positions and classify better in the process of extracting the feature. Experiments demonstrate that CANet achieves very competitive performance on the ExDark, DARK FACE, COD10K, and CURE-TSD, suggesting the effectiveness and efficiency of CANet in various challenging conditions as well as common scenarios.

In this paper, we propose a simple but effective illumination distribution prior (IDP) for images to illuminate the darkness. The illumination distribution prior is the product of a statistical approach to low-light images. It is based on a key factor - the mean value and standard deviation of images are positively correlated with the illumination. Using IDP in combination with the dual-domain feature fusion network (DFFN), we can obtain images that are more consistent with the ground truth distribution. DFFN inserts the discrete wavelet transform (DWT) into the transformer architecture, aiming to recover the detailed texture of the image through local high-frequency information and global spatial information. We have conducted extensive experiments on five widely used low-light image enhancement datasets and the experimental results show the superior performance of our proposed network (IDP-Net) compared to other state-of-the-art methods.

Estimating the illuminant from a color-biased image is an ill-posed problem without prior information or invariance about the surfaces of a scene. Based on the classical image formation model, we have developed a heuristic approach to obtain the illuminant color by computing the ratio of the average of all pixels over the color-biased scene to that over the roughly recovered scene obtained by local normalization in each channel. The computed ratio represents an estimated invariance across color channels (IACC), ranging between the average reflectance of surfaces and the maximum reflectance of surfaces of a scene, modulated by the illuminant color. This work builds a mathematical foundation for IACC and explains why it is suitable for illuminant estimation. The core discovery is that the magnitude relationship of the average reflectances of surfaces between any two color channels is opposite to that of the maximum reflectances of surfaces for most natural scenes. As a result, we have designed two approaches for explicitly learning IACC of an image, resulting in very accurate illuminant estimation. The first approach involves a novel shallow model based on diagonal or non-diagonal matrices, together with the learned model parameters, to improve IACC performance. The second approach applies IACC as a constraint to optimize a novel deep learning approach, which has achieved state-of-the-art performance on two benchmarks. An interesting finding is that the output of the learned network, constrained only by IACC loss, provides a coarse estimation of intrinsic images such as albedo from the input color-biased image

Regularization by denoising (RED) framework has shown impressive performance for many imaging inverse problems, by leveraging the denoising method in defining an explicit regularization. In this letter, we propose a novel SLN-RED scheme for image restoration by exploiting the local and nonlocal denoisers simultaneously. Theoretically, we proves that forboundeddenoisers, the SLN-RED under ADMM scheme with a continuation strategy converges to a fixed-point. Numerical experiments on deblurring and super-resolution tasks demonstrate promising performance of the proposed algorithm.

Previous work has demonstrated that incorrect white balance (WB) in the camera image signal processing pipeline has a negative impact on the performance of deep neural networks (DNNs) in high-level vision tasks, and traditional image augmentation approaches are not well suited for modeling WB errors. However, it is still unclear when this impact will occur for which kinds of images and objects. In this paper, we manually labeled 2304 images from the RECommended dataset and NUS dataset and discovered that the effect of WB on DNNs is greatly associated with object size and occlusion level among objects. In images with incorrect WB, small objects and objects with heavily occluded backgrounds are the main factors resulting in the bad performance of DNNs, indicating that the effect of WB is clearly associated with the shape of objects. Our findings may support that the functional role of some neurons in the visual cortex (e.g., V1 or V4 areas) realizing color constancy (CC) and encoding object attributes such as color and shape dependently is to contribute to high-level vision. Furthermore, based on this scientific finding, we proposed a novel augmentation strategy to address the negative impact of incorrect WB by expanding the training datasets in both color transformation and synthetic occlusion. We compared our proposed strategy with the current augmentation strategies and showed that our approach clearly improves the performance of DNNs in detection and segmentation tasks with small objects and objects with heavily occluded backgrounds.

Wavelet tight frames have been actively investigated for various image restoration problems. In this paper, we introduce an analysis-sparsity model via$\ell _2$-relaxed truncated$\ell _0$regularization and nonlocal estimation, and the resulted nonconvex minimization problem is tackled by a proximal alternating minimization strategy. Numerical experiments demonstrate that the proposed algorithm is superior to many popular methods in both objective and perceptual quality.

Edge‐preserving image smoothing aims at maintaining the fundamental constituents, i.e. salient edges of a given image, while removing the noise and insignificant details in the meantime. It is often employed at the pre‐processing step of many image processing tasks, including reconstruction, segmentation, recognition and three‐dimensional content generation, to name just a few. Recently, a sparse gradient counting scheme in an optimisation framework has attracted much attention, and it confines the discrete number of intensity changes among neighbouring pixels. This ‐regularised sparsity pursuit scheme performs favourably in a global optimisation manner. However, it often achieves unsatisfactory performance at diminishing trivial details and at smoothing discrete regions. In this study, a new image smoothing scheme with truncated regularisation is proposed, which is especially effective for sharpening critical edges. For objective evaluation of the smoothing performance, images are linearly quantised into several layers to generate the experimental images, then these quantised images are smoothed using several methods for reconstructing the smoothly changed shape and intensity of the original images. Compared with the smoothing scheme, extensive experimental results demonstrate that the proposed method performs much better at preserving main structures and removing trivial details.

The wavelet frame systems have been extensively studied due to their capability of sparsely approximating piece-wise smooth functions, such as images, and the corresponding wavelet frame-based image restoration models are mostly based on the penalization of the l1 norm of wavelet frame coefficients for sparsity enforcement. In this paper, we focus on the image inpainting problem based on the wavelet frame, propose a weighted sparse restoration model, and develop a corresponding efficient algorithm. The new algorithm combines the idea of iterative support detection method, first proposed by Wang and Yin for sparse signal reconstruction, and the split Bregman method for wavelet frame l1 model of image inpainting, and more important, naturally makes use of the specific multilevel structure of the wavelet frame coefficients to enhance the recovery quality. This new algorithm can be considered as the incorporation of prior structural information of the wavelet frame coefficients into the traditional l1 model. Our numerical experiments show that the proposed method is superior to the original split Bregman method for wavelet frame-based l1 norm image inpainting model as well as some typical l(p) (0 ≤ p < 1) norm-based nonconvex algorithms such as mean doubly augmented Lagrangian method, in terms of better preservation of sharp edges, due to their failing to make use of the structure of the wavelet frame coefficients.