Low-light image enhancement methods based on diffusion models, though effective in improving image quality, often overrely on noise sensitivity and neglect the reconstruction deviations due to the naive up- and down-sampling operations. To address this issue, we propose a novel diffusion model, MWT-Diff, which utilizes multi-layer wavelet transforms to replace up-and down-sampling based on convolutions for extracting high-order features of different scales while mitigating representation degradations. Specifically, MWT-Diff is based on the U-Net architecture; it encodes four local features after the frequency-based down-sampling at each layer and fuses the enhanced four components during the up-sampling process. Additionally, we incorporate global refinement branches to mitigate information loss and employ efficient soft gate aggregation for feature fusion and reconstruction. Extensive quantitative and qualitative experiments demonstrate that our model achieves state-of-the-art performance on benchmark datasets. Code is available at: https://github.com/lalalulao/MWT-Diff.

When restoring low-light images, most methods largely overlook the ambiguity due to dark noise and lack discrimination for region and shape representations, resulting in invalid feature enhancement. In this work, we propose a physically explainable and prior guidance model for low-light image enhancement, termed Dual-Conditional Guidance Sparse Diffusion (DCGSD). Specifically, we introduce an elaborately designed Luminance Structure Guidance Head, which can be easily plugged into the existing diffusion model to emphasize the value of the luminance and structural representation. Furthermore, for reliable noise analysis, we provide a novel Sparse Attention Enhancement Module that is adaptively empowered to exploit the most useful region-to-region dependencies. This dynamic selection makes the diffusion process from dense to sparse, thus improving the efficiency of the reasoning noise distributions. To avoid noise amplification, we further present a Skip Calibration Module, which can be used to refine the local neighborhood that contains noisy and structural information. Extensive experiments have been performed to verify the superiority of the proposed method. DCGSD shows that leveraging dual-conditional guidance can support the diffusion model to produce sharper and more realistic results.

Diffusion model has achieved excellent performance in natural image processing, which can learn the noise distribution by the degradation and restoration processes. However, the model is limited to Gaussian noises. Actually, Polarimetric Synthetic Aperture Radar(PolSAR) images have complex non-Gaussian speckle noises, for which the Gaussian diffusion model is difficult to learn their intrinsic statistical characteristics. In this paper, we propose a novel scattering mechanism inspired non-Gaussian diffusion model for PolSAR image classification. To better simulate the PolSAR speckle noise, a mixed noise distribution is defined for PolSAR covariance matrices by combining Gamma multiplicative and Gaussian additive noises. A non-Gaussian forward noising process is derived to degrade a clean PolSAR image to a noisy image by steps. Then, the U-net structure is trained to remove noises for each step, effectively extracting non-Gaussian statistical features. However, statistical features can only characterize the overall distribution of the dataset, which is insufficient to describe complicated individual objects; the original PolSAR data reflect the detailed scattering mechanism for individual pixels, which can provide complementary object information for classification. Therefore, a scattering-statistical joint learning network is further developed with a dual-branch architecture to enhance discrimination ability. In particular, a multiscale pyramid module and attention mechanism are designed to improve the ability of feature learning. Experimental results on five real PolSAR datasets demonstrate that the proposed method effectively captures edge details and preserves homogeneous regions for terrain classification, especially in heterogeneous regions.

Infrared and visible image fusion is currently an important research direction in the field of multimodal image fusion, which aims to utilize the complementary information between infrared images and visible images to generate a new image containing richer information. In recent years, many deep learning-based methods for infrared and visible image fusion have emerged.However, most of these approaches ignore the importance of semantic information in image fusion, resulting in the generation of fused images that do not perform well enough in human visual perception and advanced visual tasks.To address this problem, we propose a semantic prior knowledge-driven infrared and visible image fusion method. The method utilizes a pre-trained semantic segmentation model to acquire semantic information of infrared and visible images, and drives the fusion process of infrared and visible images through semantic feature perception module and semantic feature embedding module.Meanwhile, we divide the fused image into each category block and consider them as components, and utilize the regional semantic adversarial loss to enhance the adversarial network generation ability in different regions, thus improving the quality of the fused image.Through extensive experiments on widely used datasets, the results show that our approach outperforms current leading algorithms in both human eye visualization and advanced visual tasks.

Infrared and visible image fusion is an important multimodal image processing task that aims to enhance computer vision performance by effectively fusing infrared and visible images. Although in recent years, many deep learning-based methods for infrared and visible image fusion have emerged. Howeve, most of these methods ignore the important role of semantic information in image fusion. Therefore, this paper proposes a semantic priori guided infrared and visible image fusion network called SPGFusion. It uses an adversarial generative network framework based on semantic priors to guide the infrared and visible image fusion process by combining a semantic feature-aware module and semantic generative adversarial loss. Experimental results demonstrate that the SPG-Fusion method yields more visually appealing fusion results and outperform state-of-the-art image fusion algorithms in visual quality and quantitative evaluation. The source code is available at https://github.com/tianzhiya/SPGFusion.

Photographs taken in low-light conditions have a low signal-to-noise ratio and impaired visual quality. We observe that low-light images exhibit a lower signal-to-noise ratio, resulting in a mixture of fine details, textures, and noise, making it challenging to reconstruct small-scale textures in the image. Inspired by this observation, we propose a SNR-guided CNN-Transformer network for high frequency restoration during low light image enhancement. The proposed method first decomposes image into high-frequency and low frequency components by image decomposition module. The low-frequency image is processed by a trainable Low Frequency SNR Perception (LFSP) module, resulting in excellent denoising performance and generating SNR-enhanced images with clearer edge contours. Guided by the low-frequency SNR feature maps, the details and textures of the high-frequency components are enhanced using a combination of transformer networks and convolutional networks, thereby compensating the detail distortions in the high frequency components of the image. The subjective and objective experiments demonstrate that our proposed method outperforms existing approaches in terms of detail and structure preservation.

Dark Image Enhancement (DIE) aims to improve contrast and restore details for captured images under low illumination. Currently, traditional Transformer methods have achieved significant performance in the DIE problem; however, all-pairs correlation computation is redundant in learning key properties and restoring high-order representations. To alleviate this problem, we introduce a Channel-wise Guidance Sparse Transformer framework, namely CGSformer, which not only adaptively selects the key channel-wise representations through a threshold operator, but also keeps the most useful self-attention values for feature restoration guided by the selected information. Besides, we introduce a Bidirectional Gate Feed-Forward (BGFF) network to aggregate features to better facilitate high-quality image reconstruction. The experiments are conducted on representative datasets, showing that our CGSformer consistently achieves state-of-the-art performance on widely used benchmarks.

Deep learning networks can automatically acquire high-level semantic features for polarimetric SAR image classification, while it involves a blind learning procedure without explicit guidance. In contrast, sparse representation methods represent effective non-deep models with a robust mathematical mechanism serving as guidance. However, they can’t capture complex image features and semantic information. To address these issues, we propose a novel approach known as the CNN-enhanced Deep Sparse Representation Network (CE-DSRNet) for PolSAR image classification, which a Sparse Representation (SR) guided deep learning model. Initially, a sparse representation model is constructed for PolSAR images to capture essential features. Subsequently, to solve the sparse model, a Deep Sparse Representation Network (DSRNet) is devised by transforming the Soft Threshold Iterative (ISTA) optimization procedure into a network, enabling automatic learning of sparse coefficients as features. Finally, a CNN-enhanced DSRNet is introduced, integrating DSRNet with CNN to effectively extract deep semantic features and enhance classification accuracy. Experiments demonstrate the effectiveness of the proposed method compared to state-of-the-art approaches.

In the low light image enhancement, single exposure images contains a limited dynamic range, which hinders the restoration of contrast and texture. To address these problems, we propose a multi exposure generation and fusion method (MEGF) which simulates multi exposure images and perform feature fusion and enhancement on these images. First, we propose a Multi-Exposure Generation (MEG) method, which constructs the Gaussian Distribution for each exposure level based on multi exposure datasets. MEG can generate images with different exposure levels based on the constructed distribution. Then, the Perceptual Importance based Multi-Exposure Feature Enhancement (PIMEFE) block is developed to fuse the feature of generated multi exposure images using VGG-16 network. Before fusion, the generated images are input to Multi Scale Recursive Feature Enhancement (MSRFE) blocks and obtain the denoised and enhanced features. Finally, the fused feature are input to Curve Adjustment (CA) block for fine tuning and provide the color enhancement on fusion features. We propose the Multiple Exposure Recursive Fusion (MERF) block which estimates the adjusting factors for CA block. Experimental results demonstrate that our method outperforms other techniques in both subjective and objective evaluations on real and synthetic datasets.

The building change detection (BCD) task serves urban planning by monitoring land use. However, due to the complexity of remote-sensing images and high foreground–background similarity, it leads to inaccurate detection of building edge regions. Existing methods deal with this problem by fusing features of different layers. But the fusing operation cannot separate details information from the overall information of buildings, resulting in inaccurate detection of building edge area. To address the above challenges, we propose an iterative edge-enhancing framework (IEEF). The IEEF alleviates the building edge detection difficulty by densely implementing a detail semantic enhancement module (DSEM) in the decoding part. This module takes differential features between adjacent scales to explicitly represent the building edge information. Simultaneously, to deal with the class imbalance problem, a Density-Guided Sampling method dedicated to change detection is proposed to increase the proportion of positive samples during training. Our proposed method achieves state-of-the-art performance on the LEarning, VIsion and Remote sensing laboratory building Change Detection (LEVIR-CD) dataset and the Wuhan University (WHU) dataset and obtains accurate changed building edges.

For the remote sensing object detection task, two-stage networks are widely used due to their high accuracy. These networks roughly predict the proposal regions containing potential objects. It is assumed in these methods that the sizes of these regions are close to that of the corresponding real object. However, this assumption is not always true. Consequently, the detector is affected by the size-unfitting proposal regions. In this letter, a hierarchical information enhancing detector (HIE-Det) is advocated to deal with this issue. First, the important semantic reinjection (ISR) module is proposed to mitigate the lack of object semantics caused by the size-unfitting problem. Compared with the normal detectors, the ISR module increases the proportion of information on objects and improves the effectiveness of the detection model. Second, the object boundary enhancing (OBE) module is proposed to improve the robustness of the regression. The OBE module introduces the convolutional branch stacking multigranularity grids for the same proposal region. Multiple granularity levels improve the robustness of the model to the different degrees of proposal size unfitting. Finally, to evaluate the effectiveness of the HIE-Det on multiscale datasets in a balanced and effective manner, we propose the scale-modulating scores (S-scores), i.e., scale-modulating average precision (sAP) and scale-modulating average recall (sAR). Compared with the other comprehensive scores, the S-scores are rid of the sample amounts and give priority to weaker indices. Implementing the proposed HIE-Det, S-scores {sAP, sAR} are, respectively, improved from {17.3%, 29.7%} to {34.8%, 43.2%}, reaching the state-of-the-art performance on the HRRSD dataset. These experiments verify the effectiveness of the proposed HIE-Det.

Many remarkable works have been proposed to deal with distortions problems in image fusion to date. However, the spectral distortion and the spatial distortion cannot always be well addressed at the same time. To deal with this, we propose an Adaptive Feature Pyramid Network (AFPN) to efficiently embed an Adaptive Detail Injection (ADI) module at different scales. Feature-domain injection gains are proposed in the ADI module to adaptively modulate spatial information and guide a refined detail injection. Furthermore, we propose a texture loss function to further guide our model to learn detail perception in each band. Experiments on QuickBird and GaoFen-1 datasets show that our method achieves superior performance and produces visually pleasing fusion images. Our code is available at https://github.com/yisun98/AFPN.

In remotely sensed images, it is quite common to run into small objects, such as cars and small storage tanks. However, these small objects are quite easy to get ignored because of the positioning difficulty. Thus, small objects detection is very challenging for the remote sensing object detection task. In order to deal with this challenge, theoptimized low coupling network(OLCN) is proposed. First, alow coupling robust regression(LCRR) module improves the positioning accuracy to avoid small objects getting missed. Second, areceptive field optimizing layer(RFOL) is proposed to train better classifiers by providing more accurateregions of interest(RoIs). Experimental results on the public dataset HRRSD verify the effectiveness of the proposed OLCN. Small objects detection metric is improved from 5.70% of the baseline to 22.90% of the OLCN. Moreover, the proposed method has reached state-of-the-art performance on the HRRSD dataset.

In the multicategory object detection task of high-resolution remote sensing images, small objects are always difficult to detect. This happens because the influence of location deviation on small object detection is greater than on large object detection. The reason is that, with the same intersection decrease between a predicted box and a true box, Intersection over Union (IoU) of small objects drops more than those of large objects. In order to address this challenge, we propose a new localization model to improve the location accuracy of small objects. This model is composed of two parts. First, a global feature gating process is proposed to implement a channel attention mechanism on local feature learning. This process takes full advantages of global features’ abundant semantics and local features’ spatial details. In this case, more effective information is selected for small object detection. Second, an axis-concentrated prediction (ACP) process is adopted to project convolutional feature maps into different spatial directions, so as to avoid interference between coordinate axes and improve the location accuracy. Then, coordinate prediction is implemented with a regression layer using the learned object representation. In our experiments, we explore the relationship between the detection accuracy and the object scale, and the results show that the performance improvements of small objects are distinct using our method. Compared with the classical deep learning detection models, the proposed gated axis-concentrated localization network (GACL Net) has the characteristic of focusing on small objects.

Remote sensing image (RSI) classification is one of the most important fields in RSI processing. It is well known that RSIs are very complicated due to its various kinds of contents. Therefore, it is very difficult to distinguish different scene categories with similar visual contents, like desert and bare land. To address hard negative categories, an attribute-cooperated convolutional neural network (ACCNN) is proposed to exploit attributes as additional guiding information. First, the classification branch extracts convolutional neural network feature, which is then utilized to recognize the RSI scene categories. Second, the attribute branch is proposed to make the network distinguish scene categories efficiently. The proposed attribute branch shares feature extraction layers with the classification branch and makes the classification branch aware of extra attribute information. Finally, the relationship branch constraints the relationship between the classification branch and the attribute branch. To exploit the attribute information, three attribute-classification data sets are generated (AC-AID, AC-UCM, and AC-Sydney). Experimental results show that the proposed method is competitive to state-of-the-art methods. The data sets are available at https://github.com/CrazyStoneonRoad/Attribute-Cooperated-Classification-Data sets.

Object detection is a basic issue of very high-resolution remote sensing images (RSIs) for automatically labeling objects. At present, deep learning has gradually gained the competitive advantage for remote sensing object detection, especially based on convolutional neural networks (CNNs). Most of the existing methods use the global information in the fully connected feature vector and ignore the local information in the convolutional feature cubes. However, the local information can provide spatial information, which is helpful for accurate localization. In addition, there are variable factors, such as rotation and scaling, which affect the object detection accuracy in RSIs. In order to solve these problems, this paper presents a hierarchical robust CNN. First, multiscale convolutional features are extracted to represent the hierarchical spatial semantic information. Second, multiple fully connected layer features are stacked together so as to improve the rotation and scaling robustness. Experiments on two data sets have shown the effectiveness of our method. In addition, a large-scale high-resolution remote sensing object detection data set is established to make up for the current situation that the existing data set is insufficient or too small. The data set is available athttps://github.com/CrazyStoneonRoad/TGRS-HRRSD-Dataset.