Shadows are dark areas, typically rendering low illumination intensity. Admittedly, the infrared image can provide robust illumination cues that the visible image lacks, but existing methods ignore the collaboration between heterogeneous modalities. To fill this gap, we propose a weakly supervised shadow removal network with a spherical feature space, dubbed S2-ShadowNet, to explore the best of both worlds for visible and infrared modalities. Specifically, we employ a modal translation (visible-to-infrared) model to learn the cross-domain mapping, thus generating realistic infrared samples. Then, Swin Transformer is utilized to extract strong representational visible/infrared features. Simultaneously, the extracted features are mapped to the smooth spherical manifold, which alleviates the domain shift through regularization. Well-designed similarity loss and orthogonality loss are embedded into the spherical space, prompting the separation of private visible/infrared features and the alignment of shared visible/infrared features through constraints on both representation content and orientation. Such a manner encourages implicit reciprocity between modalities, thus providing a novel insight into shadow removal. Notably, ground truth is not available in practice, thus S2-ShadowNet is trained by cropping shadow and shadow-free patches from the shadow image itself, avoiding stereotypical and strict pair data acquisition. More importantly, we contribute a largescale weakly supervised shadow removal benchmark that makes shadow removal independent of specific scenario constraints possible. Extensive experiments demonstrate that S2-ShadowNet outperforms state-of-the-art methods in both qualitative and quantitative comparisons. The code and benchmark are available at https://github.com/chi-kaichen/S2-ShadowNet.

As a bridge between the moon and human perception, the lunar optical image reflects lunar topography, geology, and evolution. Unfortunately, the permanent shadow regions (PSRs) near the lunar poles suffer from information contamination due to insufficient illumination. Low-light enhancement is a subjective process whose target is tied to human visual perception. However, existing low-light enhancement methods often operate as opaque “black box”, lacking transparency and failing to accommodate diverse perceptual preferences. To this end, we explore a PSRs Low-Light Enhancer (PSRs-LLE) that treats low-light enhancement as a Markov decision process, thereby dynamically fitting perceptual preferences. Specifically, a deep Q network as an agent integrates multiple user-friendly attributes (e.g., brightness, contrast, chroma, and detail) through actions recursion (i.e., a candidate set of image enhancement operations). Such transparent and specific action sequences satisfy customization preferences of users while providing convincing interpretability, compared with the “black box” paradigm of deep learning. More importantly, a well-designed non-reference loss function liberates PSRs-LLE from the dilemma of virtual assumptions and paired data, which further enhances usability. Extensive experiments demonstrate that PSRs-LLE outperforms state-of-the-art methods in both qualitative and quantitative comparisons.

Shadow removal is an essential task for remote sensing imagery analysis, which is tricky due to spatial irregular and inhomogeneous degradation distribution. Unfortunately, current shadow removal pipelines face challenges with suboptimal performance and insufficient interpretability. To this end, we unleash the long-sequence modeling potential of State Space Models (SSMs) in the context of shadow removal. Coupled with the accurate perception of traditional Retinex decomposition towards illumination, the well-designed RSMamba enjoys the best of both worlds between superior competitiveness and theoretical intuitiveness. Specifically, RSMamba mimics the retina and cerebral cortex to explore illumination and reflectance. The former drives the selective scan mechanism to enhance the response towards contamination, while the latter serves as a tool to preserve illumination fidelity. In addition, contour and gradient regularizations of illumination and reflectance components reflect the spatial opponency of shadows, which are consistent with the center-surround opponent receptive field of the human visual system. Such a manner incorporates the domain knowledge of neurophysiological mechanisms into neural networks, providing new insights into shadow removal. Extensive experiments demonstrate that RSMamba outperforms state-of-the-art methods.

Remote sensing shadow removal task aims to effectively restore key regions of an image obscured by shadows. However, the spatial non-uniformity of shadow distribution presents significant challenges to this task. To address this issue, we propose RMMamba, a shadow removal network based on the SS2D architecture. The core concept of RMMamba involves balancing the spatial non-uniformity of shadow distribution, thereby optimizing the utilization efficiency of non-shadow pixel information across different windows. Specifically, RMMamba employs a random pixel shuffling operation to evenly disperse pixels from shadow regions with pronounced spatial non-uniformity into non-shadow areas, ensuring a more balanced spatial distribution of shadow and non-shadow pixels within each window. Subsequently, a shared weight local State Space Model (SS2D) is employed to integrate non-shadow pixel features uniformly distributed around shadow pixels, consequently effectively relighting shadow pixels. Reverse shuffling operations are then applied to restore the processed image to its original pixel order. Coupled with CP-FFN, a lightweight feedforward network incorporating color priors, RMMamba effectively restores color in shadow regions. More importantly, given the difficulty of acquiring remote sensing shadow samples with corresponding ground truth and shadow masks, we leverage the game GTA to control its shadow renderer and create SRGTA, a synthetic fully supervised dataset, hence providing a new benchmark for the performance evaluation of remote sensing shadow removal algorithms. Extensive experiments conducted on SRGTA and UAV-SC have demonstrated the outstanding performance of RMMamba. The code and SRGTA dataset are publicly available at https://github.com/xgd-cj/RMMamba.

Haze removal in remote sensing (RS) images has become increasingly vital due to their capacity to contain essential information for accurate geospatial analysis. Notably, this phenomenon is particularly pronounced in both spatial and spectrum distributions of buildings, complex terrain, and landforms. Inspired by the success of generative models in enhancing details incrementally and suppressing noise, we propose a deep frequency-guided denoising diffusion model for RS imagery dehazing. The pixel-level generative capability of the diffusion model is fully leveraged, and the fast Fourier transform is utilized to extract frequency-domain information. This enables the separate mining of semantic information from RS images in both spatial and spectral domains. Concurrently, the continuity of the image in the frequency domain is ensured without altering the diffusion process, thus achieving detail retention while improving overall clarity. Furthermore, to address the scarcity of physically realistic training data for spatially heterogeneous atmospheric degradation, we construct a Random Haze Distribution Dataset for Remote Sensing dehazing (RHDRS). RHDRS randomly simulates the spatial distribution and thickness of haze, containing 4,500 hazy images along with the corresponding ground truths. Experiments demonstrate that our approach outperforms existing state-of-the-art techniques. The dataset and the code can be accessed at https://github.com/Junjie-LLL/DFG-DDM.

Remote sensing shadow removal is an open issue. Previous studies focus on working in the spatial dimension, ignoring the potential of the Fourier dimension, while illumination degradation typically exists in the amplitude component. To address this limitation, our insight is a fresh dual-stage Fourier-based network (NeFour), which explores the best of both worlds between frequency and spatial information. In the frequency stage, we investigate the positive correlation between amplitude and brightness from channel and spatial statistics. Coupled with implicitly defined normalization, a controllable fitting amplitude transform map recreates the illumination. In the spatial stage, the inverted dark channel prior with 3-D coordinates serves as modulation matrices that naturally reveal the spatial distribution of shadows, thus elegantly eliminating shadow remnants. With ingenious design, NeFour achieves nontrivial performance against state-of-the-art shadow removal methods in terms of both visual perception and quantitative evaluation. The code is publicly available athttps://github.com/chi-kaichen/NeFour.

Change detection is a prevalent technique in remote sensing image analysis for investigating geomorphological evolution. The modeling and analysis of difference features are crucial for the precise detection of land cover changes. In order to extract difference features, previous work has either directly computed them through differential operations or implicitly modeled them via feature fusion. However, these rudimentary strategies rely heavily on a high degree of congruence within the bitemporal feature space, which results in the model’s diminished capacity to capture subtle variations induced by factors such as differences in illumination. In response to this challenge, the concept of 3-D neighborhood difference convolution (3D-NDC) is proposed for robustly aggregating the intensity and gradient information of features. Furthermore, to delve into the deep disparities within bitemporal instance features, we propose a novel paradigm for differential feature extraction based on 3D-NDC, termed 3-D neighborhood cross-differencing. This strategy is dedicated to exploring the interplay of cross-temporal features, thereby unveiling the inherent disparities among various land cover characteristics. In addition, a detail-focused refinement (DfR) decode based on the Laplace operator has been designed to synergize with the 3-D neighborhood cross-differencing, aiming to improve the detail performance of change instances. This integration forms the basis of a new change detection framework, named ChangeLN. Extensive experiments demonstrate that ChangeLN significantly outperforms other state-of-the-art change detection methods. Moreover, the 3-D neighborhood cross-difference strategy exhibits the potential for integration into other change detection frameworks to improve detection performance. Open code is available fromhttps://github.com/weiAI1996/3DNCD_ChangeLN.

Underwater images often suffer from severe color distortion due to the challenging imaging environment. Underwater image enhancement (UIE) techniques have been developed to recover clear images, laying the foundation for various underwater research. However, existing UIE methods tend to produce fixed results without considering individual preferences for different color tones. And there is no dataset with ground truth (GT) in different tones. Therefore, we came up with the possibility of using the currently popular multimodal methods to control the color tone of enhanced images. This article proposes a method for generating underwater enhanced images with cold, warm, and normal tones using multimodal information supervision (MM-UIE). First, we leverage the relationship between text prompts and images to supervise the generation of cold or warm images. In addition, we introduce a 6-D color operator, which not only enhances the tone control of underwater images but also serves as a bridge between different tone images. Finally, we also found that multimodal supervision methods can not only control the color tone of underwater images but also improve the quality of underwater image generation. Experimental results demonstrate the superior performance of our method compared to state-of-the-art (SOTA) techniques. Our codes will be publicly available athttps://github.com/perseveranceLX/MM-UIE.

Shadow removal from remote sensing images is still an open issue. Recently, deep network training on unpaired data is preferable since corresponding ground truths of shadow images are not available in practice. Nevertheless, unsupervised shadow removal research for remote sensing imagery is limited by the scarcity of publicly available benchmarks. This paper proposes an unsupervised progressive network (UP-ShadowGAN) to jointly learn decoupled features for shadow removal and color transfer. UP-ShadowGAN explores the mapping between shadow and shadow-free domains through adversarial learning and cycle consistency constraint. In particular, we employ progressive learning to decompose the overall mapping process into more manageable shadow removal and color transfer steps. Specifically, the realistic illumination is restored by propagating spatial context between shadow and shadow-free nodes. Coupled with a multi-color space aggregation strategy, diverse color space representations alleviate color deviation caused by spatial inconsistency. More importantly, we contribute the first unpaired remote sensing shadow removal dataset (URSSR), which encourages future exploration. Extensive experiments demonstrate that UP-ShadowGAN competes favorably with state-of-the-art methods. The dataset and code are available at https://github.com/chi-kaichen/UP-ShadowGAN.

The objective of Semantic Change Detection (SCD) is to discern intricate changes in land cover while simultaneously identifying their semantic categories. Prior research has shown that using multiple independent branches for the distinct tasks of change localization and semantic recognition is a reliable approach to solving the SCD problem. Nevertheless, conventional SCD architectures rely heavily on a high degree of consistency within the bi-temporal feature space when modeling difference features, inevitably resulting in false positives or missed alerts within change areas. In this paper, we introduce a SCD framework called the Cross-Differential Semantic Consistency (CdSC) network. CdSC is designed to mine deep discrepancies in bi-temporal instance features while preserving their semantic consistency. Specifically, the 3D-Cross-Difference module, incorporating 3D convolutions, explores the interaction of cross-temporal features, revealing inherent differences among various land features. Simultaneously, deep semantic representations are further utilized to enhance the local correlation of difference information, thereby improving the model’s discriminative capabilities within change regions. Incorporating principles from contrastive learning, a Semantic Co-Alignment loss is introduced to increase intra-class consistency and inter-class distinctiveness of dual-temporal semantic features, thereby addressing the challenges posed by semantic disparities. Extensive experiments on two SCD datasets demonstrate that CdSC outperforms other state-of-the-art SCD methods significantly in both qualitative and quantitative evaluations. The code and dataset are available at https://github.com/weiAI1996/CdSC.

Since light is scattered and absorbed by water, underwater images have inherent degradation (e.g., hazing, color shift), consequently impeding the development of remotely operated vehicles (ROVs). Toward this end, we propose a novel method, referred to as${B}$est${o}\text{f}$Bo th World${s}$(Boths). With parameters of only 0.0064 M, Boths can be considered a super lightweight neural network for underwater image enhancement. On the whole, it has three levels: structure and detail features; pixel and channel dimensions; high- and low-frequency information. Each of these three levels represents “Best of Both Worlds.” Initially, by interacting with structure and detail features, Boths can focus on these two aspects at the same time. Further, our network can simultaneously consider channel and pixel dimensions through 3-D attention learning, which is more similar to human visual perception. Lastly, the proposed model can focus on high- and low-frequency information, through a novel loss function based on the wavelet transforms. Upon subsequent analysis and evaluation, Boths has shown superior performance compared with state-of-the-art (SOTA) methods. Our models and datasets are publicly available at:https://github.com/perseveranceLX/Boths.

Haze superimposes a veil over remote sensing images, which severely limits the extraction of valuable military information. To this end, we present a novel trinity model to restore realistic surface information by integrating the merits of both prior-based and deep learning-based strategies. Concretely, the critical insight of our Trinity-Net is to investigate how to incorporate prior information into CNNs and Swin Transformer for reasonable estimation of haze parameters. Then, haze-free images are obtained by reconstructing the remote sensing image formation model. Although Swin Transformer has shown tremendous potential in the dehazing task, which typically results in ambiguous details. We devise a gradient guidance module that naturally inherits structure priors of gradient maps, guiding the deep model to generate visually pleasing details. In light of the generality of image formation parameters, we successfully promote Trinity-Net to natural image dehazing and underwater image enhancement tasks. Notably, the acquisition of large-scale remote sensing hazy images and natural hazy images in military scenes is not feasible in practice. To bridge this gap, we construct aRemote Sensing Image Dehazing Benchmark(RSID) and aNatural Image Dehazing Benchmark(NID), including 1000 real-world hazy images with corresponding ground truth images, respectively. To our knowledge, this is the first exploration to develop dehazing benchmarks in the military field, alleviating the dilemma of data scarcity. Extensive experiments on three vision tasks illustrate the superiority of our Trinity-Net against multiple state-of-the-art methods. The datasets and code are available at https://github.com/chi-kaichen/Trinity-Net.