High-resolution (HR) electro-optical (EO) satellites generally obtain multispectral (MS) images of a lower spatial resolution than their corresponding panchromatic (PAN) images owing to physical constraints. Despite these challenges, HR MS images remain critical in fields such as defense, industrial monitoring, and disaster response. Therefore, PAN-sharpening techniques have been widely studied. Recent progress in PAN-sharpening has been driven by deep learning, with diffusion models (DMs) emerging as a promising direction. Recent diffusion-based PAN-sharpening methods in the pixel domain generate high-quality PAN-sharpened (PS) images, but they generally require 25–50 denoising steps, imposing high computational complexity. To mitigate these limitations, we attempted to utilize a latent DM (LDM) for the PAN-sharpening task. However, the conventional variational autoencoder (CVAE) in LDM cannot accurately reconstruct satellite EO images of high bit depths, relatively smaller dynamic ranges within the bit depths, and multichannel characteristics. In this study, we first identify the limitations of CVAE and propose a dual-decoder-VAE (DDV), which is more suitable for satellite EO images. Furthermore, we introduce a DDV-based latent diffusion PAN-sharpening model (DDV-LDP). DDV-LDP achieves 0.06 dB and 0.98 dB higher PSNR values than a state-of-the-art diffusion-based method (UKnowDif-T) on the KOMPSAT-3A and WorldView-III datasets, respectively, even with a 99.5% reduction in testing time.

Synthetic aperture radar (SAR) imagery has become increasingly vital across diverse applications, including military surveillance, environmental monitoring, and disaster response. However, interpreting SAR images poses challenges for non-experts owing to their distinct imaging characteristics, such as speckle noise and structural distortions. Additionally, inherent properties of SAR imaging and temporal disparities frequently lead to local misalignments between paired SAR and electro-optical (EO) images. To mitigate these issues, we introduce U-SET, an uncertainty-aware framework for SAR-to-EO translation that explicitly models pixel-wise uncertainty, facilitating locally adaptive learning. By leveraging the uncertainty estimation capabilities of deep-learning models, U-SET effectively prioritizes structurally complex and ambiguous regions during training. Comprehensive evaluations on our newly compiled KOMPSAT dataset demonstrate that U-SET achieves state-of-the-art performance, outperforming existing methods across six image quality metrics in both quantitative and qualitative assessments.

Electro-optical (EO) satellites are primarily used for reconnaissance, national defense, and cartography. However, most high-resolution (HR) EO satellites obtain images at a lower resolution (LR) in the multispectral (MS) band compared to the panchromatic (PAN) band due to technical limitations. Deep learning-based PAN-sharpening methods have been continuously developed to address the growing demand for MS images with the same ground sample distance (GSD) as PAN images. The improvements in deep learning-based PAN-sharpening methods have focused on enhancing the network structure and often overlooked the physical characteristics of satellite EO sensors, leading to artifacts such as noise and color distortions in the PAN-sharpened (PS) images. Thus, we propose a fully band-separable PAN-sharpening (FBS-PS) method and its more elaborate quality-centric model, called FBS-PS+, that process MS images separately, effectively considering the physical properties of the corresponding EO sensors in the acquisition when generating PS images. This helps prevent unrelated information from being mixed among MS images and enables accurate feature extractions. Therefore, the generated PS images have less noise and reduced color distortions than previous PAN-sharpening methods that typically fuse MS images from front-end layers. In addition, we design a novel training method and loss function to handle the problem of misregistered MS and PAN images. Our FBS-PS+ outperforms all other PAN-sharpening methods in most reference-based quality metrics, while our FBS-PS, lightweight and faster, achieves comparable quality performance to local-global transformer enhanced unfolding network (LGTEUN) that has$7.95\times $more parameters and requires$4.06\times $more floating-point operations per second (FLOPs).

Recently, the analysis and use of synthetic aperture radar (SAR) imagery have become crucial for surveillance, military operations, and environmental monitoring. A common challenge with SAR images is the presence of speckle noise, which can hinder their interpretability. To enhance the clarity of SAR images, this letter introduces a novel SAR-to-electro-optical (EO) image translation (SET) network, called SGCL-SET, which first incorporates EO object label information for stable translation. We use a pretrained segmentation network to provide the segmentation regions with their labels into learning the SET. Our SGCL-SET can be trained to effectively learn the translation for the regions of confusing contexts using the segmentation and label information. Through comprehensive experiments on our KOMPSAT dataset, our SGCL-SET significantly outperforms all the previous methods with large margins across nine image quality evaluation metrics.

High-resolution (HR) satellites generally transmit multispectral (MS) images at a lower resolution than that of panchromatic (PAN) images. However, satellite image users often prefer MS images to have the same resolution as the corresponding PAN images. Therefore, PAN-sharpening (PS), a technique for obtaining HRMS images by utilizing low-resolution (LR) MS images and HRPAN images, has been a subject of study for several decades. Nevertheless, in most PS methods, various considerations are often ignored, including disparities in physical sensor locations, sensor distortions, geometric variations among acquired images, and registration errors. Owing to these missed factors, increasing the resolution by generating PS images from MS images results in increased registration errors, leading to color bleeding. Furthermore, when obtaining PS images from LRMS images, interpolation of spectral information can lead to image blurring. To address these issues, we propose a novel spectral blending network (SBN) that incorporates spectral alignment blocks (SABs) and a half-instance and half-attention block (HHB) to alleviate both color bleeding and registration errors, producing high-quality PS images with low complexity, respectively. Our SBN achieves superior performance with 1.40~4.55 dB higher peak signal-to-noise ratio (PSNR) for KOMPSAT-3A data and 1.11~3.27 dB higher PSNR for WorldView-III data, as well as with significantly lower computational complexity 42.5~99.3% lower floating point operations per second (FLOPs) than other state-of-the-art methods.

In remote sensing environments, image compression is essential to efficiently transmit and store high-resolution images due to the limited bandwidth and storage capacity. However, compression often leads to image quality degradation, requiring compression artifact removal technology in the postprocessing stage. Although deep neural networks have shown remarkable performance in image restoration, most existing methods have not adequately considered the compression conditions specific to remote sensing environments and have been evaluated primarily on synthetic datasets. To solve these issues, we propose a multidomain adaptive spatial–spectral variable compression artifact removal network (MASCAR) that effectively restores the earth surface details of compressed images in remote sensing environments. We introduce a multidomain local-patch collaborative learning strategy that extracts diverse features by decomposing the input local patch into different domains. In addition, we propose a detail focusing approach to direct the network’s focus toward fine-texture detail restoration and ensure stable training of remote sensing images with significant deviations in pixel distribution of local patches. Furthermore, a detail enhancement approach is presented to enhance the details of the restored images. Moreover, we propose an incorporated compressed image quality adaptation mechanism to respond flexibly to unknown compression ratios in remote sensing environments. The performance of MASCAR applied with the proposed method is evaluated on synthetic and real-world remote sensing datasets. Experimental results demonstrate that the proposed method has better quantitative performance and visual quality than existing methods.

Satellite Synthetic Aperture Radar (SAR) images are immensely valuable because they can be obtained regardless of weather and time conditions. However, SAR images have fatal noise and less contextual information, thus making it harder and less interpretable. So, translation of SAR to Electro-Optical (EO) images is highly required for easier interpretation. In this paper, we propose a novel coarse-to-fine context-aware SAR-to-EO image translation (CFCA-SET) framework and a misalignment-resistant loss for the misaligned pairs of SAR-EO images. With our auxiliary learning of SAR-to-Near-Infrared translation, CFCA-SET consists of a two-stage training: (i) the low-resolution SAR-to-EO translation is learned in the coarse stage via a local self-attention module that helps diminish the SAR noise, and (ii) the resulting output is used as guidance in the fine stage to generate the SAR colorization of high resolution. Our proposed auxiliary learning of SAR-to-NIR translation can successfully lead CFCA-SET to learn distinguishable characteristics of various SAR objects with less confusion in a context-aware manner. To handle the inevitable misalignment problem between SAR and EO images, we newly design a misalignment-resistant loss function. Extensive experimental results show that our CFCA-SET can generate more recognizable and understandable EO-like images compared to other methods in terms of nine image quality metrics. Our CFCA-SET surpasses the state-of-the-art methods for two (QXS and CASET) datasets with the improvements: PSNR (3.6%, 29%), ERGAS (7.4%, 30%), SSIM (15%, 15%), SAM (21%, 38%), Ds (16%, 13%), QNR (1.5%, 3.1%), CHD (18%, 12%), LPIPS (4.2%, 8%), and FID (9.0%, 33%).

Multispectral satellite imaging sensors acquire various spectral band images and have a unique spectroscopic property in each band. Unfortunately, image artifacts from imaging sensor noise often affect the quality of scenes and have a negative impact on applications for satellite imagery. Recently, deep learning approaches have been extensively explored to remove noise in satellite imagery. Most deep learning denoising methods, however, follow a supervised learning scheme, which requires matched noisy image and clean image pairs that are difficult to collect in real situations. In this article, we propose a novel unsupervised multispectral denoising method for satellite imagery using a wavelet directional cycle-consistent adversarial network (WavCycleGAN). The proposed method is based on an unsupervised learning scheme using adversarial loss and cycle-consistency loss to overcome the lack of paired data. Moreover, in contrast to the standard image-domain cycleGAN, we introduce a wavelet directional learning scheme for effective denoising without sacrificing high-frequency components such as edges and detailed information. Experimental results for the removal of vertical stripes and wave noise in satellite imaging sensors demonstrate that the proposed method effectively removes noise and preserves important high-frequency features of satellite images.

We present a novel and effective learning-based frame rate upconversion (FRUC) scheme, using linear mapping. The proposed learning-based FRUC scheme consists of: 1) a new hierarchical extended bilateral motion estimation (HEBME) method; 2) a light-weight motion deblur (LWMD) method; and 3) a synthesis-based motion-compensated frame interpolation (S-MCFI) method. First, the HEBME method considerably enhances the accuracy of the motion estimation (ME), which can lead to a significant improvement of the FRUC performance. The proposed HEBME method consists of two ME pyramids with a three-layered hierarchy, where the motion vectors (MVs) are searched in a coarse-to-fine manner via each pyramid. The found MVs are further refined in an enhanced resolution of four times by jointly combining the MVs from the two pyramids. The HEBME method employs a new elaborate matching criterion for precise ME which effectively combines a bilateral absolute difference, an edge variance, pixel variances, and an MV difference among two consecutive blocks and its neighboring blocks. Second, the LWMD method uses the MVs found by the HEBME method and removes the small motion blurs in original frames via transformations by linear mapping. Third, the S-MCFI method finally generates interpolated frames by applying linear mapping kernels for the deblurred original frames. In consequence, our FRUC scheme is capable of precisely generating interpolated frames based on the HEBME for accurate ME, the S-MCFI for elaborate frame interpolation, and the LWMD for contrast enhancement. The experimental results show that our FRUC significantly outperforms the state-of-the-art non-deep learning-based schemes with an average of 1.42 dB higher in the peak signal-to-noise-ratio and shows comparable performance with the state-of-the-art deep learning-based scheme.