Clouds in optical images are inevitable and can adversely affect subsequent analysis. Given that SAR imagery remains unaffected by cloud cover, many cloud removal methods involve the fusion of SAR and optical imagery. Unfortunately, existing methods for cloud removal through SAR-optical image fusion are computationally intensive and time-consuming, limiting their practical application. To address these challenges, this paper proposes a novel multi-feature fusion network (MFFNet) for SAR-optical image fusion, aiming to remove clouds from optical images effectively. The proposed method was applied to global and all-season Sentinel-1 and Sentinel-2 images. Quantitative experiments demonstrate that MFFNet achieves high accuracy and efficiency. Specifically, our method obtains an SSIM value of 87.10 and a speed of 25.97 FPS.

Clouds in remote sensing optical images often obscure essential information. They may lead to occlusion or distortion of ground features, thereby affecting the subsequent analysis and extraction of target information. Therefore, the removal of clouds in optical images is a critical task in various applications. SAR-optical image fusion has achieved encouraging performance in the reconstruction of cloud-covered information. Such methods, however, are extremely time-consuming and computationally intensive, making them difficult to apply in practice. This letter proposes a novel Feature Pyramid Network (FPNet) that effectively reconstructs the missing optical information. FPNet enables the extraction and fusion of multi-scale features from the SAR image and the cloudy optical image, as the FPNet leverages the power of convolutional neural networks by merging the feature maps from different scales. It can learn useful features efficiently because it downsamples the input images while preserving important information, thus reducing the computational workload. Experiments are conducted on a benchmark global SEN12MS-CR dataset and a regional South Sudan dataset. Results are compared with those of state-of-the-art methods such as DSen2-CR and GLF-CR. The experimental results demonstrate that FPNet accomplishes superior performance in terms of accuracy and visual effects. Both the inference and training speeds of FPNet are fast. Specifically, it runs at 96 FPS and requires less than four hours to train a single epoch using SEN12MS-CR on two 2080ti GPUs. Therefore, it is suitable for applying to various study areas.

Semantic segmentation of remotely sensed images plays an important role in land resource management, yield estimation, and economic assessment. U-Net, a deep encoder–decoder architecture, has been used frequently for image segmentation with high accuracy. In this letter, we incorporate multiscale features generated by different layers of U-Net and design a multiscale skip connected and asymmetric-convolution-based U-Net (MACU-Net), for segmentation using fine-resolution remotely sensed images. Our design has the following advantages: (1) the multiscale skip connections combine and realign semantic features contained in both low-level and high-level feature maps; (2) the asymmetric convolution block strengthens the feature representation and feature extraction capability of a standard convolution layer. Experiments conducted on two remotely sensed data sets captured by different satellite sensors demonstrate that the proposed MACU-Net transcends the U-Net, U-Netpyramid pooling layers (PPL), U-Net 3+, among other benchmark approaches. Code is available athttps://github.com/lironui/MACU-Net.

The attention mechanism can refine the extracted feature maps and boost the classification performance of the deep network, which has become an essential technique in computer vision and natural language processing. However, the memory and computational costs of the dot-product attention mechanism increase quadratically with the spatiotemporal size of the input. Such growth hinders the usage of attention mechanisms considerably in application scenarios with large-scale inputs. In this letter, we propose a linear attention mechanism (LAM) to address this issue, which is approximately equivalent to dot-product attention with computational efficiency. Such a design makes the incorporation between attention mechanisms and deep networks much more flexible and versatile. Based on the proposed LAM, we refactor the skip connections in the raw U-Net and design a multistage attention ResU-Net (MAResU-Net) for semantic segmentation from fine-resolution remote sensing images. Experiments conducted on the Vaihingen data set demonstrated the effectiveness and efficiency of our MAResU-Net. Our code is available athttps://github.com/lironui/MAResU-Net.

The fully convolutional network (FCN) with an encoder-decoder architecture has been the standard paradigm for semantic segmentation. The encoder-decoder architecture utilizes an encoder to capture multilevel feature maps, which are incorporated into the final prediction by a decoder. As the context is crucial for precise segmentation, tremendous effort has been made to extract such information in an intelligent fashion, including employing dilated/atrous convolutions or inserting attention modules. However, these endeavors are all based on the FCN architecture with ResNet or other backbones, which cannot fully exploit the context from the theoretical concept. By contrast, we introduce the Swin Transformer as the backbone to extract the context information and design a novel decoder of densely connected feature aggregation module (DCFAM) to restore the resolution and produce the segmentation map. The experimental results on two remotely sensed semantic segmentation datasets demonstrate the effectiveness of the proposed scheme.

Semantic segmentation of remote sensing images plays an important role in a wide range of applications, including land resource management, biosphere monitoring, and urban planning. Although the accuracy of semantic segmentation in remote sensing images has been increased significantly by deep convolutional neural networks, several limitations exist in standard models. First, for encoder–decoder architectures such as U-Net, the utilization of multiscale features causes the underuse of information, where low-level features and high-level features are concatenated directly without any refinement. Second, long-range dependencies of feature maps are insufficiently explored, resulting in suboptimal feature representations associated with each semantic class. Third, even though the dot-product attention mechanism has been introduced and utilized in semantic segmentation to model long-range dependencies, the large time and space demands of attention impede the actual usage of attention in application scenarios with large-scale input. This article proposed a multiattention network (MANet) to address these issues by extracting contextual dependencies through multiple efficient attention modules. A novel attention mechanism of kernel attention with linear complexity is proposed to alleviate the large computational demand in attention. Based on kernel attention and channel attention, we integrate local feature maps extracted by ResNet-50 with their corresponding global dependencies and reweight interdependent channel maps adaptively. Numerical experiments on two large-scale fine-resolution remote sensing datasets demonstrate the superior performance of the proposed MANet. Code is available athttps://github.com/lironui/Multi-Attention-Network.