Urban Informal Settlements (UIS) denote densely populated locales characterized by inadequate urban infrastructure standards, often exhibiting an amalgamation of rural and urban attributes, primarily situated within the confines of major cities or metropolitan regions. UIS mapping is a typical task that aims to identify pixels corresponding to urban informal settlements in remote sensing images. The extremely similar visual characteristics, sample uncertainty, and highly manual costs bring large-scale UIS mapping noteworthy challenges. In this paper, we propose a sample uncertainty-aware semi-supervised learning method guided by background-knowledge from crowdsourced geospatial data (namely SemiUIS) for UIS mapping and produces a 1-meter resolution UIS map in the Urban Agglomeration in the Middle Reaches of the Yangtze River, China (UAMRYR). Our proposed SemiUIS method integrates semisupervised learning with the guidance of background knowledge from crowdsourcing data to reduce the uncertainty of the generated pseudo-semantic segmentation samples to improve the quality of the constructed data set by jointly optimizing the background (Not UIS areas) and the foreground (UIS areas). In contrast, traditional semi-supervised learning methods only focus on explicitly optimizing the foreground (UIS areas) in the pseudo-sample generation process. Experiments were conducted in the 31 prefecture-level cities of UAMRYR. Visual interpretation of labeled remote sensing samples, street view images and two novel knowledge validation indicators, Global Knowledge Precision (GKP) and Local Knowledge Precision (LKP) are proposed to verify the UIS mapping results. The proposed method reached an overall accuracy (OA) of 90.58 %, mean intersection over union (mIoU) of 75.88 %, Accuracy (Acc) of 78.77 %, Intersection over Union (IoU) of 62.97 %, GKP of 90.33 %, and LKP of 85.01 %, respectively with only 36 (5%) training samples. This work will be available at https://github.com/RunyuFan/UisYangtze.

Off-road trafficability (ORT) in complex geological environments is crucial for special operations, emergency rescue, and natural resource development. ORT is influenced by a combination of geographical and geological environmental factors. Recent studies primarily employ rule methods for ORT assessment. These approaches are labour-intensive, lack timeliness and objectivity, and are constrained to limited data sources. To address these issues, this work proposes a new method for ORT assessment that combines remote sensing imagery with geographic and geological data (RSI-factors) through a cross-modal rectified fusion network (CRFNet). This network is designed with the feature rectification module (FRM) and feature fusion mixer module (FFMM) to integrate cross-modal features. To address potential missing data in complex geological environments, this work proposes a multi-task and prompt learning strategy to improve model robustness. Experiments conducted on our Asia Dataset and Africa Dataset yielded optimal evaluation results. On the complete Asia and Africa Dataset, the CRFNet model improved overall accuracy (OA) by more than 25% and the kappa coefficient by over 43% compared to rule methods. On the incomplete Asia and Africa Dataset, the CRFNet model improved OA by more than 5.5% and the kappa coefficient by over 8% compared to suboptimal deep learning (DL) models. To the best of our knowledge, this research work is the first in which DL features have been combined with multi-modal RSI-factors data for ORT assessment, paving a new path for research in this field. The source codes of this work will be made publicly available at https://github.com/kangkanghe/CRFNet.

Urban informal settlements (UIS) are high-density population settlements with low standards of living and supply. UIS semantic segmentation, which identifies pixels corresponding to informal settlements in remote sensing images, is crucial to the estimation of poor communities, urban management, resource allocation, and future planning, particularly in megacities. However, most studies on informal settlement mapping are either based on parcels (image classification) or pixels (semantic segmentation). Few studies utilize object information to improve UIS mapping. Since informal settlements are formed by buildings (objects), utilizing object information can improve UIS semantic segmentation. Furthermore, current UIS mapping studies mainly focus on using single-modality remote sensing images, and there is a lack of related research on using multimodal data. Due to the spatial heterogeneity of informal settlements, using only a single modality of remote sensing image features limits the effectiveness and accuracy of informal settlements semantic segmentation. Aiming at achieving fine-scale UIS mapping results, this paper proposes a UIS semantic segmentation method, namely UisNet, that utilizes a transformer-based block to receive multimodal data, including high-spatial-resolution remote sensing images (parcel- and pixel-level) and building polygon data (object-level) to identify UIS. The experiments were conducted in Shenzhen City, and they confirmed the superior performance of UisNet, which achieved an overall accuracy (OA) of 94.80% and a mean intersection over union (mIoU) of 85.51% in the testing set of the manually labeled UIS semantic segmentation dataset (UIS-Shenzhen dataset) and outperformed the best models on semantic segmentation tasks. Besides, we add a set of experiments on a public dataset (GID dataset) and compare our method with the current state-of-the-art semantic segmentation methods. Experiments show that the proposed UisNet improves mIoU by 1.64% to 7.58% compared to other methods. This work will be available at https://github.com/RunyuFan/.

Urban Villages (UV) refer to areas of urban informal settlements lagging behind the rapid urbanization process. Recent studies focus on using satellite images to classify UV. However, satellite images only capture objects from a bird-eye perspective, thus cannot obtain complex spatial relationships between objects. In UV areas, buildings and objects are usually dense, small in size, and obscure each other. Therefore, it is challenging to classify UV accurately using only satellite images with bird-eye perspectives. In this paper, to solve this problem, we proposed a novel method that uses satellite images combined with streetview images to classify UV. Specifically, we propose a novel multilevel spatial-channel feature fusion network, namely FusionMixer, that integrates CNN-based feature extraction modules and a multilevel spatial-channel feature fusing layer to make an optimal UV classification. Experiments were conducted in Shenzhen City (the RsSt-ShenzhenUV dataset) and a public UV dataset (theS2UVdataset). The proposed FusionMixer achieved an increase of OA by 8.83% and 8.84%, and improves Kappa by 0.1765 and 0.1770 in the validation set and testing set, compared to the second-best fusion models in RsSt-ShenzhenUV dataset. Experiments in theS2UVdataset show that the proposed FusionMixer improves OA by 1.82% and Kappa by 0.04 compared to other methods. We also added a set of experiments on a public dataset (Houston dataset) and compare our method with the current state-of-the-art multimodal fusion methods to prove the generalization of the proposed FusionMixer in fusing other multimodality data. These experiments confirmed the superior performance of the proposed FusionMixer.

Geological remote sensing interpretation can extract elements of interest from multiple types of images, which is vital in geological survey and mapping, especially in inaccessible regions. However, due to numerous classes, high interclass similarities, complex distributions, and sample imbalances of geological elements, the interpretation results of machine-learning (ML)-based methods are understandably worse than manual visual interpretation. Additionally, scholars in remote sensing have mainly carried out their works to interpret a single geological element category, such as mineral, lithological, soil and structure. The interpretation of multiple geological elements is missing, which is more in line with the open world. To improve the interpretation results of ML-based methods and reduce the labor cost in geological survey and mapping, we propose a deep-learning (DL)-feature-based adaptive multi-source data fusion network (AMSDFNet) for the efficient interpretation of multiple geological remote sensing elements. The AMSDFNet has two branches for learning valuable spatial and spectral information from two kinds of data sources, wherein the atrous spatial pyramid pooling operation and an attention block are applied to adaptively extract and fuse multi-scale informative features. A hard example mining algorithm was also added to select important training examples to address sample imbalance. A large-scale region in western China with sufficient geological elements was set as the research area. The proposed model improved the two critical metrics by more than 2% in the experiment section. As far as we know, this research work is the first time DL features and multi-source remote sensing images have been utilized to simultaneously interpret geological elements of lithology, soil, surface water, and glaciers. The extensive experimental results demonstrated the superiority of DL features and our model in geological remote sensing interpretation.

Object detection, aiming to recognize and locate objects of interest in aerial images, has historically played a significant role in the remote sensing community. Following remarkable improvements in Earth observation technologies, high-resolution remote sensing (HRRS) images with a bird’s eye view perspective have revealed many categories of objects with sufficient variations in appearance and on complex backgrounds that make HRRS object detection an active but challenging task. The selection of positive samples and negative training instances is an essential factor in influencing detectors’ performance. Related studies have found that many low-quality negative samples in the detectors’ training process have caused training instability and low detection accuracy. In this work, a novel sampling-balance-based multistage network (SB-MSN) is presented to adaptively mine high-quality positive and negative instances for training an accurate detector. It has a series of components to ensure the selection and generation of high-quality examples for training an accurate detector, including a multiscale information retention module, an intersection over union balance sampling strategy, a balance L1 loss, and a multistage network. The proposed detector has been evaluated on three representative HRRS data sets. The extensive experimental results show that our detector can solve the problem of low-quality samples and significantly improve the detection performance of the mAP by 1.4% with the NWPU VHR-10 data set, 3.5% with the high-resolution remote sensing detection (HRRSD) data set, and 4.2% with the detection in the optical remote (DIOR) data set.1

Remote sensing image scene classification, which aims to identify the types of land cover, is a fundamental task in remote sensing image analysis. Remote sensing images contain a variety of land-cover objects. These land-cover objects form a complex and diverse scene through spatial combination and correlation, which makes remote sensing imagery scenes classification difficult. In addition, remote sensing images contain redundant information that has a negative impact on remote sensing imagery scene classification, which makes remote sensing imagery scenes classification rather challenging. Recently, there are many deep learning based methods, which have achieved remarkable performance through an end-to-end supervised training process. Existing advances in remote sensing imagery scene classification mainly focus on training multi-layer convolutional neural networks (CNNs). These CNNs do not explicitly distinguish between key information and redundant information of the image. Therefore, the ability to extract features is limited. How to focus on key information and ignore redundant information in remote sensing imagery scene classification is a valuable problem. Inspired by the attention mechanism, we propose a CNN-based network that combines residual units and attention mechanism. It automatically assigns large weights to key areas of the image and thus has the ability to adaptively ignore redundant information. We evaluated the proposed approach with some state-of-the-art methods on the UC Merced Land-Use dataset and the NWPU-RESISC45 dataset. Experimental results show that the proposed attention model has achieved the best classification performance.