Technologies such as Unmanned Aerial Vehicles (UAVs) and deep learning provide robust technical support for urban management tasks. Addressing the significant needs of UAVs in urban management, effectively integrating and utilizing UAVs, artificial intelligence, and big data systems is a key technical issue. This paper focuses on a range of problems in UAV system communication, data management, and intelligent inspection combined with target detection models in urban governance scenarios. We conduct research on the key technologies involved and, based on this, design and construct a UAV system for urban management. This system offers integrated services of system communication, management of multi-source heterogeneous data, and efficient intelligent detection, thereby facilitating intelligent urban inspection and emergency response software services.

Geological environment remote sensing (GERS) interpretation contributes to lithological mapping, disaster prediction, soil erosion monitoring, and so on. However, the rich diversity, complex distribution, interclass similarities, and uncertainties in data quality of geological elements pose challenges to GERS interpretation. In addition, current automatic feature extraction of GERS elements, which rely on deep learning (DL) and remote sensing (RS) information process technologies, often require sufficient labeled data. Due to the enormous labor cost and specialized expertise needed, labeled GERS samples are limited to training the data-driven models. To tackle the above challenges, we introduce the semi-supervised dual-model progressive self-training (DM-ProST) framework. This framework employs two DL networks with different initializations as evaluator models to correct each other. A sample filtering strategy is then implemented to evaluate the quality of unlabeled samples, selecting high-quality and reliable ones to expand the training set. In addition, a fully connected conditional random field (CRF) module is incorporated to optimize DL network prediction maps, thereby enhancing the boundary performance of segmentation results. The framework utilizes a multitask loss function that combines consistency loss with cross-entropy, enabling the models to learn discriminative GERS features. This process accurately generates pseudo-labels and achieves precise lithology mapping of GERS with a small amount of annotation samples. Finally, we conducted an experimental evaluation on the Landsat 8 dataset in Xinjiang, China, and massive experiments proved the effectiveness of DM-ProST.

Soil elements refer to different types of soil with unique colors, textures, and particle sizes. Their interpretation is essential for agriculture, ecological environment and land permeability assessment. This typically requires experts with dual knowledge in geology and remote sensing. With the increasing volume of remote sensing data, the traditional “visual interpretation" and “field survey" technique is no longer sufficient to meet the demands. Because of the challenges such as the fine structure of soil, complex and variable natural scenes, and strong spatial variability, there remains a considerable gap between the accuracy of deep learning-based methods and expert interpretation. To improve the accuracy of intelligent soil elements interpretation, this study proposes a soil interpretation framework coupling implicit knowledge with multispectral image (SIFCIM). This framework quantifies implicit knowledge, such as interpretation symbol and terrain feature, into matrix data (interpretation symbol distance field and digital elevation model). To align with the SIFCIM, an Implicit-Knowledge-Guided Adaptive Feature Fusion Network (IAFFNet) is constructed, which enhances the utilization efficiency of auxiliary features through an adaptive implicit feature fusion module and a global feature dependence module. Experimental results demonstrate that IAFFNet outperforms interpretation methods with single remote sensing image, achieving approximately 4.34% and 6.62% improvements in overall pixel accuracy and mean intersection over union, respectively. These results validate the effectiveness and robustness of the implicit-knowledge-guided approach in soil elements interpretation. To our knowledge, this work is the first to apply the concept of implicit knowledge to soil elements interpretation, providing a novel insight for related research.

Urban informal settlements (UIS) are high-density population settlements with low standards of living and supply. UIS semantic segmentation, which identifies pixels corresponding to informal settlements in remote sensing images, is crucial to the estimation of poor communities, urban management, resource allocation, and future planning, particularly in megacities. However, most studies on informal settlement mapping are either based on parcels (image classification) or pixels (semantic segmentation). Few studies utilize object information to improve UIS mapping. Since informal settlements are formed by buildings (objects), utilizing object information can improve UIS semantic segmentation. Furthermore, current UIS mapping studies mainly focus on using single-modality remote sensing images, and there is a lack of related research on using multimodal data. Due to the spatial heterogeneity of informal settlements, using only a single modality of remote sensing image features limits the effectiveness and accuracy of informal settlements semantic segmentation. Aiming at achieving fine-scale UIS mapping results, this paper proposes a UIS semantic segmentation method, namely UisNet, that utilizes a transformer-based block to receive multimodal data, including high-spatial-resolution remote sensing images (parcel- and pixel-level) and building polygon data (object-level) to identify UIS. The experiments were conducted in Shenzhen City, and they confirmed the superior performance of UisNet, which achieved an overall accuracy (OA) of 94.80% and a mean intersection over union (mIoU) of 85.51% in the testing set of the manually labeled UIS semantic segmentation dataset (UIS-Shenzhen dataset) and outperformed the best models on semantic segmentation tasks. Besides, we add a set of experiments on a public dataset (GID dataset) and compare our method with the current state-of-the-art semantic segmentation methods. Experiments show that the proposed UisNet improves mIoU by 1.64% to 7.58% compared to other methods. This work will be available at https://github.com/RunyuFan/.

Urban Villages (UV) refer to areas of urban informal settlements lagging behind the rapid urbanization process. Recent studies focus on using satellite images to classify UV. However, satellite images only capture objects from a bird-eye perspective, thus cannot obtain complex spatial relationships between objects. In UV areas, buildings and objects are usually dense, small in size, and obscure each other. Therefore, it is challenging to classify UV accurately using only satellite images with bird-eye perspectives. In this paper, to solve this problem, we proposed a novel method that uses satellite images combined with streetview images to classify UV. Specifically, we propose a novel multilevel spatial-channel feature fusion network, namely FusionMixer, that integrates CNN-based feature extraction modules and a multilevel spatial-channel feature fusing layer to make an optimal UV classification. Experiments were conducted in Shenzhen City (the RsSt-ShenzhenUV dataset) and a public UV dataset (theS2UVdataset). The proposed FusionMixer achieved an increase of OA by 8.83% and 8.84%, and improves Kappa by 0.1765 and 0.1770 in the validation set and testing set, compared to the second-best fusion models in RsSt-ShenzhenUV dataset. Experiments in theS2UVdataset show that the proposed FusionMixer improves OA by 1.82% and Kappa by 0.04 compared to other methods. We also added a set of experiments on a public dataset (Houston dataset) and compare our method with the current state-of-the-art multimodal fusion methods to prove the generalization of the proposed FusionMixer in fusing other multimodality data. These experiments confirmed the superior performance of the proposed FusionMixer.

Geological remote sensing interpretation can extract elements of interest from multiple types of images, which is vital in geological survey and mapping, especially in inaccessible regions. However, due to numerous classes, high interclass similarities, complex distributions, and sample imbalances of geological elements, the interpretation results of machine-learning (ML)-based methods are understandably worse than manual visual interpretation. Additionally, scholars in remote sensing have mainly carried out their works to interpret a single geological element category, such as mineral, lithological, soil and structure. The interpretation of multiple geological elements is missing, which is more in line with the open world. To improve the interpretation results of ML-based methods and reduce the labor cost in geological survey and mapping, we propose a deep-learning (DL)-feature-based adaptive multi-source data fusion network (AMSDFNet) for the efficient interpretation of multiple geological remote sensing elements. The AMSDFNet has two branches for learning valuable spatial and spectral information from two kinds of data sources, wherein the atrous spatial pyramid pooling operation and an attention block are applied to adaptively extract and fuse multi-scale informative features. A hard example mining algorithm was also added to select important training examples to address sample imbalance. A large-scale region in western China with sufficient geological elements was set as the research area. The proposed model improved the two critical metrics by more than 2% in the experiment section. As far as we know, this research work is the first time DL features and multi-source remote sensing images have been utilized to simultaneously interpret geological elements of lithology, soil, surface water, and glaciers. The extensive experimental results demonstrated the superiority of DL features and our model in geological remote sensing interpretation.

High-resolution (HR) remote sensing imagery plays a critical role in remote sensing image interpretation, and single image super-resolution (SISR) reconstruction technology is becoming increasingly valuable and significant. The state-of-the-art deep-learning-based SISR methods have demonstrated remarkable advantages, while reconstructing complex texture details still remains a big challenge. Besides, as a typical ill-posed inverse problem, how to determine the optimal solution is another important topic. To address these problems, in this work, a dual learning-based graph neural network (DLGNN) is proposed, in which the GNN is utilized to consider the self-similarity patches in remote sensing imagery by aggregating cross-scale neighboring feature patches, and dual learning strategy is adopted to refine the reconstruction results by constraining the mapping process in terms of the loss function, transferring the typical ill-posed problem to a well-posed one. Abundant experiments on 3K VEHICLE_SR datasets and Massachusetts Roads demonstrate the validity and outstanding performance for remote sensing image super-resolution tasks compared with other state-of-the-art super-resolution construction methods. Code is available at https://github.com/CUG-RS/DLGNN.

Dense time-series remote sensing images have transformed the traditional bitemporal land-cover change detection to continuous monitoring. Previous work mostly employs linear fitting, prediction, or decomposition methods, and the detection accuracy is not high. The latest progress of deep learning (DL) shows its advantages in time-series change monitoring. However, DL models are computationally expensive and require lots of labeled samples, resulting in often employed prediction-threshold-based unsupervised change detection method. However, the determination of a reasonable threshold has always been a big problem. Therefore, we proposed the similarity-measurement-based deep transfer learning for time-series adaptive change detection (SDTL-TSACD) model. First, a standard dynamic time warping (SDTW) distance was proposed and used to cluster large-scale time series into multiple subcategories with high time-series similarity. Second, a time convolutional network (TCN) was used for nonlinear time-series fitting and prediction, and an early stop strategy was used to prevent overfitting. Then, the trained TCN model would be transferred and performed pixel-by-pixel time-series prediction within the same category, and the SDTW was also used to evaluate the prediction accuracy. Finally, the Otsu adaptive threshold was used to detect change points, and the spatial neighbor relationship was used to eliminate the pseudo-change points. Change detection results using 132 benchmark datasets showed that the SDTL-TSACD performed well in both accuracy and efficiency. In addition, the MOD13Q1-EVI images from 2001 to 2020 were used to study the land-cover change of the Loess Plateau, and the SDTL-TSACD also showed a good ability to solve practical problems.

Due to the trade-off of temporal resolution and spatial resolution, spatiotemporal image-fusion uses existing high-spatial-low-temporal (HSLT) and high-temporal-low-spatial (HTLS) images as prior knowledge to reconstruct high-temporal-high-spatial (HTHS) images. However, some existing spatiotemporal image-fusion algorithms ignore the issue that the spatial information of HTLS images is insufficient to support the acquisition of spatial information, which leads to the unsatisfactory accuracy of the fusion result. To introduce more spatial information, the algorithm in this article uses Cycle-generative adversarial networks (GANs) to simulate the change process of two HSLT images at k-1 and k+1, and to generate some simulated images between k-1 and k+1. Then, the generated images are selected under the help of HTLS images, and the selected ones are then enhanced with wavelet transform. Finally, the image with spatial information is introduced into the Flexible Spatiotemporal DAta Fusion (FSDAF) framework to improve the performance of spatiotemporal image-fusion. Extensive experiments on two real data sets demonstrate that our proposed method outperforms current state-of-the-art spatiotemporal image-fusion methods.

Object detection, aiming to recognize and locate objects of interest in aerial images, has historically played a significant role in the remote sensing community. Following remarkable improvements in Earth observation technologies, high-resolution remote sensing (HRRS) images with a bird’s eye view perspective have revealed many categories of objects with sufficient variations in appearance and on complex backgrounds that make HRRS object detection an active but challenging task. The selection of positive samples and negative training instances is an essential factor in influencing detectors’ performance. Related studies have found that many low-quality negative samples in the detectors’ training process have caused training instability and low detection accuracy. In this work, a novel sampling-balance-based multistage network (SB-MSN) is presented to adaptively mine high-quality positive and negative instances for training an accurate detector. It has a series of components to ensure the selection and generation of high-quality examples for training an accurate detector, including a multiscale information retention module, an intersection over union balance sampling strategy, a balance L1 loss, and a multistage network. The proposed detector has been evaluated on three representative HRRS data sets. The extensive experimental results show that our detector can solve the problem of low-quality samples and significantly improve the detection performance of the mAP by 1.4% with the NWPU VHR-10 data set, 3.5% with the high-resolution remote sensing detection (HRRSD) data set, and 4.2% with the detection in the optical remote (DIOR) data set.1

Spatiotemporal fusion utilizes the complementarity of high-temporal-low-spatial (HTLS) and high-spatial-low-temporal (HSLT) resolution data to obtain high temporal and spatial (HTHS) resolution fusion data, which can effectively satisfy the demand for HTHS data. However, due to the difference of spatial resolution, it is difficult to obtain precise spatial information in spatiotemporal fusion. To solve this problem, a multi-level strategy-based spatial domain prediction algorithm is proposed to enhance the spatial information extraction in spatiotemporal remote sensing imagery fusion, which can reduce the noise superposition in the process of multiple reconstruction. By learning-based first and then interpolation-based Super resolution reconstruction, the proposed method can obtain better prediction of spatial information and improve the accuracy of spatiotemporal fusion.

Focusing on the problems of insufficient high-quality training samples to conduct an ideal detector for high-resolution remote sensing (HRRS) image object, we applied a multi-stage based detector to apply a resampling progressively strategy, which guarantees the amount of the positive training set and minimizing overfitting. The method has a sequence of regression heads training on the samples chosen by different Intersection over Union (IoU) thresholds. The first head with a low IoU threshold trained by a large number of positive samples and can prepare more high-quality samples for the remaining branches. The subsequent heads with the increasing IoU thresholds would train on more abundant positive samples and to conduct an accurate detector and avoid the problem of overfitting. The proposed method reached the best mAP value and outperformed the comparison methods by about 10%. The experimental results show that our method can significantly improve detection performance and solve the problem of lacking high-quality samples.

High-resolution remote sensing (HRRS) image scene classification takes an important role in many applications and has attracted much attention. Recently, notable efforts have been made to present massive methods for HRRS scene classification, wherein deep-learning-based methods demonstrate remarkable performance compared with state-of-the-art methods. However, HRRS images contain complex contextual relationships and large differences of object scale, which are significantly different from natural images. The existing deep-learning-based scene classification methods are originally designed for natural image processing and have not been optimized to adapt to the characteristics of HRRS images, which significantly affects the efficiency of the feature extraction and recognition accuracy. In addition, when designing a model for remote sensing tasks, the pretraining of the model is time-consuming. The enormous amount of pretraining time and computation resources necessarily increase the difficulty of producing an excellent model. In this letter, focusing on the problems above, we proposed a new convolutional neural network (CNN)-based scene classification method. The CNN-based scene classification method is constructed by spatial-scale-aware blocks and is efficient in extracting the abundant spatial features, but can also adaptively adjust feature responses to maximize the function of informative features in the classification results. In addition, an HRRS imagery-based learning strategy is utilized to obtain an initial model for fine-tuning the model parameters, which drastically reduces the pretraining time. The proposed method has been demonstrated using two HRRS data sets, and experimental results have proven the superiority of the proposed method.

High-resolution remote sensing (HRRS) image scene classification has attracted an enormous amount of attention due to its wide application in a range of tasks. Due to the rapid development of deep learning (DL), models based on convolutional neural network (CNN) have made competitive achievements on HRRS image scene classification because of the excellent representation capacity of DL. The scene labels of HRRS images extremely depend on the combination of global information and information from key regions or locations. However, most existing models based on CNN tend only to represent the global features of images or overstate local information capturing from key regions or locations, which may confuse different categories. To address this issue, a key region or location capturing method called key filter bank (KFB) is proposed in this article, and KFB can retain global information at the same time. This method can combine with different CNN models to improve the performance of HRRS imagery scene classification. Moreover, for the convenience of practical tasks, an end-to-end model called KFBNet where KFB combined with DenseNet-121 is proposed to compare the performance with existing models. This model is evaluated on public benchmark data sets, and the proposed model makes better performance on benchmarks than the state-of-the-art methods.

High-resolution remote sensing (HRRS) image scene classification has been a critical task and greatly important for many applications, wherein convolutional neural network (CNN)-based methods have achieved considerable improvements. However, the CNN-based methods have countered a severe problem that massive annotation samples are required to obtain ideal model for scene classification. There is no dataset with a comparative scale to ImageNet to meet the sample requirement and labelling samples is labor-intensive and time-consuming. To solve the problem of insufficient annotation samples, a new generative adversarial network (GAN)-based sample generation method for scene classification is implemented. The proposed method is able to generate HRRS images with specific label and improve scene classification performance for the CNN-based methods.

High-resolution remote sensing (HRRS) scene classification plays an important role in numerous applications. During the past few decades, a lot of remarkable efforts have been made to develop various methods for HRRS scene classification. In this paper, focusing on the problems of complex context relationship and large differences of object scale in HRRS scene images, we propose a deep CNN-based scene classification method, which not only enables to enhance the ability of spatial representation, but adaptively recalibrates channel-wise feature responses to suppress useless feature channels. We evaluated the proposed method on a publicly large-scale dataset with several state-of-the-art convolutional neural network (CNN) models. The experimental results demonstrate that the proposed method is effective to extract high-level category features for HRRS scene classification.

In real simulation applications, simulations often involve large volumes of three-dimensinal (3D) moving objects. With the rapid growth of the scale of simulation-problem domains, it has become a key requirement to efficiently manage massive 3D moving objects. Conventional indexing approaches for managing 3D moving objects during simulations generally sufferfrom excessive update costs. Aiming to this problem, this paper first proposes an update-efficient indexing structure by fusing a loose Octree and one update-memo structure, namely ML-Octree. ML-Octree significantly reduces the update costs of one simulation involving massive 3D moving objects. Towards providing a more efficient indexing approach, this paper has explored the feasibility of paralleling ML-Octree by employing Graphic Processing Unit (GPU). A load-balancing scheme is used to further improve the update performance of the GPU-aided ML-Octree. Finally, a distributed GPU-aided ML-Octree is proposed for large-scale simulations. The experimental results indicate that (1) ML-Octree can acquire the update-performance gain of an order of magnitude similar to that of Octree, (2) the GPU-aided ML-Octree can accelerate 5.07χ fasterthan a parallel ML-Octree with 8 CPU threads on average, (3) the load-balance scheme can improve GPU-aided ML-Octree by 2.3χ on average, and (4) the distributed GPU-aided ML-Octree can efficiently support large-scale simulations.