As spatial resolution increases in remote-sensing imagery, the challenge of semantic segmentation intensifies due to the need to discern intricate changes in terrain. Terrain, a composite of diverse geographic elements arranged in specific spatial patterns, demands a higher level of abstraction in semantic categorization. Achieving accurate semantic segmentation in high-resolution remote-sensing images necessitates a profound understanding of the semantic structures within complex scenes. In response to this imperative, this article introduces the object Markov random field with hierarchical semantics (OMRF-HSs) method. The primary contributions of this work are twofold: 1) effective representation of layered semantic information within images is achieved, enhancing the understanding of complex scenes and 2) unified under an object Markov random field (MRF) model, the method enables the joint modeling of structured semantics and spatial context information, facilitating more robust segmentation outcomes. Experimental evaluations conducted on multiscene high-resolution remote-sensing images from the aerial sensor, SPOT5, GeoEye, and Gaofen-2 satellites demonstrate that the proposed method outperforms state-of-the-art techniques, yielding superior segmentation accuracy. The availability of code and example data further facilitates the reproducibility and adoption of the OMRF-HS method, accessible athttps://github.com/FHY-146/OMRF-HS.

Remote sensing images have a large imaging field, covering a wide range of land surfaces and encompassing complex geographical scenes. More diverse land cover types and detailed structural information within geographical scenes can be observed with improved image spatial resolution. However, due to the high cost of annotating fine semantic scales, only a few semantic segmentation methods can fully utilize multi-level semantic information. This paper introduces a novel multi-semantic Markov Random Field (MRF-MS) model for semantic segmentation of high-resolution remote sensing images. This method extends the classical object-level Markov Random Field model by introducing a sub-label field to capture semantic hierarchical information in high-resolution images. Assuming the initial categories follow a mixture of Gaussian distribution, this method models the land cover types or structural details of geographical objects within the scene by decomposing the mixture of Gaussian distribution into sub-components. Furthermore, it establishes an interactive contextual structure between the sub-label and initial label fields to model the interaction across different semantic scales. Experimental results on multi-scene high-resolution remote sensing images demonstrate that this approach yields more accurate classification results compared to the latest MRF methods.

Camellia sinensis var.assamica is a unique product of Geographical Indication in Yunnan province. Centennial tea plantations within plantations of C.sinensis var.assamica can yield high-quality ecological tea. However, the limited production of ecological tea is insufficient to meet the widespread demand for high-quality tea leaves. The optimized MaxEnt model exhibits better accuracy and reliability in predicting the potential suitable distribution of small-sample species. Therefore, this study based on 80 sample data and 16 key environmental variables, optimize the MaxEnt model using the ENMeval package. The distribution and changes of the potential suitable areas of ecological tea plantations of C.sinensis var.assamica in Yunnan province were extracted for the periods 1981-2010(History), 2011-2040(Current), and 2041-2070(Future). The research results indicate that the performance of the MaxEnt model has been enhanced, with the 10P value reduced to 0.1125 and Training AUC and Test AUC reaching 0.9516 and 0.9422, respectively. In the three periods, the potential suitable areas are mainly concentrated in the western, the south and southwest regions of Yunnan Province. The area of high suitability has decreased by a total of 259 km2, while the area of moderate suitability has increased by 4031 km2. There is potential for developing ecological tea plantations of C.sinensis var.assamica in the western, the southern and southwestern parts of Yunnan Province. The managers and planners of tea plantations can further plan for the transformation and evaluation of tea gardens based on local policies. The results of this study will contribute to driving tea industry in Yunnan Province towards a more sustainable development path.

Recent advancements in fully-supervised change detection (CD) have been notable, yet applying CD effectively in unlabeled scenarios remains a challenge. Our study focuses on domain adaptation of change detection (DACD), transferring knowledge from a labeled dataset (source domain) to an unlabeled dataset (target domain). However, we’ve found that this transfer often leads to reduced discrimination between change and non-change classes because of cross-domain discrepancy, adversely affecting CD performance in the target domain. Aiming at it, we propose a source discriminative learning (SDL) method to learn more discriminative interclass feature representations in the source domain, which can improve the transferability of change knowledge across different CD domains. To evaluate its effectiveness, we established two cross-domain CD scenarios using well-known CD datasets and applied our SDL method to them. The experimental results affirm that our method effectively boosts DACD performance.

Crop identification is a fundamental task in remote sensing image interpretation. The rapid development of Unmanned Aerial Vehicle (UAV) has revolutionized the acquisition of super-high-resolution images. Compared with remote sensing ones, the fact that UAV images are easier to be flexibly acquired and contain more information brings opportunities for refined semantic segmentation. Recently deep learning methods have gained substantial popularity in the field of semantic segmentation. However, the practical application of deep learning methods is often hindered by heavy labeling tasks and computational resources. The object-based Markov random field (OMRF) offers a both time and labor cost-effective unsupervised approach. Nevertheless, the high-spatial heterogeneity exhibited by crops in UAV images brings great difficulties to the application of this method. To address these challenges, this letter introduces RA-OMRF model, an unsupervised approach that improves OMRF model through our newly proposed pre-processing step named Region Aid (RA). RA is used to increase the amount of data for categories with fewer samples to alleviate the problem of high-spatial heterogeneity of ground object categories, thereby improving the performance of the OMRF model. Compared to five deep learning models, our method achieves matching or even higher segmentation accuracy (for instance an overall accuracy of 97.43% on the UAV Dataset DWST) in crop identification tasks of UAV images, while reducing time and labor costs.

Semantic segmentation is one of the most important tasks in the field of remote sensing. As the spatial resolution increases, the remote sensing images can capture more detailed information and make hierarchical semantic interpretation possible. However, hierarchical semantic segmentation encounters high heterogeneity not only within the intra-layer classes but also among inter-layer classes. It brings challenges to semantic segmentation methods such as the convolutional neural network (CNN). In this article, a hierarchical self-learning knowledge inference model (HSKIM) based on the Markov random field (MRF) model is proposed for hierarchical semantic segmentation of remote sensing images. The HSKIM model introduces a new framework that integrates the advantages of CNN-based data feature learning and MRF-based hierarchical semantic inference. It contains three modules: data learning module ($\boldsymbol {D}$), inference units generation module ($\boldsymbol {I}$), and self-learning knowledge inference module ($\boldsymbol {S}$). The module$\boldsymbol {D}$uses CNN to learn specific data features layer by layer and extract preliminary geographical objects as the initial results. The module I refines the geographical objects using a novel boundary-preservation trick to generate more accurate inference units with clear geographical meaning. The module S introduces a hierarchical object-based MRF model to implement semantic inference among intra-layer and inter-layer inference units, guided by the spatial interactions and geographical criteria. This module can self-learn and update the relationship between classes iteratively and provide the final result. Experiments on the GID dataset with hierarchical classes, alongside 12 state-of-the-art CNN-based methods, validate the effectiveness and robustness of the proposed HSKIM model. The code of this article is available athttps://github.com/iichengzi/HSKIM.

Exposed surface for buildings (ESB), which refers to exposed surfaces with traces of building construction, often leads to urban dust. Accurate ESB detection is important for planning urban development and improving urban environment. Fine-grained monitoring of ESB typically needs massive high-quality pixel-level labels, which are demanding and expensive. In contrast, obtaining cost-efficient image-level labels is more promising. Most image-level weakly supervised methods can extract pixel-level pseudo labels using the class activation map (CAM) generated by the classification network. Subsequently, these labels are applied to train the semantic segmentation network. However, the CAM is easy to miss fine-grained information, which leads to label noise. Moreover, the downsampling in the segmentation networks will further loss the spatial information. Furthermore, the sparse distribution and irregular shape of ESB pose additional challenges. Given these problems, we propose a stepwise refining image-level weakly supervised semantic segmentation method (SRIWS): 1) we introduce a new data augmentation method called SRMix to oversample the classification dataset; 2) we propose a two-branch network with a superpixel pooling layer (SPNet) as the semantic segmentation network to capture both global semantic information and spatial details; and 3) to alleviate the impact of potential noise in the initial labels, we design the high-confidence sample filtering operation (HSF) during the SPNet training. The evaluation experiments for the SRIWS were performed on three datasets. The results confirm that our proposed SRIWS presents a superior performance in recognizing ESB compared with existing state-of-the-art methods. In addition, numerous ablation experimental results indicate the effectiveness and robustness of our SRIWS.

We propose a novel network for Multi-View Stereo reconstruction in the field of remote sensing, which considers Clustering-based Semantic Consistency into depth estimation optimization, referred to as CSC-MVS. In this approach, high-level semantic information acquired from multiple views is utilized to construct semantic consistency and assist in guiding the optimization of the MVS network. Specifically, the Non-negative Matrix Factorization (NMF) branch and the Deep Spectral Decomposition (DSD) branch, are designed to generate local and global semantic guidance, respectively. We then propose an uncertainty multi-task optimization method to adaptively combine matching and semantic metrics. The performance of CSC-MVS is evaluated on representative benchmarks, including the WHU TLC dataset and LuoJia-MVS dataset, demonstrating its effectiveness and generality across diverse remote sensing scenarios. Comprehensive experimental results show that our CSC-MVS significantly improves the performance of various MVS baseline networks and achieves notable accuracy in depth reconstruction. We also conduct ablation studies to validate the rationality of each component, and sensitivity analysis to confirm the robustness and adaptability of our proposed method. The code is available at https://github.com/zsl-whu/csc-mvs.

Most of the existing deep learning-based hyperspectral image (HSI) classification algorithms are based on supervised learning, where large number of annotated labels with high acquisition cost are required. Self-supervised learning (SSL) methods can learn abundant representations using large amount of unlabeled data, thereby reducing the reliability of labels. Particularly, SSL based on Masked Image Modeling (MIM) can extract fine-grained features, which is well-suited for HSI classification as a pixel-level interpretation task. However, MIM has scarcely been investigated in HSI classification. Current algorithms lack a comprehensive consideration of the multiscale spectral-spatial characteristics of HSI when constructing the pre-training task, and there exists high computational cost and redundancy when applied to large-scale HSI. Therefore, this paper develops an SSL framework based on Spectral-Spatial Hierarchical Masked Modeling (S2HM2) for large-scale HSI classification. Considering the spectral-spatial characteristics of HSI, 3D masking strategy and spectral-spatial consistency loss are proposed to construct MIM task. To fully exploit features at each scale, hierarchical 3D Feature Pyramid Network (3D-FPN) is designed as decoder for both pre-text and downstream tasks in a “pixel-to-pixel” manner. In addition, Multi-Scale Masked Feature Modeling (MS-MFM) task is proposed to further facilitate the multiscale feature learning. The SSL pre-training is guided by both MIM and MS-MFM. Experimental results on two large-scale hyperspectral datasets, i.e., WHU-OHS and WHU-H2SR, demonstrate the superiority of the proposed method. Furthermore, transfer learning experiments are conducted on a variety of hyperspectral datasets, where classification accuracies are boosted in most of the scenarios. Source code will be made available at https://github.com/tulilin/S2HM2.

Deep learning methods have been widely studied in the semantic segmentation field of the remote sensing image. Training images play an important role in these methods; however, each training image usually contains not only the generalization information of each land category but also the specific interclass context between different categories. The specific interclass context prevents deep learning methods from focusing on generalization information learning during training and limits the performance on different data distributions. This article proposes a generalization sampling learning method of deep convolutional neural network (GSL-CNN) to emphasize generalization information learning for the semantic segmentation of remote sensing images. The proposed method develops a new CBR sampling strategy that contains three modules: category grouping ($\mathbf {\boldsymbol {C}}$), basic unit extraction ($\mathbf {\boldsymbol {B}}$), and random combination ($\mathbf {\boldsymbol {R}}$). Module$\mathbf {\boldsymbol {C}}$collects each land category map and strips away the specific interclass context from the raw annotated image. Module$\mathbf {\boldsymbol {B}}$extracts basic units with different granularities from each land category map, and each basic unit can keep the generalization information of this category. Module$\mathbf {\boldsymbol {R}}$aims to enhance the robustness against different data distributions by randomly picking basic units of different categories and randomly generating their interclass context. The new GSL-CNN method integrates the CBR sampling strategy with the convolutional neural network (CNN) model for semantic segmentation. Experiments on different remote sensing datasets and 15 state-of-the-art CNN models validated that the proposed method has the potential of improving the generalization ability of the CNN method from a sampling perspective.

Semantic segmentation is one of the most important tasks in the field of remote sensing image processing. Many methods have been proposed to realize it at the pixel granularity or object granularity. Specifically, the pixel-based methods usually can effectively extract the detailed information and edges, and the object-based methods can keep the internal consistency of each land cover or land use. The Markov random field (MRF) model provides a statistical way to combine the advantages of both pixel and object granularities together. However, current MRF-based methods still face a problem, that is, how to ensure that the advantages of different granularities will complement each other, not that disadvantages will affect advantages. To solve this problem, a new multigranularity edge-preservation optimization is proposed in this letter. The proposed method first represents the image with a series of granularities from the object to the pixel by downsampling. Then, the MRF model is defined on each granularity. By defining an edge set for each granularity, during the process of downsampling, the proposed method can continuously correct edges while maintaining intraclass consistency. Experiments of Gaofen-2 and SPOT5 demonstrate the effectiveness of the proposed method. Moreover, the proposed method can be also used as the postprocessing step for deep learning. The experiment of the Pavia University hyperspectral image illustrates it for an instance of DeepLab v3+.

This study used multispectral satellite imagery (Sentinel-2 MSI) to evaluate forest type mapping capabilities over a mountainous area (Shangri-La, Yunnan Province, China) at regional level. Coupled with the cloud computing platform of Google Earth Engine, the sentinel-2 satellite images were used to extract multi-temporal and spectral information, and then combined with terrain information. The random forest algorithm was adopted to identify the typical forest types. Firstly, the area is classified into forest and non-forest types. Secondly, the forest cover was sub-classified into coniferous forest and broad-leaved forest. In the end, eight types of coniferous forests (Cupressus funebris forest, Abies forest, Pinus densata forest, Picea forest, Pinus yunnanensis forest, Larix forest, Pinus armandi forest, Tsuga dumosa forest) were identified within the cover of coniferous forest. As for the whole area, the overall accuracy of forest and non-forest was 95.76%, and the Kappa coefficient was 91.34%. Within the forest coverage, the overall accuracy of the coniferous forest and the broadleaf forest was 89.74%, and the Kappa coefficient was 79.26%. And within the coniferous forest, the overall accuracy of the eight types of coniferous forest was 91.59%, and its Kappa coefficient was 90.33%. The classification results indicated that topographic information is beneficial to the extraction of forest type information, and multi-temporal Sentinel-2 imagery has great potential to accurately identify forest type at regional level.

An accurate forest type map is important for the forestry resources monitor and management. Forest type mapping over a large and complicated mountain area is full of challenge due to complex forest type compositions, similar spectral characteristics between various forest types, lack of high quality images caused by clouds or cloud shadows, and the difficulties in managing and processing large amount data. This study aims to explore forest-type mapping methods over a mountain region with strong geographic and climate heterogeneity characteristic landscape (Yunnan Province, China). Based on Landsat OLI dense time series data, four median seasonal composites consist of 7 spectral bands and 5 vegetation indexes were derived on Google Earth Engine cloud platform. The Random Forest classifier was used in two-level classification, which includes the classification of forest/non-forest and the classification of forest-type. In two-level classification, three types of feature combination, single-seasonal composite in four seasons, multi-seasonal composite and the combination of multi-seasonal composite and three environmental factors, were set, respectively. We also compared our method with two commonly used methods. The resultant forest map was evaluated by using overall accuracy and Kappa and compared to the ground survey data and four public forest products. The results show that multi-seasonal multispectral variables and the combination with environment factors improved accuracy of forest type classification. Our result is also superior to stat-of-the-art result, FCS2020, in the research area.

Semantic segmentation is one of the most important tasks in remote sensing. However, as spatial resolution increases, distinguishing the homogeneity of each land class and the heterogeneity between different land classes are challenging. The Markov random field model (MRF) is a widely used method for semantic segmentation due to its effective spatial context description. To improve segmentation accuracy, some MRF-based methods extract more image information by constructing the probability graph with pixel or object granularity units, and some other methods interpret the image from different semantic perspectives by building multilayer semantic classes. However, these MRF-based methods fail to capture the relationship between different granularity features extracted from the image and hierarchical semantic classes that need to be interpreted. In this article, a new MRF-based method is proposed to incorporate the multigranularity information and the multilayer semantic classes together for semantic segmentation of remote sensing images. The proposed method develops a framework that builds a hybrid probability graph on both pixel and object granularities and defines a multiclass-layer label field with hierarchical semantic over the hybrid probability graph. A generative alternating granularity inference is suggested to provide the result by iteratively passing and updating information between different granularities and hierarchical semantics. The proposed method is tested on texture images, different remote sensing images obtained by the SPOT5, Gaofen-2, GeoEye, and aerial sensors, and Pavia University hyperspectral image. Experiments demonstrate that the proposed method shows a better segmentation performance than other state-of-the-art methods.

Markov random field model (MRF) is one widely used method in the field of segmentation of remote sensing images. The basic unit of the MRF model can be either the pixel or the over-segmented region, known as the object. The pixel-level and the object-level MRF models are often studied independently. But, advantages of these two types of MRF models are complementary. In order to combine their advantages, a new MRF model with alternating granularities (MRF-AG) is proposed in this paper. The MRF-AG model first uses two granular units, the pixel and the object, to build the probability graphs of the MRF model, and defines feature fields and label fields on these graphs to describe the feature patterns and the corresponding class labels. Then, a new probability inference is developed to integrate the pixel-level and the object-level information by alternately using one granular label result as the prior information of the other. With iterations, the convergence label result is achieved as the final segmentation result. Experiments on different remote sensing images demonstrate that the proposed method can provide a better performance than other state-of-the-art MRF-based methods.

The Markov random field (MRF) model has attracted great attention in the field of image segmentation. However, most MRF-based methods fail to resolve segmentation misclassification problems for high spatial resolution remote sensing images due to insufficiently using the hierarchical semantic information. In order to solve such a problem, this paper proposes an object-based MRF model with auxiliary label fields that can capture more macro and detailed information and apply it to the semantic segmentation of high spatial resolution remote sensing images. Specifically, apart from the label field, two auxiliary label fields are first introduced into the proposed model for interpreting remote sensing images from different perspectives, which are implemented by setting a different number of auxiliary classes. Then, the multilevel logistic model is used to describe the interactions within each label field, and a conditional probability distribution is developed to model the interactions between label fields. A net context structure is established among them to model the interactions of classes within and between label fields. A principled probabilistic inference is suggested to solve the proposed model by iteratively renewing the label field and auxiliary label fields, in which different information of auxiliary label fields can be integrated into the label field during iterations. Experiments on different remote sensing images demonstrate that our model produces more accurate segmentation than the state-of-the-art MRF-based methods. If some prior information is added, the proposed model can produce accurate results even in complex areas.

Markov random field (MRF)based spatial regularizing methodology can improve the maps by imposing a spatial smoothness prior on the image grid, but also leads to oversmoothing at image boundary areas. This problem is caused by the reason that classic isotropous smoothness prior cannot take local discontinuities into account. In this context, this paper proposes a novel two-step MRF regularization algorithm, which addresses the problem by combining both spectral and class cost in spatial modules. The developed MRF method first establishes a spatial energy function integrating local spectral dissimilarity to smooth the initial classification map while preserving object boundaries. Second, a new anisotropic spatial energy function integrating the class co-occurrence dependency is constructed to regularize pixels around object boundaries. The effectiveness of the proposed MRF method is validated by a series of remote sensing data sets. The obtained results indicate that the method can significantly improve the classification accuracy with regards to traditional MRF classification models.

With the advantages of the simpler transmitter architecture requirements, the wider swath capability and lower data rate, compact polarimetric (CP) synthetic aperture radar (SAR) was proposed in recent ten years to substitute or compensate the disadvantage of the full polarization mode SAR, especially on widening the swath width. CP mode is transmitted with one polarization and received with both of them. As there are only two channels, the decomposition methods and theories which were used for full polarization can not directly apply into it. According to the characteristics of CP mode, the decomposition theory based on partial polarized waves were developed, like the degree of polarization and phase difference (m - δ) decomposition, the degree of polarization and scattering angle (m - α), the degree polarization and the Poincare ellipticity parameter (m - χ) decomposition. Since α and χ are mutual complementary angle, the result of these two decomposition is same. Since the main purpose of decomposition is to classify the different objects, this paper focus on the classification result difference of m - δ and m - α, the interpretation of these results and how to improve the classification based on the better decomposition results. In this paper, the classification results of these two decomposition methods were compared with full polarization classification result. The classification overall accuracy of m - α is 97.17%, whereas m - δ is 88.28%. The kappa coefficient for the former is 0.8500, the latter is 0.8207. Since the volume scattering component is same, the differences were caused by surface scattering component and dihedral component. The statistics of these two components shown that α has better accumulativeness than δ, which lead to the higher classification accuracy. Both of these two method result in higher assessment of volume scattering.

Gaofen-2 is a Chinese satellite launched in August 2014, which provides high-resolution imagery for Earth observation. In this paper, we propose a novel remote sensing pansharpening method based on the guided image filtering. A lower resolution panchromatic (PAN) is simulated, then the multispectral (MS) image is employed as a guidance image to the filtering process of this synthesized image to extract the spatial information. Finally, the spatial details are further modulated and injected into each band of resampled MS image. The feasibility of this method is demonstrated by a case study of Gaofen-2 imagery. Four evaluation metrics assessing spatial and spectral qualities of pansharpened image are considered for quantitative assessment. When compared to principal component analysis (PCA), Gram Schmidt (GS) and University of New Brunswick (UNB) Pansharpen methods, this algorithm can preserve the edge of original PAN band and show good capability in spectral fidelity.

An improved mean shift segmentation method featuring adaptive parameter selection is presented in this paper. We associate the bandwidths and weight for each point in a spatial-range feature space with boundary information in an image plane. Varying weight and bandwidth for each pixel are assigned according to a boundary map, which is obtained by learning multiple edge cues. We consider two groups of edge cues and two regressing modules, approach the cue combination as a supervised learning problem from the ground truth data (manually sketched boundary maps). From our preliminary results, the provided method can combine the top-down information got from regression models with the mean shift process and constrain over-clustering of pixels belonging different land objects.

This letter proposes an improved mean-shift filtering method. The method is added as a preprocessing step for regional segmentation methods, which aims at benefiting segmentations in a more general way. Using this method, first, a logistic regression model between two edge cues and semantic object boundaries is established. Then, boundary posterior probabilities are predicted by the model and associated with weights in the mean-shift filtering iteration. Finally, the filtered image, instead of the original image, is put into segmentation methods. In experiments, the regression model is trained with an aerial image, which is tested with an aerial image and a QuickBird image. Two popular segmentation methods are adopted for evaluations. Both quantitative and qualitative evaluations reveal that the presented procedure facilitates a superior image segmentation result and higher classification accuracy.

The multiresolution technique is one of the most important techniques for image segmentation. Wavelet transformation is a pixel-based method and is widely used for multiresolution segmentation approaches, but it suffers the deficiency of modeling the macrotexture pattern of a given image. In order to overcome such a problem, this letter extends the multiresolution technique from the pixel level to the region level and proposes a new image segmentation model by incorporating the multiregion-resolution and the Markov random field model. Experiments are conducted using synthetic-aperture-radar data and remote sensing images, which demonstrates that our method can improve the segmentation accuracy compared with the multiresolution method based on the pixel level.

In the tree-structured Markov random field (TS-MRF) model, a sequence of MRFs was hierarchically defined on the single spatial resolution in the format of a tree structure which might suffer from the deficiency of modeling the nonstationary property of a given image. In order to overcome such a problem and motivated by nonredundant directional selectivity and highly discriminative nature of the wavelet representation, we attempt to introduce the TS-MRF model into the wavelet domain and propose a new image modeling method-WTS-MRF, in which each MRF is defined over a multiresolution subset of the lattice sites corresponding to the wavelet decomposition. Based on WTS-MRF, a supervised image segmentation algorithm is carried out, and experiment on a remotely sensed image proves the better performance than the supervised segmentation algorithm based on the TS-MRF model.