Individual tree segmentation from terrestrial laser scanning (TLS) point clouds is essential for precise forest inventory, instance-level tree modeling, and the estimation of forest stock volume. However, current instance-level segmentation techniques encounter significant challenges in complex forest environments, particularly those characterized by dense understory vegetation and substantial crown overlap in natural forests. These complexities reduce segmentation accuracy and limit the generalizability of existing methods across diverse forest types. This paper presents a unified method for individual tree segmentation that integrates trunk localization with crown segmentation. The trunk localization uses normal vector features to eliminate non-trunk slice points, employs an enhanced DBSCAN algorithm for trunk slice separation, and refines trunk positions by fitting circular-like trunk slices using the Hough transform. This integrated approach ensures precise segmentation and optimization of final trunk positions. Subsequently, a graph-based optimization method is applied for crown segmentation. This method incorporates supervoxel technology, an optimal Euclidean distance metric between supervoxels, and a supervoxel similarity metric to construct an optimal undirected graph. Tree crown supervoxels are segmented by tracing the shortest path from the crown supervoxels to their corresponding tree roots. We validated the proposed method on eight sample plots representing various complexities and forest types. For tree trunk localization, the proposed method achieved an average Mean accuracy of 0.761, which is 27% higher than the best result among the three traditional methods. For crown segmentation, it achieved an average mIoU of 0.645, marking a 31% improvement over the best baseline performance. The source code for our individual tree segmentation method is available at https://github.com/TLS-tree/tree-segmentation.

Integrating low-resolution hyperspectral images (LR-HSIs) with corresponding high-resolution multispectral images (HR-MSIs) for the reconstruction of HR-HSI using deep learning techniques represents a critical area of research. Although convolutional neural networks (CNNs) are widely used for HR-HSI reconstruction, their small receptive fields hinder effective global feature extraction, which limits their potential. Fusion methods that rely on traditional attention mechanisms also lack feature interaction, failing to integrate and harmonize feature information from hyperspectral image (HSI) and MSI effectively. To address these issues, this article develops a novel progressive fusion of multiscale axial attention network (PMA2-Net), which combines multiscale convolutions with axial attention (AA) and employs a progressive interaction approach to reconstruct high-resolution images. Specifically, PMA2-Net extracts the spatial and spectral information from HSI through spatial feature extraction (Spatial-FE) and spectral feature extraction (Spectral-FE). Concurrently, a feature injection module (FIM) is introduced, employing multiscale convolution to capture local features and integrating AA to enhance global feature association. Moreover, a progressive fusion module (PFM) is employed to enhance multidimensional feature collaboration and hierarchical integration. Extensive studies conducted using five significant HSI datasets verify the effectiveness of PMA2-Net, demonstrating its superior performance compared to current state-of-the-art (SOTA) fusion techniques.

Morphological methods play a crucial role in remote sensing image processing, due to their ability to capture and preserve small structural details. However, most of the existing deep learning models for semantic segmentation are based on encoder-decoder architectures including U-Net and Segment Anything Model (SAM), where the downsampling process tends to discard fine details. In this paper, we propose a new approach that integrates learnable morphological skeleton prior into deep neural networks using the variational method. To address the difficulty in backpropagation in neural networks caused by the non-differentiability presented in classical morphological operations, we provide a smooth representation of the morphological skeleton and design a variational segmentation model integrating morphological skeleton prior by employing operator splitting and dual methods. Then, we integrate this model into the network architecture of SAM, which is achieved by adding a token to mask decoder and modifying the final sigmoid layer, ensuring the final segmentation results preserve the skeleton structure as much as possible. Experimental results on remote sensing datasets, including buildings, roads and water bodies, demonstrate that our method outperforms the original SAM on slender object segmentation and exhibits better generalization capability.

The accurate extraction of traffic signs is of great significance to the digitization of traffic information and the fine management of traffic. This article introduces an innovative approach to address the challenges associated with recognizing and detecting traffic signs, considering their vulnerability to complex backgrounds, variations in illumination, and motion blur. The proposed method utilizes a semi-supervised learning (SSL) strategy, combining convolutional neural networks (CNNs) with a transformer encoder–decoder architecture, to extract traffic sign features from vehicle panoramic images. To enhance feature extraction, a hierarchical sampling method (HSM) is introduced, which facilitates the extraction of multiscale self-attention features in the transformer encoder–decoder structure. Additionally, a network module called local and global information aggregator (LGIA) is designed based on HSM, enabling the incorporation of both local and global context information. Furthermore, a SSL strategy is adopted to simultaneously train our model using both labeled and unlabeled data samples. This strategy aims to improve the extraction of traffic signs by capitalizing on the broader data set available through unlabeled data. Experimental results demonstrate the effectiveness and robustness of the proposed method in improving the detection and recognition of traffic signs. The approach showcases significant improvements in overcoming the challenges posed by complex backgrounds, variations in illumination, and motion blur. Our approach achieved a 0.9% improvement in the F1-score evaluation over the current classical object detection algorithm on the public data set Tsinghua-Tencent 100K and a 1.1% improvement on the SSW data set.

The application of deep learning-based semantic segmentation in remote sensing (RS) images has achieved considerable success. However, many supervised methods still heavily rely on a large amount of labeled data, requiring time-consuming and labor-intensive manual annotations. Besides, networks trained on labeled source domain data often perform poorly in inference tasks with target domain data due to the domain shift phenomenon. To address these challenges, we construct a novel end-to-end unsupervised domain adaptation (UDA) framework, named data augmentation and mobile threshold self-supervision (DA&MTSS), which integrates data augmentation with self-supervision. Specifically, we analyze the common factors that cause domain shifts in RS images and adopt different data augmentation techniques to attenuate the domain shift and enhance the generalization ability and robustness of the network in cross-domain inference. In the self-supervision phase, we design a new sample-based mobile threshold method to dynamically control the thresholds of both dominant and long-tail classes during training and generate stable pseudo-labels. Therefore, our method eliminates the need for additional training or expert knowledge and achieves the co-evolution of network parameters and pseudo-label quality in the training process. The results of comprehensive experiments on five tasks across the ISPRS Vaihingen, Potsdam, and LoveDA datasets demonstrate that this method consistently achieves higher mIoU scores, showcasing the performance advantage of DA&MTSS in UDA for the semantic segmentation of RS image.

The semantic segmentation network performance of large-scale outdoor point clouds is usually limited by the number of input point clouds. In the application of most methods, the point cloud is cut into small pieces as the training input, which will not only lead to heavy preprocessing burdens but also destroy the overall geometric structure of the scene. The transformer network has demonstrated remarkable advantages of attention mechanisms in focusing on crucial features and improving model performance. However, its training and inference on large-scale data are confined by computational complexity. To eliminate these challenges, the attention mechanisms are improved to enhance their performance in processing large-scale input data, while removing limitations imposed by computational complexity. Moreover, a novel local feature enhancement (LFE) module is developed to construct the LFE-Net, which can accurately and efficiently extract spatial and attribute features from local point clouds. In particular, an improved attention module, which is called simple offset attention (SOA), is adopted for local point cloud spatial feature learning. Compared with self-attention, SOA requires less memory and can better capture the fine-grained local features. Furthermore, to effectively avoid the destruction of object geometry and diminish the impact of sample imbalance, a training sample collection method based on the number of different classes is designed. To validate the effectiveness of this method, some experiments are conducted based on publicly accessible outdoor point cloud datasets. The results demonstrate that the LFE-Net can achieve substantial improvements compared with other cutting-edge network models.

The building contours, as one of the most important features for representing geometry, are widely used in various applications including urban modeling and reconstruction. Automatic extraction of high-fidelity compact contours from unstructured point clouds is rather challenging, and the existing methods are limited in generating global-aware and artifact-free building contours. Here, we approach contour extraction as a Bayesian inference problem. Two ideas are proposed to obtain building contour points and edges directly from unstructured point clouds. First, we construct a point-edge collaborative (PECo) Bayesian framework to couple the information of contour points and contour edges. The developed model fully takes local contour features, global edge structures, and global geometric priors into account. Second, given the Bayesian framework for building contouring, we leverage an expectation maximization (EM) algorithm to iteratively infer the contour edges in a maximum posteriori manner. The alternate EM iterations between point and edge domains progressively refine the local pointwise information to a global representation of contour edges. As a result, a synergistic effect between the point features and edge structures for global-aware building contouring is attained. Our approach outperforms the state-of-the-art methods in terms of geometric accuracy and structural compactness in modeling buildings with various complexities. Furthermore, the compactness of the contouring results can be more flexible and easily controlled.

High-resolution remote sensing image (HRSI) scene classification often faces challenges; for example, the intraclass similarity is low, but the interclass similarity is high due to complex backgrounds and variable scene scales. Convolutional neural networks (CNNs), the leading methods for HRSI scene classification, offer excellent performance. However, traditional CNNs require fixed-size inputs, which are a limitation when dealing with HRSI that represent large image domains, potentially degrading classification performance. To overcome these problems, we propose a CNN-based model named GLR-CNN in this article. First, to capitalize on the information from large-scale scenes adequately, VGG16 is utilized to extract the deep representative features, fine-tuned by the target HRSI of any size. Furthermore, a multilayer feature fusion block based on the channel–spatial attention algorithm is integrated into the CNN to capture more discriminative features from arbitrary-size images. Finally, to enhance the consistency between image features and similarities, a global latent relationship is used to measure the similarities among image features, then embed it into the fully connected layers (FCLs), and construct the latent relationship constraint. The model is optimized by the joint objective function including the latent relationship constraint and cross-entropy loss with label smoothing. Extensive experiments on three HRSI datasets obtained improvements of 3.93%, 6.5%, and 2.2% in overall accuracy compared to the finetuned VGG16 model, proving the effectiveness of the GLR-CNN method.

Hyperspectral image classification (HSIC) is widely used in such fields as vegetation classification and fine agriculture. Nowadays, numerous classification models based on Transformer networks are inputting data typically in the form of patches. However, such patch-based input format may weaken the spatial relationships between neighbor pixels, thus limiting the ability to obtain contextual semantic information. Additionally, hyperspectral images (HSIs) generally contain hundreds of bands, which in turn include some redundant information. The original spectral vectors may lead to redundancy of information, thus reducing the classification accuracy remarkably. Therefore, in this article, a patch-based transformer network construction with adaptive feature-interaction (AFi) for HSIC called AFinet is developed for HSIC. As an end-to-end network, AFinet consists of a feature extraction module and an AFi module. For the developed model, the feed-forward neural network (FNN) in the standard Transformer framework suffers from a limited ability in exploiting local contexts. In this article, an expression-enhanced FNN (E2FNN) is introduced, which can incorporate depthwise convolution layers to capture local contextual information, so as to enhance the correlations between neighbor pixels. Moreover, this study designs an AFi module to facilitate the sharing of interaction features among Transformer-extracted features. Subsequently, global spatial feature information can be integrated into each spectral channel. The AFi module also includes a channel attention mechanism to help focus more attention on channels that contain critical information, while paying less attention to channels with little critical information. After the approach proposed in this study was tested on three widely used datasets, the experimental results demonstrate that the AFinet developed in this study can effectively improve classification accuracy.

Unseen category prediction is a common challenge for real-world applications, especially for remote sensing (RS) imagery interpretation. Zero-shot learning (ZSL)--based scene classification methods have made significant progress recently, providing an effective solution for unseen scene recognition with semantic embeddings that link seen and unseen classes in the field of RS. However, existing ZSL methods mainly focus on semantic feature exploration, they failed to combine image features and semantic features effectively. To address the aforementioned challenges, we propose a novel knowledge graph-based zero-shot learning model that adeptly integrates both image and semantic features for disaster RS scene recognition. First, we construct an RS knowledge graph to generate semantic features of RS scenes, enhancing the reasoning ability from conventional RS scene categories to disaster RS scene categories. Second, we propose an Interactive Attention mechanism to integrate image and semantic features, focusing on the most informative regions. Finally, we introduce an RS domain adapter that enables the model to better adapt to remote sensing data, reproject common features into the remote sensing domain, and thus solve zero-shot remote sensing scene classification tasks. To demonstrate the effectiveness of our method, we construct a remote sensing disaster scene dataset, which contains 8700 high-quality disaster scenes. Extensive experiments show that our proposed method outperforms current state-of-the-art methods under zero-shot RS image scene classification settings.

Accurate and automatic detection of road surface element (such as road marking or manhole cover) information is the basis and key to many applications. To efficiently obtain the information of road surface element, we propose a content-adaptive hierarchical deep learning model to detect arbitrary-oriented road surface elements from mobile laser scanning (MLS) point clouds. In the model, we design a densely connected feature integration module (DCFM) to connect and reorganize feature maps of each stage in the backbone network. Besides, we propose a hierarchical prediction module (HPM) to innovatively use the reorganized feature maps to recognize different types of road surface elements, and thus, semantic information of road surface element can be adaptively expressed on multilevel feature maps. We also add a cascade structure (CS) in the head of model to detect the target efficiently, which can learn the offset between the predicted minimum bounding box of road surface element and ground truth. In experiments, we prove that the proposed method mainly contributed by HPM can maintain robust detection performance, even in the cases of unbalanced category number or overlapping of road surface elements. The experiments also prove that the proposed DCFM can improve the recognition effects of small targets. The CS for predicting boundary offset can detect each target more accurately. We also integrate the designed modules into some rotation detectors, e.g., the EAST and R3Det, and achieve the state-of-the-art results in three road scenes with different categories and uneven distribution of road surface elements, which further shows the effectiveness of the proposed method.

Urban functional zones (UFZs) contain abundant landscape information that can be adopted to better understand the surroundings. Various landscape compositions and configurations reflect different human activities, which may affect the particulate matter (PM2.5) concentrations. The very high-resolution (VHR) image features can reflect the physical and spatial structures of the UFZs. However, the existing PM2.5 estimation methods neither have been based on the scale of UFZs, nor have the VHR image features of UFZs as independent variables. Hence, this article proposes a spatiotemporal interpolation graph convolutional network (STI-GCN) model and introduces VHR image features to achieve PM2.5 estimation in UFZs. First, UFZs are split, and VHR image features are extracted by the visual geometry group 16 (VGG16). Subsequently, meteorological factors, aerosol optical depth (AOD), and VHR image features are used to estimate the PM2.5 concentrations at the scale of the UFZs. The two metropolises, Beijing and Shanghai, are chosen to assess the validity of the STI-GCN model. As for Beijing and Shanghai, the overall accuracy${R^{2}}$of the STI-GCN model can reach 0.96 and 0.89, the root-mean-square errors (RMSEs) are 8.15 and 6.40$\mu \text {g}/{\text {m}^{3}}$, the mean absolute errors (MAEs) are 5.51 and 4.78$\mu \text {g}/{\text {m}^{3}}$, and the relative prediction errors (RPEs) are 18.53% and 17.38%, respectively. Experiments show that the STI-GCN consistently outperforms other models. What’s more, the PM2.5 values are relatively high in commercial and official zones (COZs) and relatively low in urban green zones (UGZs).

Current deep learning methods for semantic segmentation in remote sensing heavily depend on a substantial amount of labeled data. However, obtaining pixel-level labeled data in this field is both time-consuming and laborious. To address this challenge, semi-supervised learning methods have been introduced. Pseudo supervision is one of the most effective methods, which can be adopted to enhance the performance of semi-supervised semantic segmentation of remote sensing images [1]. But incorrect pseudo labels can cause substantially distortions to the segmentation model in semi-supervised learning. Moreover, it is difficult for conventional semantic segmentation methods to deal with global-local features of the remote sensing image without adaptive context feature. In this paper, we propose a novel learning approach based on an adaptive context transformer and pseudo labeling, called Adaptive Context Transformer for semi-supervised (ACTSS) remote sensing image segmentation. We propose an adaptive context attention model with adjustable sliding windows. A small window is used to capture Query (Q) for local feature and bigger windows are used to capture Key (K) and Value (V) for global feature. Then we combine them and get the global-local feature. And we propose a point-line-plane pseudo label filter (PLP) mechanism based on clustering and boundary extraction, which can filter unreliable pseudo labels from three angles: point, line and plane. To validate the effectiveness of the model, we carried out extensive experiments on the LOVEDA, Potsdam and Vaihingen datasets, and compared ACTSS with other methods. These experiments demonstrate that ACTSS achieves state-of-the-art performance for semi-supervised semantic segmentation on all tested datasets.

Deep learning (DL) become increasingly popular in remote sensing (RS) change detection (CD), leading to the development of massive networks that surpass traditional methods in accuracy and automation. However, the need for enormous amounts of annotated data remains a major concern and accurate boundary segmentation in RS images is challenging due to their complexity and heterogeneity. Moreover, properly aggregating the bi-temporal feature pairs and creating highly discriminative change features are crucial for detection performance. This article proposes a boundary-enhanced semi-supervised network (BESSL-Net) to tackle these issues for CD tasks. The network adopts dual encoders and one decoder architecture for segmentation and incorporates pseudo-labeling, contrastive learning, along with a teacher-student scheme to leverage unlabeled data. A boundary extraction module (BEM) is used to conduct boundary segmentation, while a change segmentation feature learning module (SFLM) is employed to create discriminative change features in both channel and spatial domains by integrating multi-level features. Three publicly available CD datasets are utilized to validate the proposed BESSL-Net. Compared to the current state-of-the-art networks, the semi-supervised network demonstrates advanced performance metrics, especially regarding Intersection over Union (IoUc) of change-class, showing improvements ranging from 1% to 9%.

Deep subspace learning (DSL) plays an essential role in hyperspectral image classification, providing an effective solution tool to reduce the redundant information of hyperspectral image (HSI) pixels. Semi-supervised convolutional neural network (CNN)-based DSL methods can extract a more representative representation of latent subspace with the help of the labeled and unlabeled data. However, CNN-based DSL methods may lose the information of the class boundaries leading to misclassifications within regular input. We develop the deep subspace learning method with boundary consistency (DSL-BC) for the HSI classification to address this problem. In DSL-BC, the convolutional autoencoder (CAE) is first applied to extract the deep subspace representation (DSR). The DSR is used to model the boundary consistency. The graph convolutional network (GCN) is further adapted to enforce the boundary consistency by conducting the graph convolution on arbitrarily structured non-Euclidean data and irregular image regions. In addition, the adaptive entropy rate (ER) superpixel segmentation algorithm is applied to generate superpixels, and superpixel constraint is employed to improve the ability of DSL and GCN construction. DSL-BC integrates the DSL, the GCN, and the superpixel constraint into a unified objective function. A customized iterative algorithm is used to solve the objective function of the DSL-BC. The experimental results on three challenging public HSI datasets demonstrate that the DSL-BC can outperform the related state-of-the-art HSI classification methods.

While deep learning algorithms have achieved good results in hyperspectral image (HSI) classification, several supervised classification algorithms rely on a large number of labeled samples to get adequate performance. Collecting a large number of labeled samples is expensive in many real applications. To address this issue, a novel semisupervised HSI classification framework called deep representative feature learning (DRFL) with virtual adversarial training (DRFL-VAT) is developed in this article. By embedding the local manifold learning (LML) into the fully connected layers of a convolutional neural network (CNN), our newly developed DRFL can learn representative features. The VAT regularization is adopted to exploit the prediction label distribution of training samples and addresses the overfitting problem. Finally, the objective function of DRFL-VAT is solved by a customized algorithm. We test our method on three widely public HSI datasets and our results show that our method is competitive when compared to other state-of-the-art approaches.

Scene classification is a fundamental task for numeral remote sensing applications, which aims to assign semantic labels to image patches. Although deep neural networks (DNN) demonstrated unique strength in scene classification, their performances are still limited due to the lack of training samples in the remote sensing field. Recent studies show that the performance of scene classification can be improved by taking advantage of the knowledge transferred from models pre-trained on RGB images. However, the modalities differences between input images hinder the knowledge transfer across models, especially when the input of the models has distinct spectral bands. To tackle the challenges, we propose a cross-modal knowledge distillation framework to improve the performance of multispectral scene classification by transferring the prior knowledge from teacher models pre-trained on RGB images to the student network with limited samples. Moreover, a teacher assistant (TA) network is introduced to further improve the classification performance by bridging the gap between the teacher and student networks. The proposed strategy is evaluated on models with multimodality inputs with distinct spectral bands and demonstrates superior performance as compared to the state-of-the-art methods.

The point cloud registration is important and necessary for the applications of changing detection, deformation monitoring, and so on, which is also challenging due to the vast clustered points, irregular, and complex structures of spatial objects, and quality effects of the labeled corresponding points. Aiming at this problem, we design an end-to-end 3-D graph deep learning framework of point cloud registration, which can simultaneously learn the detector (graph attention expression) and the descriptor (graph deep feature) for point cloud registration in a weakly supervised way, so that the learned detector and descriptor promote each other in the process of model optimization. Then, the detector is used to automatically extract the keypoints, and the descriptor describes the deep feature of each keypoint. In the framework, we innovatively propose a new module (named MLP_GCN), which fuses multilayer perceptron (MLP) and graph convolutional network (GCN). The MLP_GCN module is further integrated into the detector branch and descriptor branch to fully express the detector and descriptor of the point cloud. In the training process of the framework, we rotate and translate the point cloud randomly to form the training data in a weakly supervised way, which can save plenty of manually labeling time of corresponding points. In the experiments, our method can achieve better results of point cloud registration in comparison with other methods, which verifies the advantages of the proposed method.

Land use mapping (LUM) of a coal mining subsidence area (CMSA) is a significant task. The application of convolutional neural networks (CNNs) has become prevalent in LUM, which can achieve promising performances. However, CNNs cannot process irregular data; as a result, the boundary information is overlooked. The graph convolutional network (GCN) flexibly operates with irregular regions to capture the contextual relations among neighbors. However, the global context is not considered in the GCN. In this paper, we develop the deep global context construction with enabled boundary (DGCC-EB) for the LUM of the CMSA. An original Google Earth image is partitioned into nonoverlapping processing units. The DGCC-EB extracts preliminary features from the processing unit that are further divided into nonoverlapping superpixels with irregular edges. The superpixel features are generated and then embedded into the GCN and vision transformer (ViT). In the GCN, the graph convolution is applied to superpixel features; therefore, the boundary information of objects can be preserved. In the ViT, the multihead attention blocks and positional encoding build the global context among the superpixel features. The feature constraint is calculated to fuse the advantages of the features extracted from the GCN and ViT. To improve the LUM accuracy, the cross-entropy (CE) loss is calculated. The DGCC-EB integrates all modules into a whole end-to-end framework and is then optimized by a customized algorithm. The results of case studies show that the proposed DGCC-EB obtained acceptable OA (89.06%/88.68%) and Kappa (0.86/0.87) values for Shouzhou city and Zezhou city, respectively.

Hyperspectral image (HSI) classification is essential in remote sensing image analysis. The classification methods based on deep learning have attracted more and more attention. However, classification accuracy is seriously affected by the quantity of labeled data and redundant information. Therefore, a deep semisupervised shared subspace learning (DS4L) model is developed to overcome these problems in this article. DS4L is composed of two parts. First, the basic feature extraction (BFE) network is constructed to preliminary extract high-dimensional-space features of multiscale data and fusion them to one shared subspace. Then, a deep shared subspace learning (DSSL) network is proposed to obtain a deeper and more representative low-dimensional subspace. Moreover, to obtain a more representative subspace and alleviate dependence on labeled samples, the regular, irregular constraint, and cross-entropy (CE) loss are integrated into the model. The regular constraint is adopted to reconstruct the multiscale patches to ensure the quality of the subspace in an unsupervised manner. The irregular constraint can well embed labeled and unlabeled samples into the procedure of subspace learning (SL). Then, the CE loss is used to extract more discriminative subspace using the limited labeled samples. Finally, we perform experiments on three widely used HSI datasets. Compared with the basic SL model, the DS4L’s classification accuracy on the popular Salinas, Indian Pines, and PaviaU datasets are increased by 4.68%, 4.25%, and 1.57%, respectively.

The rapid development of hyperspectral remote sensing technology, has led to an explosion in the number of available hyperspectral images (HSI). The fast and accurate characterization of HSI poses a significant challenge for remote sensing scientists. Currently, deep learning strategies with various neural networks have been successfully applied for HSI classification using the concept of the “dataset-model”. Still, there is a need to develop universal deep learning models for HSI classification using a continual updating strategy. This paper presents a life-long learning strategy to continually update model weights with the help of continual spectral-spatial feature distillation. Specifically, the proposed method introduces a spectral-spatial distillation strategy to retain knowledge of the previous well-trained model. Meanwhile, the learning metric term is integrated into a multi-level feature extraction to minimize the spectral-spatial feature discrepancy between the previous model and the new one. The experimental results indicate that our method achieves superior performance for continual HSI classification tasks without suffering from the persistent loss of characterization memory.

Metro subway systems with underground tunnels form the backbone of urban transportations and therefore, accurate monitoring and maintenance of such subway systems are extremely necessary for a hassle-free daily commutation of billions of people. Though 3-D models of tunnels are widely used for the deformation monitoring of such subway tunnels, existing model-based tunnel monitoring systems rely on coarse geometric models and hence fail to capture complete tunnel health information. We present a two-stage algorithm to create high-fidelity geometric models of tunnel lining from Terrestrial Laser Scanning (TLS) point clouds. Tunnel geometry, defined at the detailed block entity level, is constructed through a data-driven block segmentation algorithm and a model-driven assembly technique. In our approach, the 3-D tunnel block segmentation problem has been translated into a bolt and lining joint recognition problem from 2-D images unfolded from the 3-D scans. The segmented 3-D blocks are matched with a set of predefined 3-D templates from a primitive library via a constraint total least squares matching method and the matched 3-D templates are assembled to create the final watertight tunnel model. The proposed tunnel modeling method has been comprehensively evaluated on Changzhou, Nanjing, and Wuhan tunnel data sets in terms of outliers, missing data, point density, topological representation, robustness, and geometric accuracy. The experiments on Nanjing and Changzhou metro tunnels show that the geometric model fitting incurs an error of only 7 mm, which is almost consistent with a mean density of 6 mm of these two data sets. Experimental results validate the advantages and potentials of the proposed tunnel modeling method.

Subspace learning (SL) plays an essential role in hyperspectral image (HSI) classification since it can provide an effective solution to reduce the redundant information in the image pixels of HSIs. Previous works about SL aim to improve the accuracy of HSI recognition. Using a large number of labeled samples, related methods can train the parameters of the proposed solutions to obtain better representations of HSI pixels. However, the data instances may not be sufficient to learn a precise model for HSI classification in real applications. Moreover, it is well known that it takes much time, labor, and human expertise to label HSI images. To avoid the abovementioned problems, a novel SL method that includes the probability assumption called SL with the conditional random field (SLCRF) is developed. In SLCRF, the 3-D convolutional autoencoder (3DCAE) is first introduced to remove the redundant information in HSI pixels. Besides, the relationships are also constructed using spectral-spatial information among the adjacent pixels. Then, the conditional random field (CRF) framework can be constructed and further embedded into the HSI SL procedure with the semisupervised approach. Through the linearized alternating direction method termed LADMAP, the objective function of SLCRF is optimized using a defined iterative algorithm. The proposed method is comprehensively evaluated using the challenging public HSI data sets. We can achieve state-of-the-art performance using these HSI sets.

Accurate and efficient extraction of road marking plays an important role in road transportation engineering, automotive vision, and automatic driving. In this article, we proposed a dense feature pyramid network (DFPN)-based deep learning model, by considering the particularity and complexity of road marking. The DFPN concatenated its shallow feature channels with deep feature channels so that the shallow feature maps with high resolution and abundant image details can utilize the deep features. Thus, the DFPN can learn hierarchical deep detailed features. The designed deep learning model was trained end to end for road marking instance extraction with mobile laser scanning (MLS) point clouds. Then, we introduced the focal loss function into the optimization of deep learning model in road marking segmentation part, to pay more attention to the hard-classified samples with a large extent of background. In the experiments, our method can achieve better results than state-of-the-art methods on instance segmentation of road markings, which illustrated the advantage of the proposed method.

Classification of airborne laser scanning (ALS) point clouds is needed in digital cities and 3-D modeling. To efficiently recognize objects in ALS point clouds, we propose a novel hierarchical aggregated deep feature representation method, which can adequately employ spatial association of multilevel structures and deep feature discrimination. In our method, a 3-D deep learning model is constructed to represent the discriminative feature of each point cluster in a hierarchical structure by decreasing the within-class distance and increasing the between-class distance. Our method aggregates the discriminative deep features in different levels into a hierarchical aggregated deep feature that considers the spatial hierarchy and feature distinctiveness. Lastly, we build a multichannel 1-D convolutional neural network to classify the unknown points. Our tests demonstrate that the proposed hierarchical aggregated deep feature method can enhance point cloud classification results. Comparing with seven state-of-the-art methods, those results also verified the superior performance of our method.

With a small number of labeled samples for training, it can save considerable manpower and material resources, especially when the amount of high spatial resolution remote sensing images (HSR-RSIs) increases considerably. However, many deep models face the problem of overfitting when using a small number of labeled samples. This might degrade HSR-RSI retrieval accuracy. Aiming at obtaining more accurate HSR-RSI retrieval performance with small training samples, we develop a deep metric learning approach with generative adversarial network regularization (DML-GANR) for HSR-RSI retrieval. The DML-GANR starts from a high-level feature extraction (HFE) to extract high-level features, which includes convolutional layers and fully connected (FC) layers. Each of the FC layers is constructed by deep metric learning (DML) to maximize the interclass variations and minimize the intraclass variations. The generative adversarial network (GAN) is adopted to mitigate the overfitting problem and validate the qualities of extracted high-level features. DML-GANR is optimized through a customized approach, and the optimal parameters are obtained. The experimental results on the three data sets demonstrate the superior performance of DML-GANR over state-of-the-art techniques in HSR-RSI retrieval.

Urban road extraction has wide applications in public transportation systems and unmanned vehicle navigation. The high-resolution remote sensing images contain background clutter and the roads have large appearance differences and complex connectivities, which makes it a very challenging task for road extraction. In this article, we propose a novel end-to-end deep learning model for road area extraction from remote sensing images. Road features are learned from three levels, which can remove the distraction of the background and enhance feature representation. A direction-aware attention block is introduced to the deep learning model for keeping road topologies. We compare our method on public remote sensing data sets with other related methods. The experimental results show the superiority of our method in terms of road extraction and connectivity preservation.

Classification is an important application of hyperspectral image (HSI). However, it is also a challenging research topic due to the spatial variability of spectral signature and limited training samples. To address these problems, a novel unsupervised feature learning method called latent relationship guided the stacked sparse autoencoder (LRSSAE) is developed in this article, which can effectively exploit the latent relationship under feature space to improve the ability of feature learning. Moreover, the superpixels constraint is employed on the feature representation to avoid the “salt-and-pepper” problem, and it is enforced on the latent relationship to enhance the latent relationship learning additionally. In LRSSAE, combining the stacked sparse autoencoder (SSAE) with the graph regularizations of latent relationship in each hidden layer and the superpixel constraints in the top layer, we extract feature representation in an unsupervised manner. And then, we present a customized iterative algorithm to optimize the LRSSAE. We evaluate the proposed method on three widely used HSI data sets comprehensively. The results demonstrate that our method achieves promising classification performance on these data sets and obtains improvements of 5.06%, 5.77%, and 2.11% in overall accuracy compared to the best SSAE method.

Due to the large intraclass variances and complicated object distribution, recognizing objects with complex appearances and arbitrary orientations has been an active research topic and a challenging task in remote sensing fields. In this article, we formulate object recognition as a high-level feature-learning problem, and a novel supervised method is proposed to learn high-level feature representations from high-resolution remote sensing images for object recognition. Our method simultaneously and coherently achieves high-level feature learning and classifier training, which improves the recognition performance. Two constraints that enforce the label consistencies of group images and label consistencies of single images are introduced in a deep learning framework to obtain the high-level feature space. The high-level feature and a multiclass linear classifier are finally learned by an effective optimization algorithm. Experimental results demonstrate the superior performance of the proposed method over many state-of-the-art techniques in object recognition.

Multilabel remote sensing (RS) image annotation is a challenging and time-consuming task that requires a considerable amount of expert knowledge. Most existing RS image annotation methods are based on handcrafted features and require multistage processes that are not sufficiently efficient and effective. An RS image can be assigned with a single label at the scene level to depict the overall understanding of the scene and with multiple labels at the object level to represent the major components. The multiple labels can be used as supervised information for annotation, whereas the single label can be used as additional information to exploit the scene-level similarity relationships. By exploiting the dual-level semantic concepts, we propose an end-to-end deep learning framework for object-level multilabel annotation of RS images. The proposed framework consists of a shared convolutional neural network for discriminative feature learning, a classification branch for multilabel annotation and an embedding branch for preserving the scene-level similarity relationships. In the classification branch, an attention mechanism is introduced to generate attention-aware features, and skip-layer connections are incorporated to combine information from multiple layers. The philosophy of the embedding branch is that images with the same scene-level semantic concepts should have similar visual representations. The proposed method adopts the binary cross-entropy loss for classification and the triplet loss for image embedding learning. The evaluations on three multilabel RS image data sets demonstrate the effectiveness and superiority of the proposed method in comparison with the state-of-the-art methods.

Image hashing has been widely used in image retrieval tasks. Many existing methods generate hashing codes based on image feature representations. They rarely consider the rich information such as image clustering information contained in the image set as well as uncertain relationships between images and tags simultaneously. In this paper, we develop a Weighted Generative Adversarial Networks (WeGAN) to transfer the clustering information of images to construct the hashing code. WeGAN consists three modules: 1) a hashing learning process for transferring knowledge of the image set to hashing codes of single images; 2) by means of hashing codes, a module to generate image content, tag representation, and their joint information which reflects the correlation between the image and the corresponding tags; 3) a discriminator to distinguish the generated data from the original source, and then formulating three loss functions. Different weights are assigned to these loss functions in order to deal with the uncertainties between images and tags. Through introducing the image set to process the image hashing with different tags, WeGAN can naturally provide the information of clustering results, which is useful for image hashing with multi-tags. The generated hashing code has the ability to dynamically process the uncertain relationships between images and tags. Experiments on three challenging datasets show that WeGAN outperforms the state-of-the-art methods.

The performance of hyperspectral image (HSI) classification relies on the pixel information obtained from hundreds of contiguous and narrow spectral bands. Existing approaches, however, are limited to exploit an appropriate latent subspace for data representation within the pixel-level or superpixel-level. To utilize spectral information and spatial correlation among pixels in HSI and avoid the “salt-and-pepper” problem generated in the pixel-based HSI classification, a novel pixel-level and superpixel-level aware subspace learning method called PSASL is developed. The PSASL constructs the subspace learning framework based on the reconstruction independent component analysis algorithm. The spectral–spatial graph regularization and label space regularization are developed as the pixel-level constraints. To avoid the “salt-and-pepper” problem generated in the pixel-based classification methods, superpixel-level constraints are introduced for integrating the data representations defined in the subspace and class probabilities of the pixels in the same superpixel. The subspace learning and the pixel-level regularization are combined with the superpixel-level regularization to form a unified objective function. The solution to the objective function is efficiently achieved by employing a customized iterative algorithm, and it converges very fast. A discriminative data representation and a universal multiclass classifier are learned simultaneously. We test the PSASL on three widely used HSI data sets. Experimental results demonstrate the superior performance of our method over many recently proposed methods in HSI classification.

The challenges in hyperspectral image (HSI) classification lie in the existence of noisy spectral information and lack of contextual information among pixels. Considering the three different levels in HSIs, i.e., subpixel, pixel, and superpixel, offer complementary information, we develop a novel HSI feature learning network (HSINet) to learn consistent features by self-supervision for HSI classification. HSINet contains a three-layer deep neural network and a multifeature convolutional neural network. It automatically extracts the features such as spatial, spectral, color, and boundary as well as context information. To boost the performance of self-supervised feature learning with the likelihood maximization, the conditional random field (CRF) framework is embedded into HSINet. The potential terms of unary, pairwise, and higher order in CRF are constructed by the corresponding subpixel, pixel, and superpixel. Furthermore, the feedback information derived from these terms are also fused into the different-level feature learning process, which makes the HSINet-CRF be a trainable end-to-end deep learning model with the back-propagation algorithm. Comprehensive evaluations are performed on three widely used HSI data sets and our method outperforms the state-of-the-art methods.

To parse large-scale urban scenes using the supervised methods, a large amount of training data that can account for the vast visual and structural variance of urban environment is necessary. Unfortunately, such training data are mostly obtained by tedious and time-consuming manual work. To overcome the drawback, we propose a semisupervised learning framework that combines the margin, cograph, and label constraints into an objective function for point cloud parsing. Mathematically, the margin constraint is presented to learn a novel distance criterion that can effectively recognize points of different classes. The graph regularization is then employed to characterize the intrinsic geometry structure of the data manifold and explore relationships among points. The label consistency regularization is introduced to ensure the category consistency of the clustered points and single point. To classify the out-of-sample data, the framework successfully transforms the semisupervised classification results into the linear classifier by adopting a linear regression. An iterative algorithm is utilized to efficiently and effectively optimize the objective function with characteristics of multiple variables and highly nonlinear. The point clouds of four urban scenes are used to validate our method. The experimental results show that our method outperforms the state-of-the-art algorithms.

Low-rank representation (LRR) can construct the relationships among pixels for hyperspectral image (HSI) classification with a given dictionary and a noise term. However, the accuracy of HSI classification based on LRR methods is degraded with the redundant and noise information existed in pixels. The neglect of semantic information around pixels in the LRR methods may cause “salt-and-pepper” problem in HSI classification. To avoid the aforementioned problems, a novel self-supervised low-rank representation method called SSLRR is developed. In SSLRR, the LRR and spectral–spatial graph regularization are developed as the pixel-level constraints to remove the redundant and noise information in HSIs. Superpixel constraints including data structure and relationship construction are further utilized to provide supervised feedback information to the subspace learning to avoid the “salt-and-pepper” problem generated in the pixel-based classification methods, and simultaneously enhance the performance of LRR. The pixel-level and superpixel-level regularizations are explicitly integrated into a unified objective function for LRR. By means of the linearized alternating direction method with adaptive penalty, the solution to the objective function is achieved by employing a customized iterative algorithm. We perform comprehensive evaluation of the proposed method on three challenging public HSI data sets. We obtain new state-of-the-art performance on these data sets, and achieve improvements of 44.3%, 13.4%, and 30.1% in overall accuracy compared to the best LRR method.

The performance of scene classification relies heavily on the spatial and structural features that are extracted from high spatial resolution remote-sensing images. Existing approaches, however, are limited in adequately exploiting latent relationships between scene images. Aiming to decrease the distances between intraclass images and increase the distances between interclass images, we propose a latent relationship learning framework that integrates an adaptive graph with the constraints of the feature space and label propagation for high-resolution aerial image classification. To describe the latent relationships among scene images in the framework, we construct an adaptive graph that is embedded into the constrained joint space for features and labels. To remove redundant information and improve the computational efficiency, subspace learning is introduced to assist in the latent relationship learning. To address out-of-sample data, linear regression is adopted to project the semisupervised classification results onto a linear classifier. Learning efficiency is improved by minimizing the objective function via the linearized alternating direction method with an adaptive penalty. We test our method on three widely used aerial scene image data sets. The experimental results demonstrate the superior performance of our method over the state-of-the-art algorithms in aerial scene image classification.

The large number of object categories and many overlapping or closely neighboring objects in large-scale urban scenes pose great challenges in point cloud classification. Most works in deep learning have achieved a great success on regular input representations, but they are hard to be directly applied to classify point clouds due to the irregularity and inhomogeneity of the data. In this paper, a deep neural network with spatial pooling (DNNSP) is proposed to classify large-scale point clouds without rasterization. The DNNSP first obtains the point-based feature descriptors of all points in each point cluster. The distance minimum spanning tree-based pooling is then applied in the point feature representation to describe the spatial information among the points in the point clusters. The max pooling is next employed to aggregate the point-based features into the cluster-based features. To assure the DNNSP is invariant to the point permutation and sizes of the point clusters, the point-based feature representation is determined by the multilayer perception (MLP) and the weight sharing for each point is retained, which means that the weight of each point in the same layer is the same. In this way, the DNNSP can learn the features of points scaled from the entire regions to the centers of the point clusters, which makes the point cluster-based feature representations robust and discriminative. Finally, the cluster-based features are input to another MLP for point cloud classification. We have evaluated qualitatively and quantitatively the proposed method using several airborne laser scanning and terrestrial laser scanning point cloud data sets. The experimental results have demonstrated the effectiveness of our method in improving classification accuracy.

The reconstruction of urban buildings from large-scale airborne laser scanning point clouds is an important research topic in the geoscience field. Large-scale urban scenes usually contain a large number of object categories and many overlapped or closely neighboring objects, which poses great challenges for classifying and modeling buildings from these data sets. In this paper, we propose a deep reinforcement learning framework that integrates a 3-D convolutional neural network, a deep Q-network, and a residual recurrent neural network for the efficient semantic parsing of large-scale 3-D point clouds. The proposed framework provides an end-to-end automatic processing method that maps the raw point cloud to the classification results of the given categories. After obtaining the building classes, we utilize an edge-aware resampling algorithm to consolidate the point set with noise-free normals and clean preservation of sharp features. Finally, 2.5-D dual contouring, which is a data-driven approach, is introduced to generate urban building models from the consolidated point clouds. Our method can generate lightweight building models with arbitrarily shaped roofs while preserving the verticality of connecting walls.

To efficiently recognize on-ground objects in airborne laser scanning (ALS) point clouds, we design a method that jointly learns a discriminative dictionary and a classifier. In the method, the point cloud is segmented into hierarchical point clusters, which are organized by a tree structure. Then, the feature of each point cluster is extracted. The feature of a leaf node is obtained by aggregating the features of all its parent nodes. The feature of the leaf node is called the hierarchical aggregation feature. The hierarchical aggregation features are encoded by sparse coding. We introduce a new label consistency constraint called “discriminative sparse-code error,” and combine it with the reconstruction error, the classification error, and L1-norm sparsity constraint to form a unified objective function. The objective function is efficiently solved by using the proposed label consistency feature sign method. We obtain an overcomplete discriminative dictionary and an optimal linear classifier. Experiments performed on different ALS point cloud scenes have shown that the hierarchical aggregation features combined with the learned classifier can significantly enhance the classification results, and also demonstrated the superior performance of our method over other techniques in point cloud classification.

Semantic parsing of large-scale 3D point clouds is an important research topic in computer vision and remote sensing fields. Most existing approaches utilize hand-crafted features for each modality independently and combine them in a heuristic manner. They often fail to consider the consistency and complementary information among features adequately, which makes them difficult to capture high-level semantic structures. The features learned by most of the current deep learning methods can obtain high-quality image classification results. However, these methods are hard to be applied to recognize 3D point clouds due to unorganized distribution and various point density of data. In this paper, we propose a 3DCNN-DQN-RNN method which fuses the 3D convolutional neural network (CNN), Deep Q-Network (DQN) and Residual recurrent neural network (RNN)for an efficient semantic parsing of large-scale 3D point clouds. In our method, an eye window under control of the 3D CNN and DQN can localize and segment the points of the object's class efficiently. The 3D CNN and Residual RNN further extract robust and discriminative features of the points in the eye window, and thus greatly enhance the parsing accuracy of large-scale point clouds. Our method provides an automatic process that maps the raw data to the classification results. It also integrates object localization, segmentation and classification into one framework. Experimental results demonstrate that the proposed method outperforms the state-of-the-art point cloud classification methods.

Reconstruction of 3D trees from incomplete point clouds is a challenging issue due to their large variety and natural geometric complexity. In this paper, we develop a novel method to effectively model trees from a single laser scan. First, coarse tree skeletons are extracted by utilizing the L1-median skeleton to compute the dominant direction of each point and the local point density of the point cloud. Then we propose a data completion scheme that guides the compensation for missing data. It is an iterative optimization process based on the dominant direction of each point and local point density. Finally, we present a L1-minimum spanning tree (MST) algorithm to refine tree skeletons from the optimized point cloud, which integrates the advantages of both L1-median skeleton and MST algorithms. The proposed method has been validated on various point clouds captured from single laser scans. The experiment results demonstrate the effectiveness and robustness of our method for coping with complex shapes of branching structures and occlusions.

Urban model retrieval has wide applications in the geoscience field, and it is also a very challenging research topic due to the blur and background clutter in query images and the large spatial inconsistencies between query and database images. In this study, a feature extraction and similarity metric-learning framework for urban model retrieval is proposed. In the method, the selective search voting algorithm is presented to automatically localize and segment a query object from an input image with the help of the top-ranked retrieved database images. Then, the local features of object images are extracted via sparse coding, and the global features are learned using the spatial constrained convolutional neural network. We utilize a new similarity metric to match the database images with a query object image. Finally, similar 3D models are retrieved. Both qualitative and quantitative experimental results indicate that the proposed framework can localize and segment a query object from an input image precisely and that the retrieval results are better than those of other related approaches.

To achieve high scene classification performance of high spatial resolution remote sensing images (HSR-RSIs), it is important to learn a discriminative space in which the distance metric can precisely measure both similarity and dissimilarity of features and labels between images. While the traditional metric learning methods focus on preserving interclass separability, label consistency (LC) is less involved, and this might degrade scene images classification accuracy. Aiming at considering intraclass compactness in HSR-RSIs, we propose a discriminative distance metric learning method with LC (DDML-LC). The DDML-LC starts from the dense scale invariant feature transformation features extracted from HSR-RSIs, and then uses spatial pyramid maximum pooling with sparse coding to encode the features. In the learning process, the intraclass compactness and interclass separability are enforced while the global and local LC after the feature transformation is constrained, leading to a joint optimization of feature manifold, distance metric, and label distribution. The learned metric space can scale to discriminate out-of-sample HSR-RSIs that do not appear in the metric learning process. Experimental results on three data sets demonstrate the superior performance of the DDML-LC over state-of-the-art techniques in HSR-RSI classification.

The ability to classify urban objects in large urban scenes from point clouds efficiently and accurately still remains a challenging task today. A new methodology for the effective and accurate classification of terrestrial laser scanning (TLS) point clouds is presented in this paper. First, in order to efficiently obtain the complementary characteristics of each 3-D point, a set of point-based descriptors for recognizing urban point clouds is constructed. This includes the 3-D geometry captured using the spin-image descriptor computed on three different scales, the mean RGB colors of the point in the camera images, the LAB values of that mean RGB, and the normal at each 3-D point. The initial 3-D labeling of the categories in urban environments is generated by utilizing a linear support vector machine classifier on the descriptors. These initial classification results are then first globally optimized by the multilabel graph-cut approach. These results are further refined automatically by a local optimization approach based upon the object-oriented decision tree that uses weak priors among urban categories which significantly improves the final classification accuracy. The proposed method has been validated on three urban TLS point clouds, and the experimental results demonstrate that it outperforms the state-of-the-art method in classification accuracy for buildings, trees, pedestrians, and cars.

Modeling 3-D trees from terrestrial laser scanning (TLS) point clouds remains a challenging task for several well-known reasons, including their complex structure and severe occlusions. In order to accurately reconstruct 3-D tree models from TLS point clouds that typically suffer from significant occlusions, in this paper, a novel local structure and direction-aware approach is presented to successfully complete missing structures of trees. In this method, we first extract the coarse tree skeleton from the input point cloud, and thus, the branch dominant direction and the point density of each branch are obtained. By a skeleton-based Laplacian algorithm, the point cloud is further shrunk into a skeleton point cloud to highlight the branch dominant direction of each branch. For obtaining even more accurate point densities, a dictionary-based algorithm is utilized to learn and reconstruct the local structure. Finally, the branch dominant direction and point density are integrated into an iterative optimization process to recover the missing data. Extensive experimental results have shown that the proposed method is very robust to incomplete data sets, and it is capable of accurately reconstructing 3-D trees, which are partially, or even to a large extent, missing from the input point cloud.

With the emergence of huge volumes of high-resolution remote sensing images produced by all sorts of satellites and airborne sensors, processing and analysis of these images require effective retrieval techniques. To alleviate the dramatic variation of the retrieval accuracy among queries caused by the single image feature algorithms, we developed a novel graph-based learning method for effectively retrieving remote sensing images. The method utilizes a three-layer framework that integrates the strengths of query expansion and fusion of holistic and local features. In the first layer, two retrieval image sets are obtained by, respectively, using the retrieval methods based on holistic and local features, and the top-ranked and common images from both of the top candidate lists subsequently form graph anchors. In the second layer, the graph anchors as an expansion query retrieve six image sets from the image database using each individual feature. In the third layer, the images in the six image sets are evaluated for generating positive and negative data, and SimpleMKL is applied to learn suitable query-dependent fusion weights for achieving the final image retrieval result. Extensive experiments were performed on the UC Merced Land Use-Land Cover data set. The source code has been available at our website. Compared with other related methods, the retrieval precision is significantly enhanced without sacrificing the scalability of our approach.

Efficient presentation and recognition of on-ground objects from airborne laser scanning (ALS) point clouds are a challenging task. In this paper, we propose an approach that combines a discriminative-dictionary-learning-based sparse coding and latent Dirichlet allocation (LDA) to generate multilevel point-cluster features for ALS point-cloud classification. Our method takes advantage of the labels of training data and each dictionary item to enforce discriminability in sparse coding during the dictionary learning process and more accurately further represent point-cluster features. The multipath AdaBoost classifiers with the hierarchical point-cluster features are trained, and we apply them to the classification of unknown points by the heritance of the recognition results under different paths. Experiments are performed on different ALS point clouds; the experimental results have shown that the extracted point-cluster features combined with the multipath classifiers can significantly enhance the classification accuracy, and they have demonstrated the superior performance of our method over other techniques in point-cloud classification.

Point cloud classification plays a critical role in point cloud processing and analysis. Accurately classifying objects on the ground in urban environments from airborne laser scanning (ALS) point clouds is a challenge because of their large variety, complex geometries, and visual appearances. In this paper, a novel framework is presented for effectively extracting the shape features of objects from an ALS point cloud, and then, it is used to classify large and small objects in a point cloud. In the framework, the point cloud is split into hierarchical clusters of different sizes based on a natural exponential function threshold. Then, to take advantage of hierarchical point cluster correlations, latent Dirichlet allocation and sparse coding are jointly performed to extract and encode the shape features of the multilevel point clusters. The features at different levels are used to capture information on the shapes of objects of different sizes. This way, robust and discriminative shape features of the objects can be identified, and thus, the precision of the classification is significantly improved, particularly for small objects.

In 3D urban environments, features of interest (FOIs) are often occluded by clusters of buildings, which prevent a clear overview of important spatial features. State-of-the-art disocclusion methods for urban environments fall short of preserving cityscape appearance or require time-consuming computation. These methods use only one or two operators for disocclusion and might not strike a good balance between disocclusion and distortion control. We present a novel, automatic method enabling interactive context-aware visualization of urban features of interest, which combines four effective disocclusion operators including viewpoint elevation, road shifting, building scaling, and building displacement to disocclude the features of interest. Our method provides an optimum compromise among the disocclusion operators via an efficient constrained optimization and the post-polishing phrases, which minimizes the distortions while enforcing the visibility of the FOIs. The 3D views generated at interactive frame rates ensure a resemblance in the cityscape appearance to its original ones and provide a good overview of the FOIs. The experiments with real data demonstrate that our method can greatly facilitate tasks such as navigation, wayfinding, and information overlay.

The effective extraction of shape features is an important requirement for the accurate and efficient classification of terrestrial laser scanning (TLS) point clouds. However, the challenge of how to obtain robust and discriminative features from noisy and varying density TLS point clouds remains. This paper introduces a novel multiscale and hierarchical framework, which describes the classification of TLS point clouds of cluttered urban scenes. In this framework, we propose multiscale and hierarchical point clusters (MHPCs). In MHPCs, point clouds are first resampled into different scales. Then, the resampled data set of each scale is aggregated into several hierarchical point clusters, where the point cloud of all scales in each level is termed a point-cluster set. This representation not only accounts for the multiscale properties of point clouds but also well captures their hierarchical structures. Based on the MHPCs, novel features of point clusters are constructed by employing the latent Dirichlet allocation (LDA). An LDA model is trained according to a training set. The LDA model then extracts a set of latent topics, i.e., a feature of topics, for a point cluster. Finally, to apply the introduced features for point-cluster classification, we train an AdaBoost classifier in each point-cluster set and obtain the corresponding classifiers to separate the TLS point clouds with varying point density and data missing into semantic regions. Compared with other methods, our features achieve the best classification results for buildings, trees, people, and cars from TLS point clouds, particularly for small and moving objects, such as people and cars.

Fitting primitives is of great importance for remote sensing applications, such as 3-D modeling and as-built surveys. This letter presents a method for fitting primitives that fuses the Bayesian sample consensus (BaySAC) algorithm with a statistical testing of candidate model parameters for unorganized 3-D point clouds. Instead of randomly choosing initial data sets, as in the random sample consensus (RANSAC), we implement a conditional sampling method, which is the BaySAC, to always select the minimum number of data required with the highest inlier probabilities. As the primitive parameters calculated by the different inlier sets should be convergent, this letter presents a statistical testing algorithm for the histogram of the candidate model parameter to compute the prior probability of each data point. Moreover, the probability update is implemented using the simplified Bayes formula. The proposed approach is tested with the data sets of planes, tori, and curved surfaces. The results show that the proposed optimized BaySAC can achieve high computational efficiency (five times higher than the efficiency of the RANSAC for fitting a subset of 12 500 points) and high fitting accuracy (on average, 20% higher than the accuracy of the RANSAC). Moreover, the strategy of prior probability determination is proven to be model-free and, thus, highly applicable.

A 3-D tree structure plays an important role in many scientific fields, including forestry and agriculture. For example, terrestrial laser scanning (TLS) can efficiently capture high-precision 3-D spatial arrangements and structure of trees as a point cloud. In the past, several methods to reconstruct 3-D trees from the TLS point cloud were proposed. However, in general, they fail to process incomplete TLS data. To address such incomplete TLS data sets, a new method that is based on a structure-aware global optimization approach (SAGO) is proposed. The SAGO first obtains the approximate tree skeleton from a distance minimum spanning tree (DMst) and then defines the stretching directions of the branches on the tree skeleton. Based on these stretching directions, the SAGO recovers missing data in the incomplete TLS point cloud. The DMst is applied again to obtain the refined tree skeleton from the optimized data, and the tree skeleton is smoothed by employing a Laplacian function. To reconstruct 3-D tree models, the radius of each branch section is estimated, and leaves are added to form the crown geometry. The developed methodology has been extensively evaluated by employing a dozen TLS point clouds of various types of trees. Both qualitative and quantitative performance evaluation results have indicated that the SAGO is capable of effectively reconstructing 3-D tree models from grossly incomplete TLS point clouds with significant amounts of missing data.

Leaf area index (LAI) products at regional and global scales are being routinely generated from individual instrument data acquired at a specific time. As a result of cloud contamination and other factors, these LAI products are spatially and temporally discontinuous and are also inaccurate for some vegetation types in many areas. A better strategy is to use multi-temporal data. In this paper, a method was developed to estimate LAI from time-series remote sensing data using general regression neural networks (GRNNs). A database was generated from Moderate-Resolution Imaging Spectroradiometer (MODIS) and CYCLOPES LAI products as well as MODIS reflectance products of the BELMANIP sites during the period from 2001-2003. The effective CYCLOPES LAI was first converted to true LAI, which was then combined with the MODIS LAI according to their uncertainties determined from the ground-measured true LAI. The MODIS reflectance was reprocessed to remove remaining effects. GRNNs were then trained over the fused LAI and reprocessed MODIS reflectance for each biome type to retrieve LAI from time-series remote sensing data. The reprocessed MODIS reflectance data from an entire year were inputted into the GRNNs to estimate the 1-year LAI profiles. Extensive validations for all biome types were carried out, and it was demonstrated that the method is able to estimate temporally continuous LAI profiles with much improved accuracy compared with that of the current MODIS and CYCLOPES LAI products. This new method is being used to produce the Global Land Surface Satellite LAI products in China.

This article presents a spatial cognition analysis technique for automated urban building clustering based on urban morphology and Gestalt theory. The proximity graph is selected to present the urban mrphology. The proximity graph considers the local adjacency among buildings, providing a large degree of freedom in object displacement and aggregation. Then, three principles of Gestalt theories, proximity, similarity, and common directions, are considered to extract potential Gestalt building clusters. Next, the Gestalt features are further characterized with seven indicators, that is, area difference, height difference, similarity difference, orientation difference, linear arrangement difference, interval difference, and oblique degree of arrangement. A support vector machine (SVM)-based approach is employed to extract the Gestalt building clusters. This approach transforms the Gestalt cluster extraction into a supervised discrimination process. The method presents a generalized approach for clustering buildings of a given street block into groups, while maintaining the spatial pattern and adjacency of buildings during the displacement operation. In applications of urban building generalization and three-dimensional (3D) urban panoramic-like view, the method presented in this article adequately preserves the spatial patterns, distributions, and arrangements of urban buildings. Moreover, the final 3D panoramic-like views ensure the accurate appearance of important features and landscapes.

Labeling plays an important role in map production, attaching specific texts to related geographic elements to provide clear environmental references. In three-dimensional geographical information systems (3DGISs), however, cluttering happens fairly commonly because of the unexpected overlapping and occlusion among labels and related objects, and results in an ambiguous and obscure environment. It generally also takes large computing power and memory to visualize spatial entities. Aimed at both unambiguous and efficient 3D map display, this article proposes an adaptive multi-resolution labeling method to deal with point, polyline, and polygon features labeling in a 3D landscape. It implements adaptive placement and view-driven label filtering without obscuring other visual features. The experiments indicated that the display of overlapping labels and label popping are reduced significantly with less computation burden while retaining the rendering quality.

Global terrain visual systems must support real-time visualization and manipulation of huge multiresolution geographical datasets. In this letter, we focus on certain key techniques, such as construction of three-dimensional (3-D) ellipsoidal models, spatial indexing mechanisms, interactive terrain datasets simplification by the M-band wavelets and triangulation techniques, and surface browsing from different view directions. Finally, an experiment is carried out using different levels of detail data; our results suggest that the methods are effective in enhancing performance of a 3-D global visual system.

Research on GIS interoperability is very important for spatial data share and integration. GIS specifications play important roles. In This work, OpenGIS WMS (Web Map Service) specification is studied, and Web services technology is introduced for its implementation. Totally different from other implementations, the mapping Web service has advantages of cross-platform interoperability and high capability of integration etc. We also give an example of its integrated application in Spatial Information Search Engine (SISE) system using ASP.NET. Excellent performances are achieved for all the components in this distributed system.

Internet-based GIS (WebGIS) development on infectious disease is a challenge, especially during its severe outbreak. Global SARS information WebGIS we developed is a typical case application. This work discusses the methods and steps on how to design and develop the WebGIS application. A "thin-server" and Java Applet based WebGIS solution is employed to address the Internet access problem with high-visits. A data collection and management system is developed to process SARS data and provide updated data files that the WebGIS application needs. System framework design and client-side functionalities development are presented in detail.

With the rapid development of computer technology and its relative subjects, the Geographic Information Systern (GIS) has developed at very high speed in recent years. Web GIS, a trend of GIS, enable the user conveniently to access to geographic information through Internet. This research develops a high-performance Web GIS based on COM/DCOM to provide map symbol dynamic editing and network publishing through Internet. In order to reduce network traffic and achieve faster response to users, the design of the system considers three major aspects: (1) the architecture of the Web GIS including client, map server, data server and so on; (2) the design and deployment of distributed GIS component; (3) GIS data compression and transferring and its relative key technique. In the end, This work analyses the practical system and draws some conclusions as follow: fast running speed; convenient deployment and maintenance; strong extensibility; strong and flexible editing function for map symbol. In a word, this system can meet the need of different kinds of map symbols dynamic editing for network users.

In this paper, a B/S structured WebGIS-based information system of Yang Zhou city is designed and developed by integrating various data sources including aerial photos, video clips, digital pictures etc. Special focus is cast on hierarchical data structure, image pyramid, image compression and online dissemination technology in order to efficiently process and distribute the over all 700M high resolution aerial photos of the city area. The system can automatically display the exact area with the most appropriate resolution as a response to the user's operation at browser end. Also the integrated raster-and-vector Web publishing technology is utilized thus to simultaneously display necessary map legends and text notes at the according location on the aerial photo cover. The experimental results show that the system can satisfyingly meet the need of real-time map display and smooth roam of large data volume with the most proper resolution under present computer hardware configuration, network bandwidth and transfer speed thus to set a good example of WebGIS-based mass data online service

Web-based visual simulation systems have to handle very large, even huge volumes of elevation data sets. Data compression is an effective mean for reducing space storage and transmission time of these data sets on the Internet. A novel scheme for lossy compression of elevation data using Set Partitioning in Hierarchical Trees (SPIHT) is studied in this paper. An optimal linear predictor is used to identify and remove the redundant information among the neighboring samples in the first stage. Then by exploiting the prosperities of SPIHT, an effective method based on SPIHT is developed to encode the residual elevation data. The algorithm has been applied to data from USGS digital elevation model, and the compressing results strongly suggest that the proposed lossy data compression scheme provides a practical way for elevation data compression.

Global terrain visualization is a very challenging subject. In this paper, an algorithm of determining the bounds of the global terrain in the current LOD within the view volume is presented, according to the property of the ellipsoid and the position of the eye point. Picking up 3D Geo-coordinates of the spatial points accurately is the first step toward handling 3D spatial analysis and query, and is also a research difficulty. Visual field analysis, which is widely used in military and communication fields, becomes one of the important research contents. To address these issues, this paper presents more efficient and accurate algorithms for them.

Buffer Analysis is one of the most important functions of spatial analysis in GIS. This paper uses rotation transform point formula and recursion method to further improve on the vector buffer generation algorithm of double parallel lines and circular arcs, simplifies the process of the generation of parallel lines and the circular correction of sharp angles, and finds a better solution to intersection problem of borderlines of buffer zone.