Timely and accurately extracting and assessing pavement cracks is crucial for intelligent transportation systems (ITS) to improve road maintenance and safety. In this paper, we present an automated framework for crack semantic segmentation and quantification using optical images. First, a unique boundary-guided real-time high-resolution network is proposed, termed as BulletNet, for crack semantic segmentation. BulletNet is a bullet-head structure that can retain crack details while ensuring real-time inference speed, in which a Cross-Scale Global Attention (CSGA) module is designed to enhance global feature representation and pixel-level relations, as well as a Boundary-Guided Fusion (BGF) module proposed to utilize boundary features to guide the fusion of crack details and contextual information. Second, a Pixel-level Crack Quantification (PCQ) algorithm is proposed for complex cracks, incorporating an Improved Discrete Skeleton Evolution (IDSE) method to optimize skeleton pruning for accurate crack length and a normal vector correction method to adjust propagation direction for precise crack width. Comprehensive experiments on three datasets showed that the proposed BulletNet surpassed the comparative models in terms of efficiency and performance, with average F1-score, mIoU, and Frames per second (FPS) of 87.20%, 88.70%, and 125.53, respectively. In addition, tested on 200 images, the PCQ calculated the crack maximum widths and lengths with an average relative error of 6.96% and 4.62%, respectively. Finally, BulletNet was deployed on edge devices for field testing, and a system based on the PCQ algorithm was developed to validate the effectiveness of the entire framework.

With the rapid advancements of 3-D acquisition technology, 3-D change detection has gained lots of attentions recently. Existing deep learning-based point cloud change detection methods usually adopt a common encoder-decoder structure to learn pointwise features. However, these feature learning backbones are not specifically designed for change detection task, and ignore the local structure discrepancies during feature learning. To address these issues, this article proposes a multiscale difference-aware network (Ms-DANet) for 3-D point cloud change detection. First, we propose a difference-guided multiscale feature learning (DG-MsFL) module to enhance the feature differences between bi-temporal point clouds at multiple scales during feature encoding, and use these differences to guide the network focusing more on the local structures with large discrepancies. Next, we introduce a multiscale difference feature fusion (Ms-DFF) module to fuse the multiscale feature differences to learn more discriminative features during feature decoding. Finally, we treat the point cloud change detection task as a semantic classification problem, and propose a multiscale loss (Ms-Loss) function to promote the network training. We conduct experiments on the real-world street-level point cloud change detection dataset SLPCCD and the simulated airborne urban point cloud change detection dataset URB3DCD. The experimental results show that Ms-DANet obtains a significant improvement on both the real-world and simulated point cloud change detection datasets, demonstrating its effectiveness and robustness across various sensors and data modalities.

As the requirements for the downstream tasks of land cover classification (LCC) continue to increase, the category system used for LCC is constantly being refined. This causes previous land cover products and manually annotated training samples to become quickly outdated. Meanwhile, manually annotating samples with more refined categories is extremely time consuming. To address this impasse, a cross-category knowledge transfer process is needed that can directly generate land cover products under a fine-scale category system using existing training samples under a large-scale category system. Accordingly, this paper proposes a cross-category LCC method called conditional Gaussian enhanced dense correlation matching (CGE-DCM). CGE-DCM uses samples under a large-scale category system for training. It then uses only one annotated example of each fine-scale category to achieve fine-scale LCC. In cases with very little sample support, the problems caused by different spectra of the same object and different objects of the same spectrum in complex scenes can be particularly severe. To solve this problem and improve the accuracy of classifications of complex objects, CGE-DCM offers a dense correlation matching strategy. In addition, context distribution is different under fine-scale category systems than it is under large-scale category systems. For this reason, CGE-DCM features a conditional Gaussian enhancement mechanism and designs different loss functions for degenerated and nondegenerated scenarios to ensure the stability of the model. Extensive experiments on Gaofen-2 and Orbita hyperspectral satellite images demonstrate the effectiveness of each module in CGE-DCM and its superiority over existing methods.

Point cloud semantic segmentation plays a key role in scene understanding and digital twin cities tasks. This article proposed a multi-granularity feature fusion network (MGF-Net) for point cloud semantic segmentation. The model first used a cluster relation aggregation module to extract fine-grained point features and a 3D convolution module to extract coarse-grained voxel features, followed by feature aggregation via a multi-granularity feature adaptive fusion module. Finally, to further improve the model performance, MGF-Net used a global feature attention module to capture long-distance context information. The performance of MGF-Net was evaluated on three point cloud datasets of urban scenes, i.e., Toronto3D, WHU-MLS, and SensatUrban. The quantitative results showed that MGF-Net achieved 80.16%, 51.27%, and 54.20% of mIoU on these datasets, respectively. Moreover, the comparative results showed that the proposed MGF-Net outperformed the baseline for complex urban scenes, and obtained better point cloud semantic segmentation results.

In recent years, fully supervised learning based semantic segmentation algorithms for point clouds have achieved significant advancements. However, a major limitation of these traditional algorithms is their reliance on extensive labeled datasets. This impedes their practical applicability. To overcome this obstacle, this paper proposes a novel point cloud semantic segmentation framework based on weakly supervised learning. This framework is designed to segment point cloud data both efficiently and accurately, even with a limited budget (0.1%) of labeled data. The proposed approach initiates with an active learning annotation strategy. This strategy involves computing the uncertainty scores of each point and ranking them, consequently selecting the top-K points for labeling based on the labeling budget. Furthermore, this paper developed a weakly supervised learning network. This network is enhanced by the calculation of multiple consistency losses to enhance the network's performance. Experimental results demonstrate that with a mere 0.1% labeling ratio, the proposed framework achieves a mean Intersection over Union (mIoU) of 70.3% on the NPM3D dataset.

Effective road marking classification and segmentation play a pivotal role in advancing vehicle-to-everything (V2X) applications and refining road inventory databases. However, the irregular data formats and unordered permutation modes of 3D point clouds, along with the limited availability of large-scale datasets with point-level annotations, remain significant obstacles to designing deep learning-based networks with superior performance. To address these challenges, this paper proposes a novel multi-level feature optimization network structure, named MFPNet, and introduces two point cloud benchmarks, RdmkNet and Toronto-Rdmk, for road marking classification and segmentation in intricate urban environments. MFPNet is composed of three integral modules. First, the M-transformer module, consisting of three transformers obtained from different channels, fully captures rich point cloud background information and long-distance dependencies between objects. Then, the feature pooling aggregation module uses parallel structured pooling attention mechanisms to aggregate features captured by the M-transformer module, while the prediction refinement module further enhances the acquisition of semantic features. Comparative studies indicate that MFPNet can be embedded into general deep learning networks without changing their original network structures, significantly improving the accuracy of multiple baseline networks. Furthermore, extensive experiments demonstrate that the two newly-developed point cloud datasets are meaningful for road marking classification and segmentation tasks, contributing to the development of autonomous driving.

Deep learning-based algorithms detect pavement cracks in an end-to-end manner from Mobile Laser Scanning (MLS) point clouds, achieving impressive results. However, the accuracy of existing methods still has room to improve due to the difficulty of effectively encoding multiscale features and the limited training data. In this paper, we propose a novel pavement crack detection framework, Crack-U2Net, which innovatively incorporates a two-level nested U-Net architecture for feature learning. This design enables the learning of intra-stage multiscale features without introducing significant memory and computation costs, resulting in substantial improvements in accuracy. Moreover, to solve the challenge of insufficient training data, we propose a Geometry-based Data Augmentation (GDA) strategy, aiming to expand the pavement dataset while preserving the pavement geometry. Extensive experiments on the Qinghai-Tibet Highway point cloud dataset demonstrate the higher accuracy and efficiency of Crack-U2Net over the state-of-the-art methods, achieving an average precision, recall, F$1\text - $score, and accuracy of 83.8%, 77.6%, 80.1%, and 95.8%, respectively.

Deep learning-based semantic segmentation has made great progress in understanding very-high-resolution (VHR) remote sensing images (RSIs). However, large-scale applications are still limited. The main reason is that diverse imaging modes and geographical differences make it difficult to transfer a model trained in the source domain to the target domain. To solve this problem, unsupervised domain adaptation (UDA) for VHR RSIs has received some attention, but the accuracy of cross-domain semantic segmentation still needs to be improved. Currently, one reasonable proposal for improving accuracy is to take a close look at the category-level information. In this paper, we reveal an integer programming mechanism for modeling the category-level relationship between the source and target domains. The mechanism is based on the solution of the assignment problem, and thus, the proposed method is called category-level assignment for UDA (ClA-UDA). In ClA-UDA, a category-level assignment problem with additional constraints is defined for UDA tasks, and the solution is provided. Based on the solution, an assignment-based image-to-image transferring algorithm (AIT) is first proposed to transfer the source-domain images based on the style of the target-domain images. AIT minimizes a weighted discrepancy, and provides an analytical solution for the transfer. Two assignment-based alignment losses are then introduced to align the source and target domains based on the category-level relationship in a concise way. To validate the performance of ClA-UDA, three VHR remote sensing image datasets are employed, and six UDA tasks are designed. Extensive experiments are conducted, and the results demonstrate the superiority of ClA-UDA compared to the existing methods.

Accurate and effective road object semantic segmentation plays a significant role in supporting extensive intelligent transportation system (ITS)-related applications. However, most existing image-based methods and point-based methods cannot deliver promising solutions with respect to segmentation accuracy and robustness, especially in complex urban road scenes. Thus, we design a saliency-guided transformer architecture (STN) in this letter for point-wise semantic segmentation from mobile laser scanning (MLS) point clouds. First, four types of feature saliency maps are constructed to obtain more compact feature spaces for enhancing the feature encoding semantics. Then, integrated with offset attention mechanisms and edge convolutions, an effective point-wise transformer network is proposed to extract high-level features for point-wise label assignment of road objects. The STN model is evaluated on the Pairs-Lille-3D dataset and achieves satisfactory experimental results with 87.2% overall accuracy and 81.7% mean IoU, respectively. Comparative studies with five deep learning-based methods also prove the superior performance of the STN model for large-scale semantic segmentation tasks.

Periodically conducting land cover mapping plays a vital role in monitoring the status and changes of the land use. The up-to-date and accurate land use database serves importantly for a wide range of applications. This letter constructs an efficient self-attention capsule network (ESA-CapsNet) for land cover classification of multispectral light detection and ranging (LiDAR) data. First, formulated with a novel capsule encoder–decoder architecture, the ESA-CapsNet performs promisingly in extracting high-level, informative, and strong feature semantics for pixel-wise land cover classification by using the five types of rasterized feature images. Furthermore, designed with a novel capsule-based attention module, the channel and spatial feature encodings are comprehensively exploited to boost the feature saliency and robustness. The ESA-CapsNet is evaluated on two multispectral LiDAR data sets and achieves an advantageous performance with the overall accuracy, average accuracy, and kappa coefficient of over 98.42%, 95.15%, and 0.9776, respectively. Comparative experiments with the existing methods also demonstrate the effectiveness and applicability of the ESA-CapsNet in land cover classification tasks.

Accurately interpreting image contents plays a vital role in many earth observation tasks. This letter constructs a novel cross-context and cross-scale capsule vision transformer (C2-CapsViT) architecture to serve for remote sensing image scene classification. First, employed with a multi-context patch embedding strategy, the token representation quality is greatly boosted to encode different-context feature semantics. Second, designed with a multiscale transformer block, different-grained long-range global feature interactions and different-type feature self-attentions are concurrently exploited to promote the feature encoding quality. Moreover, by combining the convolution and transformer structures, local and global feature semantics are effectively fused to direct accurate predictions. The C2-CapsViT is elaborately verified on three scene classification data sets. Both quantitative evaluations and comparative analyses prove its competitive capability and advanced performance.

Convolutional neural networks (CNNs) have achieved tremendous success in computer vision tasks, such as building extraction. However, due to domain shift, the performance of the CNNs drops sharply on unseen data from another domain, leading to poor generalization. As it is costly and time-consuming to acquire dense annotations for remote-sensing (RS) images, developing algorithms that can transfer knowledge from a labeled source domain to an unlabeled target domain is of great significance. To this end, we propose a novel full-level domain adaptation network (FDANet) for building extraction by combining image-, feature-, and output-level information effectively. At the input level, a simple Wallis filter method is employed to transfer source images into target-like ones whereby alleviating radiometric discrepancy and achieving image-level alignment. To further reduce domain shift, adversarial learning is used to enforce feature distribution consistency constraints between the source and target images. In this way, feature-level alignment can be embedded effectively. At the output level, a mean-teacher model is introduced to enforce transformation-consistent constraint for the target output so that the regularization effect is enhanced and the uncertain predictions can be suppressed as much as possible. To further improve the performance, a novel self-training strategy is also employed by using pseudo labels. The effectiveness of the proposed FDANet is verified on three diverse high-resolution aerial datasets with different resolutions and scenarios. Extensive experimental results and ablation studies demonstrated the superiority of the proposed method.

Road detection plays an important role in a wide range of applications. However, due to size variations, spectral diversities, occlusions, and complex scenarios, it is still challenging to accurately extract roads from very-high resolution (VHR) optical remote sensing images. This paper proposes a capsule feature pyramid network for extracting road networks from VHR optical images, termed as RoadCapsFPN. By designing a capsule feature pyramid network, the RoadCapsFPN extracts and integrates multiscale capsule features to recover a high-resolution and semantically strong road feature representation. Next, we also design a contextual feature module, including dense atrous convolution (DAC) and residual multi-kernel pooling (RMP) units, to further exploit rich contextual properties of the roads at a high-resolution perspective. Benefitting from the multiscale feature abstraction and context augmentation, our RoadCapsFPN shows impressing results in processing variedly-sized and diversely-spectral roads in complex environments. Two testing datasets, Google and Massichusate Roads Datasets, are used for evaluating the proposed RoadCapsFPN via four testing indicators -precision,recall, intersection-over-union (IoU), and$F_{1}$-score. Comparative studies also confirm the superior performance of the RoadCapsFPN in accurately extracting road networks.

Periodically monitoring the pavement conditions is of great importance to many intelligent transportation activities. Timely and correctly identifying the distresses or anomalies on pavement surfaces can help to smooth traffic flows and avoid potential threats to pavement securities. In this paper, we develop a novel context-augmented capsule feature pyramid network (CCapFPN) to detect cracks from pavement images. The CCapFPN adopts vectorial capsules to represent high-level, intrinsic, and salient features of cracks. By designing a feature pyramid architecture, the CCapFPN can fuse different levels and different scales of capsule features to provide a high-resolution, semantically strong feature representation for accurate crack detection. To take advantage of the context properties, a context-augmented module is embedded into each stage of the CCapFPN to rapidly enlarge the receptive field. The CCapFPN performs effectively and efficiently in processing pavement images of diverse conditions and detecting cracks of different topologies. Quantitative evaluations show that an overall performance with a precision, a recall, and an F-score of 0.9200, 0.9149, and 0.9174, respectively, were achieved on the test datasets. Comparative studies with some existing deep learning and edge based crack detection methods also confirm the superior performance of the CCapFPN in crack detection tasks.

Building footprint extraction plays an important role in a wide range of applications. However, due to size and shape diversities, occlusions, and complex scenarios, it is still challenging to accurately extract building footprints from aerial images. This letter proposes a capsule feature pyramid network (CapFPN) for building footprint extraction from aerial images. Taking advantage of the properties of capsules and fusing different levels of capsule features, the CapFPN can extract high-resolution, intrinsic, and semantically strong features, which perform effectively in improving the pixel-wise building footprint extraction accuracy. With the use of signed distance maps as ground truths, the CapFPN can extract solid building regions free of tiny holes. Quantitative evaluations on an aerial image data set show that a precision, recall, intersection-over-union (IoU), and F-score of 0.928, 0.914, 0.853, and 0.921, respectively, are obtained. Comparative studies with six existing methods confirm the superior performance of the CapFPN in accurately extracting building footprints.

Change detection (CD) is one of the main applications of remote sensing. With the increasing popularity of deep learning, most recent developments of CD methods have introduced the use of deep learning techniques to increase the accuracy and automation level over traditional methods. However, when using supervised CD methods, a large amount of labeled data is needed to train deep convolutional networks with millions of parameters. These labeled data are difficult to acquire for CD tasks. To address this limitation, a novel semisupervised convolutional network for CD (SemiCDNet) is proposed based on a generative adversarial network (GAN). First, both the labeled data and unlabeled data are input into the segmentation network to produce initial predictions and entropy maps. Then, to exploit the potential of unlabeled data, two discriminators are adopted to enforce the feature distribution consistency of segmentation maps and entropy maps between the labeled and unlabeled data. During the competitive training, the generator is continuously regularized by utilizing the unlabeled information, thus improving its generalization capability. The effectiveness and reliability of our proposed method are verified on two high-resolution remote sensing data sets. Extensive experimental results demonstrate the superiority of the proposed method against other state-of-the-art approaches.

Vehicle logo recognition provides an important supplement to vehicle make and model analysis. Some of the existing vehicle logo recognition methods depend on the detection of license plates to roughly locate vehicle logo regions using prior knowledge. The vehicle logo recognition performance is greatly affected by the license plate detection techniques. This paper presents a cascaded deep convolutional network for directly recognizing vehicle logos without depending on the existence of license plates. This is a two-stage processing framework composed of a region proposal network and a convolutional capsule network. First, potential region proposals that might contain vehicle logos are generated by the region proposal network. Then, the convolutional capsule network classifies these region proposals into the background and different types of vehicle logos. We have evaluated the proposed framework on a large test set towards vehicle logo recognition. Quantitative evaluations show that a detection rate, a recognition rate, and an overall performance of 0.987, 0.994, and 0.981, respectively, are achieved. Comparative studies with the Faster R-CNN and other three existing methods also confirm that the proposed method performs effectively and robustly in recognizing vehicle logos of various conditions.

Traffic-sign recognition plays an important role in road transportation systems. This letter presents a novel two-stage method for detecting and recognizing traffic signs from mobile Light Detection and Ranging (LiDAR) point clouds and digital images. First, traffic signs are detected from mobile LiDAR point cloud data according to their geometrical and spectral properties, which have been fully studied in our previous work. Afterward, the traffic-sign patches are obtained by projecting the detected points onto the registered digital images. To improve the performance of traffic-sign recognition, we apply a convolutional capsule network to the traffic-sign patches to classify them into different types. We have evaluated the proposed framework on data sets acquired by a RIEGL VMX-450 system. Quantitative evaluations show that a recognition rate of 0.957 is achieved. Comparative studies with the convolutional neural network (CNN) and our previous supervised Gaussian-Bernoulli deep Boltzmann machine (GB-DBM) classifier also confirm that the proposed method performs effectively and robustly in recognizing traffic signs of various types and conditions.

Land cover mapping is an effective way to quantify land resources and monitor their changes. It plays an important role in a wide range of applications. This letter proposes a hybrid capsule network for land cover classification using multispectral light detection and ranging (LiDAR) data. First, the multispectral LiDAR data were rasterized into a set of feature images to exploit the geometrical and spectral properties of different types of land covers. Then, a hybrid capsule network composed of an encoder network and a decoder network is trained to extract both high-level local and global entity-oriented capsule features for accurate land cover classification. Quantitative classification evaluations on two data sets show that the overall accuracy, average accuracy, and kappa coefficient of over 97.89%, 94.54%, and 0.9713, respectively, are obtained. Comparative studies with five existing methods confirm that the proposed method performs robustly and accurately in land cover classification using the multispectral LiDAR data.

Effective object extraction plays an important role in many point cloud-based applications. This letter proposes a 3-D feature matching framework for point cloud object extraction. To determine the optimal affine transformation parameters for each template feature point, a convex dissimilarity function and the locally affine-invariant geometric constraints are designed to construct the overall objective function. The 3-D feature matching framework is integrated into a point cloud object extraction workflow. Extraction results on six test data sets show that average completeness, correctness, quality, and F1-measure of 0.96, 0.97, 0.93, and 0.96, respectively, are obtained in extracting light poles, vehicles, and palm trees. Comparative studies also confirm that the proposed method performs effectively and robustly, and exhibits superior or compatible performance over the other compared methods.

Periodical road manhole cover measurement is extremely important to ensure road safety and reduce traffic disasters. This letter proposes an effective method for delineating road manhole covers from mobile laser scanning point cloud data. To improve processing efficiency, first, road surface points are segmented and rasterized into georeferenced intensity images. Then, object-oriented patches are generated through superpixel segmentation and further fed to a convolutional capsule network classifier for manhole cover detection. Finally, manhole covers are accurately delineated through a marked point process of disks. Quantitative evaluations on three data sets show that an average completeness, correctness, quality, and F1-measure of 0.965, 0.961, 0.929, and 0.963, respectively, are obtained. Comparative studies with three existing methods confirm that the proposed method performs superiorly in delineating manhole covers of varying conditions and on complex road surface environments.

Probabilistic registration algorithms [e.g., coherent point drift, (CPD)] provide effective solutions for point cloud alignment. However, using the original CPD algorithm for automatic registration of terrestrial laser scanner (TLS) point clouds is highly challenging because of density variations caused by scanning acquisition geometry. In this letter, we propose a new global registration method, introducing the use of the CPD framework for TLS point clouds. We first consider the measurement geometry and the intrinsic characteristics of the scene to simplify points. In addition to the Euclidean distance, we incorporate geometric information as well as structural constraints in the probabilistic model to optimize the so-called matching probability matrix. Among the structural constraints, we use a spectral graph to measure the structural similarity between matches at each iteration. The method is tested on three data sets collected by different TLS scanners. Experimental results demonstrate that the proposed method is robust to density variations and can decrease iterations effectively. The average registration errors of the three data sets are 0.05, 0.12, and 0.08 m, respectively. It is also shown that our registration framework is superior to the state-of-the-art methods in terms of both registration errors and efficiency. The experiments demonstrate the effectiveness and efficiency of the proposed probabilistic global registration.

Vehicle detection plays an important role in a variety of traffic-related applications. However, due to the scale and orientation variations and partial occlusions of vehicles, it is still challengeable to accurately detect vehicles from remote sensing images. This letter proposes a convolutional capsule network for detecting vehicles from high-resolution remote sensing images. First, a test image is segmented into superpixels to generate meaningful and nonredundant patches. Then, these patches are input to a convolutional capsule network to label them into vehicles or the background. Finally, nonmaximum suppression is adopted to eliminate repetitive detections. Quantitative evaluations on four test data sets show that average completeness, correctness, quality, and F1-measure of 0.93, 0.97, 0.90, and 0.95, respectively, are obtained. Comparative studies with three existing methods confirm that the proposed method effectively performs in detecting vehicles of various conditions.

This letter addresses the problem of detecting pole-like road objects (including light poles and traffic signposts) from mobile light detection and ranging (LiDAR) data for transportation-related applications. The method consists of two consecutive stages: training and pole-like object detection. At the training stage, a contextual visual vocabulary is created from the feature regions generated from a training data set by supervoxel segmentation. At the pole-like object detection stage, a bag-of-contextual-visual-words representation is generated for each semantic object segmented from mobile LiDAR data. The experimental results show that the proposed method achieves correctness, omission, and commission of 88.9%, 11.1%, and 2.8%, respectively, in detecting pole-like road objects. Computational complexity analysis demonstrates that our method provides a promising and effective solution to rapid and accurate detection of pole-like objects from large volumes of mobile LiDAR data.

This paper presents a study of vehicle detection from high-resolution aerial images. In this paper, a superpixel segmentation method designed for aerial images is proposed to control the segmentation with a low breakage rate. To make the training and detection more efficient, we extract meaningful patches based on the centers of the segmented superpixels. After the segmentation, through a training sample selection iteration strategy that is based on the sparse representation, we obtain a complete and small training subset from the original entire training set. With the selected training subset, we obtain a dictionary with high discrimination ability for vehicle detection. During training and detection, the grids of histogram of oriented gradient descriptor are used for feature extraction. To further improve the training and detection efficiency, a method is proposed for the defined main direction estimation of each patch. By rotating each patch to its main direction, we give the patches consistent directions. Comprehensive analyses and comparisons on two data sets illustrate the satisfactory performance of the proposed algorithm.

This paper presents an automated algorithm for rapidly and effectively detecting cars directly from large-volume 3-D point clouds. Rather than using low-order descriptors, a multilayer feature generation model is created to obtain high-order feature representations for 3-D local patches through deep learning techniques. To handle cars with different levels of incompleteness caused by data acquisition ways and occlusions, a hierarchical visibility estimation model is developed to augment Hough voting. Considering scale and orientation variations in the azimuth direction, a set of multiscale Hough forests is constructed to rotationally cast votes to estimate cars' centroids. Quantitative assessments show that the proposed algorithm achieves average completeness, correctness, quality, and F1-measure of 0.94, 0.96, 0.90, and 0.95, respectively, in detecting 3-D cars. Comparative studies also demonstrate that the proposed algorithm outperforms the other four existing algorithms in accurately and completely detecting 3-D cars from large-scale 3-D point clouds.

This paper proposes a novel algorithm for detecting road scene objects (e.g., light poles, traffic signposts, and cars) from 3-D mobile-laser-scanning point cloud data for transportation-related applications. To describe local abstract features of point cloud objects, a contextual visual vocabulary is generated by integrating spatial contextual information of feature regions. Objects of interest are detected based on the similarity measures of the bag of contextual-visual words between the query object and the segmented semantic objects. Quantitative evaluations on two selected data sets show that the proposed algorithm achieves an average recall, precision, quality, and F-score of 0.949, 0.970, 0.922, and 0.959, respectively, in detecting light poles, traffic signposts, and cars. Comparative studies demonstrate the superior performance of the proposed algorithm over other existing methods.

This paper presents a tensor voting approach to automated detection of dark spots in RADARSAT-1 ScanSAR Narrow Beam mode images. First, a thresholding algorithm that well maximizes the ratio of between-class variance to within-class variance is used to detect potential dark spot candidates. Next, a tensor voting framework integrated with sparse and dense ball votings is carried out to suppress noise while maintaining dark spots. Then, a saliency map that reflects the probability of a pixel being located within a dark spot is generated using the saliencies of ball tensors. Finally, a segmentation method is applied to ascertain dark spots based on the saliency map. The proposed approach has been tested on a set of RADARSAT-1 ScanSAR Narrow Beam intensity images. Quantitative evaluations demonstrate that the proposed approach achieves an average commission error, omission error, and quality of 0.003, 0.037, and 0.956, respectively, for detecting dark spots in SAR intensity imagery.

This letter presents a rotation-invariant method for detecting geospatial objects from high-resolution satellite images. First, a superpixel segmentation strategy is proposed to generate meaningful and nonredundant patches. Second, a multilayer deep feature generation model is developed to generate high-level feature representations of patches using deep learning techniques. Third, a set of multiscale Hough forests with embedded patch orientations is constructed to cast rotation-invariant votes for estimating object centroids. Quantitative evaluations on the images collected from Google Earth service show that an average completeness, correctness, quality, and F1- measure values of 0.958, 0.969, 0.929, and 0.963, respectively, are obtained. Comparative studies with three existing methods demonstrate the superior performance of the proposed method in accurately and correctly detecting objects that are arbitrarily oriented and of varying sizes.

This letter presents a 3-D object matching framework to support information extraction directly from 3-D point clouds. The problem of 3-D object matching is to match a template, represented by a group of 3-D points, to a point cloud scene containing an instance of that object. A locally affine-invariant geometric constraint is proposed to effectively handle affine transformations, occlusions, incompleteness, and scales in 3-D point clouds. The 3-D object matching framework is integrated into 3-D correspondence computation, 3-D object detection, and point cloud object classification in mobile laser scanning (MLS) point clouds. Experimental results obtained using the 3-D point clouds acquired by a RIEGL VMX-450 system showed that completeness, correctness, and quality of over 0.96, 0.94, and 0.91 are achieved, respectively, with the proposed framework in 3-D object detection. Comparative studies demonstrate that the proposed method outperforms the two existing methods for detecting 3-D objects directly from large-volume MLS point clouds.

The assessment of pavement cracks is one of the essential tasks for road maintenance. This paper presents a novel framework, called ITVCrack, for automated crack extraction based on iterative tensor voting (ITV), from high-density point clouds collected by a mobile laser scanning system. The proposed ITVCrack comprises the following: 1) the preprocessing involving the separation of road points from nonroad points using vehicle trajectory data; 2) the generation of the georeferenced feature (GRF) image from the road points; and 3) the ITV-based crack extraction from the noisy GRF image, followed by an accurate delineation of the curvilinear cracks. Qualitatively, the method is applicable for pavement cracks with low contrast, low signal-to-noise ratio, and bad continuity. Besides the application to GRF images, the proposed framework demonstrates much better crack extraction performance when quantitatively compared to existing methods on synthetic data and pavement images.

This paper proposes a novel algorithm for extracting street light poles from vehicleborne mobile light detection and ranging (LiDAR) point-clouds. First, the algorithm rapidly detects curb-lines and segments a point-cloud into road and nonroad surface points based on trajectory data recorded by the integrated position and orientation system onboard the vehicle. Second, the algorithm accurately extracts street light poles from the segmented nonroad surface points using a novel pairwise 3-D shape context. The proposed algorithm is tested on a set of point-clouds acquired by a RIEGL VMX-450 mobile LiDAR system. The results show that road surfaces are correctly segmented, and street light poles are robustly extracted with a completeness exceeding 99%, a correctness exceeding 97%, and a quality exceeding 96%, thereby demonstrating the efficiency and feasibility of the proposed algorithm to segment road surfaces and extract street light poles from huge volumes of mobile LiDAR point-clouds.

This paper presents a survey of literature about road feature extraction, giving a detailed description of a Mobile Laser Scanning (MLS) system (RIEGL VMX-450) for transportation-related applications. This paper describes the development of automated algorithms for extracting road features (road surfaces, road markings, and pavement cracks) from MLS point cloud data. The proposed road surface extraction algorithm detects road curbs from a set of profiles that are sliced along vehicle trajectory data. Based on segmented road surface points, we create Geo-Referenced Feature (GRF) images and develop two algorithms, respectively, for extracting the following: 1) road markings with high retroreflectivity and 2) cracks containing low contrast with their surroundings, low signal-to-noise ratio, and poor continuity. A comprehensive comparison illustrates satisfactory performance of the proposed algorithms and concludes that MLS is a reliable and cost-effective alternative for rapid road inspection.

Updating road markings is one of the routine tasks of transportation agencies. Compared with traditional road inventory mapping techniques, vehicle-borne mobile light detection and ranging (LiDAR) systems can undertake the job safely and efficiently. However, current hurdles include software and computing challenges when handling huge volumes of highly dense and irregularly distributed 3-D mobile LiDAR point clouds. This paper presents the development and implementation aspects of an automated object extraction strategy for rapid and accurate road marking inventory. The proposed road marking extraction method is based on 2-D georeferenced feature (GRF) images, which are interpolated from 3-D road surface points through a modified inverse distance weighted (IDW) interpolation. Weighted neighboring difference histogram (WNDH)-based dynamic thresholding and multiscale tensor voting (MSTV) are proposed to segment and extract road markings from the noisy corrupted GRF images. The results obtained using 3-D point clouds acquired by a RIEGL VMX-450 mobile LiDAR system in a subtropical urban environment are encouraging.

This paper proposes a novel framework for automated detection of urban road manhole covers using mobile laser scanning (MLS) data. First, to narrow searching regions and reduce the computational complexity, road surface points are segmented from a raw point cloud via a curb-based road surface segmentation approach and rasterized into a georeferenced intensity image through inverse distance weighted interpolation. Then, a supervised deep learning model is developed to construct a multilayer feature generation model for depicting high-order features of local image patches. Next, a random forest model is trained to learn mappings from high-order patch features to the probabilities of the existence of urban road manhole covers centered at specific locations. Finally, urban road manhole covers are detected from georeferenced intensity images based on the multilayer feature generation model and random forest model. Quantitative evaluations show that the proposed algorithm achieves an average completeness, correctness, quality, and F1-measure of 0.955, 0.959, 0.917, and 0.957, respectively, in detecting urban road manhole covers from georeferenced intensity images. Comparative studies demonstrate the advantageous performance of the proposed algorithm over other existing methods for rapid and automated detection of urban road manhole covers using MLS data.

This paper proposes a novel, automated algorithm for rapidly extracting urban road facilities, including street light poles, traffic signposts, and bus stations, for transportation-related applications. A detailed description and implementation of the proposed algorithm is provided using mobile laser scanning data collected by a state-of-the-art RIEGL VMX-450 system. First, to reduce the quantity of data to be handled, a fast voxel-based upward growing method is developed to remove ground points. Then, off-ground points are clustered and segmented into individual objects via Euclidean distance clustering and voxel-based normalized cut segmentation, respectively. Finally, a 3-D object matching framework, benefiting from a locally affine-invariant geometric constraint, is developed to achieve the extraction of 3-D objects. Quantitative evaluations show that the proposed algorithm attains an average completeness, correctness, quality, and F1-measure of 0.949, 0.971, 0.922, and 0.960, respectively, in extracting 3-D light poles, traffic signposts, and bus stations. Comparative studies demonstrate the efficiency and feasibility of the proposed algorithm for automated and rapid extraction of urban road facilities.

This paper presents a stepwise algorithm for extracting power-lines from mobile laser scanning (MLS) data. This algorithm first extracts non-road points from MLS data by estimating road ranges with regard to scanning mechanism and applying elevation-difference and slope criteria to the road ranges scan-line by scan-line. Then, three filters, in terms of height, spatial density, and size-and-shape, are proposed to extract power-line points in the identified non-road points, followed by Hough transform and Euclidean distance clustering. Finally, a 3D power line is modelled as a horizontal line in X-Y plane and a vertical catenary curve defined by a hyperbolic cosine function in X-Z plane. The proposed algorithm has been tested on a sample of point clouds acquired by a RIEGL VMX-450 MLS system. The results demonstrate the applicability of the proposed algorithm in extracting power transmission lines.

Optical sensors onboard an unmanned aerial vehicle (UAV) can collect high resolution images with small dimensions. Image mosaicking is necessary to cover a larger geographic area. This paper presents a novel approach to mosaicking UAV images automatically. The "Orthophoto Map" is based on Structure From Motion (SFM). This method can fully automatic mosaic generate a wide range, and with well visual effects, and no evident deformation. This method can not only get a panoramic image of wide range of areas, and can get the corresponding three-dimensional terrain model.

This paper presents a novel algorithm for extracting 3D crack skeletons from 3D point clouds acquired by a mobile Light Detection and Ranging (LiDAR) system. This algorithm uses intensity information of cloud clouds to identify pavement cracks that usually exhibit lower intensities compared to their surroundings. First, crack candidates are extracted by applying the Otsu thresholding algorithm. Then, a spatial density filter is used to remove outliers. Next, crack points are grouped into crack-lines using a Euclidean distance clustering method. Finally, crack skeletons are extracted based on an L1-medial skeleton extraction method. The proposed algorithm has been tested on a set of mobile LiDAR point clouds acquired by a state-of-the-art RIEGL VMX-450 mobile LiDAR system. The results demonstrate the efficiency and reliability of the proposed algorithm in extracting 3D crack skeletons.

This letter presents an effective approach to reduce the ambiguity of matching results for image registration based on a coarse-to-fine strategy. In the coarse registration stage, we compute initial transformation parameters via the descriptors. In the fine registration stage, we propose a new matching strategy for an iterative closest point framework, in which the matching pairs are determined by a bidirectional matching criterion in terms of feature similarity and spatial consistency. In this letter, the spatial consistency includes not only spatial distance but also local structure constraints on reference and sensed images. Comparative experiments on multispectral and viewpoint-altered images show that the proposed algorithm achieves higher performance in accuracy and robustness.

A novel object detection algorithm is developed for automatically detecting road manhole and sewer well covers from mobile light detection and ranging point clouds. This algorithm takes advantage of a marked point process of disks and rectangles to model the locations of manhole and sewer well covers and their geometric dimensions. A reversible jump Markov chain Monte Carlo algorithm is implemented for simulating the posterior distribution obtained using a Bayesian paradigm. The detection results obtained from the road surface point clouds acquired by a RIEGL VMX-450 system show that the manhole and sewer well covers can be detected automatically and accurately. The performance achieved using the proposed algorithm is much more accurate and effective than those of the other three existing algorithms.

A novel pairwise 3-D shape context for partial object matching and retrieval is developed for extracting 3-D light poles and trees from mobile laser scanning (MLS) point clouds in a typical urban street scene. Unlike the single-point shape context describing only the local topology of a shape, the pairwise 3-D shape context can simultaneously model the local and global geometric structures of a shape in manifold space. By using histogram descriptors, the pairwise 3-D shape context has such characteristics as invariance to scale, invariance to orientation, and partial insensitivity to topological changes. Our results show that 3-D light poles and individual trees can be extracted from the RIEGL VMX-450 MLS point clouds and the performance achieved using our algorithm is much more accurate and effective than those of the other two existing algorithms.