Timely and accurately extracting and assessing pavement cracks is crucial for intelligent transportation systems (ITS) to improve road maintenance and safety. In this paper, we present an automated framework for crack semantic segmentation and quantification using optical images. First, a unique boundary-guided real-time high-resolution network is proposed, termed as BulletNet, for crack semantic segmentation. BulletNet is a bullet-head structure that can retain crack details while ensuring real-time inference speed, in which a Cross-Scale Global Attention (CSGA) module is designed to enhance global feature representation and pixel-level relations, as well as a Boundary-Guided Fusion (BGF) module proposed to utilize boundary features to guide the fusion of crack details and contextual information. Second, a Pixel-level Crack Quantification (PCQ) algorithm is proposed for complex cracks, incorporating an Improved Discrete Skeleton Evolution (IDSE) method to optimize skeleton pruning for accurate crack length and a normal vector correction method to adjust propagation direction for precise crack width. Comprehensive experiments on three datasets showed that the proposed BulletNet surpassed the comparative models in terms of efficiency and performance, with average F1-score, mIoU, and Frames per second (FPS) of 87.20%, 88.70%, and 125.53, respectively. In addition, tested on 200 images, the PCQ calculated the crack maximum widths and lengths with an average relative error of 6.96% and 4.62%, respectively. Finally, BulletNet was deployed on edge devices for field testing, and a system based on the PCQ algorithm was developed to validate the effectiveness of the entire framework.

Recently, in the field of Synthetic Aperture Radar (SAR) ship detection, deep learning-based methods have made significant strides in terms of detection accuracy and speed. However, small-scale targets and complex backgrounds remain a formidable obstacle to SAR ship detection. To overcome the aforementioned challenges, this letter proposes LiteSAR-Net, a lightweight SAR ship detector, with the aim of enhancing ship detection capabilities in SAR imagery. In detail, an End-to-end Image Preprocessing Network (E2IPNet) is proposed to strengthen context information and expand the network’s effective receptive field. In addition, to prevent the dilution of semantic information, the Channel Feature Guided Spatial Pyramid Pooling (CFGSPP) is proposed, which can adjust the parameters adaptively based on inter-channel information. The proposed LiteSAR-Net achieved an Average Precision (AP) of 98.61% on the SAR Ship Detection Dataset (SSDD) and 93.33% on the High-resolution SAR Images Dataset (HRSID), with a parameter of only 5.247M, outperformed many state-of-the-art (SOTA) detectors. Our code can be found at https://github.com/ZYMCCX/LiteSAR-Net.

Bitemporal remote sensing (RS) semantic change detection (SCD) involves discerning and categorizing changes in the same geographical area across two RS images taken at different times. High-performance SCD approaches typically address this task using multitask networks that simultaneously handle binary change detection (BCD) and semantic segmentation (SS). Despite significant advancements in SCD research, constructing a multitask network that fully explores the correlation between BCD and SS remains challenging. To address this, we propose a novel approach called the multitask CNN-transformer network (MCTNet), tailored for SCD using bitemporal RS images. Our Siamese network simultaneously tackles SS and BCD via three subnetworks: two for SS and one for BCD. The methodology begins with a multiscale convolutional neural network (CNN) extracting local features from input images, and converting them into tokens. A Transformer module with an encoder-decoder architecture then captures long-range dependencies among these visual tokens. The extracted features are subsequently passed to multitask heads, generating predicted outputs. To ensure that the BCD results remain consistent regardless of the order of images in the input pair, we introduce spatiotemporal feature learning (SFL), enabling the acquisition of temporal-symmetric representations for BCD. Extensive experimental validation on the WHU-CD, SECOND, and HRSCD datasets demonstrates the effectiveness and efficiency of MCTNet for both SS and BCD tasks. The source code for this article will be published on GitHub in the futurehttps://github.com/kangziwen1/MCTNet.

Deep learning-based algorithms detect pavement cracks in an end-to-end manner from Mobile Laser Scanning (MLS) point clouds, achieving impressive results. However, the accuracy of existing methods still has room to improve due to the difficulty of effectively encoding multiscale features and the limited training data. In this paper, we propose a novel pavement crack detection framework, Crack-U2Net, which innovatively incorporates a two-level nested U-Net architecture for feature learning. This design enables the learning of intra-stage multiscale features without introducing significant memory and computation costs, resulting in substantial improvements in accuracy. Moreover, to solve the challenge of insufficient training data, we propose a Geometry-based Data Augmentation (GDA) strategy, aiming to expand the pavement dataset while preserving the pavement geometry. Extensive experiments on the Qinghai-Tibet Highway point cloud dataset demonstrate the higher accuracy and efficiency of Crack-U2Net over the state-of-the-art methods, achieving an average precision, recall, F$1\text - $score, and accuracy of 83.8%, 77.6%, 80.1%, and 95.8%, respectively.

Abstract Occlusion has always been a challenging research topic in the field of multi‐target tracking. The invisibility of the target in full occlusion increases the difficulty of continuous tracking, which makes the recovery failure when the target is re‐visible, and ultimately leads to a decrease in tracking accuracy. To address full occlusion problem, an effective multi‐object tracking algorithm with full occlusion handle and adaptive fusion features is proposed. Firstly, a spatio‐temporal model is established for full occlusion, and a simple, efficient and training‐free method is proposed to find full occluded targets. Secondly, local high discrimination features with better stability and independence is proposed to realize effective correlation between targets before and after the full occlusion. Finally, an adaptive feature fusion mechanism is proposed, which can adjust feature structure dynamically according to the occlusion state. The experimental results show that most evaluation metrics of the proposed algorithm are superior to those of some typical algorithms proposed in recent years under full occlusion tracking scenes. The proposed algorithm can realize accurate occluded targets identification and improve tracking robustness under short‐term, long‐term and frequent full occlusion.

Accurate and effective road object semantic segmentation plays a significant role in supporting extensive intelligent transportation system (ITS)-related applications. However, most existing image-based methods and point-based methods cannot deliver promising solutions with respect to segmentation accuracy and robustness, especially in complex urban road scenes. Thus, we design a saliency-guided transformer architecture (STN) in this letter for point-wise semantic segmentation from mobile laser scanning (MLS) point clouds. First, four types of feature saliency maps are constructed to obtain more compact feature spaces for enhancing the feature encoding semantics. Then, integrated with offset attention mechanisms and edge convolutions, an effective point-wise transformer network is proposed to extract high-level features for point-wise label assignment of road objects. The STN model is evaluated on the Pairs-Lille-3D dataset and achieves satisfactory experimental results with 87.2% overall accuracy and 81.7% mean IoU, respectively. Comparative studies with five deep learning-based methods also prove the superior performance of the STN model for large-scale semantic segmentation tasks.

The up-to-date and accurate building database serves as an important prerequisite to many applications. However, caused by the issues of shape and size variations, texture and distribution diversities, and occlusion and shadow covers of buildings in remote sensing images, it is still challenging to well guarantee the integrity and accuracy of the extracted building instances. This letter proposes a high-resolution capsule network (HR-CapsNet) to conduct building extraction. First, designed with an HR-CapsNet architecture assisted by multiresolution feature propagation and fusion, the HR-CapsNet can provide semantically strong and spatially accurate feature representations to promote the pixel-wise building extraction accuracy. In addition, integrated with an efficient capsule feature attention module, the HR-CapsNet can attend to channel-wise informative and class-specific spatial features to boost the feature encoding quality. Quantitative evaluations, visual inspections, and comparative experiments on two large remote sensing image datasets demonstrate that the HR-CapsNet provides a feasible and competitive solution to building extraction tasks.

Periodically conducting land cover mapping plays a vital role in monitoring the status and changes of the land use. The up-to-date and accurate land use database serves importantly for a wide range of applications. This letter constructs an efficient self-attention capsule network (ESA-CapsNet) for land cover classification of multispectral light detection and ranging (LiDAR) data. First, formulated with a novel capsule encoder–decoder architecture, the ESA-CapsNet performs promisingly in extracting high-level, informative, and strong feature semantics for pixel-wise land cover classification by using the five types of rasterized feature images. Furthermore, designed with a novel capsule-based attention module, the channel and spatial feature encodings are comprehensively exploited to boost the feature saliency and robustness. The ESA-CapsNet is evaluated on two multispectral LiDAR data sets and achieves an advantageous performance with the overall accuracy, average accuracy, and kappa coefficient of over 98.42%, 95.15%, and 0.9776, respectively. Comparative experiments with the existing methods also demonstrate the effectiveness and applicability of the ESA-CapsNet in land cover classification tasks.

High-definition map building and map navigation systems often require detailed, complete, and up-to-date data of road markings. The real-time and accurate recognition of road markings also serves significantly to the autonomous vehicles. This letter designs a self-attention (SA)-guided high-resolution capsule network to conduct road marking extraction from aerial images. First, by combining the superiorities of capsule formulation and high-resolution network architecture, this model behaves advantageously in providing fine-grained and strong feature semantics for promoting pixel-wise marking extraction accuracy. Furthermore, boosted by the capsule-based SA and adversarial learning mechanisms, the feature encoding quality and robustness are positively enhanced. Quantitative assessments, qualitative inspections, and comparative analyses on two aerial image datasets prove the excellent feasibility and effectiveness of the proposed model in road marking extraction tasks.

Accurately interpreting image contents plays a vital role in many earth observation tasks. This letter constructs a novel cross-context and cross-scale capsule vision transformer (C2-CapsViT) architecture to serve for remote sensing image scene classification. First, employed with a multi-context patch embedding strategy, the token representation quality is greatly boosted to encode different-context feature semantics. Second, designed with a multiscale transformer block, different-grained long-range global feature interactions and different-type feature self-attentions are concurrently exploited to promote the feature encoding quality. Moreover, by combining the convolution and transformer structures, local and global feature semantics are effectively fused to direct accurate predictions. The C2-CapsViT is elaborately verified on three scene classification data sets. Both quantitative evaluations and comparative analyses prove its competitive capability and advanced performance.

Road detection plays an important role in a wide range of applications. However, due to size variations, spectral diversities, occlusions, and complex scenarios, it is still challenging to accurately extract roads from very-high resolution (VHR) optical remote sensing images. This paper proposes a capsule feature pyramid network for extracting road networks from VHR optical images, termed as RoadCapsFPN. By designing a capsule feature pyramid network, the RoadCapsFPN extracts and integrates multiscale capsule features to recover a high-resolution and semantically strong road feature representation. Next, we also design a contextual feature module, including dense atrous convolution (DAC) and residual multi-kernel pooling (RMP) units, to further exploit rich contextual properties of the roads at a high-resolution perspective. Benefitting from the multiscale feature abstraction and context augmentation, our RoadCapsFPN shows impressing results in processing variedly-sized and diversely-spectral roads in complex environments. Two testing datasets, Google and Massichusate Roads Datasets, are used for evaluating the proposed RoadCapsFPN via four testing indicators -precision,recall, intersection-over-union (IoU), and$F_{1}$-score. Comparative studies also confirm the superior performance of the RoadCapsFPN in accurately extracting road networks.

Periodically monitoring the pavement conditions is of great importance to many intelligent transportation activities. Timely and correctly identifying the distresses or anomalies on pavement surfaces can help to smooth traffic flows and avoid potential threats to pavement securities. In this paper, we develop a novel context-augmented capsule feature pyramid network (CCapFPN) to detect cracks from pavement images. The CCapFPN adopts vectorial capsules to represent high-level, intrinsic, and salient features of cracks. By designing a feature pyramid architecture, the CCapFPN can fuse different levels and different scales of capsule features to provide a high-resolution, semantically strong feature representation for accurate crack detection. To take advantage of the context properties, a context-augmented module is embedded into each stage of the CCapFPN to rapidly enlarge the receptive field. The CCapFPN performs effectively and efficiently in processing pavement images of diverse conditions and detecting cracks of different topologies. Quantitative evaluations show that an overall performance with a precision, a recall, and an F-score of 0.9200, 0.9149, and 0.9174, respectively, were achieved on the test datasets. Comparative studies with some existing deep learning and edge based crack detection methods also confirm the superior performance of the CCapFPN in crack detection tasks.

Building footprint extraction plays an important role in a wide range of applications. However, due to size and shape diversities, occlusions, and complex scenarios, it is still challenging to accurately extract building footprints from aerial images. This letter proposes a capsule feature pyramid network (CapFPN) for building footprint extraction from aerial images. Taking advantage of the properties of capsules and fusing different levels of capsule features, the CapFPN can extract high-resolution, intrinsic, and semantically strong features, which perform effectively in improving the pixel-wise building footprint extraction accuracy. With the use of signed distance maps as ground truths, the CapFPN can extract solid building regions free of tiny holes. Quantitative evaluations on an aerial image data set show that a precision, recall, intersection-over-union (IoU), and F-score of 0.928, 0.914, 0.853, and 0.921, respectively, are obtained. Comparative studies with six existing methods confirm the superior performance of the CapFPN in accurately extracting building footprints.

Accurate road marking extraction and classification play a significant role in the development of autonomous vehicles (AVs) and high-definition (HD) maps. Due to point density and intensity variations from mobile laser scanning (MLS) systems, most of the existing thresholding-based extraction methods and rule-based classification methods cannot deliver high efficiency and remarkable robustness. To address this, we propose a capsule-based deep learning framework for road marking extraction and classification from massive and unordered MLS point clouds. This framework mainly contains three modules. Module I is first implemented to segment road surfaces from 3D MLS point clouds, followed by an inverse distance weighting (IDW) interpolation method for 2D georeferenced image generation. Then, in Module II, a U-shaped capsule-based network is constructed to extract road markings based on the convolutional and deconvolutional capsule operations. Finally, a hybrid capsule-based network is developed to classify different types of road markings by using a revised dynamic routing algorithm and large-margin Softmax loss function. A road marking dataset containing both 3D point clouds and manually labeled reference data is built from three types of road scenes, including urban roads, highways, and underground garages. The proposed networks were accordingly evaluated by estimating robustness and efficiency using this dataset. Quantitative evaluations indicate the proposed extraction method can deliver 94.11% in precision, 90.52% in recall, and 92.43% in F1-score, respectively, while the classification network achieves an average of 3.42% misclassification rate in different road scenes.

Although significant improvement has been achieved in fully autonomous driving and semantic high-definition map (HD) domains, most of the existing 3D point cloud segmentation methods cannot provide high representativeness and remarkable robustness. The principally increasing challenges remain in completely and efficiently extracting high-level 3D point cloud features, specifically in large-scale road environments. This paper provides an end-to-end feature extraction framework for 3D point cloud segmentation by using dynamic point-wise convolutional operations in multiple scales. Compared to existing point cloud segmentation methods that are commonly based on traditional convolutional neural networks (CNNs), our proposed method is less sensitive to data distribution and computational powers. This framework mainly includes four modules. Module I is first designed to construct a revised 3D point-wise convolutional operation. Then, a U-shaped downsampling-upsampling architecture is proposed to leverage both global and local features in multiple scales in Module II. Next, in Module III, high-level local edge features in 3D point neighborhoods are further extracted by using an adaptive graph convolutional neural network based on the K-Nearest Neighbor (KNN) algorithm. Finally, in Module IV, a conditional random field (CRF) algorithm is developed for postprocessing and segmentation result refinement. The proposed method was evaluated on three large-scale LiDAR point cloud datasets in both urban and indoor environments. The experimental results acquired by using different point cloud scenarios indicate our method can achieve state-of-the-art semantic segmentation performance in feature representativeness, segmentation accuracy, and technical robustness.

Vehicle logo recognition provides an important supplement to vehicle make and model analysis. Some of the existing vehicle logo recognition methods depend on the detection of license plates to roughly locate vehicle logo regions using prior knowledge. The vehicle logo recognition performance is greatly affected by the license plate detection techniques. This paper presents a cascaded deep convolutional network for directly recognizing vehicle logos without depending on the existence of license plates. This is a two-stage processing framework composed of a region proposal network and a convolutional capsule network. First, potential region proposals that might contain vehicle logos are generated by the region proposal network. Then, the convolutional capsule network classifies these region proposals into the background and different types of vehicle logos. We have evaluated the proposed framework on a large test set towards vehicle logo recognition. Quantitative evaluations show that a detection rate, a recognition rate, and an overall performance of 0.987, 0.994, and 0.981, respectively, are achieved. Comparative studies with the Faster R-CNN and other three existing methods also confirm that the proposed method performs effectively and robustly in recognizing vehicle logos of various conditions.

Traffic-sign recognition plays an important role in road transportation systems. This letter presents a novel two-stage method for detecting and recognizing traffic signs from mobile Light Detection and Ranging (LiDAR) point clouds and digital images. First, traffic signs are detected from mobile LiDAR point cloud data according to their geometrical and spectral properties, which have been fully studied in our previous work. Afterward, the traffic-sign patches are obtained by projecting the detected points onto the registered digital images. To improve the performance of traffic-sign recognition, we apply a convolutional capsule network to the traffic-sign patches to classify them into different types. We have evaluated the proposed framework on data sets acquired by a RIEGL VMX-450 system. Quantitative evaluations show that a recognition rate of 0.957 is achieved. Comparative studies with the convolutional neural network (CNN) and our previous supervised Gaussian-Bernoulli deep Boltzmann machine (GB-DBM) classifier also confirm that the proposed method performs effectively and robustly in recognizing traffic signs of various types and conditions.

Land cover mapping is an effective way to quantify land resources and monitor their changes. It plays an important role in a wide range of applications. This letter proposes a hybrid capsule network for land cover classification using multispectral light detection and ranging (LiDAR) data. First, the multispectral LiDAR data were rasterized into a set of feature images to exploit the geometrical and spectral properties of different types of land covers. Then, a hybrid capsule network composed of an encoder network and a decoder network is trained to extract both high-level local and global entity-oriented capsule features for accurate land cover classification. Quantitative classification evaluations on two data sets show that the overall accuracy, average accuracy, and kappa coefficient of over 97.89%, 94.54%, and 0.9713, respectively, are obtained. Comparative studies with five existing methods confirm that the proposed method performs robustly and accurately in land cover classification using the multispectral LiDAR data.

Effective object extraction plays an important role in many point cloud-based applications. This letter proposes a 3-D feature matching framework for point cloud object extraction. To determine the optimal affine transformation parameters for each template feature point, a convex dissimilarity function and the locally affine-invariant geometric constraints are designed to construct the overall objective function. The 3-D feature matching framework is integrated into a point cloud object extraction workflow. Extraction results on six test data sets show that average completeness, correctness, quality, and F1-measure of 0.96, 0.97, 0.93, and 0.96, respectively, are obtained in extracting light poles, vehicles, and palm trees. Comparative studies also confirm that the proposed method performs effectively and robustly, and exhibits superior or compatible performance over the other compared methods.

Periodical road manhole cover measurement is extremely important to ensure road safety and reduce traffic disasters. This letter proposes an effective method for delineating road manhole covers from mobile laser scanning point cloud data. To improve processing efficiency, first, road surface points are segmented and rasterized into georeferenced intensity images. Then, object-oriented patches are generated through superpixel segmentation and further fed to a convolutional capsule network classifier for manhole cover detection. Finally, manhole covers are accurately delineated through a marked point process of disks. Quantitative evaluations on three data sets show that an average completeness, correctness, quality, and F1-measure of 0.965, 0.961, 0.929, and 0.963, respectively, are obtained. Comparative studies with three existing methods confirm that the proposed method performs superiorly in delineating manhole covers of varying conditions and on complex road surface environments.

Vehicle detection plays an important role in a variety of traffic-related applications. However, due to the scale and orientation variations and partial occlusions of vehicles, it is still challengeable to accurately detect vehicles from remote sensing images. This letter proposes a convolutional capsule network for detecting vehicles from high-resolution remote sensing images. First, a test image is segmented into superpixels to generate meaningful and nonredundant patches. Then, these patches are input to a convolutional capsule network to label them into vehicles or the background. Finally, nonmaximum suppression is adopted to eliminate repetitive detections. Quantitative evaluations on four test data sets show that average completeness, correctness, quality, and F1-measure of 0.93, 0.97, 0.90, and 0.95, respectively, are obtained. Comparative studies with three existing methods confirm that the proposed method effectively performs in detecting vehicles of various conditions.

Using mobile Light Detection and Ranging point clouds to accomplish road scene labeling tasks shows promise for a variety of applications. Most existing methods for semantic labeling of point clouds require a huge number of fully supervised point cloud scenes, where each point needs to be manually annotated with a specific category. Manually annotating each point in point cloud scenes is labor intensive and hinders practical usage of those methods. To alleviate such a huge burden of manual annotation, in this paper, we introduce an active learning method that avoids annotating the whole point cloud scenes by iteratively annotating a small portion of unlabeled supervoxels and creating a minimal manually annotated training set. In order to avoid the biased sampling existing in traditional active learning methods, a neighbor-consistency prior is exploited to select the potentially misclassified samples into the training set to improve the accuracy of the statistical model. Furthermore, lots of methods only consider short-range contextual information to conduct semantic labeling tasks, but ignore the long-range contexts among local variables. In this paper, we use a higher order Markov random field model to take into account more contexts for refining the labeling results, despite of lacking fully supervised scenes. Evaluations on three data sets show that our proposed framework achieves a high accuracy in labeling point clouds although only a small portion of labels is provided. Moreover, comparative experiments demonstrate that our proposed framework is superior to traditional sampling methods and exhibits comparable performance to those fully supervised models.

This paper presents a supervoxel-based approach for automated localization and extraction of street light poles in point clouds acquired by a mobile LiDAR system. The method consists of five steps: preprocessing, localization, segmentation, feature extraction, and classification. First, the raw point clouds are divided into segments along the trajectory, the ground points are removed, and the remaining points are segmented into supervoxels. Then, a robust localization method is proposed to accurately identify the pole-like objects. Next, a localization-guided segmentation method is proposed to obtain pole-like objects. Subsequently, the pole features are classified using the support vector machine and random forests. The proposed approach was evaluated on three datasets with 1,055 street light poles and 701 million points. Experimental results show that our localization method achieved an average recall value of 98.8%. A comparative study proved that our method is more robust and efficient than other existing methods for localization and extraction of street light poles.

This letter addresses the problem of detecting pole-like road objects (including light poles and traffic signposts) from mobile light detection and ranging (LiDAR) data for transportation-related applications. The method consists of two consecutive stages: training and pole-like object detection. At the training stage, a contextual visual vocabulary is created from the feature regions generated from a training data set by supervoxel segmentation. At the pole-like object detection stage, a bag-of-contextual-visual-words representation is generated for each semantic object segmented from mobile LiDAR data. The experimental results show that the proposed method achieves correctness, omission, and commission of 88.9%, 11.1%, and 2.8%, respectively, in detecting pole-like road objects. Computational complexity analysis demonstrates that our method provides a promising and effective solution to rapid and accurate detection of pole-like objects from large volumes of mobile LiDAR data.

This paper presents an automated algorithm for rapidly and effectively detecting cars directly from large-volume 3-D point clouds. Rather than using low-order descriptors, a multilayer feature generation model is created to obtain high-order feature representations for 3-D local patches through deep learning techniques. To handle cars with different levels of incompleteness caused by data acquisition ways and occlusions, a hierarchical visibility estimation model is developed to augment Hough voting. Considering scale and orientation variations in the azimuth direction, a set of multiscale Hough forests is constructed to rotationally cast votes to estimate cars' centroids. Quantitative assessments show that the proposed algorithm achieves average completeness, correctness, quality, and F1-measure of 0.94, 0.96, 0.90, and 0.95, respectively, in detecting 3-D cars. Comparative studies also demonstrate that the proposed algorithm outperforms the other four existing algorithms in accurately and completely detecting 3-D cars from large-scale 3-D point clouds.

This paper presents an algorithm for vehicle detection in high-resolution aerial images through a fast sparse representation classification method and a multiorder feature descriptor that contains information of texture, color, and high-order context. To speed up computation of sparse representation, a set of small dictionaries, instead of a large dictionary containing all training items, is used for classification. To extract the context information of a patch, we proposed a high-order context information extraction method based on the proposed fast sparse representation classification method. To effectively extract the color information, the RGB color space is transformed into color name space. Then, the color name information is embedded into the grids of histogram of oriented gradient feature to represent the low-order feature of vehicles. By combining low- and high-order features together, a multiorder feature is used to describe vehicles. We also proposed a sample selection strategy based on our fast sparse representation classification method to construct a complete training subset. Finally, a set of dictionaries, which are trained by the multiorder features of the selected training subset, is used to detect vehicles based on superpixel segmentation results of aerial images. Experimental results illustrate the satisfactory performance of our algorithm.

Semantic labeling of road scenes using colorized mobile LiDAR point clouds is of great significance in a variety of applications, particularly intelligent transportation systems. However, many challenges, such as incompleteness of objects caused by occlusion, overlapping between neighboring objects, interclass local similarities, and computational burden brought by a huge number of points, make it an ongoing open research area. In this paper, we propose a novel patch-based framework for labeling road scenes of colorized mobile LiDAR point clouds. In the proposed framework, first, three-dimensional (3-D) patches extracted from point clouds are used to construct a 3-D patch-based match graph structure (3D-PMG), which transfers category labels from labeled to unlabeled point cloud road scenes efficiently. Then, to rectify the transferring errors caused by local patch similarities in different categories, contextual information among 3-D patches is exploited by combining 3D-PMG with Markov random fields. In the experiments, the proposed framework is validated on colorized mobile LiDAR point clouds acquired by the RIEGL VMX-450 mobile LiDAR system. Comparative experiments show the superior performance of the proposed framework for accurate semantic labeling of road scenes.

This paper presents a spatial-related traffic sign inspection process for sign type, position, and placement using mobile laser scanning (MLS) data acquired by a RIEGL VMX-450 system and presents its potential for traffic sign inventory applications. First, the paper describes an algorithm for traffic sign detection in complicated road scenes based on the retroreflectivity properties of traffic signs in MLS point clouds. Then, a point cloud-to-image registration process is proposed to project the traffic sign point clouds onto a 2-D image plane. Third, based on the extracted traffic sign points, we propose a traffic sign position and placement inspection process by creating geospatial relations between the traffic signs and road environment. For further inventory applications, we acquire several spatial-related inventory measurements. Finally, a traffic sign recognition process is conducted to assign sign type. With the acquired sign type, position, and placement data, a spatial-associated sign network is built. Experimental results indicate satisfactory performance of the proposed detection, recognition, position, and placement inspection algorithms. The experimental results also prove the potential of MLS data for automatic traffic sign inventory applications.

This paper proposes a novel algorithm for detecting road scene objects (e.g., light poles, traffic signposts, and cars) from 3-D mobile-laser-scanning point cloud data for transportation-related applications. To describe local abstract features of point cloud objects, a contextual visual vocabulary is generated by integrating spatial contextual information of feature regions. Objects of interest are detected based on the similarity measures of the bag of contextual-visual words between the query object and the segmented semantic objects. Quantitative evaluations on two selected data sets show that the proposed algorithm achieves an average recall, precision, quality, and F-score of 0.949, 0.970, 0.922, and 0.959, respectively, in detecting light poles, traffic signposts, and cars. Comparative studies demonstrate the superior performance of the proposed algorithm over other existing methods.

This paper presents a tensor voting approach to automated detection of dark spots in RADARSAT-1 ScanSAR Narrow Beam mode images. First, a thresholding algorithm that well maximizes the ratio of between-class variance to within-class variance is used to detect potential dark spot candidates. Next, a tensor voting framework integrated with sparse and dense ball votings is carried out to suppress noise while maintaining dark spots. Then, a saliency map that reflects the probability of a pixel being located within a dark spot is generated using the saliencies of ball tensors. Finally, a segmentation method is applied to ascertain dark spots based on the saliency map. The proposed approach has been tested on a set of RADARSAT-1 ScanSAR Narrow Beam intensity images. Quantitative evaluations demonstrate that the proposed approach achieves an average commission error, omission error, and quality of 0.003, 0.037, and 0.956, respectively, for detecting dark spots in SAR intensity imagery.

This paper proposes a method for automated extraction of street trees in a typical urban environment from 3D point cloud data acquired by the mobile laser scanning system. First, the algorithm utilizes the voxel-based method to remove the ground points from the scene. Second, the Euclidean distance clustering is adopted to cluster points into individual objects. The eigenvalues of neighborhood covariance matrix and the corresponding normalized centroid distance are computed for each point to obtain the subdivided dimensional features. Finally, the statistical component features and horizontal information are calculated for object detection. The experiment results show the feasibility of the proposed algorithm.

This paper presents a novel approach for extracting street lighting poles directly from MLS point clouds. The approach includes four stages: 1) elevation filtering to remove ground points, 2) Euclidean distance clustering to cluster points, 3) voxel-based normalized cut (Ncut) segmentation to separate overlapping objects, and 4) statistical analysis of geometric properties to extract 3D street lighting poles. A Dataset acquired by a RIEGL VMX-450 MLS system are tested with the proposed approach. The results demonstrate the efficiency and reliability of the proposed approach to extract 3D street lighting poles.

This letter presents a rotation-invariant method for detecting geospatial objects from high-resolution satellite images. First, a superpixel segmentation strategy is proposed to generate meaningful and nonredundant patches. Second, a multilayer deep feature generation model is developed to generate high-level feature representations of patches using deep learning techniques. Third, a set of multiscale Hough forests with embedded patch orientations is constructed to cast rotation-invariant votes for estimating object centroids. Quantitative evaluations on the images collected from Google Earth service show that an average completeness, correctness, quality, and F1- measure values of 0.958, 0.969, 0.929, and 0.963, respectively, are obtained. Comparative studies with three existing methods demonstrate the superior performance of the proposed method in accurately and correctly detecting objects that are arbitrarily oriented and of varying sizes.

This letter presents a 3-D object matching framework to support information extraction directly from 3-D point clouds. The problem of 3-D object matching is to match a template, represented by a group of 3-D points, to a point cloud scene containing an instance of that object. A locally affine-invariant geometric constraint is proposed to effectively handle affine transformations, occlusions, incompleteness, and scales in 3-D point clouds. The 3-D object matching framework is integrated into 3-D correspondence computation, 3-D object detection, and point cloud object classification in mobile laser scanning (MLS) point clouds. Experimental results obtained using the 3-D point clouds acquired by a RIEGL VMX-450 system showed that completeness, correctness, and quality of over 0.96, 0.94, and 0.91 are achieved, respectively, with the proposed framework in 3-D object detection. Comparative studies demonstrate that the proposed method outperforms the two existing methods for detecting 3-D objects directly from large-volume MLS point clouds.

The assessment of pavement cracks is one of the essential tasks for road maintenance. This paper presents a novel framework, called ITVCrack, for automated crack extraction based on iterative tensor voting (ITV), from high-density point clouds collected by a mobile laser scanning system. The proposed ITVCrack comprises the following: 1) the preprocessing involving the separation of road points from nonroad points using vehicle trajectory data; 2) the generation of the georeferenced feature (GRF) image from the road points; and 3) the ITV-based crack extraction from the noisy GRF image, followed by an accurate delineation of the curvilinear cracks. Qualitatively, the method is applicable for pavement cracks with low contrast, low signal-to-noise ratio, and bad continuity. Besides the application to GRF images, the proposed framework demonstrates much better crack extraction performance when quantitatively compared to existing methods on synthetic data and pavement images.

This paper proposes a novel algorithm for extracting street light poles from vehicleborne mobile light detection and ranging (LiDAR) point-clouds. First, the algorithm rapidly detects curb-lines and segments a point-cloud into road and nonroad surface points based on trajectory data recorded by the integrated position and orientation system onboard the vehicle. Second, the algorithm accurately extracts street light poles from the segmented nonroad surface points using a novel pairwise 3-D shape context. The proposed algorithm is tested on a set of point-clouds acquired by a RIEGL VMX-450 mobile LiDAR system. The results show that road surfaces are correctly segmented, and street light poles are robustly extracted with a completeness exceeding 99%, a correctness exceeding 97%, and a quality exceeding 96%, thereby demonstrating the efficiency and feasibility of the proposed algorithm to segment road surfaces and extract street light poles from huge volumes of mobile LiDAR point-clouds.

This paper presents a survey of literature about road feature extraction, giving a detailed description of a Mobile Laser Scanning (MLS) system (RIEGL VMX-450) for transportation-related applications. This paper describes the development of automated algorithms for extracting road features (road surfaces, road markings, and pavement cracks) from MLS point cloud data. The proposed road surface extraction algorithm detects road curbs from a set of profiles that are sliced along vehicle trajectory data. Based on segmented road surface points, we create Geo-Referenced Feature (GRF) images and develop two algorithms, respectively, for extracting the following: 1) road markings with high retroreflectivity and 2) cracks containing low contrast with their surroundings, low signal-to-noise ratio, and poor continuity. A comprehensive comparison illustrates satisfactory performance of the proposed algorithms and concludes that MLS is a reliable and cost-effective alternative for rapid road inspection.

Updating road markings is one of the routine tasks of transportation agencies. Compared with traditional road inventory mapping techniques, vehicle-borne mobile light detection and ranging (LiDAR) systems can undertake the job safely and efficiently. However, current hurdles include software and computing challenges when handling huge volumes of highly dense and irregularly distributed 3-D mobile LiDAR point clouds. This paper presents the development and implementation aspects of an automated object extraction strategy for rapid and accurate road marking inventory. The proposed road marking extraction method is based on 2-D georeferenced feature (GRF) images, which are interpolated from 3-D road surface points through a modified inverse distance weighted (IDW) interpolation. Weighted neighboring difference histogram (WNDH)-based dynamic thresholding and multiscale tensor voting (MSTV) are proposed to segment and extract road markings from the noisy corrupted GRF images. The results obtained using 3-D point clouds acquired by a RIEGL VMX-450 mobile LiDAR system in a subtropical urban environment are encouraging.

This paper proposes a novel framework for automated detection of urban road manhole covers using mobile laser scanning (MLS) data. First, to narrow searching regions and reduce the computational complexity, road surface points are segmented from a raw point cloud via a curb-based road surface segmentation approach and rasterized into a georeferenced intensity image through inverse distance weighted interpolation. Then, a supervised deep learning model is developed to construct a multilayer feature generation model for depicting high-order features of local image patches. Next, a random forest model is trained to learn mappings from high-order patch features to the probabilities of the existence of urban road manhole covers centered at specific locations. Finally, urban road manhole covers are detected from georeferenced intensity images based on the multilayer feature generation model and random forest model. Quantitative evaluations show that the proposed algorithm achieves an average completeness, correctness, quality, and F1-measure of 0.955, 0.959, 0.917, and 0.957, respectively, in detecting urban road manhole covers from georeferenced intensity images. Comparative studies demonstrate the advantageous performance of the proposed algorithm over other existing methods for rapid and automated detection of urban road manhole covers using MLS data.

This paper proposes a novel, automated algorithm for rapidly extracting urban road facilities, including street light poles, traffic signposts, and bus stations, for transportation-related applications. A detailed description and implementation of the proposed algorithm is provided using mobile laser scanning data collected by a state-of-the-art RIEGL VMX-450 system. First, to reduce the quantity of data to be handled, a fast voxel-based upward growing method is developed to remove ground points. Then, off-ground points are clustered and segmented into individual objects via Euclidean distance clustering and voxel-based normalized cut segmentation, respectively. Finally, a 3-D object matching framework, benefiting from a locally affine-invariant geometric constraint, is developed to achieve the extraction of 3-D objects. Quantitative evaluations show that the proposed algorithm attains an average completeness, correctness, quality, and F1-measure of 0.949, 0.971, 0.922, and 0.960, respectively, in extracting 3-D light poles, traffic signposts, and bus stations. Comparative studies demonstrate the efficiency and feasibility of the proposed algorithm for automated and rapid extraction of urban road facilities.

This paper presents a stepwise algorithm for extracting power-lines from mobile laser scanning (MLS) data. This algorithm first extracts non-road points from MLS data by estimating road ranges with regard to scanning mechanism and applying elevation-difference and slope criteria to the road ranges scan-line by scan-line. Then, three filters, in terms of height, spatial density, and size-and-shape, are proposed to extract power-line points in the identified non-road points, followed by Hough transform and Euclidean distance clustering. Finally, a 3D power line is modelled as a horizontal line in X-Y plane and a vertical catenary curve defined by a hyperbolic cosine function in X-Z plane. The proposed algorithm has been tested on a sample of point clouds acquired by a RIEGL VMX-450 MLS system. The results demonstrate the applicability of the proposed algorithm in extracting power transmission lines.

This paper presents a novel method for estimating earthwork volumes in asphalt pavement reconstruction using a mobile laser scanning (MLS) system. First, based on the static targets, this method registers two point cloud datasets into the same coordinate system, which respectively are acquired in the reconstructing road before and after asphalting. Next, road surface points are detected from each point cloud using a curb-based method, and further divided into a set of blocks. Afterwards, the blocks are perpendicularly partitioned into grids, where two surface features are extracted using the RANSAC. Finally, the volume of each grid is calculated according to these two surface features. The proposed algorithm has been tested on two sets of point clouds acquired by a RIEGL VMX-450 MLS system in the reconstructing road before and after asphalting. The results demonstrate the accuracy and efficiency of the proposed algorithm in estimating earthwork volumes.

This paper presents a novel algorithm for extracting 3D crack skeletons from 3D point clouds acquired by a mobile Light Detection and Ranging (LiDAR) system. This algorithm uses intensity information of cloud clouds to identify pavement cracks that usually exhibit lower intensities compared to their surroundings. First, crack candidates are extracted by applying the Otsu thresholding algorithm. Then, a spatial density filter is used to remove outliers. Next, crack points are grouped into crack-lines using a Euclidean distance clustering method. Finally, crack skeletons are extracted based on an L1-medial skeleton extraction method. The proposed algorithm has been tested on a set of mobile LiDAR point clouds acquired by a state-of-the-art RIEGL VMX-450 mobile LiDAR system. The results demonstrate the efficiency and reliability of the proposed algorithm in extracting 3D crack skeletons.

A novel object detection algorithm is developed for automatically detecting road manhole and sewer well covers from mobile light detection and ranging point clouds. This algorithm takes advantage of a marked point process of disks and rectangles to model the locations of manhole and sewer well covers and their geometric dimensions. A reversible jump Markov chain Monte Carlo algorithm is implemented for simulating the posterior distribution obtained using a Bayesian paradigm. The detection results obtained from the road surface point clouds acquired by a RIEGL VMX-450 system show that the manhole and sewer well covers can be detected automatically and accurately. The performance achieved using the proposed algorithm is much more accurate and effective than those of the other three existing algorithms.

A novel pairwise 3-D shape context for partial object matching and retrieval is developed for extracting 3-D light poles and trees from mobile laser scanning (MLS) point clouds in a typical urban street scene. Unlike the single-point shape context describing only the local topology of a shape, the pairwise 3-D shape context can simultaneously model the local and global geometric structures of a shape in manifold space. By using histogram descriptors, the pairwise 3-D shape context has such characteristics as invariance to scale, invariance to orientation, and partial insensitivity to topological changes. Our results show that 3-D light poles and individual trees can be extracted from the RIEGL VMX-450 MLS point clouds and the performance achieved using our algorithm is much more accurate and effective than those of the other two existing algorithms.

BACKGROUND: As an obligate intracellular parasite, Apicomplexa interacts with the host in the special living environment, competing for energy and nutrients from the host cells by manipulating the host metabolism. Previous studies of host-parasite interaction mainly focused on using cellular and biochemical methods to investigate molecular functions in metabolic pathways of parasite infected hosts. Computational approaches taking advantage of high-throughput biological data and topology of metabolic pathways have a great potential in revealing the details and mechanism of parasites-to-host interactions. A new analytical method was designed in this work to study host-parasite interactions in human cells infected with Plasmodium falciparum and Cryptosporidium parvum. RESULTS: We introduced a new method that analyzes the host metabolic pathways in divided parts: host specific subpathways and host-parasite common subpathways. Upon analysis on gene expression data from cells infected by Plasmodium falciparum or Cryptosporidium parvum, we found: (i) six host-parasite common subpathways and four host specific subpathways were significantly altered in plasmodium infected human cells; (ii) plasmodium utilized fatty acid biosynthesis and elongation, and Pantothenate and CoA biosynthesis to obtain nutrients from host environment; (iii) in Cryptosporidium parvum infected cells, most of the host-parasite common enzymes were down-regulated, whereas the host specific enzymes up-regulated; (iv) the down-regulation of common subpathways in host cells might be caused by competition for the substrates and up-regulation of host specific subpathways may be stimulated by parasite infection. CONCLUSION: Results demonstrated a significantly coordinated expression pattern between the two groups of subpathways. The method helped expose the impact of parasite infection on host cell metabolism, which was previously concealed in the pathway enrichment analysis. Our approach revealed detailed subpathways and metabolic information are important to the symbiosis in two kinds of the apicomplex parasites, and highlighted its significance in research and understanding of parasite-host interactions.

BACKGROUND: Embryogenesis is the process by which the embryo is formed, develops, and establishes developmental hierarchies of tissues. The recent advance in microarray technology made it possible to investigate the tissue specific patterns of gene expression and their relationship with tissue lineages. This study is focused on how tissue specific functions, tissue lineage, and cell differentiation are correlated, which is essential to understand embryonic development and organism complexity. RESULTS: We performed individual gene and gene set based analysis on multiple tissue expression data, in association with the classic topology of mammalian fate maps of embryogenesis. For each sub-group of tissues on the fate map, conservatively, differentially and correlatively expressed genes or gene sets were identified. Tissue distance was found to correlate with gene expression divergence. Tissues of the ectoderm or mesoderm origins from the same segments on the fate map shared more similar expression pattern than those from different origins. Conservatively expressed genes or gene sets define common functions in a tissue group and are related to tissue specific diseases, which is supported by results from Gene Ontology and KEGG pathway analysis. Gene expression divergence is larger in certain human tissues than in the mouse homologous tissues. CONCLUSION: The results from tissue lineage and gene expression analysis indicate that common function features of neighbor tissue groups were defined by the conservatively expressed genes and were related to tissue specific diseases, and differentially expressed genes contribute to the functional divergence of tissues. The difference of gene expression divergence in human and mouse homologous tissues reflected the organism complexity, i.e. distinct neural development levels and different body sizes.