To provide a comprehensive representation of 3-Dimensional (3D) information in large-scale outdoor scenes, multi-platform, multi-sensor, and multi-temporal laser point cloud acquisition and registration technologies have experienced rapid development. However, due to the complexity of outdoor environments and the differences in hardware performance across various observation platforms, significant challenges arise in accurately and efficiently registering multi-source heterogeneous point clouds with inconsistent spatial coordinate systems. These challenges include substantial noise interference, occlusions, missing data, and geometric heterogeneity. In this paper, we propose a heterogeneous point cloud fine registration method based on fully connected graph and heat conduction model. Specifically, we first establish initial correspondences for the classified feature primitives using a Gaussian probability distribution framework. Subsequently, low-level semantic association and rigid transformation compatibility check are employed to rapidly eliminate erroneous matching relationships caused by outliers. As a core step, we develop a homonymous point selection algorithm based on heat conduction simulation to accurately estimate robust correspondences for point cloud candidate pairs with limited overlap and discrete noise. The effectiveness of this approach is attributed to the similarity measure of spatial local geometric structures and global topological distributions provided by the nonlinear heat diffusion Laplacian matrix. Finally, a least-squares model weighted by a residual robust loss function is designed, incorporating facade information to solve for the optimal spatial transformation. Extensive experiments on multiple real-world datasets demonstrate that the proposed method inherits the effectiveness and robustness of geometry-based registration strategies, achieving precise fusion of edge positions in multi-source heterogeneous point clouds, with an average RMSE (Root Mean Square Error) below 0.06m. Compared to existing advanced registration methods (e.g., VGICP, Teaser++, et al.), the proposed method demonstrates outstanding registration performance and holds promising application prospects in fields such as fine-grained 3D reconstruction.

3D object detection is an essential scene perception capability for autonomous vehicles. In intelligent transportation systems, autonomous vehicles require minimal inference latency to sense their surroundings in real-time. However, advanced 3D detection methods often suffer from high inference latency. This limits the real-time deployment of 3D detection models in the real world. To address this problem, this paper proposes a fast and deployable 3D object detection method from the LiDAR point cloud for autonomous driving, namedFade3D. Firstly, we propose a Lightweight Input Encoder (LIE) to extract the most critical features from point clouds. Then, we develop a Spatial Feature Enhancement BEV backbone (SFENet) that efficiently encodes geometry features into compact representations. Additionally, we design an IoU-aware Loss Re-weighting (ILR) that enhances performance by shifting more attention to hard samples. Leveraging LIE and SFENet, our approach is independent of point cloud density and number, achieving significant speed advantages in processing large-scale point clouds and being deployment-friendly. Extensive experiments on KITTI and Waymo Open Dataset (WOD) datasets comparing various baseline detectors demonstrate its universality and superiority. Specifically, our method demonstrates impressive real-time inference capabilities, achieving 51.5 Hz on an RTX3090 GPU and 12.4 Hz on a Jetson Orin embedded development board. Code will be available at https://github.com/wayyeah/Fade3D

Semi-supervised change detection is increasingly becoming an interesting and challenging topic for the remote sensing image processing community. As the application of deep learning in change detection becomes more and more widespread, there is a growing lack of labeled training data, which substantially limits the practical application of change detection. In order to discuss a more effective semi-supervised change detection approach and to make more reasonable use of the large amount of remote sensing data, we propose a semi-supervised change detection framework in this paper, which utilizes two different networks to cross-supervise and provide information to each other. Unlike most existing semi-supervised change detection, the proposed framework also incorporates a new filtering algorithm to find better pseudo-labels for the retraining of the two networks in the paper. Then, the computation of the loss functions of the two networks is crossed and the two networks are used for Transformer and CNN different learning paradigms, respectively, while simplifying the classical deep collaborative learning for consistency regularization. In addition, we add two markers to record the highest MIoU of training during retraining, and dynamically update the pseudo-labels as the training metrics progressively improve, which significantly improves the training effect. Our approach is tested on public dataset and achieves very good results that effectively demonstrate the effectiveness of the proposed framework.

Accurate and automatic detection of road surface element (such as road marking or manhole cover) information is the basis and key to many applications. To efficiently obtain the information of road surface element, we propose a content-adaptive hierarchical deep learning model to detect arbitrary-oriented road surface elements from mobile laser scanning (MLS) point clouds. In the model, we design a densely connected feature integration module (DCFM) to connect and reorganize feature maps of each stage in the backbone network. Besides, we propose a hierarchical prediction module (HPM) to innovatively use the reorganized feature maps to recognize different types of road surface elements, and thus, semantic information of road surface element can be adaptively expressed on multilevel feature maps. We also add a cascade structure (CS) in the head of model to detect the target efficiently, which can learn the offset between the predicted minimum bounding box of road surface element and ground truth. In experiments, we prove that the proposed method mainly contributed by HPM can maintain robust detection performance, even in the cases of unbalanced category number or overlapping of road surface elements. The experiments also prove that the proposed DCFM can improve the recognition effects of small targets. The CS for predicting boundary offset can detect each target more accurately. We also integrate the designed modules into some rotation detectors, e.g., the EAST and R3Det, and achieve the state-of-the-art results in three road scenes with different categories and uneven distribution of road surface elements, which further shows the effectiveness of the proposed method.

In the change detection (CD) task, the UNet architecture has achieved superior results. However, due to the inherent limitation of convolution operations, UNet is inadequate in learning global context and long-range spatial relations. Transformers can capture long-range feature dependencies, but the lack of low-level details may result in limited localization capabilities. Therefore, this article proposes an end-to-end encoding–decoding hybrid transformer model for CD, TransUNetCD, which has the advantages of both transformers and UNet. The model encodes the tokenized image patches from the convolutional neural network (CNN) feature map to extract rich global context information. The decoder upsamples the encoded features, connects them with higher-resolution multiscale features through skip connections to learn local–global semantic features, and restores the full spatial resolution of the feature map to achieve precise localization. The model proposed in this article not only solves the problem that redundant information is generated when extracting low-level features under the UNet framework, but also solves the problem that the relationship between each feature layer cannot be fully modeled and the optimal feature difference representation cannot be obtained. On this basis, we introduce a difference enhancement module to generate a difference feature map containing rich change information. By weighting each pixel and selectively aggregating features, the effectiveness of the network and the accuracy of extracting changing features are improved. The results on multiple datasets demonstrate that, compared to state-of-the-art methods, the TransUNetCD can further reduce false alarms and missed alarms, and the edge of the changing area is more accurate. The model has the highest score in each metric than other baseline models and has a robust generalization ability.

The maintenance of subway tunnels is vital to ensure the safety of their daily operation. Issues experienced by shield subway tunnels, especially the water leakages, require rapid and accurate detection and diagnosis. Due to the large number of disturbances in the tunnels, conventional algorithms face limitations when extracting discriminative features. To solve this problem, we propose a novel and efficient deep learning model for extracting multiscale and discriminative features of water leakages based on mobile laser scanning (MLS) point cloud intensity images. A new residual network module (Res2Net) is integrated with a cascade structure to form a unified model to extract the multiscale features of water leakages. The model can fully consider geometric characteristics of water leakages and grade the residual connections in a single residual block. This expands the size of receptive field in each network layer and can better facilitate the extraction of geometric characteristics of water leakages. Finally, we verify the advantages of the proposed method via experiments on five water leakage datasets of tunnel intensity images converted from point clouds obtained by a self-developed MLS system and compare its performance with other methods.

The point cloud registration is important and necessary for the applications of changing detection, deformation monitoring, and so on, which is also challenging due to the vast clustered points, irregular, and complex structures of spatial objects, and quality effects of the labeled corresponding points. Aiming at this problem, we design an end-to-end 3-D graph deep learning framework of point cloud registration, which can simultaneously learn the detector (graph attention expression) and the descriptor (graph deep feature) for point cloud registration in a weakly supervised way, so that the learned detector and descriptor promote each other in the process of model optimization. Then, the detector is used to automatically extract the keypoints, and the descriptor describes the deep feature of each keypoint. In the framework, we innovatively propose a new module (named MLP_GCN), which fuses multilayer perceptron (MLP) and graph convolutional network (GCN). The MLP_GCN module is further integrated into the detector branch and descriptor branch to fully express the detector and descriptor of the point cloud. In the training process of the framework, we rotate and translate the point cloud randomly to form the training data in a weakly supervised way, which can save plenty of manually labeling time of corresponding points. In the experiments, our method can achieve better results of point cloud registration in comparison with other methods, which verifies the advantages of the proposed method.

Tunnel dislocation affects the stability and waterproof of the structure and endangers its service life. This paper presents a dislocation calculation method of shield tunnel based on cross sectional point clouds, which contain three main steps. First, a longitudinal joint detection method is proposed based on the circumferential joints detected by the existing gradient accumulation method. The point clouds containing capping block in each ring are converted into gray image to obtain the center line position of the block, and the longitudinal joints are calculated through combining with the tunnel design data. Second, tunnel appendages are removed quickly through unfolding the tunnel point clouds by cylinder projection and adjusting the parameters of the cloth simulation filtering algorithm. Finally, the circumferential dislocations are calculated by selecting multiple denoised sections on both sides of the joint and clustering the points at each angle. Meanwhile, the longitudinal dislocations are calculated through fitting the segments in each ring separately. Experimental results show that the automatic extraction rate of longitudinal joints is higher than 94%. The point clouds filtering method can quickly separate the very long tunnel lining from the close appendages attached to it. Meanwhile, the RMSE of the repeated circumferential and longitudinal dislocation are evaluated to be 1.56 mm and 0.57 mm respectively. Compared with the existing state of art methods, these values reach 1.75 mm and 0.63 mm. Through the method proposed, the dislocation value at any mileage and any angle of tunnel can be displayed intuitively.

Accurate and efficient extraction of road marking plays an important role in road transportation engineering, automotive vision, and automatic driving. In this article, we proposed a dense feature pyramid network (DFPN)-based deep learning model, by considering the particularity and complexity of road marking. The DFPN concatenated its shallow feature channels with deep feature channels so that the shallow feature maps with high resolution and abundant image details can utilize the deep features. Thus, the DFPN can learn hierarchical deep detailed features. The designed deep learning model was trained end to end for road marking instance extraction with mobile laser scanning (MLS) point clouds. Then, we introduced the focal loss function into the optimization of deep learning model in road marking segmentation part, to pay more attention to the hard-classified samples with a large extent of background. In the experiments, our method can achieve better results than state-of-the-art methods on instance segmentation of road markings, which illustrated the advantage of the proposed method.

This study presents a new end-to-end change detection network, called difference-enhancement dense-attention convolutional neural network (DDCNN), that is designed for detection of changes in the bitemporal optical remote sensing images. To model the internal correlation between high-level and low-level features, a dense attention method consisting of several up-sampling attention units is proposed. Both the up-sampling spatial and up-sampling channel attention are adopted by the unit. The unit, which can use high-level features with rich category information to guide the selection of low-level features, can use the spatial context information to capture the changed features of ground objects. Furthermore, DDCNN also pays attention to the differentiating features of the bitemporal images. By introducing a DE unit, each pixel is weighted and the features are selectively aggregated. The combination of dense attention and the DE unit improves the effectiveness of the network and its accuracy in extracting the change features. The effectiveness of the proposed approach is demonstrated via five challenge data sets. The experimental results show that DDCNN achieves new state-of-the-art change detection performance on these five challenging data sets. For the seasonal change detection data set in particular, compared with the best existing change detection model, the proposed method increases the F1 score and IoU by 2.96% and 5.17%, respectively; compared with the baseline method, our method improved 3.75% and 6.50% on the F1 score and IoU, respectively.

This article is focused on a challenging topic emerging from the registration of point clouds, specifically the registration of dynamic objects with low overlapping ratio. This problem is especially difficult when the static scanner is installed on a floating platform, and the objects it scans are also floating. These issues make most of the automatic registration methods and software solutions invalid. To solve this problem, explicit exploration of the static region is necessary for both the coarse and fine registration steps. Fortunately, determining the corresponding regions can be eased by the intuitive realization that in urban environments, natural objects neither present straight boundaries nor stack vertically. This intuition has guided the authors to develop a robust approach for the detection of static regions using planar structures. Then, silhouettes of the objects are extracted from the planar structures, which assist in the determination of an SE(2) transformation in the horizontal direction by a novel line matching method. The silhouettes also enable identification of the correspondences of planes in the step of fine registration using a variant of the iterative closest point method. Experimental evaluations using point clouds of cargo ships with different sizes and shapes reveal the robustness and efficiency of the proposed method, which gives 100% success and reasonable accuracy in rapid time, suitable for an online system. In addition, the proposed method is evaluated systematically with regard to several practical situations caused by the floating platform, and it demonstrates good robustness to limited scanning time and noise.

Classification of airborne laser scanning (ALS) point clouds is needed in digital cities and 3-D modeling. To efficiently recognize objects in ALS point clouds, we propose a novel hierarchical aggregated deep feature representation method, which can adequately employ spatial association of multilevel structures and deep feature discrimination. In our method, a 3-D deep learning model is constructed to represent the discriminative feature of each point cluster in a hierarchical structure by decreasing the within-class distance and increasing the between-class distance. Our method aggregates the discriminative deep features in different levels into a hierarchical aggregated deep feature that considers the spatial hierarchy and feature distinctiveness. Lastly, we build a multichannel 1-D convolutional neural network to classify the unknown points. Our tests demonstrate that the proposed hierarchical aggregated deep feature method can enhance point cloud classification results. Comparing with seven state-of-the-art methods, those results also verified the superior performance of our method.

In the maintenance work of tunnels, images are often used to detect diseases, but collections of tunnel images are limited by the tunnel environment and working time. Three-dimensional laser scanning technology can acquire high-precision tunnel information efficiently, and the main problem to be solved by using this technology to collect tunnel inner wall images is the dimensionality reduction of the laser tunnel point cloud data. This letter proposes a tunnel surface parameterization algorithm based on a harmonic map, where a 3-D tunnel point cloud is used as a data source to reconstruct a triangle mesh model of the tunnel and then generate a harmonic map depth map of the tunnel inner wall on the triangle mesh. We can obtain the spatial distribution and position information of the appendages and detect whether there are cracks, water leakage, falling pieces, and other diseases by the depth images. The results of this study indicate that the proposed algorithm is suitable for tunnels of various shapes and has low area distortion, which can better avoid the loss of information during dimensionality reduction. Compared with other existing methods, the algorithm has higher efficiency and applicability.

Due to errors in sensors and positioning, there exist mismatches between different phases of mobile laser scanning point clouds, which impede the application of point cloud, such as changing detection and deformation monitoring. To rectify such mismatches, we designed a 3-D deep feature construction method for point cloud registration. The proposed method combines two 3-D convolutional neural networks into a uniform deep learning model to extract 3-D deep features. First, the corresponding points and noncorresponding points are set to train the deep learning model to minimize the distance between corresponding points’ features and maximize the distance between features of noncorresponding points. Second, in the test phase, the 3-D deep feature for each keypoint was extracted by the trained deep learning model. This could be used to determine the corresponding points by the$k$-dimensional tree and random sample consensus (RANSAC) algorithm. Finally, a transformation matrix was calculated based on the corresponding points and was then applied to point cloud registration. The experimental results illustrated that the proposed method of using 3-D deep features is more efficient at a corresponding point search than representatives of three existing methods. It also improved registration accuracy.

To efficiently recognize on-ground objects in airborne laser scanning (ALS) point clouds, we design a method that jointly learns a discriminative dictionary and a classifier. In the method, the point cloud is segmented into hierarchical point clusters, which are organized by a tree structure. Then, the feature of each point cluster is extracted. The feature of a leaf node is obtained by aggregating the features of all its parent nodes. The feature of the leaf node is called the hierarchical aggregation feature. The hierarchical aggregation features are encoded by sparse coding. We introduce a new label consistency constraint called “discriminative sparse-code error,” and combine it with the reconstruction error, the classification error, and L1-norm sparsity constraint to form a unified objective function. The objective function is efficiently solved by using the proposed label consistency feature sign method. We obtain an overcomplete discriminative dictionary and an optimal linear classifier. Experiments performed on different ALS point cloud scenes have shown that the hierarchical aggregation features combined with the learned classifier can significantly enhance the classification results, and also demonstrated the superior performance of our method over other techniques in point cloud classification.

This paper aims to extend the generalized split-window (GSW) algorithm in land surface temperature (LST) retrieval to atmospheres with air temperature inversion (ATI) near the Earth surface boundary. Simulation analysis shows that the influence of ATI on the LST retrieval of the GSW algorithm becomes larger when the ATI intensity increases. To further analyze the influence, all ATI atmospheric profiles are extracted from the Thermodynamic Initial Guess Retrieval (TIGR) cloud-free database. Combining the ATI atmospheric profiles and the GSW coefficients, we find that the LST retrieval error caused by ATI is larger than 0.3 K. To reduce the LST retrieval error associated with the ATI in the GSW algorithm, a quadratic equation as a function of ATI intensity is proposed. To validate the proposed method, some in situ measurements observed at the Hailar site are used. The results show that the proposed method could improve the LST retrieval accuracy by 0.47 K for atmospheres under ATI conditions.

This paper attempts to retrieve leaf water content (LWC) by developing new vegetation indices from the combination of the near-infrared (NIR) and middle-infrared (MIR) spectral reflectances. The expanded vegetation leaf model PROSPECT-VISIR and the widely validated four-stream scattering by arbitrarily inclined leaves (4SAIL) model are employed to simulate canopy reflectance in 0.4–5.7 μm region with various leaf water content scenarios. Change of standard deviation of the canopy reflectance with respect to wavelength is used to analyze the sensitive of the spectral reflectance to the LWC. The results show that the spectral reflectances at 1.405μm, 1.875μm, 2.015μm, and 4.375μm are most sensitive to the change of LWC, and the difference vegetation index (DVI) combined by spectral reflectances in 1.405μm and 4.375μm is the best index to retrieve LWC with root mean square error (RMSE) of 0.0008 g/cm2.

Three-dimensional laser scanning can acquire point cloud data with high spatial resolution. However, for practical applications, such as point cloud fitting and 3-D reconstruction, there is usually significant data redundancy, which reduces the operational efficiency. In this letter, we propose a fast point cloud fitting algorithm that uses point cloud simplification to preserve feature boundaries and threshold-independent Bayesian sampling consensus (BaySAC) to fit planar features. We first extract the point features, such as corner points and contour points, using a smoothing analysis of the vicinities of scattered points and an angle analysis of vectors based on search points and their adjacent points. Then, keeping all the feature points, we thin the nonfeature points by constructing a cube grid. Finally, based on the least median squares and the BaySAC algorithm, we propose a robust nonthreshold-dependent method to perform the rapid fitting of planar features in the point cloud after thinning. We used three sets of point cloud data acquired using a 3-D laser scanner to verify the accuracy and efficiency of the planar feature fitting method. The experimental results indicate that the method can extract finer planar features and has significantly better accuracy and computational efficiency than the classical random sample consensus algorithm for the fitting of planar features without using a threshold.

This paper is intended to investigate the sensing of surface parameters by microwave radiometry. A Extended Kalman filter(EKF) is developed to manage the nonlinear relationship between surface parameters and radiometric signatures. Its performance of retrieving plant water content (PWC) and soil moisture content (SMC) from brightness temperatures is examined by using both predictions from model simulations and measurements from field experiments. It calculates background error covariance matrix using EKF method and is able to resolve the nonlinearity and discontinuity exist within model operator and observation operator. We optimize the observing scheme for sensing surface soil moisture (SM) from simulated brightness temperatures by the EKF. The frequencies of interest include 6.9 and 10.7 GHz of the Advanced Microwave Scanning Radiometer (AMSR), and 1.4 GHz (L-band) of the Soil Moisture and Ocean Salinity (SMOS) sensor. The Land Surface Process/Radiobrightness (LSP/R) model is used to provide time series of both SM and brightness temperatures at 6.9 and 10.7 GHz for AMSR's viewing angle of 55 degrees, and at L-band for SMOS's multiple viewing angles of 0, 10, 20, 30, 40, and 50 degrees. These multiple frequencies and viewing angles allow us to design a variety of observation modes to examine their sensitivity to SM. For example, L-band brightness temperature at any single look angle is regarded as an L-band 1D observation mode. Meanwhile, it can be combined with either the observation at other angles to become an L-band 2D mode or a multiple dimensional observation mode, or with the observation at 6.9 or 10.7 GHz to become a multiple frequency/dimensional observation mode. In this study, it is shown that the L-band 1D radiometric observation is sensitive to SM. The sensitivity can be increased by incorporating radiometric observation either from a second angle, or from multiple look angles, or from any of the two lowest AMSR channels. In addition, the advantage of an L-band 2D mode or a multiple dimensional observation mode over an L-band 1D observation mode is demonstrated. Moreover, we investigate the best observing configuration for sensing plant water content (PWC) and soil moisture content (SMC) profiles from the measured H- and V-polarized brightness temperatures at 1.4 (L-band) by the EKF. The brightness temperatures were taken by the ESTAR radiometer in SGP97 experiment. The radiometer was used to measure brightness temperatures at incident angles from −45 to 45 degrees at L-band. The SMC profiles were measured to the depths of 10 cm. The VWC was computed by Normalized Difference Vegetation Index (NDVI) values for the entire region using the TM data collected on July 2 5, 1997. The EKF was trained with observations randomly chosen from the ESTAR data of SGP97, and evaluated by the remaining data from the same set. The results indicate that the EKF can significantly improve the soil moisture estimation in the surface Iayer. And we think that the Extended Kalman filter is both practical and effective for assimilating in situ observation into land surface models.

With the fast development of Internet technology, human society is moving into an information age. Digital city, as well as digital earth, has become a new symbol of the progress of society, and at the meanwhile, in order to represent the real world, as we know, GIS has been proved to be the most effective technical platform. Traditional GIS map service generally provides some simple spatial data analysis, such as the enquiry of the location, the shortest path analysis, analysis of the buffer zone, and so on. People online ask for more services with a growing demand nowadays, so these services can no longer meet the people's needs, not only the shortest path between the two places what the computer is giving, they would also like to know more detail about information of this strange street, such as the special features along the street, so that they can have a direct sense of the street. In order to meet people's needs, and promote the development of GIS map service, the author studies on this area. As we know, there are so many ways to design a WebGIS platform, and this paper chooses a popular one. Based on the technology of asp.net, we can use the ArcIMS to distribute our maps, which is a product of the ESRI, and the activeX to distribute our video, which has been edited. Generally, when you establish a geographic information system, the big problem is the acquisition of data.

The main objective of this research is to analyses the accuracy and calibrate the sensors to develop a mobile mapping system for automatic surveying of the 3D objects. This system has become quite popular in recent years due to it's capability of providing information directly in three dimension. In our system, we have use laser scanners as the main data acquisition device, supplemented by line CCD cameras for texture information and as usual combination of GPS, INS and odometer for position and attitude information. Because every sensor and device has it's own local coordinate system. For example, GPS output is based on WGS84 coordinates system, Laser data is based on it's own local coordinate system, the origin of which lies at the laser scanning head and so on for other sensors and device. The major problem is to identify the spatial position of the objects scanned by the laser at any time while the vehicle is moving with reference to a common coordinate system. It involves the integration of all the sensors and devices to a common coordinate system, which is the local mapping coordinate system. The integration process mainly involves the computation of fixed rotation and shift vectors between the INS body and sensors. As the GPS and INS are physically located in two different places, we also need to know the shift vector between the GPS and INS.

Five spacecrafts named in the Shenzhou series (SZ in abbreviation) have been launched during 1999 to 2003 in China. Earth observation is a major scientific mission of the SZ spacecrafts. SZ-3 carried a 34-band medium resolution spectrometer, and SZ-4 was equipped with multimode microwave sensors composing a microwave scatterometer, radiometer, and radar altimeter. Our research group made "spacecraft-aircraft-ground" synchronous measurements. This paper presents the review of the mission of SZ spacecrafts, and gives the results of data processing and environment and geoscience applications.

The multiple model microwave remote sensor is an important part of the payload carried by Shenzhou IV spacecraft launched on 30 December 2002 and is also China's first experimental microwave remote sensing system operating in space. The system includes three kinds of microwave remote sensors, namely microwave radiometer, radar altimeter and radar scatterometer. The radiometer instrument provides global passive microwave measurements of terrestrial, oceanic, and atmospheric variables for the investigation of water and energy cycles. Here we discussed the retrieval of land surface parameters by using the Shenzhou IV brightness data aimed at evaluating the performance of this microwave radiometer. The results shows that the microwave radiometer observations of Shenzhou IV can be used to study seasonal and interannual changes of land surface parameters. Although compared with other satellite passive microwave sensor, the radiometer of Shenzhou IV has it's limitation.