UAVs have a broad application prospect in the field of railway inspection due to their excellent mobility and flexibility. However, it still faces challenges, such as high human labor costs and low intelligence levels. Therefore, it is of great significance to develop a real-time intelligent rail recognition algorithm that can be deployed on the onboard computing device to guide the UAV’s camera to follow the target rail area and complete the inspection automatically. However, a significant issue is that rails from the perspective of UAVs may appear with changing pixel widths and various inclination angles. Concerning the issue, a general and adaptive rail representation method based on projection length discrimination (RRM-PLD) is proposed. It can always select the optimal representation direction, horizontal or vertical, to represent any kind of rails. With the RRM-PLD, a novel architecture (Real-Time Rail Recognition Network, TriRNet) is proposed. In TriRNet, a designed inter-rail attention (IRA) mechanism is presented to fuse local features of single rails and global features of other rails to accurately discriminate the geometric distribution of all rails in the image in a regressive way and thus improve the final recognition accuracy. Further, one-to-one mapping from anchor points to final feature maps is established. It greatly simplifies the model design process and improves the model’s interpretability. Besides, detailed model training strategies are also presented. Extensive experiments have verified the effectiveness and superiority of the proposed formulation in terms of both network reasoning latency and recognition accuracy.

Point cloud semantic segmentation (PCSS) is crucial for digital twins of high-speed railways. By now, the concerned subjects are confined within the interior infrastructures of railways. However, the surrounding environments are also important for the safe operation. Concerning this issue, a full-range high-speed railway scanning scheme based on unmanned-aerial-vehicle-borne LiDAR is utilized. However, the massive data volume and data distribution imbalance pose great challenges for PCSS. To address these issues, a novel PCSS framework called 3DGraphSeg is proposed in this article. To cope with the massive data volume, a structural representation algorithm named local embedding super-point graph is proposed to represent the vast point cloud into a concise graph while retain the data's inherent topology structure by local spatial embedding. Then, the gated integration graph convolutional network (GIGCN) is proposed to contextual segment the graph. In the GIGCN, to prevent the gradients from vanishing or exploding, the hidden states of gated recurrent units in every layer are integrated using a new layer named gated hidden states integration (GHSI). GHSI strengthens the back propagation by giving the loss function direct access to each layer and absorbs the features of different layers comprehensively, which enables the network to produce a smoother decision boundary and prevents the overfitting problem. Besides, to enhance its robustness to data imbalance, we propose a loss function: adaptive weighted cross entropy. Finally, five experiments are designed for verification. The proposed framework has excelled in different datasets and outperforms state-of-the-art approaches on the SemanticRail dataset.

Point cloud semantic segmentation for railway infrastructures is an essential step towards establishing railway digital twins. Deep learning-based methods have shown great potential in this field compared to traditional methods that rely on hand-crafted features. However, deep learning-based methods for railway point clouds still face typical challenges that need to be addressed. In this regard, we propose a novel learning framework named SALAProNet, which consists of a set of effective and concise modular solutions. The first challenge addressed is the massive data scale of railway point clouds, which makes it difficult to directly process large-scale point clouds due to memory limitations. To solve this problem, we adapt efficient random sampling in the network and propose the Self-Attentive Aggregation (SAA) module based on an attention mechanism to greatly expand the receptive field, which covers the unsampled points and successfully retains information in a high-dimensional feature space. The second challenge is fine-grained segmentation, where we propose the Local Geometry Embedding (LGE) module to embed local geometry. With the help of context information provided by SAA, the network can perform fine-grained segmentation for railway infrastructures. The third challenge is the insufficient generalization ability of the network, where we propose a Prototype Guided Regularization (PGR) method to guide the network to segment the point cloud among railways with different construction standards. This method enhances the network’s interpretability and improves its generalization ability. We have validated our proposed framework through experiments on different datasets, and it outperforms state-of-the-art approaches.

The height and stagger of the contact wire directly affect the energy supply of high-speed trains. To ensure the operation safety, there is an urgent demand for high-speed railways to measure the static parameters of contact wires all over the line with high precision and efficiency. However, this issue is barely discussed. Concerning the issue, this paper proposes a UAV-LiDAR-based measuring framework for the static height and stagger of high-speed railway contact wire. By mounting LiDAR on the UAV, the framework can efficiently collect data from the lines in service without occupying the train operating-diagrams. It is extremely significant for the high-speed and high-density railways. Then, we present self-adaptive extraction algorithms to extract critical infrastructures (rails, contact wires, masts and other suspensions) based on their specific geometric characteristics as well as the continuity and consistency of the spatial distributions along the line. Finally, the height and stagger are calculated by formulas automatically. To verify the framework in practice, we tested it on Beijing-Shanghai high-speed railway, which is the busiest high-speed railway in China. It is shown that the measurement error is within 9mm and the framework has potential to reform the inspection of high-speed railways.

UAV-based automatic railway inspection is expected to have the potential to reform the inspection of railways. In this area, real-time railway scene parsing is quite essential. However, the limited computation resources of the UAV onboard computer pose a huge challenge for the algorithm to juggle a precise prediction with strong timeliness. Concerning this issue, this paper proposes a novel algorithm named deep fully decoupled residual convolutional network, which consists of fully decoupled residual blocks (Non-bottleneck-FDs) to deal with the dilemma between the high demand of real-time and limited resources. The residual block is constructed based on a new convolution which divides the standard convolution into three sequential convolutions to decouple the conventional operational correlations fully. Furthermore, a customized auxiliary line loss (LL) function is proposed to constrain the segmentation of railway and non-railway simultaneously without increasing the computation complexity. The proposed LL can force the predicted railway areas to concentrate in long strip areas precisely and inhibit their appearances in other impossible local areas. Subsequently, an integrated loss backpropagation strategy of the LL and cross-entropy function is presented. A comprehensive set of experiments are conducted for verification. Experiments demonstrate the superior performance of our approach with a more than$2\times $reduction in parameters and computation cost. Moreover, our approach also has a faster inference speed than the most existing lightweight architectures while providing comparable or higher accuracy. It is proven that our approach can reconcile the precise prediction with strong timeliness for railway scene parsing within the limitation of onboard computers. Besides, the results also imply its highest performance in terms of local details and edges of railway areas.