Efficiently updating real-time road maps is essential for autonomous driving and significantly influences human driving decisions. Existing high-definition (HD) maps are often based on static data and neglect real-time road information, particularly temporary traffic control details related to construction. In this study, we propose a method to identify and update lane-level information for roads in restricted areas using only a digital video recorder and a low-cost global navigation satellite system (GNSS). This research proposes a Road Temporary Traffic Control Network (RTTCNet) to detect control devices and lane markings and to optimize a 3D reconstruction method that can accurately position control devices on HD maps. The method identifies areas with restricted road access and analyzes the spatial relationship between control devices and lane lines to update the HD map. Our method effectively addresses the high costs and inefficiencies associated with conventional map-updating methods. Experimental tests on various road types in Shanghai demonstrated 94.85% accuracy in identifying restricted areas for road access from single images, with a mean positioning error of 3.86 meters for temporary traffic control devices on HD maps. This method enables real-time lane-level updates for road information on HD maps and more effective decision-making support for autonomous driving.

Accurate three-dimensional (3D) object detection in maritime environments is critical for autonomous navigation. However, it remains challenging because of sparse point clouds, complex vertical structures, and extreme object scale variations. Existing 3D detectors are primarily designed for road scenes and often perform poorly in such conditions. Therefore, we propose a Hierarchical Vertical-aware and Adaptive Multi-scale Network (HVAM-Net), an anchor-free, single-stage deep learning framework tailored for maritime scenarios. HVAM-Net integrates three core modules: (1) a Hierarchical Pillar Encoding module that enhances vertical representation via exponential stratification and semantic-aware fusion; (2) an Adaptive Multi-scale Feature Extraction module that captures diverse spatial contexts via parallel atrous convolutions and attention-guided fusion; and (3) an Attention-Guided Dynamic Sampling module that refines upsampling by learning adaptive spatial offsets, enhancing semantic consistency in sparse regions. The effectiveness of HVAM-Net is validated through comprehensive comparisons with state-of-the-art 3D object detection methods. Experiments show that HVAM-Net achieves mean Average Precision scores of 86.7 %, 78 %, and 88 % on the self-collected, Thames River vessel, and simulated datasets, respectively, outperforming all baseline methods. Moreover, its resilience under adverse weather conditions and varying light detection and ranging configurations further confirms the strong generalization capability of this artificial intelligence-based approach in real-world maritime environments.

Nighttime light (NTL) data provides an excellent opportunity for continuous spatiotemporal monitoring of global urbanization. However, in the two extensively employed NTL datasets (DMSP/OLS and NPP/VIIRS), there were also problems such as spatiotemporal inconsistencies, different spatial resolution, and inconsistent brightness with that of light-sensitive ground objects, which limited the application of NTL data. To address this issue, we proposed a framework integrating super-resolution reconstruction model and ground-object feature constraints algorithm, and generated a consistent global NTL dataset (1992–2023, 500 m), namely Tongji-NTL. First, the NTL data quality enhancement operation was performed to enhance the accuracy of NTL data. Second, a super-resolution reconstruction model was developed to convert the DMSP/OLS data (1 km) into the NPP-like data (500 m). Subsequently, a novel ground-object feature constraint algorithm was constructed to enhance the interpretability of the NTL data. The assessment showed that the proposed super-resolution model achieved superior performance. And the NTL data constrained by ground-objects feature were more sensitive to road network and water. To evaluate the performance of Tongji-NTL, we conducted evaluations and results showed that: 1) Tongji-NTL has maintained a high consistency, and the overall situation has exhibited a steady upward trend. 2) The average correlation coefficients of Tongji-NTL with four statistical indicators were significantly higher compared to original data. 3) Tongji-NTL exhibits the most extensive temporal coverage, widest spatial extent and highest level of spatial resolution by comparing with other nine NTL products. Tongji-NTL is expected to make outstanding contributions to global urbanization monitoring in the future.

Distributed machine learning has emerged as a promising data processing technology for next-generation communication systems. It leverages the computational capabilities of local nodes to efficiently handle large datasets, creating highly accurate data-driven models for analysis and prediction purposes. However, the performance of distributed machine learning can be significantly hampered by communication bottlenecks and node dropouts. In this article, a novel unmanned aerial vehicle (UAV)-enabled hierarchical distributed learning architecture is proposed to support machine learning applications, e.g., regional monitoring. Multiple UAV receivers (URs) are introduced as wireless relays to improve the communication between the UAV transmitters (UTs) and the cloud server. Our objective is to identify the optimal UT-UR association to maximize the social welfare of the network, which is distinctly different from the existing works that focus on the unilateral profit-maximizing problem. We formulate a two-side many-to-one matching game to model the UT-UR association problem, and a two-phase many-to-one matching algorithm is designed to identify the stable matching. The validity of our proposed scheme is verified through in-depth numerical simulations.

Scene recognition is a fundamental task in 3D scene understanding. It answers the question ‘What is this place?’. In an indoor environment, the answer can be an office, kitchen, lobby, and so on. As the number of point clouds increases, using embedded point information in scene recognition becomes computationally heavy to process. To achieve computational efficiency and accurate classification, our idea is to use indoor scene graph that represents the 3D spatial structures via object instances. The proposed method comprises two parts, namely, (i) construction of indoor scene graphs leveraging object instances and their spatial relationships and (ii) classification of these graphs using a deep learning network. Specifically, each indoor scene is represented by a graph, where each node represents either a structural element (like a ceiling, a wall, or a floor) or a piece of furniture (like a chair or a table) and each edge encodes the spatial relationship between these elements. Then these graphs are used as input for our proposed graph classification network to learn different scene representations. The public indoor dataset, ScanNet v2, with 625.53 million points is selected to test our method. Experiments yield good results with up to 88.00% accuracy and 82.30% F1-score in the fixed validation dataset, and 90.46% accuracy and 81.45% F1-score in 10-fold cross validation method. Moreover, if some indoor objects can’t be successfully identified, the scene classification accuracy depends sub-linearly on the rate of missing objects in the scene.

Different types of roads in complex road networks may run side-by-side or across in 2D or 3D spaces, which causes mismatched segments using existing online map-matching algorithms. A driving scenario that represents the driving environment can inform map-matching algorithms. Images from vehicle cameras contain extensive information about driving scenarios, such as surrounding key objects. This research utilized vehicle images and developed an object-based method to classify driving scenarios (Object-Based Driving-Scenario Classification: OBDSC) to calculate the probabilities of the current image in predefined types of driving scenarios. We implemented an online map-matching algorithm with the OBDSC method (OMM-OBDSC) to obtain optimal matching segments. The algorithm was tested on nine trajectories and OpenStreetMap data in Shanghai and compared with five benchmark algorithms in terms of the match rate, recall and accuracy. The OBDSC method is also applied to the benchmark algorithms to verify the effectiveness of map matching. The results show that our algorithm outperforms the benchmark algorithms with both the original interval and downsampled intervals (96.6%, 96.5%, 93.7% on average with 1–20 s intervals for the three metrics, respectively). The average match rate has improved by 8.9% for all benchmark algorithms after the addition of the OBDSC method.

High definition (HD) maps offer precise positioning and dependable navigation capabilities, which are essential to guaranteeing the safety of autonomous vehicles. Lane-level road network, as a crucial component of HD maps, can provide perception, positioning, local planning, and vehicle control services. In urban scenarios, the effectiveness of using sensor-equipped mapping vehicles to construct HD maps on a large scale is hindered by complex road conditions, heavy traffic flow, and limited sensor measurement range. In this paper, we propose a method for generating a lane-level road network from unmanned aerial vehicle LiDAR data, which is flexible, maneuverable, and not limited by the constraints of road traffic conditions. The proposed method employs a Segformer model to acquire road areas and eliminates pavement interferential objects through a DBSCAN clustering and RANSAC plane fitting algorithm. Subsequently, the PP-LiteSeg model is utilized to extract road symbols from a relatively clean pavement point clouds, and the lane-level road network is generated. We tested our method on the inner ring elevated road section of Yangpu District, Shanghai. The experimental results demonstrate the effectiveness and robustness of our method for generating lane-level road network in high-density urban scene.

Street trees providing ecological and cultural functions may block traffic signs from the driver’s view if they are not properly maintained. Aiming at the problems of low automation and poor refinement of the detection of tree areas blocking traffic signboards, and the inability to make pruning simulation and quantification of the occlude branches, in this paper, the detection of trees to be maintained and the simulation of branch pruning are studied based on mobile laser scanning point clouds. First, the tree areas that block the cantilever traffic signs from the driver’s view are automatically and accurately detected based on line of sight analysis. Then, via structured modeling, the pruning effect of trees under different pruning degrees is simulated in three dimensions, and four indicators (the number of pruning positions, the pruning branch volume, the pruning ratio of branch volume, and the pruning ratio of leaves) are proposed to quantify the pruning. The application in four different scenes proves the effectiveness of the proposed method. This research is helpful to automatically and accurately identify the street tree branches to be maintained, and provides an important reference for the formulation of branch pruning schemes that can comprehensively balance the pruning cost, tree ecological benefits and cultural benefits.

With the rapid development of Deepfake synthesis technology, our information security and personal privacy have been severely threatened in recent years. To achieve a robust Deepfake detection, researchers attempt to exploit the joint spatial-temporal information in the videos, like using recurrent networks and 3D convolutional networks. However, these spatial-temporal models remain room to improve. Another general challenge for spatial-temporal models is that people do not clearly understand what these spatial-temporal models really learn. To address these two challenges, in this paper, we propose an Interpretable Spatial-Temporal Video Transformer (ISTVT), which consists of a novel decomposed spatial-temporal self-attention and a self-subtract mechanism to capture spatial artifacts and temporal inconsistency for robust Deepfake detection. Thanks to this decomposition, we propose to interpret ISTVT by visualizing the discriminative regions for both spatial and temporal dimensions via the relevance (the pixel-wise importance on the input) propagation algorithm. We conduct extensive experiments on large-scale datasets, including FaceForensics++, FaceShifter, DeeperForensics, Celeb-DF, and DFDC datasets. Our strong performance of intra-dataset and cross-dataset Deepfake detection demonstrates the effectiveness and robustness of our method, and our visualization-based interpretability offers people insights into our model.

Point cloud semantic segmentation in urban scenes plays a vital role in intelligent city modeling, autonomous driving, and urban planning. Point cloud semantic segmentation based on deep learning methods has achieved significant improvement. However, it is also challenging for accurate semantic segmentation in large scenes due to complex elements, variety of scene classes, occlusions, and noise. Besides, most methods need to split the original point cloud into multiple blocks before processing and cannot directly deal with the point clouds on a large scale. We propose a novel context-aware network (CAN) that can directly deal with large-scale point clouds. In the proposed network, a Local Feature Aggregation Module (LFAM) is designed to preserve rich geometric details in the raw point cloud and reduce the information loss during feature extraction. Then, in combination with a Global Context Aggregation Module (GCAM), capture long-range dependencies to enhance the network feature representation and suppress the noise. Finally, a Context-Aware Upsampling Module (CAUM) is embedded into the proposed network to capture the global perception from a broad perspective. The ensemble of low-level and high-level features facilitates the effectiveness and efficiency of 3D point cloud feature refinement. Comprehensive experiments were carried out on three large-scale point cloud datasets in both outdoor and indoor environments to evaluate the performance of the proposed network. The results show that the proposed method outperformed the state-of-the-art representative semantic segmentation networks, and the overall accuracy (OA) of Tongji-3D, Semantic3D, and S3DIS is 96.01%, 95.0%, and 88.55%, respectively.

Vehicle and pedestrian detection is one of the critical tasks in autonomous driving. Since heterogeneous techniques have been proposed, the selection of a detection system with an appropriate balance among detection accuracy, speed and memory consumption for a specific task has become very challenging. To deal with this issue and to provide guidance for model selection, this paper analyzes several mainstream object detection architectures, including Faster R-CNN, R-FCN, and SSD, along with several typical feature extractors, such as ResNet50, ResNet101, MobileNet_V1, MobileNet_V2, Inception_V2 and Inception_ResNet_V2. By conducting extensive experiments using the KITTI benchmark, which is a commonly used street dataset, we demonstrate that Faster R-CNN ResNet50 obtains the best average precision (AP) (58%) for vehicle and pedestrian detection, with a speed of 8.6 FPS. Faster R-CNN Inception_V2 performs best for detecting cars and detecting pedestrians respectively (74.5% and 47.3%). ResNet101 consumes the highest memory (9907 MB) and has the largest number of parameters (64.42 millions), and Inception_ResNet_V2 is the slowest model (3.05 FPS). SSD MobileNet_V2 is the fastest model (70 FPS), and SSD MobileNet_V1 is the lightest model in terms of memory usage (875 MB), both of which are suitable for applications on mobile and embedded devices.

A band-weighted support vector machine (BWSVM) method is proposed to classify hyperspectral imagery (HSI). The BWSVM presents an L1penalty term of band weight vector to regularize the regular SVM model. The L1norm regularization term guarantees the sparsity of band weights and describes potentially divergent contributions from different bands in modeling the binary SVM model. The BWSVM adopts the KerNel iterative feature extraction algorithm to minimize the nonconvex program. It linearizes nonlinear kernels and iteratively optimizes two convex subproblems with respect to both sample coefficients and band weights. The class label is determined by picking the largest sample coefficients from all its binary models of BWSVM. Two popular HSI data sets are utilized to testify the classification performance of BWSVM. Experimental results show that the BWSVM outperforms three state-of-the-art classifiers including SVM, random forest, and k-nearest neighbor.