The effectiveness of transportation systems is significantly influenced by the field of view (FOV) of cameras, with larger FOVs yielding more reliable performance. Conventional cameras with restricted FOV face difficulties in scenarios characterized by field-of-view shake or low-texture imaging areas, complicating feature extraction and matching tracking. Conversely, 360° panoramic cameras, which provide an expansive field of view and high pixel density, are emerging as promising alternatives. While many studies have explored motion calculation based on panoramic vision, most resort to conventional image processing methods for panoramic images, leading to inefficiencies and reduced accuracy. This article proposes a novel approach to panoramic vision motion estimation and reconstruction. The method involves projecting the panoramic image onto a fitting sphere and establishing a mapping relationship between a uniform spherical grid and the pixels of the panoramic image. This enables distortion-free mapping imaging for the entire panorama, addressing issues with matching consecutive frame images caused by distortion in the polar regions. Going beyond conventional epipolar constraints, this paper introduces a geometric constraint of epipolarity within a three-dimensional sphere and derives pose-solving equations tailored for panoramic vision, enabling motion estimation between frames. Additionally, it derives the polar arc equation of panoramic images, accelerating three-dimensional reconstruction of dense point clouds. The sliding window estimation method is used for iterative algorithm optimization, along with an optimization mechanism aligned with the panoramic camera model. Validation on public datasets and hardware experimental platforms demonstrates the proposed method’s significantly improved accuracy compared to existing optimal algorithms.

Light detection and ranging (LiDAR) positioning has received great attention especially when satellites fail. However, the positioning accuracy is still subjected to the following. First, the LiDAR positioning accuracy is affected by the sparsity due to LiDAR beams and environment. Second, the existing methods for suppressing cumulative errors are based on ego-vehicle sensing that requires the motion of repeated paths. Moreover, the output frequency of LiDAR is low. To solve the above problems, an enhanced-LiDAR/ultrawideband (UWB)/inertial navigation system integrated positioning methodology is proposed. First, the enhanced LiDAR odometry module is designed to improve the resolution of LiDAR beams for more accurate odometry. Then, the cooperative optimization module is proposed to introduce UWB observation to suppress the accumulated error without relying on ego-vehicle sensing. Finally, the factor graph fusion module is used to fuse multisensor information dynamically and improve the output frequency. Experimental results prove the effectiveness of our methodology.

Place recognition plays a crucial role in simultaneous localization and mapping. Unfortunately, however, changes in viewpoints and conditions in large-scale environments impose tricky challenges for PR. To this end, this article specifically proposes an explicit points-of-interest driven PR method, which consists of a road segmentation module based on grid-wise patch U-transformer and a PR module based on regions of interest siamese transformer NetVLAD (RI_STV). Especially for RI_STV, in the individual dimension, it is dedicated to exploring the local topological features of nonroad regions of interest. In the spatial dimension, an improved Transformer is introduced to capture the global interactions between features of interest. In the cluster dimension, NetVLAD embedded with weighted pooling is created to perform weighted aggregation of feature clusters to generate discriminative and general descriptors. Evaluation on various datasets shows that our customized method is not only impressively competitive, but also strikes the best balance between accuracy and real-time performance.

Pedestrian trajectory prediction in dynamic scenes remains a challenging and critical problem in numerous applications, such as self-driving cars and socially aware robots. Challenges concentrate on capturing pedestrians' motion patterns and social interactions, as well as handling the future uncertainties. Recent studies focus on modeling pedestrians' motion patterns with recurrent neural networks, capturing social interactions with pooling- or graph-based methods, and handling future uncertainties by using the random Gaussian noise as the latent variable. However, they do not integrate specific obstacle avoidance experiences (OAEs) that may improve prediction performance. For example, pedestrians' future trajectories are always influenced by others in front. Here, we propose the Graph-based Trajectory Predictor with Pseudo-Oracle (GTPPO), an encoder-decoder-based method conditioned on pedestrians' future behaviors. Pedestrians' motion patterns are encoded with a long short-term memory unit, which introduces temporal attention to highlight specific time steps. Their interactions are captured by a graph-based attention mechanism, which draws OAE into the data-driven learning process of graph attention. Future uncertainties are handled by generating multimodal outputs with an informative latent variable. Such a variable is generated by a novel pseudo-oracle predictor, which minimizes the knowledge gap between historical and ground-truth trajectories. Finally, the GTPPO is evaluated on ETH, UCY, and Stanford Drone datasets, and the results demonstrate state-of-the-art performance. Besides, the qualitative evaluations show successful cases of handling sudden motion changes in the future. Such findings indicate that GTPPO can peek into the future.

This article proposes a novel visual measurement framework, multimodule cascaded deep neural network (MMC-DNN), to achieve accurate, reliable, and cost-effective vehicle positioning in complex urban environments. The MMC-DNN is inspired by the mechanism of the human eyes' lateral positioning, which consists of three modules called siamesed fully convolutional network (S-FCN), skip-connection fully convolutional autoencoder (SC-FCAE), and multitask neural network regressor (MT-NNR), respectively. The S-FCN is first designed to accurately detect the road area. Then, the segmented road was executed inverse perspective mapping and the result is fed to the developed SC-FCAE for extracting equivalent positioning features. Furthermore, the MT-NNR is proposed to efficiently estimate lateral position and yaw angle with the help of a road map. Based on the estimation results, the MEMS INS/GPS integration is significantly augmented by extended Kalman filter. Experimental results validate the effectiveness of the proposed framework in enhancing positioning performance.

Reliable estimation of vehicle lateral position plays an essential role in enhancing the safety of autonomous vehicles. However, it remains a challenging problem due to the frequently occurred road occlusion and the unreliability of employed reference objects (e.g., lane markings, curbs, etc.). Most existing works can only solve part of the problem, resulting in unsatisfactory performance. This paper proposes a novel deep inference network (DINet) to estimate vehicle lateral position, which can adequately address the challenges. DINet integrates three deep neural network (DNN)-based components in a human-like manner. A road area detection and occluding object segmentation (RADOOS) model focuses on detecting road areas and segmenting occluding objects on the road. A road area reconstruction (RAR) model tries to reconstruct the corrupted road area to a complete one as realistic as possible, by inferring missing road regions conditioned on the occluding objects segmented before. A lateral position estimator (LPE) model estimates the position from the reconstructed road area. To verify the effectiveness of DINet, road-test experiments were carried out in the scenarios with different degrees of occlusion. The experimental results demonstrate that DINet can obtain reliable and accurate (centimeter-level) lateral position even in severe road occlusion.