Trajectory prediction is crucial for autonomous vehicles to make safe and informed decisions. However, the lack of transparency in current trajectory prediction models introduces significant security risks, because their output contains almost no explanatory details. To address these challenges and bridge this research gap, we propose a novel approach IGTPT that not only predicts vehicle trajectories but also generates textual descriptions of the vehicle’s intent. The vehicle’s intent can be broadcast to nearby vehicles to enhance decision-making transparency and increase user trust. Our approach directly confronts the opaque nature of existing systems by providing clear, understandable explanations for autonomous decisions, thereby enhancing both the security and reliability of these systems. We enhanced the BDD-X dataset to create the BDD-XE, a specialized dataset for trajectory prediction and behavior description, which our framework uses to achieve superior results in both prediction accuracy and behavioral interpretation compared to established baseline methods. We demonstrate the practical applicability of our framework through a complete system that processes past raw driving videos and trajectory observations to deliver real-time predictions along with insightful behavioral narrations and reasoning.

Low-light image enhancement restores the colors and details of a single image and improves high-level visual tasks. However, restoring the lost details in the dark area is still a challenge relying only on the RGB domain. In this paper, we delve into frequency as a new clue into the model and propose a DCT-driven enhancement transformer (DEFormer) framework. First, we propose a learnable frequency branch (LFB) for frequency enhancement contains DCT processing and curvature-based frequency enhancement (CFE) to represent frequency features. Additionally, we propose a cross domain fusion (CDF) to reduce the differences between the RGB domain and the frequency domain. Our DEFormer has achieved superior results on the LOL and MIT-Adobe FiveK datasets, improving the dark detection performance.

Cameras and LiDARs are fundamental sensors for autonomous driving, with wide applicability. This paper comprehensively studies and analyzes the calibration of the multi-terminal camera-LiDAR devices from the perspectives of vehicle, roadside, and road cooperation, and outlines its related applications and far-reaching significance. Different from previous studies that focused on sensor calibration of a single platform, ignoring the differences between vehicle-end and roadside, this paper particularly emphasizes the difference between vehicle-end and roadside, as well as heterogeneous calibration challenges emerging in vehicle-road cooperation scenarios. These challenges include cross-sensor temporal synchronization and spatial alignment in dynamic environments. It reveals the advantages and differences of different methods in dealing with constraints such as the fixed installation postures of vehicle-mounted and roadside devices and the large-scale observation domain. This paper further points out key research gaps, such as the adaptability of online calibration and the maintenance of calibration consistency among multiagents, providing important references for the technological evolution from isolated calibration to a sensor-collaborative calibration framework in traffic scenarios.

In recent years, methods leveraging global context modeling for continuous road extraction from remote sensing images have garnered significant attention. These methods can effectively address the issues of road discontinuity and missed detection caused by complex background interference, but they generally suffer from high computational complexity. Therefore, we propose CGCNet, a simple yet effective road extraction network based on global context modeling. CGCNet adopts an encoder–decoder architecture with a compact global context-aware block (CGCB) embedded in the center part to capture long-range dependencies among road segments. This block effectively enhances the model’s global modeling capability and significantly reduces computational complexity using a compact representation of the embedded Gaussian nonlocal block (NLB). Furthermore, we introduce SW-XJU, the first remote sensing road dataset constructed explicitly for the unique landscapes of Western China. Extensive experiments show that CGCNet achieves a superior balance between efficiency and accuracy compared with the state-of-the-art methods, and the extracted roads exhibit stronger connectivity. The source code and dataset are available athttps://github.com/LPeng625/CGCNet.

In recent years, there has been a notable increase in interest in image-based surface normal estimation. These approaches are capable of predicting the surface normal of real scenes using only an image, thereby facilitating a more profound comprehension of the actual scene and providing assistance with other perceptual tasks. However, dense regression predictions are susceptible to misdirection when encountering intricate details, which presents a paradoxical challenge for image-based surface normal estimation in reconciling detail and density. By introducing quaternion rotations as fusion module with geometric property, we propose a quaternion-based refined network structure that fuses detailed and structural information. Specifically, we design a high-resolution surface normal baseline with a streamlined structure, to extract fine-grained features while reducing the angular error in surface normal regression values caused by downsampling. Additionally, we propose a subtle angle loss function that prevents subtle changes from being overlooked without extra information, further enhancing the model's ability to learn detailed information. The proposed method demonstrates state-of-the-art performance compared to existing techniques on three real-world datasets comprising indoor and outdoor scenes. The results demonstrate the robust effectiveness of our deep learning approach that incorporates geometric prior guidance, highlighting improved robustness in applying deep learning methods. The source code will be released upon acceptance.

While the diffusion transformer (DiT) has become a focal point of interest in recent years, its application in low-light image enhancement remains a blank area for exploration. Current methods recover the details from low-light images while inevitably amplifying the noise in images, resulting in poor visual quality. In this paper, we firstly introduce DiT into the low-light enhancement task and design a novel Structure-guided Diffusion Transformer based Low-light image enhancement (SDTL) framework. We compress the feature through wavelet transform to improve the inference efficiency of the model and capture the multi-directional frequency band. Then we propose a Structure Enhancement Module (SEM) that uses structural prior to enhance the texture and leverages an adaptive fusion strategy to achieve more accurate enhancement effect. In Addition, we propose a Structure-guided Attention Block (SAB) to pay more attention to texture-riched tokens and avoid interference from noisy areas in noise prediction. Extensive qualitative and quantitative experiments demonstrate that our method achieves SOTA performance on several popular datasets, validating the effectiveness of SDTL in improving image quality and the potential of DiT in low-light enhancement tasks.

Coarse-to-fine schemes are widely used in traditional single-image motion deblur; however, in the context of deep learning, existing multi-scale algorithms not only require the use of complex modules for feature fusion of low-scale RGB images and deep semantics, but also manually generate low-resolution pairs of images that do not have sufficient confidence. In this work, we propose a multi-scale network based on single-input and multiple-outputs(SIMO) for motion deblurring. This simplifies the complexity of algorithms based on a coarse-to-fine scheme. To alleviate restoration defects impacting detail information brought about by using a multi-scale architecture, we combine the characteristics of real-world blurring trajectories with a learnable wavelet transform module to focus on the directional continuity and frequency features of the step-by-step transitions between blurred images to sharp images. In conclusion, we propose a multi-scale network with a learnable discrete wavelet transform (MLWNet), which exhibits state-of-the-art performance on multiple real-world deblurred datasets, in terms of both subjective and objective quality as well as computational efficiency. Our code is available on https://github.com/thqiu0419/MLWNet.

Driving behavior classification is an important direction in the field of social transportation systems and advanced driving assistance system (ADAS), which has attracted more and more attention in recent years. An accurate driving behavior classification algorithm plays a great role in traffic safety, energy saving, and other fields. In this article, we propose a novel vehicle speed and image fusion for driving behavior classification network (SIFDriveNet), which classifies driver behaviors into normal driving, aggressive driving, and drowsy driving. Our method has the following key advantages. First, in the research of driving behavior classification, we are the first to introduce a 2-D image with rich roadside information and convert speeds into a 2-D spectrogram expressing the time–frequency characteristics of speeds through short-time Fourier transform (STFT) while unifying the data space of image information and speed information. Second, we propose a tensor fusion method based on weight decomposition to fully fuse the vectors of the two modalities. This method maps the tensor outer product results to the low-dimensional space through weight decomposition and has a low computational cost while maintaining the fusion effect of the tensor outer product. In addition, we evaluated our model on the public UAH-DriveSet and compared it with the most advanced model. Experimental results show that our model has a better performance, and F1-score is 97.9% on all roads. Especially on the secondary road, our F1-score is 99.4%. Also, our model has strong generalization, and we have reached 99.3% F1 in distracted driving multimodal dataset. In addition, the inference speed reaches 411 FPS, enabling real-time needs. The code is available onhttps://github.com/alu222/SIFDriveNet.

Noise has always been nonnegligible trouble in object detection by creating confusion in model reasoning, thereby reducing the informativeness of the data. It can lead to inaccurate recognition due to the shift in the observed pattern, that requires a robust generalization of the models. To implement a general vision model, we need to develop deep learning models that can adaptively select valid information from multimodal data. This is mainly based on two reasons. Multimodal learning can break through the inherent defects of single-modal data, and adaptive information selection can reduce chaos in multimodal data. To tackle this problem, we propose a universal uncertainty-aware multimodal fusion model. It adopts a multipipeline loosely coupled architecture to combine the features and results from point clouds and images. To quantify the correlation in multimodal information, we model the uncertainty, as the inverse of data information, in different modalities and embed it in the bounding box generation. In this way, our model reduces the randomness in fusion and generates reliable output. Moreover, we conducted a completed investigation on the KITTI 2-D object detection dataset and its derived dirty data. Our fusion model is proven to resist severe noise interference like Gaussian, motion blur, and frost, with only slight degradation. The experiment results demonstrate the benefits of our adaptive fusion. Our analysis on the robustness of multimodal fusion will provide further insights for future research.