We present TUMTraf VideoQA, a novel dataset and benchmark designed for spatio-temporal video understanding in complex roadside traffic scenarios. The dataset comprises 1,000 videos, featuring 85,000 multiple-choice QA pairs, 2,300 object captioning, and 5,700 object grounding annotations, encompassing diverse real-world conditions such as adverse weather and traffic anomalies. By incorporating tuple-based spatio-temporal object expressions, TUMTraf VideoQA unifies three essential tasks—multiple-choice video question answering, referred object captioning, and spatio-temporal object grounding—within a cohesive evaluation framework. We further introduce the TraffiX-Qwen baseline model, enhanced with visual token sampling strategies, providing valuable insights into the challenges of fine-grained spatio-temporal reasoning. Extensive experiments demonstrate the dataset’s complexity, highlight the limitations of existing models, and position TUMTraf VideoQA as a robust foundation for advancing research in intelligent transportation systems. The dataset and benchmark are publicly available to facilitate further exploration.

Low-light image enhancement (LLIE) aims to enhance the illumination of images that are captured under dark conditions, which is critical for various applications in dim environments, such as robotics and autonomous driving. Existing convolutional neural network (CNN)-based methods usually struggle to capture long-range dependencies, while transformer-based methods, despite their effectiveness, are resource-consuming. Besides, the frequency domain includes important lightness degradation information. To this end, we propose a Mamba-based framework called MambaSFLNet to effectively address LLIE by integrating spatial and frequency features. Our approach utilizes the Visual State Space Module to establish relationships across different regions of the input image while maintaining low model complexity. Furthermore, The spatial module not only balances illumination distribution but also suppresses noise and artifacts during enhancement. In addition, the frequency module enhances image contrast and sharpness by leveraging frequency-domain information. Extensive experiments on nine widely used benchmarks demonstrate that our approach achieves superior performance and exhibits strong generalization capabilities compared to existing methods. The codes are available at https://github.com/MingyuLiu1/MambaSFLNet.git

In this paper, we introduce a novel method for the estimation of vehicle pose and extent in traffic surveillance scenarios based on camera data. The state estimation is performed in a common world frame, enabling the seamless integration of the image data from different viewpoints. Our approach incorporates the non-linear transformation between the measurements and the states directly into the framework of an Unscented Kalman filter. Two measurement models are proposed: one designed for bounding boxes and another for discretized object contours extracted from segmentation masks. The method is evaluated using data from a real-world traffic surveillance system, demonstrating the high effectiveness and good feasibility of our approach for localizing passing cars.

Most vision-based sensing and localization infrastructure today employ conventional area scanning cameras due to the high information density and cost efficiency offered by them. While the information-rich two-dimensional images provided by such sensors make it easier to detect and classify traffic objects with the help of deep neural networks, their accurate localization in the three-dimensional real world also calls for a reliable calibration methodology, that maintains accuracy not just during installation, but also under continuous operation over time. In this paper, we propose a camera calibration architecture that extracts and uses corresponding targets from high definition maps, augment it with an efficient stabilization mechanism in order to compensate for the errors arising out of fast transient vibrations and slow orientational drifts. Finally, we evaluate its performance on a real-world test site.

In this paper, we present a novel design for a multi-camera tracking system with occlusion-handling capabilities and its application to a highway traffic surveillance system. The fundamental concept follows the tracking-by-detection principle with monocular detectors and an LMB tracker for tracking the objects in the world frame. All data from the multi-view setup is combined into one consistent representation of the real-time traffic situation. In order to assess the inter-target occlusion scenarios in 3D, the vehicles are modeled as cuboids and their extents are estimated from the bounding boxes provided by the detectors. We re-transform the 3D occlusion estimation problem into the 2D camera space and present two methods for quantifying the occlusion state of the objects. Moreover, we propose a modification to the computation of the existence probability of undetected and occluded targets. Based on this, the tracking system is extended by an occlusion-aware detection model. We evaluate our occlusion-handling approach on a real-world traffic dataset from the Providentia++ project and show an improved tracking performance. We find that the number of misdetected targets is reduced and more track identities are preserved.

Data-intensive machine learning based techniques increasingly play a prominent role in the development of future mobility solutions - from driver assistance and automation functions in vehicles, to real-time traffic management systems realized through dedicated infrastructure. The availability of high quality real-world data is often an important prerequisite for the development and reliable deployment of such systems in large scale. Towards this endeavour, we present the A9-Dataset based on roadside sensor infrastructure from the 3 km long Providentia++ test field near Munich in Germany. The dataset includes anonymized and precision-timestamped multi-modal sensor and object data in high resolution, covering a variety of traffic situations. As part of the first set of data, which we describe in this paper, we provide camera and LiDAR frames from two overhead gantry bridges on the A9 autobahn with the corresponding objects labeled with 3D bounding boxes. The first set includes in total more than 1000 sensor frames and 14000 traffic objects. The dataset is available for download at https://a9-dataset.com.