The current state-of-the-art methods in domain adaptive object detection (DAOD) use Mean Teacher self-labelling, where a teacher model, directly derived as an exponential moving average of the student model, is used to generate labels on the target domain which are then used to improve both models in a positive loop. This couples learning and generating labels on the target domain, and other recent works also leverage the generated labels to add additional domain alignment losses. We believe this coupling is brittle and excessively constrained: there is no guarantee that a student trained only on source data can generate accurate target domain labels and initiate the positive feedback loop, and much better target domain labels can likely be generated by using a large pretrained network that has been exposed to much more data. Vision foundational models are exactly such models, and they have shown impressive task generalization capabilities even when frozen. We want to leverage these models for DAOD and introduce DINO Teacher, which consists of two components. First, we train a new labeller on source data only using a large frozen DINOv2 backbone and show it generates more accurate labels than Mean Teacher. Next, we align the student’s source and target image patch features with those from a DINO encoder, driving source and target representations closer to the generalizable DINO representation. We obtain state-of-the-art performance on multiple DAOD datasets. Code available at https://github.com/TRAILab/DINO_Teacher.

Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed Aly, Beidi Chen, Carole-Jean Wu. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Vision-Language Models (VLMs) are pretrained on large, diverse, and noisy web-crawled datasets. This underscores the critical need for dataset pruning, as the quality of these datasets is strongly correlated with the performance of VLMs on downstream tasks. Using CLIPScore from a pretrained model to only train models using highly-aligned samples is one of the most successful methods for pruning. We argue that this approach suffers from multiple limitations including: false positives and negatives due to CLIP's pretraining on noisy labels. We propose a pruning signal, Sieve, that employs synthetic captions generated by image-captioning models pretrained on small, diverse, and well-aligned image-text pairs to evaluate the alignment of noisy image-text pairs. To bridge the gap between the limited diversity of generated captions and the high diversity of alternative text (alt-text), we estimate the semantic textual similarity in the embedding space of a language model pretrained on unlabeled text corpus. Using DataComp, a multimodal dataset filtering benchmark, when evaluating on 38 downstream tasks, our pruning approach, surpasses CLIPScore by 2.6% and 1.7% on medium and large scale respectively. In addition, on retrieval tasks, Sieve leads to a significant improvement of 2.7% and 4.5% on medium and large scale respectively.

An effective framework for learning 3D representations for perception tasks is distilling rich self-supervised image features via contrastive learning. However, image-to-point representation learning for autonomous driving datasets faces two main challenges: 1) the abundance of self-similarity, which results in the contrastive losses pushing away semantically similar point and image regions and thus disturbing the local semantic structure of the learned representations, and 2) severe class imbalance as pretraining gets dominated by over-represented classes. We propose to alleviate the self-similarity problem through a novel semantically tolerant image-to-point contrastive loss that takes into consideration the semantic distance between positive and negative image regions to minimize contrasting semantically similar point and image regions. Additionally, we address class imbalance by designing a class-agnostic balanced loss that approximates the degree of class imbalance through an aggregate sample-to-samples semantic similarity measure. We demonstrate that our semantically-tolerant contrastive loss with class balancing improves state-of-the-art 2D-to-3D representation learning in all evaluation settings on 3D semantic segmentation. Our method consistently outperforms state-of-the-art 2D-to-3D representation learning frameworks across a wide range of 2D self-supervised pretrained models.

Camera and LiDAR sensor modalities provide complementary appearance and geometric information useful for detecting 3D objects for autonomous vehicle applications. However, current end-to-end fusion methods are challenging to train and underperform state-of-the-art LiDAR-only detectors. Sequential fusion methods suffer from a limited number of pixel and point correspondences due to point cloud sparsity, or their performance is strictly capped by the detections of one of the modalities. Our proposed solution, Dense Voxel Fusion (DVF) is a sequential fusion method that generates multi-scale dense voxel feature representations, improving expressiveness in low point density regions. To enhance multi-modal learning, we train directly with projected ground truth 3D bounding box labels, avoiding noisy, detector-specific 2D predictions. Both DVF and the multi-modal training approach can be applied to any voxel-based LiDAR backbone. DVF ranks 3rdamong published fusion methods on KITTI’s 3D car detection benchmark without introducing additional trainable parameters, nor requiring stereo images or dense depth labels. In addition, DVF significantly improves 3D vehicle detection performance of voxel-based methods on the Waymo Open Dataset.

In the era of autonomous cars, accurate vehicular positioning becomes very essential. The global navigation satellite systems (GNSS) suffer from signal blockage and severe multipath in urban canyons, which degrades the positioning accuracy and availability. Therefore, vehicles solely relying on positioning from GNSS receivers have limited performance. In this research, we present a novel unified cooperative positioning solution which enhances positioning accuracy and availability in urban canyons. The proposed system exploits the fact that vehicles have different positioning resources and is based on angle approximation, which artificially generates the hindered pseudorange by sharing angle information between vehicles using dedicated short-range communication. In addition, we propose a system that employs the proposed cooperative technique to assist the loose integration between the inertial navigation system (INS) and the GPS system (using extended Kalman filter) during partial GPS outages. Using raw data from inertial sensors and GPS receivers in the real road trajectories, we implement the cooperative INS/GPS loose integration and show that our cooperative integrated system outperforms the non-cooperative integrated system. The performance metrics used are the 2-D positioning root-mean-square error, the maximum 2-D positioning error, and the positioning accuracy gain (PAG). Specifically, the PAG gain is around 88%, 80%, and 60% when the number of blocked satellites is one, two, and three, respectively.

This paper addresses the challenge of vehicle selection in a Vehicular Ad-hoc Network (VANET) used to assist vehicles with limited satellite visibility in urban environments. In [1], we proposed a Non-Range cooperative positioning system which uses pseudoranges from only one assisting vehicle at any given time. However, many vehicles are within the communication zone of the target vehicle especially in dense urban canyons. In this paper, we elevate the performance of our cooperative system by proposing a distributed vehicle selection criterion named Absolute Sum of Single Differencing (ASOSD). To test the viability of the proposed system, we design a cooperative experiment using three NovAtel receivers and show that the Positioning Accuracy Gain (PAG) of our system has increased by 60% compared to a system that averages Artificial Candidate Pseudoranges (ACPs) from two assisting receivers. Moreover, we study the effect of; the number of assisting vehicles, multipath, receiver noise, satellite clock bias, ionospheric and tropospheric errors on the selectivity of the proposed system. We show that as the number of assisting vehicles increase, the Root-Mean-Square-Error (RMSE) of the generated ACP decreases. Moreover, the selectivity of the ASOSD selector is not affected by the common errors in the shared pseudoranges.

Location-Based Services (LBS) and Intelligent Transportation Systems (ITS) demand positioning accuracy and availability requirements. In urban canyons, Global Navigation Satellite Systems (GNSS) suffer from signal blockage, jamming , severe multipath and low Carrier-to-Noise (C/No) ratio which degrade location accuracy and availability. Therefore, applications solely relying on GNSS have limited performance. In this paper, we present a novel unified Cooperative Positioning (CP) solution which enhances positioning accuracy and availability in urban canyons. Our proposed approach is named Angle Approximation (AA). AA requires no infrastructure or other aiding sensors, AA is distributed and addresses two core challenges (limited positioning accuracy and availability) in a unified solution. AA artificially generates the hindered pseudorange by sharing angle information between vehicles using Dedicated Short Range Communication (DSRC). To enhance the performance of the AA technique, we propose and analytically derive the Absolute Sum of Double Differencing (ASODD) method which increases the probability of selecting the most accurate generated pseudorange. We experimentally evaluate the performance of the proposed system through real measurements using NovAtel receivers. We also carry out extensive simulations to demonstrate the ability of the proposed system to increase solution availability. Our experimental and simulation results demonstrate that the solution accuracy of our approach is inversely proportional to the distance between vehicles. Specifically, the mean error of the generated pseudorange is limited to 14 percent of the distance between vehicles.