Many tracking companies collect user data and sell it to data markets and advertisers. While they claim to protect user privacy by anonymizing the data, our research reveals that significant privacy risks persist even with anonymized data. Attackers can exploit this data to identify users' accounts on other websites and perform targeted identity alignment. In this paper, we propose an effective identity alignment scheme for accurately identifying targeted users. We develop a data collector to obtain the necessary datasets, an algorithm for identity alignment, and, based on this, construct two types of de-anonymization attacks: thepassive attack, which analyzes tracker data to align identities, and theactive attack, which induces users to interact online, leading to higher success rates. Furthermore, we introduce, for the first time, a novel evaluation framework for online tracking-based identity alignment. We investigate the key factors influencing the effectiveness of identity alignment. Additionally, we provide an independent assessment of our generated dataset and present a fully functional system prototype applied to a cryptocurrency use case.

Discretionary lane changing is one of the most important behaviors in vehicle dynamics. The analysis of discretionary lane changing can provide support for human-like autonomous driving system and microscopic traffic simulation. The rule based discretionary lane change decision-making model has the advantages of good interpretability and being in line with human intuitiveness, but the performance of the rule based models are relatively poor. The learning-based discretionary lane changing decision-making models perform better in lane changing analysis than that of the rule based models, while they require a large amount of training data and have poor interpretability. This study proposes a framework for discretionary lane changing decision-making based on cumulative prospect theory and decision tree model, and uses genetic algorithm to calibrate the parameters of the framework. The framework proposed in this study has good interpretability for discretionary lane changing behavior similar to rule-based models, and it can achieve good lane changing prediction performance. The Next Generation Simulation (NGSIM) dataset is used to validate the framework proposed in this study. The results show that the framework proposed in this study can achieve better performance of discretionary lane change prediction than the analysis model based on decision tree and the analysis model based on cumulative prospect theory. When predicting discretionary lane changing behavior 2–0 seconds in advance, the accuracy of the framework in the test set can reach 85.6%.

Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Daniel Egert, Ellie Evans, Hoo-Chang Shin, Felipe Soares, Yi Dong, Oleksii Kuchaiev. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.

Reward models are critical for aligning models to follow instructions, and are typically trained following one of two popular paradigms: Bradley-Terry style or Regression style. However, there is a lack of evidence that either approach is better than the other, when adequately matched for data. This is primarily because these approaches require data collected in different (but incompatible) formats, meaning that adequately matched data is not available in existing public datasets. To tackle this problem, we release preference annotations (designed for Bradley-Terry training) to complement existing ratings (designed for Regression style training) in the HelpSteer2 dataset. To improve data interpretability, preference annotations are accompanied with human-written justifications. Using this data, we conduct the first head-to-head comparison of Bradley-Terry and Regression models when adequately matched for data. Based on insights derived from such a comparison, we propose a novel approach to combine Bradley-Terry and Regression reward modeling. A Llama-3.1-70B-Instruct model tuned with this approach scores 94.1 on RewardBench, emerging top of more than 140 reward models as of 1 Oct 2024. This reward model can then be used with REINFORCE algorithm (RLHF) to align an Instruct model to reach 85.0 on Arena Hard, which is No. 1 as of 1 Oct 2024. We open-source this dataset (CC-BY-4.0 license) at https://huggingface.co/datasets/nvidia/HelpSteer2#preferences-new---1-oct-2024 and openly release the trained Reward and Instruct models at https://huggingface.co/nvidia/Llama-3.1-Nemotron-70B-Reward and https://huggingface.co/nvidia/Llama-3.1-Nemotron-70B-Instruct .

Preference datasets are essential for training general-domain, instruction-following language models with Reinforcement Learning from Human Feedback (RLHF). Each subsequent data release raises expectations for future data collection, meaning there is a constant need to advance the quality and diversity of openly available preference data. To address this need, we introduce HelpSteer3-Preference, a permissively licensed (CC-BY-4.0), high-quality, human-annotated preference dataset comprising of over 40,000 samples. These samples span diverse real-world applications of large language models (LLMs), including tasks relating to STEM, coding and multilingual scenarios. Using HelpSteer3-Preference, we train Reward Models (RMs) that achieve top performance on RM-Bench (82.4%) and JudgeBench (73.7%). This represents a substantial improvement (~10% absolute) over the previously best-reported results from existing RMs. We demonstrate HelpSteer3-Preference can also be applied to train Generative RMs and how policy models can be aligned with RLHF using our RMs.

As the Internet of Things (IoT) technology advances, human activity recognition (HAR) using IoT devices, including wearable sensors has become prevalent in various applications. Nevertheless, many sensor-based HAR methods still struggle to balance recognition accuracy with network complexity. Meanwhile, most existing sensor-based HAR networks fail to achieve an effective fusion of multidimensional features. To address the above issues, a hierarchical multiscale time-frequency and channel feature adaptive fusion (HMTF-CFAF) network is put forward. The HMTF-CFAF efficiently extracts unique multiscale time-frequency and channel features in sensor data using hierarchical connectivity. Furthermore, it incorporates a feature fusion mechanism to integrate and exchange multiscale and multidimensional features, providing more comprehensive and richer features. To evaluate the HMTF-CFAF network, we utilize three datasets: 1) the University of California Irvine HAR (UCI-HAR); 2) physical activity monitoring for aging people (PAMAP2); and 3) self-collected household behavior (HB) dataset. The HMTF-CFAF network achieves the accuracies of 97.66%, 98.75%, and 98.80% on the above three datasets, respectively, demonstrating its excellent performance.

Mobile Phone Signaling (MPS) data record the daily traces of urban residents, offering a cost-effective means to obtain travel information for urban traffic management and planning at low cost. However, despite the vast amount of data available, there remains a lag in the development of techniques for identifying fine-grained information such as travel modes. On the one hand, the high positioning error and irregular collection frequency make the identification of fine-grained modes challenging. On the other hand, the difficulties in collecting real labeled data limit the training and evaluation of advanced models. In this paper, we present an advanced Travel Mode Identification (TMI) framework and collect real labeled MPS data for evaluation. Specifically, a fast smoothing method is proposed to enhance noise reduction in large-scale trajectories while effectively mitigating positioning errors. We propose novel point-level bus route alignment features for advanced deep-learning models to improve differentiation between motorized modes. Furthermore, a deep learning model with ensembled feature encoding modules is designed to overcome training instability due to limited data amount. Our proposed framework achieves an accuracy of 83.06% in identifying fine-grained travel modes, including walking, riding, bus, car, and metro, with recall rates exceeding 78% for all modes except walking. We analyze the relationship between accuracy and the spatiotemporal characteristics of trajectories, revealing a significant impact from the collection frequency, while showing insensitivity to distance gaps and positioning errors. This study demonstrates the potential of MPS for TMI and promotes its application in intelligent transportation systems.

Parameter-Efficient Fine-Tuning (PEFT) and Retrieval-Augmented Generation (RAG) have become popular methods for adapting large language models while minimizing compute requirements.In this paper, we apply PEFT methods (P-tuning, Adapters, and LoRA) to a modified Retrieval-Enhanced Transformer (RETRO) and a baseline GPT model across several sizes, ranging from 823 million to 48 billion parameters.We show that RETRO models outperform GPT models in zero-shot settings due to their unique pre-training process but GPT models have higher performance potential with PEFT.Additionally, our study indicates that 8B parameter models strike an optimal balance between cost and performance and P-tuning lags behind other PEFT techniques.We further provide a comparative analysis between applying PEFT to an Instruction-tuned RETRO model and base RETRO model.This work presents the first comprehensive comparison of various PEFT methods integrated with RAG, applied to both GPT and RETRO models, highlighting their relative performance.

Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Scowcroft, Neel Kant, Aidan Swope, Oleksii Kuchaiev. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.

High-quality preference datasets are essential for training reward models that can effectively guide large language models (LLMs) in generating high-quality responses aligned with human preferences.As LLMs become stronger and better aligned, permissively licensed preference datasets, such as Open Assistant, HH-RLHF, and HelpSteer need to be updated to remain effective for reward modeling.Methods that distil preference data from proprietary LLMs such as GPT-4 have restrictions on commercial usage imposed by model providers.To improve upon both generated responses and attribute labeling quality, we release HelpSteer2, a permissively licensed preference dataset (CC-BY-4.0). Using a powerful Nemotron-4-340B base model trained on HelpSteer2, we are able to achieve the SOTA score (92.0%) on Reward-Bench's primary dataset, outperforming currently listed open and proprietary models, as of June 12th, 2024.Notably, HelpSteer2 consists of only ten thousand response pairs, an order of magnitude fewer than existing preference datasets (e.g., HH-RLHF), which makes it highly efficient for training reward models. Our extensive experiments demonstrate that reward models trained with HelpSteer2 are effective in aligning LLMs. Additionally, we propose SteerLM 2.0, a model alignment approach that can effectively make use of the rich multi-attribute score predicted by our reward models. HelpSteer2 is available at https://huggingface.co/datasets/nvidia/HelpSteer2 and code is available at https://github.com/NVIDIA/NeMo-Aligner

Vehicle re-identification aims to match and identify the same vehicle crossing multiple surveillance cameras and obtain traffic information such as travel time. The Automatic License Plate Recognition (ALPR) data are widely employed in urban surveillance. However, vehicle re-identification based on ALPR data is challenging due to license plate recognition errors and unrecognized issues. This paper proposes a vehicle matching algorithm designed to maximize the travel time probability using ALPR data, while accounting for recognition errors and unrecognized issues. The proposed algorithm consists of several modules, including the estimation of travel time distribution, computation of travel time probability, calculation of travel time confidence intervals and matching time window size, restricted fuzzy matching, and vehicle matching optimization. To evaluate the effectiveness of the proposed algorithm across varying lighting and weather conditions, ALPR data was collected from a survey road in four scenarios: sunny day, sunny night, rainy day, and rainy night. The results indicate that when compared to a sunny day scenario, severe lighting and adverse weather conditions lead to decreased matching accuracy and increased matching accuracy errors for all methods evaluated. However, our proposed model outperforms benchmark algorithms in both scenarios, demonstrating its superior performance.

The collection of individual GPS data, as a substitute for traditional travel surveys, is hindered by low response rates and high costs. Meanwhile, passive mobile phone data, such as Location-Based Service (LBS) data, has high user penetration but remains unexplored for travel behavior analysis. The irregular frequency of data collection results in distorted features and non-uniformly distributed information of trajectories in the Travel Mode Identification (TMI) task. The diversity of trajectories also poses challenges for TMI models. To fill the gap, we propose a TMI framework named Trajectory-as-a-Sequence for Non-uniform data (TaaSN). Specifically, we incorporate GIS features to address the sparsity of motion features. Then, we design a model structure that accounts for time gaps, capturing non-uniform information of trajectory due to irregular frequency. To further enhance model’s generalizability to diverse trajectory data, we propose a trajectory point dropout training strategy. The experimental results demonstrate that the TaaSN framework can greatly exploit the potential of LBS data in travel behavior mining. The proposed model achieves high accuracy on both non-uniform and uniform trajectories. On pseudo-LBS data, the accuracy reaches 84.9% when applied to the trajectories of existing travelers, and achieves 83.5% when applied to the trajectories of new travelers. On uniform data, it reaches 86.8% and 85.4%, respectively. Furthermore, we carry out comprehensive experiments and conclude valuable insights for future TMI framework design.

Graph classification is a widely studied problem and has broad applications. In many real-world problems, the number of labeled graphs available for training classification models is limited, which renders these models prone to overfitting. To address this problem, we propose two approaches based on contrastive self-supervised learning (CSSL) to alleviate overfitting. In the first approach, we use CSSL to pretrain graph encoders on widely-available unlabeled graphs without relying on human-provided labels, then finetune the pretrained encoders on labeled graphs. In the second approach, we develop a regularizer based on CSSL, and solve the supervised classification task and the unsupervised CSSL task simultaneously. To perform CSSL on graphs, given a collection of original graphs, we perform data augmentation to create augmented graphs out of the original graphs. An augmented graph is created by consecutively applying a sequence of graph alteration operations. A contrastive loss is defined to learn graph encoders by judging whether two augmented graphs are from the same original graph. Experiments on various graph classification datasets demonstrate the effectiveness of our proposed methods. The code is available at https://github.com/UCSD-AI4H/GraphSSL.

Information Extraction (IE) aims to extract structural information from unstructured texts.In practice, long-tailed distributions caused by the selection bias of a dataset, may lead to incorrect correlations, also known as spurious correlations, between entities and labels in the conventional likelihood models.This motivates us to propose counterfactual IE (CFIE), a novel framework that aims to uncover the main causalities behind data in the view of causal inference.Specifically, 1) we first introduce a unified structural causal model (SCM) for various IE tasks, describing the relationships among variables; 2) with our SCM, we then generate counterfactuals based on an explicit language structure to better calculate the direct causal effect during the inference stage; 3) we further propose a novel debiasing approach to yield more robust predictions.Experiments on three IE tasks across five public datasets show the effectiveness of our CFIE model in mitigating the spurious correlation issues.

Text-guided image inpainting aims to complete the corrupted patches coherent with both visual and textual context. On one hand, existing works focus on surrounding pixels of the corrupted patches without considering the objects in the image, resulting in the characteristics of objects described in text being painted on non-object regions. On the other hand, the redundant information in text may distract the generation of objects of interest in the restored image. In this paper, we propose an adversarial learning framework with mask reconstruction (ALMR) for image inpainting with textual guidance, which consists of a two-stage generator and dual discriminators. The two-stage generator aims to restore coarse-grained and fine-grained images, respectively. In particular, we devise a dual-attention module (DAM) to incorporate the word-level and sentence-level textual features as guidance on generating the coarse-grained and fine-grained details in the two stages. Furthermore, we design a mask reconstruction module (MRM) to penalize the restoration of the objects of interest with the given textual descriptions about the objects. For adversarial training, we exploit global and local discriminators for the whole image and corrupted patches, respectively. Extensive experiments conducted on CUB-200-2011, Oxford-102 and CelebA-HQ show the outperformance of the proposed ALMR (e.g., FID value is reduced from 29.69 to 14.69 compared with the state-of-the-art approach on CUB-200-2011). Codes are available at https://github.com/GaranWu/ALMR

Guangtao Zeng, Wenmian Yang, Zeqian Ju, Yue Yang, Sicheng Wang, Ruisi Zhang, Meng Zhou, Jiaqi Zeng, Xiangyu Dong, Ruoyu Zhang, Hongchao Fang, Penghui Zhu, Shu Chen, Pengtao Xie. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020.

In software-defined datacenter networks, there are bandwidth-demanding elephant flows without deadline and delay-sensitive mice flows with strict deadline. They compete with each other for limited network resources, and how to effectively schedule such mix-flow is a huge challenge. We propose DRL-PLink (deep reinforcement learning with private link) that combines software-defined network and deep reinforcement learning (DRL) to schedule mix-flow. It divides the link bandwidth and establishes some corresponding private links for different types of flows respectively to isolate them. DRL is used to adaptively allocate bandwidth resources for these private links. Furthermore, DRL-PLink introduces Clipped Double Q-learning and parameter exploration NoisyNet technology to improve the scheduling policy for overestimated value estimates and action exploration problems in DRL. The simulation results show that DRL-PLink can effectively schedule mix-flow. Compared with ECMP and pFabric, the average flow completion time of DRL-PLink has decreased by 68.87% and 52.18% respectively. At the same time, it maintains a high deadline meet rate (>96.6%) close to pFabric and Karuna very much.