With the broader adoption and highly successful development of large language models (LLMs), there has been growing interest and demand for applying LLMs to autonomous driving technology. Driven by their natural language (NL) understanding and reasoning capabilities, LLMs have the potential to enhance various aspects of autonomous driving systems, from perception and scene understanding to interactive decision-making. This article first introduces the novel concept of designing LLMs for autonomous driving (LLM4AD), followed by a review of existing LLM4AD studies. Then, a comprehensive benchmark is proposed for evaluating the instruction-following and reasoning abilities of LLM4AD systems, which includes LaMPilot-Bench, CARLA Leaderboard 1.0 Benchmark in simulation and NuPlanQA for multiview visual question answering (VQA). Furthermore, extensive real-world experiments are conducted on autonomous vehicle platforms, examining both on-cloud and on-edge LLM deployment for personalized decision-making and motion control. Next, the future trends of integrating language diffusion models into autonomous driving are explored, exemplified by the proposed vision-language diffusion (ViLaD) framework. Finally, the main challenges of LLM4AD are discussed, including latency, deployment, security and privacy, safety, trust and transparency, and personalization.

Vision-Language Models (VLMs) have demonstrated significant promise for autonomous driving due to their powerful multimodal reasoning capabilities. However, adapting VLMs from generic data to safety-critical driving contexts introduces a notable challenge known as domain shift. Existing simulation-based and dataset-driven evaluation approaches struggle to accurately replicate real-world complexities, lacking repeatable closed-loop evaluation and flexible scenario manipulation. Furthermore, current real-world testing platforms typically focus on isolated modules and do not support comprehensive interaction with VLM-based systems. Consequently, there is a critical need for a holistic testing architecture capable of integrating perception, planning, and control modules, accommodating VLM-based systems, and supporting configurable real-world testing scenarios. In this article, we address this critical gap by proposing a hierarchical real-world test platform specialized in the rigorous evaluation of VLM-integrated autonomous driving systems. Specifically, our platform features have: a lightweight, structured, and low-latency middleware pipeline specialized for seamless VLM integration; a hierarchical modular architecture enabling flexible substitution between conventional and VLM-based autonomy components, providing exceptional deployment flexibility for rapid experimentation; and sophisticated closed-loop scenario-based testing capabilities on a controlled test track, facilitating comprehensive evaluation of the entire full-stack VLM-integrated autonomous driving pipeline, from perception, reasoning, decision-making, and planning to final vehicle maneuvers. Through an extensive real-world case study, we demonstrate the effectiveness of our platform in evaluating the performance and robustness of VLM-integrated autonomous driving under diverse realistic conditions. Project page and codes: https://github.com/YupengZhouPurdue/VLMTest .

Perception is a crucial component of autonomous driving systems. However, single-agent setups often face limitations due to sensor constraints, especially under challenging conditions like severe occlusion, adverse weather, and long-range object detection. Multi-agent collaborative perception (CP) offers a promising solution that enables communication and information sharing between connected vehicles. Yet, the heterogeneity among agents—in terms of sensors, models, and tasks—significantly hinders effective and efficient cross-agent collaboration. To address these challenges, we propose STAMP, a scalable task- and model-agnostic collaborative perception framework tailored for heterogeneous agents. STAMP utilizes lightweight adapter-reverter pairs to transform Bird's Eye View (BEV) features between agent-specific domains and a shared protocol domain, facilitating efficient feature sharing and fusion while minimizing computational overhead. Moreover, our approach enhances scalability, preserves model security, and accommodates a diverse range of agents. Extensive experiments on both simulated (OPV2V) and real-world (V2V4Real) datasets demonstrate that STAMP achieves comparable or superior accuracy to state-of-the-art models with significantly reduced computational costs. As the first-of-its-kind task- and model-agnostic collaborative perception framework, STAMP aims to advance research in scalable and secure mobility systems, bringing us closer to Level 5 autonomy. Our project page is at https://xiangbogaobarry.github.io/STAMP and the code is available at https://github.com/taco-group/STAMP.

Personalized driving refers to an autonomous vehicle’s ability to adapt its driving behavior or control strategies to match individual users’ preferences and driving styles while maintaining safety and comfort standards. However, existing works either fail to capture every individual’s preference precisely or become computationally inefficient as the user base expands. Vision-Language Models (VLMs) offer promising solutions to this front through their natural language understanding and scene reasoning capabilities. In this work, we propose a lightweight yet effective on-board VLM framework that provides low-latency personalized driving performance while maintaining strong reasoning capabilities. Our solution incorporates a Retrieval-Augmented Generation (RAG)-based memory module that enables continuous learning of individual driving preferences through human feedback. Through comprehensive real-world vehicle experiments, our system has demonstrated the ability to provide safe, comfortable, and personalized driving experiences across various scenarios and significantly reduce takeover rates by up to 76.9%. To the best of our knowledge, this work represents the first personalized VLM motion control system in real-world autonomous vehicles. The demo video can be watched at https://tinyurl.com/4xsnz79n.

The rapid iteration of autonomous vehicle (AV) deployments leads to increasing needs for building realistic and scalable multi-agent traffic simulators for efficient evaluation. Recent advances in this area focus on closed-loop simulators that enable generating diverse and interactive scenarios. This paper introduces Neural Interactive Agents (NIVA), a probabilistic framework for multi-agent simulation driven by a hierarchical Bayesian model that enables closed-loop, observation-conditioned simulation through autoregressive sampling from a latent, finite mixture of Gaussian distributions. We demonstrate how NIVA unifies preexisting sequence-to-sequence trajectory prediction models and emerging closed-loop simulation models trained on Next-token Prediction (NTP) from a Bayesian inference perspective. Experiments on the Waymo Open Motion Dataset demonstrate that NIVA attains competitive performance compared to the existing method while providing embellishing control over intentions and driving styles.

Multimodal large language models (MLLMs) have shown significant potential in enhancing driving scene understanding and visual question answering (VQA) through advanced logical reasoning capabilities. These tasks support driving action generation and explanation, especially in end-to-end autonomous driving applications. However, deploying these models poses a significant challenge due to their substantial parameter sizes and computational demands, which often exceed onboard computational limits. A key limitation stems from the large number of visual tokens needed to capture detailed, long-context visual information, resulting in increased latency and memory use. To address this, we propose Video Token Sparsification (VTS), a novel approach that leverages redundancy in consecutive video frames to reduce visual tokens while preserving critical information. VTS employs a lightweight CNN-based model to identify key frames and prune less informative tokens, mitigating hallucinations and boosting inference throughput without performance loss. Comprehensive experiments on the LingoQA and DRAMA benchmarks show that VTS achieves up to a 33% improvement in inference throughput and a 28% reduction in memory usage compared to baselines, maintaining comparable performance.

Timed regular expressions serve as a formalism for specifying real-time behaviors of Cyber-Physical Systems. In this paper, we consider the synthesis of timed regular expressions, focusing on generating a timed regular expression consistent with a given set of system behaviors including positive and negative examples, i.e., accepting all positive examples and rejecting all negative examples. We first prove the decidability of the synthesis problem through an exploration of simple timed regular expressions. Subsequently, we propose our method of generating a consistent timed regular expression with minimal length, which unfolds in two steps. The first step is to enumerate and prune candidate parametric timed regular expressions. In the second step, we encode the requirement that a candidate generated by the first step is consistent with the given set into a Satisfiability Modulo Theories (SMT) formula, which is consequently solved to determine a solution to parametric time constraints. Finally, we evaluate our approach on benchmarks, including randomly generated behaviors from target timed models and a case study.

Concentric cable-driven manipulators have the flexibility and typical advantages of working in confined environments. However, its configuration optimization in confined three-dimensional (3-D) space is very complicated due to infinite configurations of inverse kinematics solutions. This article proposes a spatial triarc planning method in response to the issue mentioned above. A reasonable triarc configuration can be optimized by this method based on six input parameters, i.e., the proximal control point and tangent vector, distal control point and tangent vector, and two centers of curvature circles in both two-dimensional (2-D) and 3-D space. This method has the following three advantages. First, the task configuration of a spatial triarc can be predicted by presetting the relationship between the proximal and distal tangent vectors. Furthermore, the configuration of the middle and inner concentric cable-driven mechanism can be controlled by adjusting the direction of the distal tangent vector. Additionally, the proportion of each concentric cable-driven mechanism can be controlled by changing centers of curvature circles. Finally, the proposed spatial triarc planning method is verified by simulations and experiments. Results show that the maximum errors of the C-shaped spatial triarc and the S-shaped spatial triarc experiments are 1.68 mm and 1.36 mm, respectively.

Vision-language generative AI has demonstrated re-markable promise for empowering cross-modal scene understanding of autonomous driving and high-definition (HD) map systems. However, current benchmark datasets lack multi-modal point cloud, image, and language data pairs. Recent approaches utilize visual instruction learning and cross-modal prompt engineering to expand vision-language models into this domain. In this paper, we pro-pose a new vision-language benchmark that can be used to finetune traffic and HD map domain-specific foundation models. Specifically, we annotate and leverage large-scale, broad-coverage traffic and map data extracted from huge HD map annotations, and use CLIP and LLaMA-2 / Vi-cuna to finetune a baseline model with instruction-following data. Our experimental results across various algorithms reveal that while visual instruction-tuning large language models (LLMs) can effectively learn meaningful represen-tations from MAPLM-QA, there remains significant room for further advancements. To facilitate applying LLMs and multi-modal data into self-driving research, we will release our visual-language QA data, and the baseline models at GitHub.com/LLVM-AD/MAPLM.

Safety and robustness are crucial factors in developing trustworthy autonomous vehicles. One essential aspect of addressing these factors is to equip vehicles with the capability to predict future trajectories for all moving objects in the surroundings and quantify prediction uncertainties. In this paper, we propose the Sequential Neural Variational Agent (SeNe VA), a generative model that describes the distribution of future trajectories for a single moving object. Our approach can distinguish Out-of-Distribution data while quantifying uncertainty and achieving competitive performance compared to state-of-the-art methods on the Argoverse 2 and INTERACTION datasets. Specifically, a 0.446 meters minimum Final Displacement Error, a 0.203 meters minimum Average Displacement Er-ror, and a 5.35% Miss Rate are achieved on the INTERACTION test set. Extensive qualitative and quantitative analy-sis is also provided to evaluate the proposed model. Our open-source code is available at https://github.com/PurdueDigitalTwin/seneva.

Autonomous driving (AD) has made significant strides in recent years. However, existing frameworks struggle to interpret and execute spontaneous user instructions, such as "overtake the car ahead.” Large Language Models (LLMs) have demonstrated impressive reasoning capabilities showing potential to bridge this gap. In this paper, we present LaMPilot, a novel framework that integrates LLMs into AD systems, enabling them to follow user instructions by generating code that leverages established functional primitives. We also introduce LaMPilot-Bench, the first bench-mark dataset specifically designed to quantitatively evaluate the efficacy of language model programs in AD. Adopting the LaMPilot framework, we conduct extensive experiments to assess the performance of off-the-shelf LLMs on LaMPilot-Bench. Our results demonstrate the potential of LLMs in handling diverse driving scenarios and following user instructions in driving. To facilitate further research in this area, we release our code and data at GitHub.com/PurdueDigitalTwin/LaMPilot.

Ensuring traffic safety and mitigating accidents in modern driving is of paramount importance, and computer vision technologies have the potential to significantly contribute to this goal. This paper presents a multi-modal Vision Transformer for Driver Distraction Detection (termed ViT-DD), which incorporates inductive information from training signals related to both distraction detection and driver emotion recognition. Additionally, a self-learning algorithm is developed, allowing for the seamless integration of driver data without emotion labels into the multi-task training process of ViT-DD. Experimental results reveal that the proposed ViT-DD surpasses existing state-of-the-art methods for driver distraction detection by 6.5% and 0.9% on the SFDDD and AUCDD datasets, respectively.

Vehicle-to-vehicle (V2V) communications have greatly enhanced the perception capabilities of connected and automated vehicles (CAVs) by enabling information sharing to "see through the occlusions", resulting in significant performance improvements. However, developing and training complex multi-agent perception models from scratch can be expensive and unnecessary when existing single-agent models show remarkable generalization capabilities. In this paper, we propose a new framework termed MACP, which equips a single-agent pre-trained model with cooperation capabilities. We approach this objective by identifying the key challenges of shifting from single-agent to cooperative settings, adapting the model by freezing most of its parameters and adding a few lightweight modules. We demonstrate in our experiments that the proposed framework can effectively utilize cooperative observations and outperform other state-of-the-art approaches in both simulated and real-world cooperative perception benchmarks while requiring substantially fewer tunable parameters with reduced communication costs. Our ource code is available at https://github.com/PurdueDigitalTwin/MACP.

Federated learning (FL) has been gaining attention for its ability to share knowledge while maintaining user data, protecting privacy, increasing learning efficiency, and reducing communication overhead. Decentralized FL (DFL) is a decentralized network architecture that eliminates the need for a central server in contrast to centralized FL (CFL). DFL enables direct communication between clients, resulting in significant savings in communication resources. In this paper, a comprehensive survey and profound perspective are provided for DFL. First, a review of the methodology, challenges, and variants of CFL is conducted, laying the background of DFL. Then, a systematic and detailed perspective on DFL is introduced, including iteration order, communication protocols, network topologies, paradigm proposals, and temporal variability. Next, based on the definition of DFL, several extended variants and categorizations are proposed with state-of-the-art (SOTA) technologies. Lastly, in addition to summarizing the current challenges in the DFL, some possible solutions and future research directions are also discussed.

The development of autonomous vehicles is dramatically reshaping the transportation landscape, bringing new challenges and opportunities in human-machine interaction. As autonomous vehicles evolve, understanding and responding to human intent becomes significant, and therefore require new ways of human-autonomy teaming. A human digital twin (HDT) is a virtual representation of an individual driver, capturing their preferences, behaviors, and physiological states, enabling machines to better understand and predict human needs and responses. In this paper, we explore how large language models (LLMs), like GPT-4 and LLaMA, together with HDTs are changing the way humans team up with autonomous vehicles. These LLMs help make our conversations with vehicles more natural and intuitive. By pairing them in HDTs, we can get real-time feedback and smarter responses. This combination offers not just easier control but also safer driving experiences. We will break down how this works, why it matters, and what we might expect in the future.

In order to enhance driving safety and identify potential hazards, next-generation intelligent vehicles will need to understand human drivers’ intentions and predict their potential maneuvers correctly. In a lane-change scenario, a driver’s head rotation measured by the in-cabin driver monitoring camera can serve as a reliable indicator to predict his/her intention. However, using a general model to predict each driver’s maneuver is not accurate, while directly sharing the personalized monitoring data to other intelligent vehicles raises the privacy concern. In this paper, we propose a clustering-based personalized federated learning framework (CPFL) to predict lane-change maneuver based on driver monitoring data. Personalization is added on top of the traditional federated learning (FL) through clustering, which separates and groups similar driving behaviors based on clustering parameters: head position threshold and average pre-lane-change preparation time. Long-Short Term Memory (LSTM) networks with different sequence lengths are deployed to predict lane changes in different clusters based on the lane-change preparation time. CPFL framework is trained and tested using the data collected from several human drivers under different driving scenarios through the Unity simulation platform. According to the results, CPFL’s average training efficiency is 7.6 times higher than the classic FedAvg approach, and CPFL also offers better adaptability to different driving behaviors than FedAvg with 4% higher accuracy, 0.2% fewer false positives, and 27.8% fewer false negatives.

Connected and automated vehicles (CAVs) are supposed to share the road with human-driven vehicles (HDVs) in a foreseeable future. Therefore, considering the mixed traffic environment is more pragmatic, as the well-planned operation of CAVs may be interrupted by HDVs. In the circumstance that human behaviors have significant impacts, CAVs need to understand HDV behaviors to make safe actions. In this study, we develop a driver digital twin (DDT) for the online prediction of personalized lane-change behavior, allowing CAVs to predict surrounding vehicles’ behaviors with the help of the digital twin technology. DDT is deployed on a vehicle-edge–cloud architecture, where the cloud server models the driver behavior for each HDV based on the historical naturalistic driving data, while the edge server processes the real-time data from each driver with his/her digital twin on the cloud to predict the personalized lane-change maneuver. The proposed system is first evaluated on a human-in-the-loop co-simulation platform, and then in a field implementation with three passenger vehicles driving along an on/off ramp segment connecting to the edge server and cloud through the 4G/LTE cellular network. The lane-change intention can be recognized in 6 s on average before the vehicle crosses the lane separation line, and the Mean Euclidean Distance between the predicted trajectory and GPS ground truth is 1.03 m within a 4-s prediction window. Compared to the general model, using a personalized model can improve prediction accuracy by 27.8%. The demonstration video of the proposed system can be watched athttps://youtu.be/5cbsabgIOdM.

Federated learning (FL) shines through in the Internet of Things (IoT) with its ability to realize collaborative learning and improve learning efficiency by sharing client model parameters trained on local data. Although FL has been successfully applied to various domains, including driver monitoring applications (DMAs) on the Internet of Vehicles (IoV), its usages still face some open issues, such as data and system heterogeneity, large-scale parallelism communication resources, malicious attacks, and data poisoning. This article proposes a federated transfer–ordered–personalized learning (FedTOP) framework to address the above problems and test on two real-world data sets with and without system heterogeneity. The performance of the three extensions, transfer, ordered, and personalized, is compared by an ablation study and achieves 92.32% and 95.96% accuracy on the test clients of two data sets, respectively. Compared to the baseline, there is a 462% improvement in accuracy and a 37.46% reduction in communication resource consumption. The results demonstrate that the proposed FedTOP can be used as a highly accurate, streamlined, privacy-preserving, cybersecurity-oriented, and personalized framework for DMA.

In the foreseeable future, connected and auto-mated vehicles (CAVs) and human-driven vehicles will share the road networks together. In such a mixed traffic environment, CAVs need to understand and predict maneuvers of surrounding vehicles for safer and more efficient interactions, especially when human drivers bring in a wide range of uncertainties. In this paper, we propose a learning-based lane-change prediction algorithm that considers the driving behaviors of the target human driver. To provide accurate maneuver prediction, we adopt a hierarchical structure that seamlessly seals both the lane-change decision prediction and the vehicle trajectory pre-diction together. Specifically, we propose a lane-change decision prediction method based on a Long-Short Term Memory (LSTM) network, and a trajectories prediction considering driver preference and vehicular interactions based on Inverse Reinforcement Learning (IRL). To validate the performance of the proposed methodology, a case study of an on-ramp merging scenario is conducted on a uniquely built human-in-the-loop simulation platform that can provide an immersive driving environment, collect data of lane-change behaviors, and test drivers' reactions to the prediction results in real time. It is shown in the simulation results that we can predict the lane-change decision 3 seconds before the vehicle crosses the line to another lane, and the Mean Euclidean Distance between the predicted trajectory and ground truth is 0.39 meters within a 4-second prediction window.

Driving automation is gradually replacing human driving maneuvers in different applications such as adaptive cruise control and lane keeping. However, contemporary driving automation applications based on expert systems or prede-fined control strategies are not in line with individual human driver's preference. To overcome this problem, we propose a Personalized Adaptive Cruise Control (P-ACC) system that can learn the driver's car-following preferences from historical data using model-based maximum entropy Inverse Reinforcement Learning (IRL). Once activated in real-time, the P-ACC system first classifies the driver type and the weather type (at that moment). The vehicle is then controlled using the pre-trained IRL model on the cloud of the associated class. The personalized IRL model on the cloud will be updated as more human driving data is collected from various scenarios. Numerical simulation with real-world naturalistic driving data shows that, the accuracy of reproducing the real-world driving profile improves up to 30.1% in terms of speed and 36.5% in terms of distance gap, when P-ACC is compared with the Intelligent Driver Model (IDM). Game engine-based human-in-the-loop simulation demonstrates that, the takeover frequency of the driver during the usage of P-ACC decreases up to 93.4%, compared with that during the usage of IDM-based ACC.

Road network models form the foundation of road network analyses, route planning, navigation and traffic predictions. However, existing models cannot effectively represent the dynamic topological relationships that exist among lanes due to the effects of time-dependent traffic control measures. To address this problem, we propose a time-dependent road network model (TRNM) to represent these topological relationships, and present its construction method based on a traditional carriageway network model. We constructed two TRNMs in Changzhou and Shanghai and then conducted path-planning experiments to verify the effectiveness of the models. Our results showed that TRNMs could be constructed readily from traditional road networks without introducing large volumes of data, while effectively representing the time-dependent topological relationships among lanes. It is particularly beneficial to path planning, as it not only provides valid and shorter paths but also lane-level navigation information. Time-dependent road network models mirror real-world road networks and can represent more time-dependent traffic controls, such as non-periodic changes at different frequencies. The TRNM developed here can provide support for applications based on road network models, as well as a useful reference for the geographic information system (GIS) and complex networks.

A Digital Twin is a digital replica of a living or nonliving physical entity, and this emerging technology attracted extensive attention from different industries during the past decade. Although a few Digital Twin studies have been conducted in the transportation domain very recently, there is no systematic research with a holistic framework connecting various mobility entities together. In this study, a mobility digital twin (MDT) framework is developed, which is defined as an artificial intelligence (AI)-based data-driven cloud–edge–device framework for mobility services. This MDT consists of three building blocks in the physical space (namely,Human,Vehicle, andTraffic), and their associated Digital Twins in the digital space. An example cloud–edge architecture is built with Amazon Web Services (AWS) to accommodate the proposed MDT framework and to fulfill its digital functionalities of storage, modeling, learning, simulation, and prediction. A case study of the personalized adaptive cruise control (P-ACC) system is conducted, which integrates the key microservices of all three digital building blocks of the MDT framework: 1) theHuman Digital Twinwith user management and driver type classification; 2) theVehicle Digital Twinwith cloud-based advanced driver-assistance systems (ADAS); and 3) theTraffic Digital Twinwith traffic flow monitoring and variable speed limit. Future challenges of the proposed MDT framework are discussed toward the end of the article, including standardization, AI for computing, public or private cloud service, and network heterogeneity.

Recent work has considered personalized route planning based on user profiles, but none of it accounts for human trust. We argue that human trust is an important factor to consider when planning routes for automated vehicles. This article presents a trust-based route-planning approach for automated vehicles. We formalize the human-vehicle interaction as a partially observable Markov decision process (POMDP) and model trust as a partially observable state variable of the POMDP, representing the human’s hidden mental state. We build data-driven models of human trust dynamics and takeover decisions, which are incorporated in the POMDP framework, using data collected from an online user study with 100 participants on the Amazon Mechanical Turk platform. We compute optimal routes for automated vehicles by solving optimal policies in the POMDP planning and evaluate the resulting routes via human subject experiments with 22 participants on a driving simulator. The experimental results show that participants taking the trust-based route generally reported more positive responses in the after-driving survey than those taking the baseline (trust-free) route. In addition, we analyze the trade-offs between multiple planning objectives (e.g., trust, distance, energy consumption) via multi-objective optimization of the POMDP. We also identify a set of open issues and implications for real-world deployment of the proposed approach in automated vehicles.

Ramp merging is considered as one of the most difficult driving scenarios due to the chaotic nature in both longitudinal and lateral driver behaviors (namely lack of effective coordination) in the merging area. In this study, we have designed a cooperative ramp merging system for connected vehicles, allowing merging vehicles to cooperate with others prior to arriving at the merging zone. Different from most of the existing studies that utilize dedicated short-range communication, we adopt a Digital Twin approach based on vehicle-to-cloud communication. On-board devices upload the data to the cloud server through the 4G/LTE cellular network. The server creates Digital Twins of vehicles and drivers whose parameters are synchronized in real time with their counterparts in the physical world, processes the data with the proposed models in the digital world, and sends advisory information back to the vehicles and drivers in the physical world. A real-world field implementation has been conducted in Riverside, California, with three passenger vehicles. The results show the proposed system addresses the issues of safety and environmental sustainability with an acceptable communication delay, compared to the baseline scenario where no advisory information is provided during the merging process.

Advanced driver-assistance systems (ADAS) have matured over the past few decades with the dedication to enhance user experience and gain a wider market penetration. However, personalization features, as an approach to make the current technologies more acceptable and trustworthy for users, have been gaining momentum only very recently. In this work, we aim to learn personalized longitudinal driving behaviors via a Gaussian Process (GP) model. The proposed method learns from individual driver’s naturalistic car-following behavior, and outputs a desired acceleration profile that suits the driver’s preference. The learned model, together with a predictive safety filter that prevents rear-end collision, is used as a personalized adaptive cruise control (PACC) system. Numerical experiments show that GP-based PACC (GP-PACC) can almost exactly reproduce the driving styles of an intelligent driver model. Additionally, GP-PACC is further validated by human-in-the-loop experiments on the Unity game engine-based driving simulator. Trips driven by GP-PACC and two other baseline ACC algorithms with driver override rates are recorded and compared. Results show that on average, GP-PACC reduces the human override duration by 60% and 85% as compared to two widely-used ACC models, respectively, which shows the great potential of GP-PACC in improving driving comfort and overall user experience.

Ramp merging is considered to be one of the major causes of traffic accidents and congestion due to its inherent chaotic nature. With the development of the connected and automated vehicle (CAV) technology, CAVs can conduct cooperative merging using communication, and can also handle complicated situations even with legacy vehicles. In this article, a game theory-based ramp merging strategy has been developed for the optimal merging coordination of CAVs in mixed traffic, which can determine the dynamic merging sequence and corresponding longitudinal/lateral control. This strategy improves the safety and efficiency of the merging process by ensuring a safe intervehicle distance and harmonizing the speeds of CAVs in the traffic stream. To verify the proposed strategy, mixed traffic simulation runs under different penetration rates and different congestion levels have been carried out on an innovative Unity-SUMO integrated platform, which connects a game engine-based driving simulator with a state-of-the-art microscopic traffic simulator. The results show that the average speed of traffic flow can be increased up to 210%, while the fuel consumption can be reduced up to 53.9%. In addition, the driving volatility can be stabilized to a level with 0% extreme values.

With the rapid development of intelligent vehicles and Advanced Driving Assistance Systems (ADAS), a mixed level of human driver engagements is involved in the transportation system. Visual guidance for drivers is essential under this situation to prevent potential risks. To advance the development of visual guidance systems, we introduce a novel sensor fusion methodology, integrating camera image and Digital Twin knowledge from the cloud. Target vehicle bounding box is drawn and matched by combining results of object detector running on ego vehicle and position information from the cloud. The best matching result, with a 79.2% accuracy under 0.7 Intersection over Union (IoU) threshold, is obtained with depth image served as an additional feature source. Game engine-based simulation results also reveal that the visual guidance system could improve driving safety significantly cooperate with the cloud Digital Twin system.

Behavior prediction plays an essential role in both autonomous driving systems and Advanced Driver Assistance Systems (ADAS), since it enhances vehicle's awareness of the imminent hazards in the surrounding environment. Many existing lane change prediction models take as input lateral or angle information and make short-term (<; 5 seconds) maneuver predictions. In this study, we propose a longer-term (5~10 seconds) prediction model without any lateral or angle information. Three prediction models are introduced, including a logistic regression model, a multilayer perceptron (MLP) model, and a recurrent neural network (RNN) model, and their performances are compared by using the real-world NGSIM dataset. To properly label the trajectory data, this study proposes a new time-window labeling scheme by adding a time gap between positive and negative samples. Two approaches are also proposed to address the unstable prediction issue, where the aggressive approach propagates each positive prediction for certain seconds, while the conservative approach adopts a roll-window average to smooth the prediction. Evaluation results show that the developed prediction model is able to capture 75% of real lane change maneuvers with an average advanced prediction time of 8.05 seconds.

Our current transportation system suffers from a number of problems in terms of safety, mobility, and environmental sustainability. The emergence of innovative intelligent transportation systems (ITS) technologies, and in particular connected and automated vehicles (CAVs), provides many opportunities to address the aforementioned issues. In this paper, we propose a hierarchical ramp merging system that not only generates microscopic cooperative maneuvers for CAVs on the ramp to merge into the mainline traffic flow, but also provides controllability of the ramp inflow rate, thereby enabling macroscopic traffic flow control. A centralized optimal control-based approach is proposed to smooth the merging flow, improve the system-wide mobility, and decrease the overall fuel consumption of the network. Linear quadratic trackers in both finite horizon and receding horizon forms are developed to solve the optimization problem in terms of path planning and sequence determination, where a microscopic vehicle fuel consumption model is applied. Extensive traffic simulation runs have been conducted using PTV VISSIM to evaluate the impact of the proposed system on a segment of SR-91 E in Corona, California. The results confirm that under the regulated inflow rate, the proposed system can avoid potential traffic congestion and improve mobility (e.g., VMT/VHT) up to 147%, with a 47% fuel savings compared to the conventional ramp metering and the ramp without any control approach.

With the development of advanced communication technology, connected vehicles become increasingly popular in our transportation systems, which can conduct cooperative maneuvers with each other as well as road entities through vehicle-to-everything communication. A lot of research interests have been drawn to other building blocks of a connected vehicle system, such as communication, planning, and control. However, less research studies were focused on the human-machine cooperation and interface, namely how to visualize the guidance information to the driver as an advanced driver-assistance system (ADAS). In this study, we propose an augmented reality (AR)-based ADAS, which visualizes the guidance information calculated cooperatively by multiple connected vehicles. An unsignalized intersection scenario is adopted as the use case of this system, where the driver can drive the connected vehicle crossing the intersection under the AR guidance, without any full stop at the intersection. A simulation environment is built in Unity game engine based on the road network of San Francisco, and human-in-the-loop (HITL) simulation is conducted to validate the effectiveness of our proposed system regarding travel time and energy consumption.

Digital twin, an emerging representation of cyberphysical systems, has attracted increasing attentions very recently. It opens the way to real-time monitoring and synchronization of real-world activities with the virtual counterparts. In this study, we develop a digital twin paradigm using an advanced driver assistance system (ADAS) for connected vehicles. By leveraging vehicle-to-cloud (V2C) communication, on-board devices can upload the data to the server through cellular network. The server creates a virtual world based on the received data, processes them with the proposed models, and sends them back to the connected vehicles. Drivers can benefit from this V2C based ADAS, even if all computations are conducted on the cloud. The cooperative ramp merging case study is conducted, and the field implementation results show the proposed digital twin framework can benefit the transportation systems regarding mobility and environmental sustainability with acceptable communication delays and packet losses.

The emergence of connected and automated vehicle (CAV) technology has the potential to bring a number of benefits to our existing transportation systems. Specifically, when CAVs travel along an arterial corridor with signalized intersections, they can not only be driven automatically using pre-designed control models but can also communicate with other CAVs and the roadside infrastructure. In this paper, we describe a cooperative eco-driving (CED) system targeted for signalized corridors, focusing on how the penetration rate of CAVs affects the energy efficiency of the traffic network. In particular, we propose a role transition protocol for CAVs to switch between a leader and following vehicles in a string. Longitudinal control models are developed for conventional vehicles in the network and for different CAVs based on their roles and distances to intersections. A microscopic traffic simulation evaluation has been conducted using PTV VISSIM with realistic traffic data collected for the City of Riverside, CA, USA. The effects on traffic mobility are evaluated, and the environmental benefits are analyzed by the U.S. Environmental Protection Agency's MOtor Vehicle Emission Simulator (MOVES) model. The simulation results indicate that the energy consumption and pollutant emissions of the proposed system decrease, as the penetration rate of CAVs increases. Specifically, more than 7% reduction on energy consumption and up to 59% reduction on pollutant emission can be achieved when all vehicles in the proposed system are CAVs.

In[1], the traffic volume of the left-turn movement was set to be extremely high (with significant queue spill back from the left-turn bay) in the microscopic traffic simulator. Although some explanation has been provided in the paper, we consider the simulation results to be non-representative. Therefore, we reran the simulation with more balanced values across different movements, and updated the simulation results inTABLE Von page 8 of[1]with the one shown below.

Connected and automated vehicle (CAV) technology has become increasingly popular. As an example, Cooperative Adaptive Cruise Control (CACC) systems are of high interest, allowing CAVs to communicate and cooperate with each other to form platoons, where one vehicle follows another with a predefined spacing or time gap. Although numerous studies have been conducted on CACC systems, very few have examined the protocols from the perspective of environmental sustainability, not to mention from a platoon-wide consideration. In this study, we propose a vehicle-to-vehicle (V2V) communication based Eco-CACC system, aiming to minimize the platoon-wide energy consumption and pollutant emissions at different stages of the CACC operation. A full spectrum of environmentally-friendly CACC maneuvers are explored and the associated protocols are developed, including sequence determination, gap closing and opening, platoon cruising with gap regulation, and platoon joining and splitting. Simulation studies of different scenarios are conducted using MATLAB/Simulink. Compared to an existing CACC system, the proposed one can achieve additional 2% energy savings and additional 17% pollutant emissions reductions during the platoon joining scenario.