Large-scale scenario databases may contain hundreds of thousands of scenarios for the verification and validation (V&V) of autonomous vehicles (AV). Scenarios in the database are often labelled with semantic Operational Design Domain (ODD) tags (e.g., WeatherRainy, RoadTypeHighway and ActorTypeTruck) to be queried via exact tag matching. Such a scenario database design has two major limitations, i.e. combinatorial scenario generation inevitably leads to many redundant scenarios, and each ODD query matches only a small number of scenarios in the database (0.2% in our case study), rendering most of the database wealth wasted. We propose a novel scenario database design and the first ODD-based query-time scenario mutation framework to address the limitations. Our case study results show that the proposed framework has the potential to fully utilize all the database scenarios at query time while eliminating scenario redundancy in the database (in our case study, given the same ODD query, the number of final matched scenarios increased by 36 times, diversity increased by 99 times, and scenario database utilization rate increased from 0.2% to 36%).

This paper presents a novel solution to the demand of a realistic virtual test environment (VTE) for the development and safety assurance of Automated Driving Systems (ADSs). The current VTE offerings suffer limitations when it comes to creating a rich environment (from the Operational Design Domain perspective) based on the ASAM OpenDRIVE files (a formatted description of the scenery), and hence there is no automatic OpenDRIVE scenery generation available that resembles the richness required to thoroughly test an ADS in a virtual environment. Therefore, through the use of Unreal Engine, leveraging the power of esmini’s roadmanager and developing on a primitive OpenDRIVE plugin, a comprehensive scenery with complex lighting, dynamic environmental conditions and photoscanned meshes can be achieved. There is then a demonstration of how this is incorporated into an end-to-end testing framework, using just one user interface to take the user from the creation and retrieval of scenarios through to the execution.

Scenario-based testing for automated driving systems (ADS) is an industry norm for safety assurance. A scenario describes situations that an automated driving systems may encounter during its operation. To ensure accurate representation of real-world situations, including human behavior and system interactions, a formal language is essential. It ensures consistent testing across diverse scenarios and facilitates compatibility with simulation tools. However, while existing scenario languages excel in describing environmental and road structure aspects, they lack the same detail for road users and drivers. We have developed a methodology to identify and incorporate relevant human factors elements into scenario languages. Our methodology focuses on understanding diverse individuals and their interactions with ADS on the road, enabling their representation in scenarios. We offer practical examples to improve language representation of human elements and actions, in WMG-SDL Level-2 for logical scenarios and BSI Flex 1889 for abstract scenario descriptions. This methodology serves as a starting point for language designers to accurately represent all road users and their interactions with ADS.

Scenario-based verification and validation (V&V) has emerged as the predominant approach for the performance evaluation of automated driving systems (ADSs). Many scenario-generation methods have been proposed to search for critical scenarios, i.e. disengagement or traffic rule violations. However, the widely adopted binary (pass/fail) criterion suffers from two main limitations, i.e., the difficulty of locating root causes and the lack of statistical guarantee of testing sufficiency. Recently, new scenario engineering approaches focusing on the intelligence of ADSs enlightened a promising pathway via dynamic driving task decomposition and function atom constraints. However, none of the state-of-the-art scenario description languages support such approaches. To fill this gap and facilitate further research into this promising direction, in this work, we propose a generic architecture to extend the existing scenario description languages for the intelligence testing of ADSs. The case study with WMG SDL demonstrates the capability and flexibility of the proposed extension design in defining intelligence function constraints.

Automated Driving Systems (ADSs), like human drivers, must be compliant with the rules of the road. However, current rules of the road are not well defined. They use inconsistent and ambiguous language. As a result, they are not sufficiently formal for machine interpretability, a necessity for applications of verification and validation (V&V) of ADSs. Rules must be defined in a way that make them usable to a variety of stakeholders. While first-order and temporal logic forms of rules of the road are needed for monitoring and verification during simulation and testing, a structured natural language for these rules is necessary for consistent definition. They must also adhering to standard vocabulary taxonomies of Operational Design Domain (ODD) and behaviour. This paper contributes a structured natural language based on formal logic, that allows rules of the road to be defined in a natural, yet precise manner, using concepts of ODD and behaviour, making them usable in the V&V of ADSs. We evaluate the effectiveness of the language on a selection of rules from the Vienna Convention on Road Traffic and the UK Highway Code.

This paper presents a scenario-based safety assurance process for Automated Driving Systems (ADSs) combined with the Vehicle-to-Everything (V2X) connectivity aspect, system with connectivity capability is benchmarked with system without such capabilities. In addition, a novel approach to V2X modelling is introduced and implemented to obtain the required configuration of the V2X parameters for individual system, such modelling approach ensures that the V2X is effective within a system during testing by using a distance-based V2X parameter that correlates to the speed of the system. Such parameter is then used as the configuration of the V2X model when carrying out the ADS and V2X combined test. Using a pedestrian crossing scenario, a reduction in failure cases is demonstrated when combining the V2X and the ADS. Therefore, a synchronised approach of vehicle, sensor and communication sub-system can improve the overall safety function of the system.

The move towards connected and autonomous vehicles (CAVs) has gained a strong focus in recent years due to the many benefits they provide. While the autonomous aspect has seen substantial advancement in both the development and testing methodologies, the connected aspect has lagged behind, especially in the verification and validation (V&V) discussions. Integrating connectivity into the development and testing framework for CAVs is a necessity for ensuring the early deployment of cooperative driving systems. A key element within such a framework is a test scenario, which represents a set of scenery, environmental conditions, and dynamic conditions, that a system needs to be tested in. However, the connectivity element is not present in any of the current state of the art scenario description languages (SDLs) that are publicly available. This leaves a gap within the CAV development ecosystem. To accommodate for, and accelerate the development of, connected vehicle systems and their verification and validation methods, this paper proposes a novel V2X extension to the previously published two-level abstraction SDL. The extension enables communications between vehicles, infrastructures, and further additional entities to be specified as part of the scenario and be subsequently tested in virtual testing or real-world testing. Eight new V2X attributes have been added to the SDL. An example set of syntax and semantic definitions are presented in this paper targeting two different abstraction levels – level 1 aims at the abstract scenario level for non-technical end-users such as regulators, and level 2 aims at the logical and concrete scenario level for end-users such as simulation test engineers.

One of the main challenges for the introduction of Automated Driving Systems (ADSs) is their verification and validation (V&V). Simulation based testing has been widely accepted as an essential aspect of the ADS V&V processes. Simulations are especially useful when exposing the ADS to challenging driving scenarios, as they offer a safe and efficient alternative to real world testing. It is thus suggested that evidence for the safety case for an ADS will include results from both simulation and real-world testing. However, for simulation results to be trusted as part of the safety case of an ADS for its safety assurance, it is essential to prove that the simulation results are representative of the real world, thus validating the simulation platform itself. In this paper, we propose a novel methodology for validating the simulation environments focusing on comparing point cloud data from real LiDAR sensor and a simulated LiDAR sensor model. A 3D object dissimilarity metric is proposed to compare between the two maps (real and simulated), to quantify how accurate the simulation is. This metric is tested on collected LiDAR point cloud data and the simulated point cloud generated in the simulated environment.

For Automated Driving Systems (ADSs), vehicle safety and functional correctness are assessed against scenarios the ADS would encounter - within or outside its operational design domain (ODD). A scenario specifies conditions and events that an ADS is expected to respond to when deployed. Scenario specifications underpin the verification and validation (V&V) life-cycle, and are used by a diverse set of stakeholders - from engineers to regulators. Due to the diversity in stakeholder expertise, scenarios must be available at different levels of detail. Further, the chance of writing syntactically or semantically incorrect scenarios is high. Due to the high reliance of V&V on scenarios, their correctness is crucial. Therefore, present-day scenario-description languages (SDLs) need to be supported by technologies to help authors compose scenarios and provide a mechanism for easy translation into executable forms for virtual or real-life testing. This paper addresses these issues by building on the existing two-level abstraction WMG-SDL in the following ways, (1) introducing a human-readable, natural language SDL, replacing the former Level-1 SDL, and complementing the more detailed Level-2 SDL, which is now syntax aligned with the ODD Taxonomy defined in ISO 34503, and (2) providing a tool consisting of a parser and a validator to assist writing syntactically and semantically correct scenarios. Our tool may be used within a graphical scenario editing interface or on the command-line. Further, for developers, an object-oriented interface for parsed scenarios enables further development and integration with off-the-shelf ADS simulation and language tools. The tools and technologies described in this paper are to be made open-source.

Behaviour prediction function of an autonomous vehicle predicts the future states of the nearby vehicles based on the current and past observations of the surrounding environment. This helps enhance their awareness of the imminent hazards. However, conventional behavior prediction solutions are applicable in simple driving scenarios that require short prediction horizons. Most recently, deep learning-based approaches have become popular due to their promising performance in more complex environments compared to the conventional approaches. Motivated by this increased popularity, we provide a comprehensive review of the state-of-the-art of deep learning-based approaches for vehicle behavior prediction in this article. We firstly give an overview of the generic problem of vehicle behavior prediction and discuss its challenges, followed by classification and review of the most recent deep learning-based solutions based on three criteria: input representation, output type, and prediction method. The article also discusses the performance of several well-known solutions, identifies the research gaps in the literature and outlines potential new research directions.

Partially automated vehicles present a large range of information to the driver in order to keep them in-the-loop and engaged with monitoring the vehicle’s actions. However, existing research shows that this causes cognitive overload and disengagement from the monitoring task. Adaptive Human Machine Interfaces (HMIs) are an emerging technology that might address this problem, by prioritising the information presented. To date, research aiming to define the driver’s glance fixation behaviour in a partially automated vehicle to contribute towards an adaptive interface is scarce. This study used a unique three-day longitudinal driving simulator study design to explore which information drivers in a partially automated vehicle require. Twenty-seven participants experienced nine partially automated driving simulations over three consecutive days. Nine information types, developed from standards, previous studies and industry collaboration, were displayed as discrete icons and presented on a surrogate in-vehicle display. Unique to the literature, this study showed that the recorded eye-tracking data demonstrated that usage of the information types changed with longitudinal driving simulator use. This study provides three key contributions: first, the longitudinal study design suggest that single exposure HMI evaluations may be limited in their assessment. Secondly, this study has methodologically shortlisted a list of nine information types that can be used in future studies to represent future partially automated vehicle interfaces. Finally, this is one of the first studies to characterise glance behaviour for partially automated vehicles. With this knowledge, this study contributes important design recommendations for the development of adaptive interfaces.

Identification of test scenarios for Automated Driving Systems (ADSs) remains a key challenge for the Verification & Validation of ADSs. Various approaches including data based approaches and knowledge based approaches have been proposed for scenario generation. Identifying the conditions that lead to high severity traffic accidents can help us not only identify test scenarios for ADSs, but also implement measures to save lives and infrastructure resources. Taking a data based approach, in this paper, we introduce a novel accident data analysis method for generating test scenarios where we analyze UK's Stats19 accident data to identify trends in high severity accidents for test scenario generation. This paper first focuses on the severity of the accidents with the goal of relating it to static and time-dependent internal and external factors in a comprehensive way taking into account Operational Design Domain (ODD) properties, e.g. road, environmental conditions, and vehicle properties and driver characteristics. For this purpose, the paper utilizes a data grouping strategy (coarse-graining) and builds a logistic regression approach, derived from conventional regression models, in which emerging features become more pronounced, while uninteresting features and noise weaken. The approach makes the relationship between the factors and outcome variable more visible and hence well suited for the severity analysis. The method shows superior performance as compared to ordinary logistic models measured by goodness of fit and accounting for model variance$(R^{2}=0.05$for the ordinary model,$R^{2}=0.85$for the current model). The model is then used to solve the inverse problem of constructing high-risk pre-crash conditions as test scenarios for simulation based testing of ADSs.

The development of Automated Driving Systems (ADSs) is driven by the many benefits they offer. However, the complexities associated with ADSs and their interactions with the environment pose challenges for their safety assurance. A key aspect during its development process is knowing the capabilities, limitations, and being able to convey them in a clear manner for various types of stakeholders. The Operational Design Domain (ODD) concept was introduced to define the operating boundaries where a system can operate safely. It is therefore a key element for the safety assurance of ADSs. Efforts have been made to define the scope and the content an ODD for ADSs should cover, however there remains the need for a common, exchangeable, executable, and human-readable format for the description. This paper presents a language for the description of the ODD of ADSs, in a textual format that leans on natural language influence. Such format is intended to be both human and machine-readable and would be relevant to end users such as regulators and systems designers. The two-level abstraction approach – a structured natural language representation and a formal representation (covered across two papers) -- has been developed to have the ability to describe complex ODD conditionalities and utilize a well-defined domain ontology to achieve rich semantics. It is aimed to support ODD related activities throughout the development cycle of ADSs (specification as well as verification and validation), while covering a diverse range of stakeholders.

A formal representation for the Operational Design Domain (ODD) of Automated Driving Systems (ADSs) is presented in this paper. An ODD specification determines for every situation whether it is included or excluded from the ODD. We focus on methods to provide unambiguous specification in a programmatic format which are simultaneously machine and human readable. We present a logical framework with intuitive and well-defined semantics which directly supports safety engineering process through specification stage to defining uncertainty and acceptable risk. Its rich and diverse feature set include 1) parsimonious permissive and restrictive constraints, 2) the ability to import OWL ontologies, 3) ability to bind to non-uniform complex variable length structures within situation data, and 4) ability for components to control the scope of usage by integrators. The presentation of syntax and formal semantics is illustrated with example demonstrating key concepts and language capabilities. \n

Latest generations of automobiles are gradually being equipped with technologies that have increasing automation, a trend which had led to increase in the system complexity as well as increased human-automation interactions. Failures in such complex human-automation interactions increasingly occur due to the mismatch between what operators know about the system and what the designers expect operators to know. Causes of road accidents also change due to role shift of drivers from controlling the vehicle to monitoring the in-vehicle controllers. Failures in such complex systems involving human-automation interactions increasingly occur due to the emergent behaviours from the interactions, and are less likely due to reliability of individual components. Traditional safety analysis methods fall short in identifying such emergent failures. This paper focuses on using a systems thinking inspired safety analysis method called System Theoretic Process Analysis (STPA) to identify potential failures. The analysis focuses on a SAE Level-4 Vehicle that is in the development phase, and is controlled partially by a safety driver and its built-in Autonomous Driving System (ADS). The analysis yields that while increase in complexity does increase system functionality, it also brings a challenge to evaluate the safety of the system and potentially causes incorrect human-automation interactions, leading to an accident. After the possible inadequate driver-vehicle interactions are identified by STPA, corresponding requirements were then proposed in order to avoid the unsafe behaviour and thus preventing the hazards.

The complexities associated with Automated Driving Systems (ADSs) and their interaction with the environment pose a challenge for their safety evaluation. Number of miles driven has been suggested as one of the metrics to demonstrate technological maturity. However, the experiences or the scenarios encountered by the ADSs is a more meaningful metric, and has led to a shift to scenario-based testing approach in the automotive industry and research community. Variety of scenario generation techniques have been advocated, including real-world data analysis, accident data analysis and via systems hazard analysis. While scenario generation can be done via these methods, there is a need for a scenario description language format which enables the exchange of scenarios between diverse stakeholders (as part of the systems engineering lifecycle) with varied usage requirements. In this paper, we propose a two-level abstraction approach to scenario description language (SDL) - SDL level 1 and SDL level 2. SDL level 1 is a textual description of the scenario at a higher abstraction level to be used by regulators or system engineers. SDL level 2 is a formal machine-readable language which is ingested by testing platform e.g. simulation or test track. One can transform a scenario in SDL level 1 into SDL level 2 by adding more details or from SDL level 2 to SDL level 1 by abstracting.

This paper simulates traffic at the Dartford-Thurrock Crossing Tunnel, Kent, UK. Using a traffic simulation model, Connected and Autonomous Freight Vehicles (CAV-F) are simulated alongside conventional light goods vehicles, to determine the feasibility of increasing the traffic throughput at the tunnel. The results show that with the use of CAV-F, the overall traffic flow is increased by --33% from current flow of --5,000 vehicles/hr. With the reduction in the headway and standstill distance and increase in scope of intelligent connectivity and traffic speed limit, the average congestion and travel time are reduced even at a higher traffic concentration. By analysing the results, it has thus been possible to highlight the benefits to traffic management and road utilisation by introducing CAV-F into our road network, in the long term.

Localization accuracy obtainable from global navigation satellites systems in built up areas like urban canyons and multi-storey car parks is severely impaired due to multipath and non-line-of-sight signal propagation. In this paper, a simple classifier was used in discriminating between multipath and line-of-sight GNSS signals. By using the carrier to noise ratio which characterizes the received signal strength of the GNSS signals, and the rate of change of the epochs of the satellite vehicles in view, a prediction accuracy of 98% was attained from the classifier. Also investigated in this paper is the effect of antenna placement on localization accuracy. Our measurement campaign using a Nissan Leaf hatch back model showed that the centre longitudinal line of the roof generated the least localization errors for an urbanized route.

Understanding how best to present information inside a semi-automated vehicle is a prevalent challenge in HMI design. There is an understanding that a driver's trust and previous driving experience can affect the information they require inside a semi-automated vehicle. However, to date little is known about how these predispositions specifically affect the types of information that should be presented and importantly, how this changes with increased exposure to an automated system. In this paper, seventeen participants experienced twenty-six minutes of an automated driving simulation once every day for a week. The information to display was carefully chosen in accordance with the Skills, Rules, Knowledge model. The information was synchronized to the driving simulation and presented on a tablet in the driving simulator. Eye tracking was used to measure the information looked at. The results showed that trust increased significantly with increased exposure, but this had no correlation to any specific piece of information viewed. Drivers who were more prone to making lapses or errors (as measured by the Driver Behavior Questionnaire) tended towards using information that was less cognitively demanding. Finally, a driver's propensity to making lapses was found to be a potential early predictor of trust, but this became less accurate with increased exposure to the semi-automated vehicle.

This paper presents results from a series of focus groups, aimed at enhancing technical engineering system requirements, for a public transport system, encompassing a fleet of platooning low-speed autonomous vehicles (LSAV; aka pods) in urban areas. A critical review of the pods was conducted, as part of a series of technical workshops, to examine the key areas of the system that could affect users and other stakeholders, such as businesses and the public. These initial findings were used to inform a series of focus groups, aimed at identifying the public's views of multiple autonomous vehicles being deployed in a pedestrianised area that can join and form platoons. Analysis of findings from the focus groups suggests that while people view platooning public transport vehicles favourably as a passenger, they have some concerns from a pedestrian perspective. Thematic analysis was applied to these findings and a systematic approach was used to identify where subjective outputs could be formalised to inform requirements. Finally, a step-by-step requirements elicitation process is presented that illustrates the method used to convert qualitative user data to objective engineering requirements.

With increasing automation, occupants of fully autonomous vehicles are likely to be completely disengaged from the driving task. However, even with no driving involved, there are still activities that will require interfaces between the vehicle and passengers. This study evaluated different configurations of screens providing operational-related information to occupants for tracking the progress of journeys. Surveys and interviews were used to measure trust, usability, workload and experience after users were driven by an autonomous low speed pod. Results showed that participants want to monitor the state of the vehicle and see details about the ride, including a map of the route and related information. There was a preference for this information to be displayed via an onboard touchscreen device combined with an overhead letterbox display versus a smartphone-based interface. This paper provides recommendations for the design of devices with the potential to improve the user interaction with future autonomous vehicles.

It is important to equip future vehicles with an onboard system capable of tracking and analyzing driver state in real-time in order to mitigate the risk of human error occurrence in manual or semi-autonomous driving. This study aims to provide some supporting evidence for adoption of consumer grade electronic devices in driver state monitoring. The study adopted repeated measure design and was performed in high-fidelity driving simulator. Total of 39 participants of mixed age and gender have taken part in the user trials. The mobile application was developed to demonstrate how a mobile device can act as a host for a driver state monitoring system, support connectivity, synchronization, and storage of driver state related measures from multiple devices. The results of this study showed that multiple physiological measures, sourced from consumer grade electronic devices, can be used to successfully distinguish task complexities across common driving activities. For instance, galvanic skin response and some heart rate derivatives were found to be correlated to overall subjective workload ratings. Furthermore, emotions were captured and showed to be affected by extreme driving situations.

The purpose of this paper is to bring together multiple literature sources which present innovative methodologies for the assessment of driver state, driving context and performance by means of technology within a vehicle and consumer electronic devices. It also provides an overview of ongoing research and trends in the area of driver state monitoring. As part of this review a model of a hybrid driver state monitoring system is proposed. The model incorporates technology within a vehicle and multiple brought-in devices for enhanced validity and reliability of recorded data. Additionally, the model draws upon requirement of data fusion in order to generate unified driver state indicator(-s) that could be used to modify in-vehicle information and safety systems hence, make them driver state adaptable. Such modification could help to reach optimal driving performance in a particular driving situation. To conclude, we discuss the advantages of integrating hybrid driver state monitoring system into a vehicle and suggest future areas of research.

This paper presents the results from a preliminary study where a wearable consumer electronic device was used to assess driver's state by capturing human physiological response in non-intrusive manner. Majority of state of the art studies have employed medical equipment drivers' state evaluation. Despite the potential gain in road safety this method of measuring physiology is unlikely to be accepted by private vehicle consumers due to its invasiveness, complexity, and high cost. This study was aiming to investigate possibility of employing a consumer grade wearable device to measure physiological parameters related to cognitive workload in real-time while driving i.e., drivers' heart rate. Furthermore, validity of captured heart activity metrics was analyzed to determine if wearable devices could be embedded into driving at its current technological state. The driving context was reproduced in desktop driving simulator, with 14 participants agreeing to take part in the study (μ = 28, σ = 8.5 years). Drivers were exposed to various road types, including pure Motorway, Rural, and Urban scenario modes. An accident was simulated in order to generate sudden cognitive arousal and capture participants' physiological response to the generated distress. It was found that a smartwatch is capable of reliable heart activity tracking in driving context. The results, supporting the relationship between cognitive workload level, generated by various complexity driving tasks, and Heart Rate Variability, were also presented.

Recently there has been a growth in the incorporation of autonomous features within vehicles. From being perceived as a comfort feature, autonomous features in vehicles have now become a safety feature which are foreseen to reduce accidents. This has led to a new trend within the automotive industry of focussing on autonomous features for driver safety, which might ultimately lead to fully autonomous vehicles. Considering the fact that most of the accidents on UK roads occur due to driver error, driver-less vehicles would prove to be a benefit. However with automation, an even greater challenge of system validation in all scenarios needs to be addressed. For this, various methods of validation have been developed by different research organizations and manufacturers, but a standardized process still evades the industry. Some of the existing methods have been discussed in this paper to critically compare their quality of results and ease of execution. Subsequently, a new test platform has been proposed using the 3xD driving simulator which encompasses most requirements of a general testing method. A standardized process which would benefit the industry both in terms of reducing costs of having varied processes, and by increasing customer confidence can be developed using a non-invasive platform like the 3xD driving simulator. The novelty of the 3xD simulator is the ability to drive-in any vehicle (production/prototype) and develop testing methodologies in an immersive wireless environment.

This paper gives an overview of the new Drive-in Driver-in-Loop simulator at WMG, University of Warwick, UK, which has been conceptualized to serve as a standard platform for virtual verification and validation of autonomous features. This front loading approach is key to the development of some of the upcoming technologies which have been lagging behind in terms of mass acceptance due to lack of proper simulation environment for testing. One of the key areas for the simulator is the study of driver acceptance of autonomous features in cars. Additionally, the simulator would help in the development of a validation methodology for autonomous systems keeping in mind the industry safety regulations and standards. The drive-in component on this scale adds to the novelty of the simulator, as it's a first of its kind. This enhances the challenge of making the communication interfaces of the simulator general, in order to communicate with any vehicle driving into the simulator. In order to achieve this, emphasis was laid on software architecture to enable modularity and re-configuration. A brief about various applications of the simulator has been provided in this paper.

A smart driving system (providing both safety and fuel-efficient driving advice in real time in the vehicle) was evaluated in real-world on-road driving trials to see if any measurable beneficial changes in driving performance would be observed. Forty participants drove an instrumented vehicle over a 50-min mixed-route driving scenario. Two conditions were adopted: one is a control with no smart driving feedback offered and the other is with advice being presented to the driver via a smartphone in the vehicle. Key findings from the study showed a 4.1% improvement in fuel efficiency when using the smart driving aid, importantly with no increase in journey time or reduction in average speed. Primarily, these efficiency savings were enabled by limiting the use of lower gears (facilitated by planning ahead to avoid unnecessary stops) and an increase in the use of the fifth gear (as advised by the in-vehicle system). Significant and important changes in driving safety behaviors were also observed, with an increase in mean headway to 2.3 s and an almost threefold reduction in time spent traveling closer than 1.5 s to the vehicle in front. This paper has shown that an in-vehicle smart driving system specifically developed and designed with the drivers' information requirements in mind can lead to significant improvements in driving behaviors in the real world on real roads with real users.

Laws mandate that electric vehicles emit sounds to ensure pedestrians' safety by alerting pedestrians of the vehicles' approach. Additionally, manufacturers want these sounds to promote positive impressions of the vehicle brand. A reliable and valid methodology is needed to evaluate electric vehicles' exterior sounds. To help develop such a methodology, this paper examines automotive exterior sound evaluation methods in the context of experimental design and cognitive psychology. Currently, such evaluations are usually conducted on road or inside a laboratory; however, a virtual environment provides advantages of both these methods but none of their limitations. The stimuli selected for evaluations must satisfy legislative guidelines. Methods for presenting and measuring the stimuli can affect study outcomes. A methodology is proposed for conducting evaluations of an electric vehicle's exterior sounds, testing its detectability and emotional evaluation. An experiment tested the methodology. Thirty-one participants evaluated an electric car in a virtual environment of a town's T-junction with 15 exterior sounds as stimuli. The car's arrival time, direction of approach, and, thus, distance to pedestrian varied across conditions. Detection time of the sound and pleasantness and powerfulness evaluations of the car were recorded. The vehicle's arrival time and approach direction affected its detectability and emotional evaluation; thus, these are important elements to vary and control in studies. Overall, the proposed methodology increases the realistic context and experimental control than in existing listening evaluations. It benefits by combining two competing elements necessary for assessing electric vehicle exterior sounds, namely, pedestrians' safety and impressions of the vehicle brand.