Level 3 automated driving systems (ADS) have attracted significant attention and are being commercialized. A level 3 ADS prompts the driver to take control by issuing a request to intervene (RtI) when its operational design domains (ODD) are exceeded. However, complex traffic situations can cause drivers to perceive multiple potential triggers of RtI simultaneously, causing hesitation or confusion during take-over. Therefore, drivers need to clearly understand the ADS's system limitations to ensure safe take-over. This study proposes a voice-based educational human machine interface~(HMI) for providing RtI trigger cues and reason to help drivers understand ADS's system limitations. The results of a between-group experiment using a driving simulator showed that incorporating effective trigger cues and reason into the RtI was related to improved driver comprehension of the ADS's system limitations. Moreover, most participants, instructed via the proposed method, could proactively take over control of the ADS in cases where RtI fails; meanwhile, their number of collisions was lower compared with the other RtI HMI conditions. Therefore, using the proposed method to continually enhance the driver's understanding of the system limitations of ADS through the proposed method is associated with safer and more effective real-time interactions with ADS.

Autonomous personal mobility vehicle (APMV) is an innovative small autonomous transportation device designed for individual use in mixed-traffic environments, such as shared spaces and indoor environments. To enhance the interaction experience between pedestrians and APMVs and to prevent potential risks, it is crucial to investigate pedestrians’ walking behaviors when interacting with APMVs and to understand the psychological processes underlying these behaviors. This study aims to investigate the causal relations between subjective evaluations of pedestrians and their walking behaviors during interactions with an APMV equipped with an external human-machine interface (eHMI). An experiment of pedestrian-APMV interaction was conducted with 42 pedestrian participants, in which various eHMIs on the APMV were designed to induce participants to experience different levels of subjective evaluations and generate the corresponding walking behaviors. Based on the hypothesized model of the pedestrian’s cognition-decision-behavior process, the results of causal discovery align with the previously proposed model. Furthermore, this study further analyzes the direct and total causal effects of each factor and investigates the causal processes affecting several important factors in the field of human-vehicle interaction, such as situation awareness, trust in vehicle, risk perception, hesitation in decision making, and walking behaviors.

As autonomous personal mobility vehicles (APMVs) are increasingly integrated into shared spaces, short-distance interactions between pedestrians and APMVs will become more frequent. To facilitate communication in shared spaces, APMVs equipped with external human-machine interfaces (eHMIs). Although the eHMI is primarily designed to communicate with pedestrians, its communication also affects the APMV passenger due to the short-distance interaction. This paper focused on the effect of passengers’ personality traits on their user experience when the APMV exhibits different eHMIs. An experiment was conducted in the field with 24 participants as APMV passengers who experienced three distinct eHMI types: eHMI-T (text-based), eHMI-NV (neutral voice-based), and eHMI-AV (affective voice-based). Through causal discovery analysis, our findings revealed that when the APMV is equipped with eHMI-T, various personality traits of passengers collectively influenced their user experience. In contrast, the eHMI-NV design demonstrated that personality traits had no direct influence on user experience. The eHMI-AV design primarily showed that agreeableness and extraversion negatively influenced concerns about drawing attention, which subsequently affected other user experience. Based on the results, this paper recommends designing different eHMIs based on the APMV ownerships, such as private or public shared APMVs.

Autonomous Personal Mobility Vehicles (APMVs) are designed to address the “last-mile” transportation challenge for everyone. When an APMV encounters a pedestrian, it uses an external Human-Machine Interface (eHMI) to negotiate road rights. Through this interaction, passengers are also passively exposed to the process. This study examines passengers' gaze behavior toward pedestrians during such interactions, focusing on whether passengers' personality traits influence their gaze patterns towards pedestrians when using different eHMI designs. When using a visual-based eHMI, which caused passengers to struggle in perceiving the communication content, the results suggested that passengers with higher Neuroticism scores, who were more sensitive to communication details, might seek cues from pedestrians' reactions. In addition, a multimodal eHMI (visual and voice) using neutral voice did not significantly affect the gaze behavior of passengers toward pedestrians, regardless of personality traits. In contrast, a multimodal eHMI using affective voice encouraged passengers with high Openness to Experience scores to focus on pedestrians' heads. In summary, this study revealed how different eHMI designs influence passengers' gaze behavior and highlighted the effects of personality traits on their gaze patterns toward pedestrians, providing new insights for personalized eHMI designs.

Autonomous personal mobility vehicle (APMV) is a miniaturized autonomous vehicle designed for short-distance mobility to everyone. Due to its open design, APMV’s passengers are exposed to communications between the external human-machine interface (eHMI) on APMV and pedestrians. Therefore, effective eHMI designs for APMV need to consider potential impacts of APMV-pedestrian interactions on passengers’ subjective feelings. This study from the perspective of APMV passengers discussed three eHMI designs: (1) graphical user interface (GUI)-based eHMI with text message (eHMI-T), (2) multimodal user interface (MUI)-based eHMI with neutral voice (eHMI-NV), and (3) MUI-based eHMI with affective voice (eHMI-AV). In a riding field experiment (N = 24), eHMI-T made passengers feel awkward during the “silent time” when eHMI-T conveyed information exclusively to pedestrians, not passengers. MUI-based eHMIs with voice cues showed advantages, with eHMI-NV excelling in pragmatic quality and eHMI-AV in hedonic quality. Considering passengers’ personalities and genders in APMV eHMI design is also highlighted.

Eye typing interfaces enable a person to enter text into an interface using only their own eyes. But despite the inherent advantages of touchless operation and intuitive design, such eye-typing interfaces often suffer from slow typing speeds, resulting in slow words per minute (WPM) counts. In this study, we add word and letter prediction to the eye-typing interface and investigate users’ typing performance as well as their subjective experience while using the interface. In experiment 1, we compared three typing interfaces with letter prediction (LP), letter + word prediction (L + WP), and no prediction (NoP), respectively. We found that the interface with L + WP achieved the highest average text entry speed (5.48 WPM), followed by the interface with LP (3.42 WPM), and the interface with NoP (3.39 WPM). Participants were able to quickly understand the procedural design for word prediction and perceived this function as very helpful. Compared to LP and NoP, participants needed more time to familiarize themselves with L + WP in order to reach a plateau regarding text entry speed. Experiment 2 explored training effects in L + WP interfaces. Two moving speeds were implemented: slow (6.4°/s same speed as in experiment 1) and fast (10°/s). The study employed a mixed experimental design, incorporating moving speeds as a between-subjects factor, to evaluate its influence on typing performance throughout 10 consecutive training sessions. The results showed that the typing speed reached 6.17 WPM for the slow group and 7.35 WPM for the fast group after practice. Overall, the two experiments show that adding letter and word prediction to eye-typing interfaces increases typing speeds. We also find that more extended training is required to achieve these high typing speeds.

This study takes a fresh perspective by focusing on the drivers of surrounding cars near to level 3 automated vehicles (AVs). We advocates for level 3 AVs using an external human-machine interface (eHMI) to provide high-risk warning information to drivers of surrounding cars during AVs issuing a request-to-intervene (RtI). Through a driving simulator-based subjects experiments, we have established that the proposed eHMI can assist the surrounding MV’s driver in better comprehending the AV’s driving intentions and predicting its driving behavior. This leads to increased the surrounding MV’s driver confidence in handling potential risks from the AV during the take-over period. Although we did not observe a significant impact of the proposed eHMI on the driving behavior of the MV drivers, i.e. participants, they reported a greater willingness to have AVs equipped with the proposed eHMI drive around them in their daily life.

This study aims to investigate the causal relationships from pedestrians’ psychological states to their walking behavior during interactions with an autonomous personal mobility vehicle (APMV) featuring automation capabilities ranging from SAE levels 3 to 5. A subjective experiment was conducted, where various external human-machine interfaces (eHMIs) were designed to induce participants to experience different levels of subjective feelings and generate corresponding walking behaviors. By employing a structural equation model named DirectLiNGAM to analyze the collected data for causal discovery, the results of causal discovery align with the hypothesized model of the pedestrian’s cognition-decision-behavior process. Furthermore, the experimental results have enriched the detailed causal relationships within the hypothesized model, i. e., the outcomes of situation awareness lead to a sense of danger, trust in APMV and a sense of relief; the outcomes of situation awareness, the sense of danger and trust in APMV lead to hesitation in decision-making; and the outcomes of situation awareness, the sense of danger and hesitation lead to walking behaviors.

Passengers of level 3-5 autonomous personal mobility vehicles (APMV) can perform non-driving tasks, such as reading books and smartphones, while driving. It has been pointed out that such activities may increase motion sickness, especially when frequently avoiding pedestrians or obstacles in shared spaces. Many studies have been conducted to build countermeasures, of which various computational motion sickness models have been developed. Among them, models based on subjective vertical conflict (SVC) theory, which describes vertical changes in direction sensed by human sensory organs v.s. those expected by the central nervous system, have been actively developed. To model motion sickness due to conflict between visual vertical information and vestibular sensation, we proposed a 6 DoF SVC-VV model which added a visually perceived vertical block into a conventional 6 DoF SVC model to predict visual vertical directions from image data simulating the visual input of a human. In a driving experiment, 27 participants rode on the APMV and experienced slalom driving with two visual conditions: looking ahead (LAD) and working with a tablet device (WAD). We verified that passengers got motion sickness while riding the APMV, and the symptoms were severer when especially working on it, by simulating the frequent pedestrian avoidance scenarios of the APMV in the experiment. In addition, the results of the experiment demonstrated that the proposed 6 DoF SVC-VV model could describe the increased motion sickness experienced when the visual vertical and gravitational acceleration directions were different.

Interactions between pedestrians and autonomous personal mobility vehicle (APMV) will increase with the popularity of autonomous driving systems. However, when the APMVs are applied in a mixed traffic environment after manual driving PMV (MPMV) have been popular, pedestrians may feel unsafe in the interactions when they are uncertain about the driving intention of the APMV. This study seeks to find a surrogate measure for pedestrians’ understanding of driving intention and perceived safety during the interaction with an APMV. We conducted an experiment to measure the gaze duration and subjective evaluations of the participants when they interacted with a PMV in manual and autonomous driving modes. Pedestrians fixed their gaze at the APMV longer when they did not accurately understand the driving intention than when they understood it. Furthermore, the pedestrians perceived danger when they did not clearly understand the driving intention of the APMV. Besides, these factors were different when pedestrians interact with an MPMV and an APMV.

Passengers (drivers) of level 3-5 autonomous personal mobility vehicles (APMV) and cars can perform non-driving tasks, such as reading books and smartphones, while driving. It has been pointed out that such activities may increase motion sickness. Many studies have been conducted to build countermeasures, of which various computational motion sickness models have been developed. Many of these are based on subjective vertical conflict (SVC) theory, which describes vertical changes in direction sensed by human sensory organs vs. those expected by the central nervous system. Such models are expected to be applied to autonomous driving scenarios. However, no current computational model can integrate visual vertical information with vestibular sensations. We proposed a 6 DoF SVC-VV model which add a visually perceived vertical block into a conventional six-degrees-of freedom SVC model to predict VV directions from image data simulating the visual input of a human. Hence, a simple image-based VV estimation method is proposed. As the validation of the proposed model, this paper focuses on describing the fact that the motion sickness increases as a passenger reads a book while using an AMPV, assuming that visual vertical (VV) plays an important role. In the static experiment, it is demonstrated that the estimated VV by the proposed method accurately described the gravitational acceleration direction with a low mean absolute deviation. In addition, the results of the driving experiment using an APMV demonstrated that the proposed 6 DoF SVC-VV model could describe that the increased motion sickness experienced when the VV and gravitational acceleration directions were different.

Human drivers (HDs) need to negotiate the right-of-way when they encounter in bottleneck roads. Since driver-less autonomous vehicles (AVs) in the foreseeable future will lead to mixed traffic, the communication between AV and HD may become more difficult than the two HDs communication. How to make an effective communication method to convey the intention of AV to HD is an unsolved problem. In this study, we investigate the usefulness of three HMI-based communication methods from the viewpoint of HDs, i.e., the internal HMI (iHMI) based on vehicle-to-vehicle communications on head-up display in manually driven vehicle, the external HMI (eHMI) on AV, as well as using both of them synchronously. Participants, as the HDs, experienced these three HMIs through videos on the online-survey and performed subjective evaluations. We found that although there were no significant differences in the subjective evaluation results of participants for the three HMIs, iHMI gave better evaluations of understanding, safety, and stress, while eHMI had the lowest evaluation of trust.

Compared to a manual driving vehicle (MV), an automated driving vehicle lacks a way to communicate with the pedestrian through the driver when it interacts with the pedestrian because the driver usually does not participate in driving tasks. Thus, an external human machine interface (eHMI) can be viewed as a novel explicit communication method for providing driving intentions of an automated driving vehicle (AV) to pedestrians when they need to negotiate in an interaction, e.g., an encountering scene. However, the eHMI may not guarantee that the pedestrians will fully recognize the intention of the AV. In this paper, we propose that the instruction of the eHMI's rationale can help pedestrians correctly understand the driving intentions and predict the behavior of the AV, and thus their subjective feelings (i. e., dangerous feeling, trust in the AV, and feeling of relief) and decision-making are also improved. The results of an interaction experiment in a road-crossing scene indicate that the participants were more difficult to be aware of the situation when they encountered an AV w/o eHMI compared to when they encountered an MV; further, the participants' subjective feelings and hesitation in decision-making also deteriorated significantly. When the eHMI was used in the AV, the situational awareness, subjective feelings and decision-making of the participants regarding the AV w/ eHMI were improved. After the instruction, it was easier for the participants to understand the driving intention and predict driving behavior of the AV w/ eHMI. Further, the subjective feelings and the hesitation related to decision-making were improved and reached the same standards as that for the MV.

Interaction detection between vehicles and vulnerable road users (e.g. pedestrians and cyclists) is important for e.g. safety control and autonomous driving. However, there are many challenges for automatically detecting interactions, such as the ambiguity of defining when interaction is required in dynamic traffic activities among different road users and the lack of labeled data for training a machine learning detector. To overcome the challenges, we introduce a way to define whether or not interaction is required in various traffic scenes and create a large real-world dataset from a very challenging intersection. A sequence-to-sequence method that uses the object information and motion information of the traffic scenes extracted by a state-of-the-art object detector and from optical flow, respectively, is proposed for automatic interaction detection. The proposed method generates a probability of interaction at each short interval (<; 0.1 s) that represents the changing of interaction along a sequence. We obtain a baseline model that differentiates no interaction from interaction on the basis of the location and road user type from the detected object information. Compared with the baseline model, the empirical results of the proposed method demonstrate very accurate predictions for vehicle turning sequences with varying length.

This paper presents a comparison of driving behavior modeling methods based on hidden Markov models (HMMs) with driver's eye-gaze measurement and ego-vehicle localization. Original HMMs are sometimes insufficient to model real-world scenarios. To overcome these limitations, extended HMMs have been proposed, e.g., autoregressive input-output HMMs (AIOHMMs). This paper first details AIOHMMs and presents ways to use them for driving behavior modeling. We compare the performance for behavior modeling and maneuver discrimination for six types of HMMs. The driving data for this work was gathered in our university campus with a car-like vehicle. Experimental results suggest that the hidden states can properly represent the average of the driving actions when the driving behaviors are accurately modeled by the HMMs. It is also suggested that surrounding and past information can be used to flexibly model the relationship between driving actions and related information.

In this paper, we propose a visualization method for driving behavior that helps people to recognize distinctive driving behavior patterns in continuous driving behavior data. Driving behavior can be measured using various types of sensors connected to a control area network. The measured multi-dimensional time series data are called driving behavior data. In many cases, each dimension of the time series data is not independent of each other in a statistical sense. For example, accelerator opening rate and longitudinal acceleration are mutually dependent. We hypothesize that only a small number of hidden features that are essential for driving behavior are generating the multivariate driving behavior data. Thus, extracting essential hidden features from measured redundant driving behavior data is a problem to be solved to develop an effective visualization method for driving behavior. In this paper, we propose using deep sparse autoencoder (DSAE) to extract hidden features for visualization of driving behavior. Based on the DSAE, we propose a visualization method called adriving color mapby mapping the extracted 3-D hidden feature to the red green blue (RGB) color space. A driving color map is produced by placing the colors in the corresponding positions on the map. The subjective experiment shows that feature extraction method based on the DSAE is effective for visualization. In addition, its performance is also evaluated numerically by using pattern recognition method. We also provide examples of applications that use driving color maps in practical problems. In summary, it is shown the driving color map based on DSAE facilitates better visualization of driving behavior.

In-vehicle speech-based interaction between a driver and a driving agent should be performed without affecting the driving behavior. A driving agent provides information to the driver and helps his/her driving behavior and non-driving-related tasks, e.g., selecting music and giving weather information. In this paper, we focus on a method for determining utterance timings when a driving agent provides non-driving-related information. If a driving agent provides a driver with non-driving-related information at an inappropriate moment, it will distract his/her driving behavior and deteriorate his/her safety driving. To solve or to mitigate the problem, we propose a novel method for determining the utterance timing of a driving agent on the basis of a double articulation analyzer, which is an unsupervised nonparametric Bayesian machine learning method for detecting contextual change points. To verify the effectiveness of the method, we conduct two experiments. One is an experiment on a short circuit around a park in an urban area, and the other is an experiment on a long course in a town. The results show that the proposed method enables a driving agent to avoid inappropriate timing better than baseline methods.

Driving behavior can be represented by many different types of measured sensor information obtained through a control area network. We assume that the measured sensor information is generated from several hidden time-series data through multiple nonlinear transformations. These hidden time-series data are statistically independent of each other and capture essential driving behavior. Driving behavior information is usually generated by multiple nonlinear transformations that fuse essential features, e.g., "Yaw rate" is generated by fusing the velocity of the vehicle and the change of driving direction. However, driving behavior data is often redundant because such data includes multivariate information and involves duplicated essential features. In this paper, we propose a feature extraction method to extract essential features from redundant driving behavior data using a deep sparse autoencoder (DSAE), which is a deep learning method. Two-dimensional features are extracted from seven-dimensional artificial data using a DSAE and are determined experimentally to be highly correlated with the prepared essential features. DSAEs are also used to extract features from an actual driving behavior data set. To verify a DSAE's ability to extract essential driving behavior features and filter out redundant information, we prepare twelve data sets that include some or all of the driving behavior information. Twelve DSAEs are used to independently extract features from the twelve prepared data sets, and canonical correlation analysis is used to analyze the canonical correlation coefficients between extracted features. Furthermore, we verify DSAEs' ability to extract essential driving behavior features from the redundant driving behavior data sets.

Driving behavioral data is too high-dimensional for people to review their driving behavior. It includes accelerator opening rate, steering angle, brake Master-Cylinder pressure and other various information. The high-dimensional data is not very intuitive for drivers to understand their driving behavior when they take a look back on their recorded driving behavior. We used a deep sparse autoencoder to extract the low-dimensional high-level representation from high-dimensional raw driving behavioral data obtained from a control area network. Based on this low-dimensional representation, we propose two visualization methods called Driving Cube and Driving Color Map. Driving Cube is a cubic representation displaying extracted three-dimensional features. Driving Color Map is a colored trajectory shown on a road map representing the extracted features. The trajectory is colored using the RGB color space, which corresponds to the extracted three-dimensional features. To evaluate the proposed method for extracting low-dimensional feature, we conducted an experiment and found several differences with recorded driving behavior by viewing the visualized Driving Color Map and that our visualization methods can help people to recognize different driving behavior. To evaluate the effectiveness of low-dimensional representation, we compared deep sparse autoencoder with other conventional methods from the viewpoint of linear separability of elemental driving behavior. As a result, our methods outperformed other conventional methods.