Panoramic view generation and articulation angle estimation are essential for enabling active safety and control in complex dynamic systems such as long combination vehicles (LCVs). Traditional approaches rely on mechanical sensors for angle measurement, which are susceptible to noise and require frequent maintenance, potentially degrading lateral control performance and causing misalignment in panoramic views. Moreover, panoramic view generation for LCVs remains underexplored due to their extended length and complex articulated structures. To address these challenges, this study proposes a real-time, weak-supervised framework with offline training and online inference modules to achieve simultaneous panoramic view generation and articulation angle estimation. In the offline stage, the angle estimation network is trained using noisy angle data as weak supervision, guided by image-level self-supervision signals like photometric consistency. In the online stage, the network directly infers articulation angles from raw fisheye image pairs, eliminating the need for preprocessing or computationally expensive online optimization. Field experiments on a three-carbody-articulated LCV validate the effectiveness of the framework in accurately estimating articulation angles and generating holistic, color-balanced panoramic views under varying conditions. The system addresses information loss and photometric inconsistency, especially during large-angle turns, offering a practical, sensor-independent vision solution to enhance safety and deployment flexibility in complex vehicle systems.

For better lateral dynamic stability of an A-double vehicle (comprising a tractor, semitrailer, dolly, and another semitrailer) at high speeds, it is a practical and economical way to control the dolly, a linking structure for long combination vehicles (LCVs). This paper proposes a simple differential brake control strategy based on the feedback of the yaw rate of the dolly. A stability analysis approach is employed by exploring the relationship between damping ratios and speeds, and the critical speed of an A-double vehicle is found. Next, the relationship between the A-double vehicle's damping ratios and the feedback parameter employed in this differential brake control approach is examined, showing that a more favorable feedback value can raise the vehicle's minimum damping ratio. Finally, the method of selecting feedback parameters is also provided, considering the adhesion utilization demanded by the dolly. The time-domain simulation confirms that the stability analysis utilizing the damping ratio is accurate. Using MATLAB/Simulink at various speeds, the effectiveness of this control strategy is tested under single lane-change operations. According to these simulation results, the A-double vehicle using the differential control strategy has a higher critical speed and maintains stability during single lane-change operations at high speeds. This verified that the simple differential brake control strategy has the potential to significantly increase the lateral dynamic stability of the A -double vehicle at high speeds during single-lane change operations economically and practically.

Articulation angle plays a significant role in determining the motion of a complex dynamic system such as a multi-articulated vehicle. By engineering practice, articulation angles are measured using mechanical angle sensors that are delicate to physical damage. To overcome this problem, this study proposed a supervised articulation angle estimation method based on the panoramic camera system of multi-articulated vehicles. By constructing neural network that takes images of surrounding environment captured by spatially adjacent cameras as input, and takes temporal dependency as well as data imbalanced distribution into consideration, we show that the proposed vision-only method could make accurate estimations either on collected dataset or field experiment. Results of our experiments verified the validity and feasibility of the proposed method in playing as an alternative to mechanical angle sensors without bringing additional hardware setting expenses.

In-vehicle human-machine interface (HMI) plays a significant role in accomplishing effective interactions between driving automation systems and drivers, especially during the transition of control. For this reason, different in-vehicle HMIs have been designed to convey additional semantic information from the driving automation systems to the drivers to realize safer, smoother, and better control transitions. This review summarizes and analyses 86 previously published studies that researched the effects of additional semantic information delivered through in-vehicle HMIs just before, during and right after takeover request (TOR). The additional semantic information mentioned in this review refer to the information beyond simple alerts to not only gain drivers’ attention but also additionally communicate contextual content and explanation to the drivers regarding its own purpose. In this review, the additional semantic information are categorized according to their purposes and effects into three aspects: mode awareness enhancement, situation awareness enhancement, and takeover maneuver assistance. The specificities and the corresponding concerns when applying additional semantic information to in-vehicle HMIs have been detailed analyzed throughout the entire article. Further suggestions are proposed for what should be carefully considered when adding additional information for better takeover. Prospects into future in-vehicle HMI possibilities are also raised that could be applied in both academic research and industry.

Although latent hazard notification for highly automated driving is expected to enhance traffic safety, its practical effects have yet to be verified. This study systemically investigated the expected safety benefits and driver behavioral adaptation based on structural equation modeling. First, we developed a notification system to inform drivers of latent hazards with auditory alerts and conducted a driving simulation experiment involving eyes-off-road situations. To test the system, we adopted two types of events (i.e., the collision avoidance function working or failure) in which latent hazards transform into immediate risks. Then, a measurement model was developed to evaluate driver trust, driver attention, and traffic safety. Subsequently, we examined the corresponding causal relationships. On the one hand, latent hazard notification significantly improves driver attention (i.e., more fixations on latent hazards, less engagement in non-driving-related tasks, and faster notice of immediate risks), which significantly enhances traffic safety. On the other hand, latent hazard notification significantly increases driver trust, which lowers driver attention and consequently impairs traffic safety. This causality reveals driver behavioral adaptation, although driver trust does not directly affect traffic safety. Overall, we find that latent hazard notification for highly automated driving can improve traffic safety, but the consequent driver behavioral adaptation impairs 15.12% of the expected safety benefits.

For highly automated vehicles, effective take-over performance measures are essential for establishing quantitative take-over models and exploring approaches to improve take-over performance. However, there is a lack of comprehensive take-over performance measures that suitably combine multiple objective metrics based on an average evaluation from human drivers. In this study, we proposed a human-centered comprehensive measure of take-over performance (HCMTP). There are four main building blocks for the HCMTP. First, we adopted sparse principal component analysis to identify the main aspects of take-over performance based on multiple original objective take-over performance metrics. Second, we developed a scale of take-over performance assessment to obtain drivers’ original subjective self-assessments of take-over performance. Third, we established nonlinear individual mapping functions to acquire different drivers’ evaluation criteria for take-over performance. Fourth, we proposed a relabeling algorithm to obtain drivers’ average evaluation of take-over performance. To verify the effectiveness of the HCMTP, we conducted a verification experiment involving 68 participants. The results indicate that the HCMTP is effective and able to reduce the interference of individual differences, stochasticity, and data imbalance. This study contributes to identifying the main aspects of take-over performance, systematically understanding how human drivers subjectively evaluate take-over performance, and evaluating drivers’ take-over performance comprehensively.

As human-machine collaborative driving systems, highly automated driving vehicles require human drivers to take over when take-over requests are triggered. Extensive studies have shown that drivers’ take-over performance is affected by their fatigue state, traffic conditions, and the take-over time budget (TB). However, there is still a paucity of a systematic understanding of how these factors affect take-over performance, which prevents the implementation of adaptive take-over systems. This study establishes a highly accurate take-over performance prediction model to systematically explore the effects of these factors on take-over performance and to propose an adaptive TB adjustment strategy for highly automated driving vehicles. First, we propose metrics to evaluate drivers’ fatigue states and the relative positions of surrounding traffic. Second, a generalized additive model is established to predict take-over performance and accurately evaluate the influence of the aforementioned factors on take-over performance. Based on the model, we propose an adaptive adjustment strategy of the TB for take-over systems and demonstrate its effectiveness by a verification experiment. This study contributes to understanding the influence of drivers’ passive fatigue states, the relative positions of surrounding traffic, and the TB on drivers’ take-over performance as well as to the development of adaptive take-over systems for highly automated vehicles.

Due to the growing interest in automated driving, a deep understanding on the characteristics of human driving behavior is critical for human-like autonomous vehicles. Among various driving behaviors, lane change is the most important one for vehicle lateral driving safety. This study proposes an unsupervised method to extract and discover the behavioral patterns of lane change maneuvers for the purpose of exploring the composed behavioral patterns during lane change. This method involves two phases: Firstly, the lane change sequences will be segmented into blocks using time-series segmentation algorithms. Three segmentation algorithms were utilized in this study. In the second phase, the segments will be clustered to find the corresponding behavioral pattern of each segment. Two extended latent Dirichlet allocation (LDA) models were adopted to cluster the segments. The combination of different segmentation and clustering algorithms were evaluated and compared by employing entropy and perplexity as the evaluation criteria. Collected lane change data from naturalistic driving were applied to examine its effectiveness. The results show that this method could effectively mine descriptive behavioral patterns from lane change data. This study provides a promising data mining solution to facilitating deep and comprehensive understanding on driver lane change behaviors, which will promote the development of human-like autonomous vehicles.

Cooperative driving of human driver and automated system can effectively reduce the necessity of extremely accurate environment perception of highly automated vehicles, and enhance the robustness of decision-making and motion control. However, due to the two players’ different intentions, severe conflicts may exist during the cooperation, which often result in negative consequences on driving safety and maneuverability. This paper presents an indirect shared control method to model the situation and improve the driving performance, which focus on the affine input nonlinear vehicle dynamic system for shared controller design under the framework of non-zero sum differential game. The Nash equilibria strategy indicates the best response for the automated system, which can guide the automated controller to act more safely and comfortably. Aimed to obtain fast solution for practical application, approximate dynamic programming is utilized to find the Nash equilibria, which is represented by deep neural networks and solved iteratively. Driver-in-the-loop tests on a driving simulator were conducted to verify the performance of the proposed method under highway driving scenarios. The results show that the designed controller is able to reduce the driving workload and ensure the driving safety.

Accurate trajectory prediction of surrounding road users is critical to autonomous driving systems. In mixed traffic flows, road users with different kinds of behaviors and styles bring complexity to the environment, which requires considering interactions among road users when anticipating their future trajectories. This paper presents a long-term interactive trajectory prediction method for surrounding vehicles using a hierarchical multi-sequence learning network. In contrast to non-interactive method which assumes that road users are independent of each other, this method can automatically learn high-level dependencies among multiple interacting vehicles through the proposed structural-LSTM (long short-term memory) network. Specifically, structural-LSTM first assigns one LSTM for each interacting vehicle. Then these LSTMs share their cell states and hidden states with their spatial-neighboring LSTMs by a radial connection, and recurrently analyze the output state of itself as well as the other LSTMs in a deeper layer. Finally based on all output states, the network predicts trajectories for surrounding vehicles. The proposed method is evaluated on the NGSIM dataset, and its results show that satisfyingly accurate prediction performance of long-term trajectories of surrounding vehicles is accessible, e.g., longitudinal and lateral RMS error can be reduced to less than 1.93m and 0.31m over 5s time horizon, respectively.

Detection of driver cognitive distraction is critical for active safety systems of road vehicles. Compared with visual distraction, cognitive distraction is more challenging for detection due to the lack of apparent exterior features. This paper presents a novel real-time detection algorithm for driver cognitive distraction by using support vector machine (SVM). Data are collected from 26 subjects, driving in typical urban and highway scenarios in a simulator. The chosen urban scenario is the stop-controlled intersection and the highway scenario is the speed-limited highway. Driver cognitive distraction while driving is induced by clock tasks which compete with the main driving tasks for visuospatial short working memory. For each subject, distracted driving instances and the equal number of non-distracted driving instances were collected (24 for urban scenario and 20 for highway scenario in total). Features concerning both driving performance and eye movement are used for training and validation. The proposed algorithm have correct rate of 93.0% and 98.5% for highway and urban scenarios respectively. Results also show that driver distraction can be recognized 6.5 s to 9.0 s after its happening, indicating good performance of the detection algorithm.

Driver distraction has been identified as one major cause of unsafe driving. The existing studies on cognitive distraction detection mainly focused on high-speed driving situations, but less on low-speed traffic in urban driving. This paper presents a method for the detection of driver cognitive distraction at stop-controlled intersections and compares its feature subsets and classification accuracy with that on a speed-limited highway. In the simulator study, 27 subjects were recruited to participate. Driver cognitive distraction is induced by the clock task that taxes visuospatial working memory. The support vector machine (SVM) recursive feature elimination algorithm is used to extract an optimal feature subset out of features constructed from driving performance and eye movement. After feature extraction, the SVM classifier is trained and cross-validated within subjects. On average, the classifier based on the fusion of driving performance and eye movement yields the best correct rate and F-measure (correctrate = 95.8 ± 4.4%; for stop-controlled intersections and correct rate = 93.7 ± 5.0%; for a speed-limited highway) among four types of the SVM model based on different candidate features. The comparisons of extracted optimal feature subsets and the SVM performance between two typical driving scenarios are presented.

Lane change maneuver recognition is critical in driver characteristics analysis and driver behavior modeling for active safety systems. This paper presents an enhanced classification method to recognize lane change maneuver by using optimized features exclusively extracted from vehicle state and driver operation signals. The sequential forward floating selection (SFFS) algorithm was adopted to select the optimized feature set to maximize the k-nearest-neighbor classifier performance. The hidden Markov models (HMMs), based on the optimized feature set, were developed to classify driver lane change and lane keeping maneuvers. Fifteen drivers participated in the road test for validation with an accumulation of 2,200 km naturalistic driving data, from which 372 lane changes were extracted. Results show that the recognition rate of lane change maneuver achieves 88.2%. The numbers are 87.6% and 88.8% for left and right lane change maneuvers, respectively, superior to the results from conventional classifiers.

Driver distraction has been identified as an important driving safety issue. However, existed studies focused less on low-speed condition, especially at intersections. This paper aims to find the impact of driver cognitive distraction on vehicle performance at stop-controlled intersections. Eight subjects (young adult: 4, older adult: 4) participated in this study and each of them drove through 40 stop-controlled intersections. The intersections were presented randomly at two levels of FOV (field of view). Driver cognitive distraction was induced by a one-back task and a clock task. Results showed that the cognitive tasks led to more abrupt steering in both age groups while significant influence on lane-keeping capability was only observed in the young group. Steering smoothness was mainly influenced by the cognitive tasks at brake on-restart phase in the young group while at after-restart phase in the older group. Impaired longitudinal control (stop for watching) was observed in the older adult group. These findings can be applied to automatically recognize driver distraction at stop-controlled intersections in future.