This editorial serves as an extended introduction to the Special Issue on Technologies for Risk Mitigation and Support of Impaired Drivers. It gives the context to recent advances in assisted and automated driving and the new challenges that arise when modern technology meets human users. The Special Issue focuses on the development of robust sensors and detection algorithms for driver state monitoring of fatigue, stress, and inattention, and on the development of personalized multimodal, user-oriented, and adaptive information, warning, actuation, and handover strategies. A summary of more recent developments serves as a motivation for each article that follows.

This paper presents initial work on a context-dependent driver distraction detection algorithm called AttenD2.0, which extends the original AttenD algorithm with elements from the Minimum Required Attention (MiRA) theory. Central to the original AttenD algorithm is a time buffer which keeps track of how often and for how long the driver looks away from the forward roadway. When the driver looks away the buffer is depleted and when looking back the buffer fills up. If the buffer runs empty the driver is classified as distracted. AttenD2.0 extends this concept by adding multiple buffers, thus integrating situation dependence and visual time-sharing behaviour in a transparent manner. Also, the increment and decrement of the buffers are now controlled by both static requirements (e.g. the presence of an on-ramp increases the need to monitor the sides and the mirrors) as well as dynamic requirements (e.g., reduced speed lowers the need to monitor the speedometer). The algorithm description is generic, but a real-time implementation with concrete values for different parameters is showcased in a driving simulator experiment with 16 bus drivers, where AttenD2.0 was used to ensure that drivers are attentive before taking back control after an automated bus stop docking and depot procedure. The scalability of AttenD2.0 relative to available data sources and the level of vehicle automation is demonstrated. Future work includes expanding the concept to real-world environments by automatically integrating situational information from the vehicles environmental sensing and from digital maps.

Hours of service regulations govern the working hours of commercial motor vehicle drivers, but these regulations may become more flexible as highly automated vehicles have the potential to afford periods of in-cab rest or even sleep while the vehicle is moving. A prerequisite is robust continuous monitoring of when the driver is resting (to account for reduced time on task) or sleeping (to account for the reduced physiological drive to sleep). The overall aims of this paper are to raise a discussion of whether it is possible to obtain successful rest during automated driving, and to present initial work on a hypothetical data driven algorithm aimed to estimate if it is possible to gain driving time after resting under fully automated driving. The presented algorithm consists of four central components, a heart rate-based relaxation detection algorithm, a camera-based sleep detection algorithm, a fatigue modelling component taking time awake, time of day and time on task into account, and a component that estimates gained driving time. Real-time assessment of driver fitness is complicated, especially when it comes to the recuperative value of in-cab sleep and rest, as it depends on sleep quality, time of day, homeostatic sleep pressure and on the activities that are carried out while resting. The monotony that characterizes for long-haul truck driving is clearly interrupted for a while, but the long-term consequences of extended driving times, including user acceptance of the key stakeholders, requires further research.

Driver fatigue is a contributing factor in about 20% of all fatal road crashes worldwide. Countermeasures are urgently needed and one of the most promising and currently available approaches for that are in-vehicle systems for driver fatigue detection. The main objective of this paper is to present a video-based driver sleepiness detection system set up as a two-stage model with (1) a generic deep feature extraction module combined with (2) a personalised sleepiness detection module. The approach was designed and evaluated using data from 13 drivers, collected during naturalistic driving conditions on a motorway in Sweden. Each driver performed one 90-minute driving session during daytime (low sleepiness condition) and one session during night-time (high sleepiness condition). The sleepiness detection model outputs a continuous output representing the Karolinska Sleepiness Scale (KSS) scale from 1–9 or a binary decision as alert (defined as KSS 1–6) or sleepy (defined as KSS 7–9). Continuous output modelling resulted in a mean absolute error (MAE) of 0.54 KSS units. Binary classification of alert or sleepy showed an accuracy of 92% (sensitivity = 91.7%, specificity = 92.3%, F1 score = 90.4%). Without personalisation, the corresponding accuracy was 72%, while a standard fatigue detection PERCLOS-based baseline method reached an accuracy of 68% on the same dataset. The developed real-time sleepiness detection model can be used in the management of sleepiness/fatigue by detecting precursors of severe fatigue, and ultimately reduce sleepiness-related road crashes by alerting drivers before high levels of fatigue are reached.

Driver sleepiness is a contributing factor in many road fatalities. A long-standing goal in driver state research has therefore been to develop a robust sleepiness detection system. It has been suggested that various heart rate variability (HRV) metrics can be used for driver sleepiness classification. However, since heart rate is modulated not only by sleepiness but also by several other time-varying intra-individual factors such as posture, distress, boredom and relaxation, it is relevant to highlight not only the possibilities but also the difficulties involved in HRV-based driver sleepiness classification. This paper investigates the reliability of HRV as a standalone feature for driver sleepiness detection in a realistic setting. Data from three real-road driving studies were used, including 86 drivers in both alert and sleep-deprived conditions. Subjective ratings based on the Karolinska sleepiness scale (KSS) were used as ground truth when training four binary classifiers ( k-nearest neighbours, support vector machine, AdaBoost, and random forest). The best performance was achieved with the random forest classifier with an accuracy of 85%. However, the accuracy dropped to 64% for three-class classification and to 44% for subject-independent, leave-one-participant-out classification. The worst results were obtained in the severely sleepy class. The results show that in realistic driving conditions, subject-independent sleepiness classification based on HRV is poor. The conclusion is that more work is needed to control for the many confounding factors that also influence HRV before it can be used as input to a driver sleepiness detection system.

The objective of this paper is to investigate if signal analysis and machine learning can be used to develop an accurate sleepiness warning system. The developed system was trained using the supposedly most reliable sleepiness indicators available, extracted from electroencephalography, electrocardiography, electrooculography, and driving performance data (steering behavior and lane positioning). Sequential forward floating selection was used to select the most descriptive features, and five different classifiers were tested. A unique data set with 86 drivers, obtained while driving on real roads in real traffic, both in alert and sleep deprived conditions, was used to train and test the classifiers. Subjective ratings using the Karolinska sleepiness scale (KSS) was used to split the data as sufficiently alert (KSS ≤ 6) or sleepy (KSS ≥ 8). KSS = 7 was excluded to get a clearer distinction between the groups. A random forest classifier was found to be the most robust classifier with an accuracy of 94.1% (sensitivity 86.5%, specificity 95.7%). The results further showed the importance of personalizing a sleepiness detection system. When testing the classifier on data from a person that it had not been trained on, the sensitivity dropped to 41.4%. One way to improve the sensitivity was to add a biomathematical model of sleepiness amongst the features, which increased the sensitivity to 66.2% for participant-independent classification. Future works include taking contextual features into account, using classifiers that takes full advantage of sequential data, and to develop models that adapt to individual drivers.

Automated analyses of electroencephalographic (EEG) signals acquired in naturalistic environments are becoming increasingly important in areas such as brain-computer interfaces and behavior science. However, the recorded EEG in such environments is often heavily contaminated by motion artifacts and eye movements. This poses new requirements on artifact handling. The objective of this paper is to present an automated EEG artifacts handling algorithm, which will be used as a preprocessing step in a driver monitoring application. The algorithm, named Automated aRTifacts handling in EEG (ARTE), is based on wavelets, independent component analysis, and hierarchical clustering. The algorithm is tested on a dataset obtained from a driver sleepiness study including 30 drivers and 540 30-min 30-channel EEG recordings. The algorithm is evaluated by a clinical neurophysiologist, by quantitative criteria (signal quality index, mean square error, relative error, and mean absolute error), and by demonstrating its usefulness as a preprocessing step in driver monitoring, here exemplified with driver sleepiness classification. All results are compared with a state-of-the-art algorithm called FORCe. The quantitative and expert evaluation results show that the two algorithms are comparable, and that both algorithms significantly reduce the impact of artifacts in recorded EEG signals. When artifact handling is used as a preprocessing step in driver sleepiness classification, the classification accuracy increased by 5% when using ARTE and by 2% when using FORCe. The advantage with ARTE is that it is data driven and does not rely on additional reference signals or manually defined thresholds, making it well suited for use in dynamic settings where unforeseen and rare artifacts are commonly encountered.

Indicators based on visual time-sharing have been used to investigate drivers' visual behaviour during additional task execution. However, visual time-sharing analyses have been restricted to additional tasks with well-defined temporal start and end points and a dedicated visual target area. We introduce a method to automatically extract visual time-sharing sequences directly from eye tracking data. This facilitates investigations of systems, providing continuous information without well-defined start and end points. Furthermore, it becomes possible to investigate time-sharing behavior with other types of glance targets such as the mirrors. Time-sharing sequences are here extracted based on between-glance durations. If glances to a particular target are separated by less than a time-based threshold value, we assume that they belong to the same information intake event. Our results indicate that a 4-s threshold is appropriate. Examples derived from 12 drivers (about 100 hours of eye tracking data), collected in an on-road investigation of an in-vehicle information system, are provided to illustrate sequence-based analyses. This includes the possibility to investigate human-machine interface designs based on the number of glances in the extracted sequences, and to increase the legibility of transition matrices by deriving them from time-sharing sequences instead of single glances. More object-oriented glance behavior analyses, based on additional sensor and information fusion, are identified as the next future step. This would enable automated extraction of time-sharing sequences not only for targets fixed in the vehicle's coordinate system, but also for environmental and traffic targets that move independently of the driver's vehicle.

Driver distraction is a contributing factor to many crashes; therefore, a real-time distraction warning system should have the potential to mitigate or circumvent many of these crashes. The objective of this paper is to investigate the usefulness of a real-time distraction detection algorithm called AttenD. The evaluation is based on data from an extended field study comprising seven drivers who drove on an average of 4351 ± 2181 km in a naturalistic setting. Visual behavior was investigated both on a global scale and on a local scale in the surroundings of each warning. An increase in the percentage of glances at the rear-view mirror and a decrease in the amount of glances at the center console were found. The results also show that visual time sharing decreased in duration from 9.94 to 9.20 s due to the warnings, that the time from fully attentive to warning decreased from 3.20 to 3.03 s, and that the time from warning to full attentiveness decreased from 6.02 to 5.46 s. The limited number of participants does not allow any generalizable conclusions, but a trend toward improved visual behavior could be observed. This is a promising start for further improvements of the algorithm and the warning strategy.

Driver distraction and driver inattention are frequently recognized as leading causes of crashes and incidents. Despite this fact, there are few methods available for the automatic detection of driver distraction. Eye tracking has come forward as the most promising detection technology, but the technique suffers from quality issues when used in the field over an extended period of time. Eye-tracking data acquired in the field clearly differs from what is acquired in a laboratory setting or a driving simulator, and algorithms that have been developed in these settings are often unable to operate on noisy field data. The aim of this paper is to develop algorithms for quality handling and signal enhancement of naturalistic eye- and head-tracking data within the setting of visual driver distraction. In particular, practical issues are highlighted. Developed algorithms are evaluated on large-scale field operational test data acquired in the Sweden-Michigan Field Operational Test (SeMiFOT) project, including data from 44 unique drivers and more than 10 000 trips from 13 eye-tracker-equipped vehicles. Results indicate that, by applying advanced data-processing methods, sensitivity and specificity of eyes-off-road glance detection can be increased by about 10%. In conclusion, postenhancement and quality handling is critical when analyzing large databases with naturalistic eye-tracking data. The presented algorithms provide the first holistic approach to accomplish this task.

The 3rdheart sound (S3) is normally heard during auscultation of younger individuals, but it is also common in many patients with heart failure. Compared to the 1stand 2ndheart sounds, S3 has low amplitude and low frequency content, making it hard to detect (both manually for the physician and automatically by a detection algorithm). We present an algorithm based on a recurrence time statistic which is sensitive to changes in a reconstructed state space, particularly for detection of transitions with very low energy. Heart sound signals from ten children were used in this study. Most S3 occurrences were detected (98%), but the amount of false extra detections was rather high (7% of the heart cycles). In conclusion, the method seems capable of detecting S3 with high accuracy and robustness

Recurrence plots (RP) visualize multi-dimensional state spaces and represent the recurrence of states of a system. Recurrence points can be divided into true recurrence points and false recurrence points (also called sojourn points). We introduce the true recurrence point recurrence plot, TRP, a variant of the traditional RP excluding the sojourn points. This is a cleaned up RP free from recurrence points originating from tangential motion, and hence a more robust representation of unstable periodic orbits. The method is demonstrated with three simple systems, a periodic sine wave, a quasi-periodic torus and the x-component of the chaotic Lorenz system

Heart sounds (HS) obscure the interpretation of lung sounds (LS). This letter presents a new method to detect and remove this undesired disturbance. The HS detection algorithm is based on a recurrence time statistic that is sensitive to changes in a reconstructed state space. Signal segments that are found to contain HS are removed, and the arising missing parts are replaced with predicted LS using a nonlinear prediction scheme. The prediction operates in the reconstructed state space and uses an iterated integrated nearest trajectory algorithm. The HS detection algorithm detects HS with an error rate of 4% false positives and 8% false negatives. The spectral difference between the reconstructed LS signal and an LS signal with removed HS was 0.34/spl plusmn/0.25, 0.50/spl plusmn/0.33, 0.46/spl plusmn/0.35, and 0.94/spl plusmn/0.64 dB/Hz in the frequency bands 20-40, 40-70, 70-150, and 150-300 Hz, respectively. The cross-correlation index was found to be 99.7%, indicating excellent similarity between actual LS and predicted LS. Listening tests performed by a skilled physician showed high-quality auditory results.