This Camera-only 3D occupancy estimation aims to cost-effectively reconstruct the occupancy state of a grid of voxels in a three-dimensional space, based on input from several cameras. One of the limitations of this approach is detecting occupied voxels of objects located far away, since the cameras resolution at such distances is relatively low. In this work, we boost performance by introducing gaze map estimation. Specifically, we show that although no additional sensor is used, gaze map estimation is strong enough and can be used to enhance basic occupancy estimation networks, yielding better Chamfer distance (CD), F-score and intersection over union (IoU) metrics. At long distances, we found an improvement over the baseline of more than 20% in CD, 24% in F-score and 15% in IoU.

In the process of training Neural Networks, pre-training is an unsupervised training process that uses automatically generated labels for real end-goal task inputs. It usually precedes a supervised training stage, can improve neural network performance, and can reduce training loss. In this work, we used pre-training in the automotive domain where the setup was composed of a camera aimed outside the vehicle and an eye tracking system observing the driver. Our pre-training process used images from the camera as input and eye gaze direction as the automatic generated label. The eye gaze pre-training goal was to initiate and thus improve disparity estimation networks.Selecting eye gaze as labels is the best of both worlds. On one hand, it is somewhat similar to supervised training. Labels are generated by humans, by drivers who have deep understanding of the scenes and driving situation. On the other hand, it is similar to unsupervised training. The labels can be generated automatically. Large quantities of data can be collected easily. Overall, the eye gaze pre-train helped reduce the L1 loss from 0.65 when not using pre-train to 0.45 when using it on the validation set.

During training, artificial neural networks might not converge to a global minimum. Usually, using gradient descent, the training procedure cause the network to stroll in the high-dimensional weights’ space. This stroll passes adjacently to local minima and locations in the geometry of loss landscape associated with low loss. Overall, the network moves from one low loss area to a lower loss area.In this work, we explored those low loss areas and minima, and tried to understand them. A U-Net was trained based on a gaze prediction task. A network was presented with images of different scenes, and the purpose of the network was to predict the expected human gaze distribution over those images. The driving task was selected since it involves relatively strong goal-oriented behaviors. It was shown that the training had two stages: (1) At the beginning, the network selected area was associated with saliency distributions (bottom-up behavior); (2) Later, the network selected area had the characteristics of goal-oriented distributions (top-down behavior) and it shifted away from the saliency distributions.

We describe a novel model of human eye gaze behavior under workload, derived from the basic principle of information constrained control. The model assumes two distributions over the visual field: A saliency distribution, which is nongoal oriented, and a reward task-related distribution. The eye gaze behavior is determined by the tradeoff between these two distributions, where the goal is to preserve the task-related constraints, while remaining as close as possible to the saliency distribution representing a comfort zone. Based on minimum Kullback-Liebler divergence principles, the model gives rise to a family of gaze distributions controlled by a single tradeoff parameter. The model was evaluated experimentally in a driving simulator that consisted of an immersive environment with clear tasks and accurate monitoring capabilities. The findings confirm the theoretical predictions with respect to the low rank manifold and order relations in the data. We show that the model can be used to visualize the unknown reward function associated with a task, and predict human workload based on gaze pattern.

Preventive maintenance is essential for the smooth operation of any equipment. Still, people occasionally do not maintain their equipment adequately. Maintenance alert systems attempt to remind people to perform maintenance. However, most of these systems do not provide alerts at the optimal timing, and nor do they take into account the time required for maintenance or compute the optimal timing for a specific user. We model the problem of maintenance performance, assuming maintenance is time consuming. We solve the optimal policy for the user, i.e., the optimal timing for a user to perform maintenance. This optimal strategy depends on the value of user's time, and thus it may vary from user to user and may change over time. %We present a game Based on the solved optimal strategy we present a personalized maintenance agent, which, depending on the value of user's time, provides alerts to the user when she should perform maintenance. In an experiment using a spaceship computer game, we show that receiving alerts from the personalized alert agent significantly improves user performance.

Drivers' situation awareness is known to be remarkably low in the automated driving mode, which can result in a delayed and inefficient response when requested to resume control of the vehicle. The present study examined the usefulness of a haptic seat that projects spatial information on approaching vehicles to facilitate drivers' preparedness to take control of the vehicle. The results of a simulator study on 26 participants using behavioral and eye tracking techniques showed that when required to regain control, having haptic seat led to faster reactions in scenarios requiring lane changing. The haptic seat also reduced the probability that the participants would slow down below acceptable speeds on a freeway. Eye tracking showed that drivers had a more systematic scan of the environment in the first two seconds following the transition of control with a haptic seat. Overall, these findings suggest that the haptic seat can play a significant role in keeping drivers aware of surrounding traffic during automated driving, and consequently facilitate the control transitions between the vehicle and the driver.

Reducing energy consumption of climate control systems is important in order to reduce human environmental footprint. The need to save energy becomes even greater when considering an electric car, since heavy use of the climate control system may exhaust the battery. In this paper we consider a method for an automated agent to provide advice to drivers which will motivate them to reduce the energy consumption of their climate control unit. Our approach takes into account both the energy consumption of the climate control system and the expected comfort level of the driver. We therefore build two models, one for assessing the energy consumption of the climate control system as a function of the system’s settings, and the other, models human comfort level as a function of the climate control system’s settings. Using these models, the agent provides advice to the driver considering how to set the climate control system. The agent advises settings which try to preserve a high level of comfort while consuming as little energy as possible. We empirically show that drivers equipped with our agent which provides them with advice significantly save energy as compared to drivers not equipped with our agent.

Adaptive Cruise Control (ACC) is a technology that allows a vehicle to automatically adjust its speed to maintain a preset distance from the vehicle in front of it based on the driver’s preferences. Individual drivers have different driving styles and preferences. Current systems do not distinguish among the users. We introduce a method to combine machine learning algorithms with demographic information and expert advice into existing automated assistive systems. This method can save on the interactions between drivers and automated systems by adjusting parameters relevant to the operation of these systems based on their specific drivers and context of drive. We also learn when users tend to engage and disengage the automated system. This method sheds light on the kinds of dynamics that users develop while interacting with automation and can teach us how to improve these systems for the benefit of their users. While accepted packages such as Weka were successful in learning drivers’ behavior, we found that improved learning models could be developed by adding information on drivers’ demographics and a previously developed model about different driver types. We present the general methodology of our learning procedure and suggest applications of our approach to other domains as well.

We examine how people perceive visual properties of new concepts for the design of animated vehicle instrument clusters, with emphasis on aesthetic aspects. The project is placed in the context of animations for eco-conscious driving. It consists of two stages: Creating animations and studying drivers' reactions to them. Two studies were conducted which provide various insights regarding tradeoff in the design process and drivers' preferences. The second study also serves as a first step towards the study of people's aesthetic perceptions of in-vehicle animations.

Empirical data modeling can be used to model human performance and explore the relationships between diverse sets of variables. A major challenge of empirical data modeling is how to generalize or extrapolate the findings with a limited amount of observed data to a broader context. In this paper, we introduce an approach from machine learning, known as support vector regression (SVR), which can help address this challenge. To demonstrate the method and the value of modeling human performance with SVR, we apply SVR to a real-world human factors problem of night vision system design for passenger vehicles by modeling the probability of pedestrian detection as a function of image metrics. The results indicate that the SVR-based model of pedestrian detection shows good performance. Some suggestions on modeling human performance by using SVR are discussed.

Natural language speech user interfaces offer a compelling choice of user interaction for the automotive market. With the increasing number of domains in which speech applications are applied, drivers must currently memorize many command words to control traditional speech interfaces. In contrast, natural language interfaces demand only a basic understanding of the system model instead of memorizing keywords and predefined patterns. To utilize natural language interfaces optimally, designers need to better comprehend how people utter their requests to express their intentions. In this study, we collected a corpus of utterances from users who interacted freely with an automotive natural language speech application. We analyzed the corpus by employing a corpus linguistic technique. As a result, natural language utterances can be classified into three components: information data, context relevant words, and non context relevant vocabulary. Applying this classification, users tended to repeat similar utterance patterns composed from a very limited set of different words. Most of the vocabulary in longer utterances was found to be non context restrictive providing no information. Moreover, users could be distinguished by their language patterns. Finally, this information can be used for the development of natural language speech applications. Some initial ideas are discussed in the paper.

It is well established in the literature that secondary tasks adversely affect driving behavior. Previous research has focused on discovering the general trends by analyzing the average effects of secondary tasks on a population of drivers. This paper conjectures that there may also be individual effects, i.e., different effects of secondary tasks on individual drivers, which may be obscured within the average behavior of the population, and proposes a model-based approach to analyze them. Specifically, a radial-basis neural-network-based modeling framework is developed to characterize the normal driving behavior of a driver when driving without secondary tasks. The model is then used in a scenario of driving with a secondary task to predict the hypothetical actions of the driver, had there been no secondary tasks. The difference between the predicted normal behavior and the actual distracted behavior gives individual insight into how the secondary tasks affect the driver. It is shown that this framework can help uncover the different effects of secondary tasks on each driver, and when used together with support vector machines, it can help systematically classify normal and distracted driving conditions for each driver.

This paper introduces a queueing network-based computational model to explain driver performance in a pedestrian-detection task assisted with night-vision-enhancement systems. The computational cognitive model simulated the pedestrian-detection task using images displayed by two night-vision systems as input stimuli. The system equipped with a far-infrared (FIR) sensor generated less-cluttered images than the system equipped with a near-infrared (NIR) sensor. Using a reinforcement learning process, the model developed eye-movement strategies for each night-vision system. The differences in eye-movement strategies generated different eye-movement behaviors, in accord with the empirical findings.

This paper describes two studies in which two night-vision enhancement systems were examined to compare nighttime driver performance in pedestrian detection. In the first study, the levels of clutter in the images displayed by the two types of night-vision enhancement systems were measured objectively and subjectively. The subjective ratings of clutter changed as a power function of the objective measure of clutter intensity. In the second study, the effect of clutter on glance behavior during pedestrian detection was examined in a driving simulator. Night-vision images with less clutter required shorter search times and fewer glances to detect the pedestrian, but the duration of each glance remained relatively constant.

The primary purpose of night-vision systems in civilian vehicles is to help drivers detect pedestrians. Pedestrian detection distance with night-vision systems has been modeled based on image metrics. However, the probability of pedestrian detection, in particular considering the factor of distance, has not been modeled based on image metrics. In this paper, we first describe a model of the probability of pedestrian detection, which compares several combinations of image-based clutter, contrast, and pedestrian size metrics using a simple mathematical equation. Next, we describe a model of the probability of pedestrian detection as a function of distance and image-based metrics by combining the model of pedestrian-detection probability and a model that represents the relationship between the distance to a pedestrian and an image-based pedestrian size metric. In the final model, image-based metrics are used to predict pedestrian-detection performance and can also be used to evaluate and support the development of night-vision systems in vehicles.

The risk of vehicle collisions significantly increases when drivers are overloaded with information from in-vehicle systems. One of the solutions to this problem is developing adaptive workload management systems (AWMSs) to dynamically control the rate of messages from these in-vehicle systems. However, existing AWMSs do not use a model of the driver cognitive system to estimate workload and only suppress or redirect in-vehicle system messages, without changing their rate based on driver workload. In this paper, we propose a prototype of a new queueing network-model human processor AWMS (QN-MHP AWMS), which includes a queueing network model of driver workload that estimates the driver workload in several driving situations and a message controller that determines the optimal delay times between messages and dynamically controls the rate of messages presented to drivers. Given the task information of a secondary task, the QN-MHP AWMS adapted the rate of messages to the driving conditions (i.e., speeds and curvatures) and driver characteristics (i.e., age). A corresponding experimental study was conducted to validate the potential effectiveness of this system in reducing driver workload and improving driver performance. Further development of the QN-MHP AWMS, including its use in in-vehicle system design and possible implementation in vehicles, is discussed.

Queueing Network-Model Human Processor (QN-MHP) is a computational architecture that integrates two complementary approaches to cognitive modeling: the queueing network approach and the symbolic approach (exemplified by the MHP/GOMS family of models, ACT-R, EPIC, and SOAR). Queueing networks are particularly suited for modeling parallel activities and complex structures. Symbolic models have particular strength in generating a person's actions in specific task situations. By integrating the two approaches, QN-MHP offers an architecture for mathematical modeling and real-time generation of concurrent activities in a truly concurrent manner. QN-MHP expands the three discrete serial stages of MHP, of perceptual, cognitive, and motor processing, into three continuous-transmission subnetworks of servers, each performing distinct psychological functions specified with a GOMS-style language. Multitask performance emerges as the behavior of multiple streams of information flowing through a network, with no need to devise complex, task-specific procedures to either interleave production rules into a serial program (ACT-R), or for an executive process to interactively control task processes (EPIC). Using QN-MHP, a driver performance model was created and interfaced with a driving simulator to perform a vehicle steering, and a map reading task concurrently and in real time. The performance data of the model are similar to human subjects performing the same tasks.

In this study 24 participants drove a simulator while listening to three types of messages in both synthesized speech and recorded human speech. The messages consisted of short navigation messages, medium length (approximately 100 words) email messages, and longer news stories (approximately 200 words). After each message the participant was presented with a series of multiple choice questions to measure comprehension of the message. Driving performance was recorded. Findings show that for the low driving workload conditions in the study, (cruise control, predictable two-lane road with no intersections, invariant lead car) driving performance was not affected by listening to messages. This was true for both the synthesized speech and natural speech. Comprehension of messages in synthetic speech was significantly lower than for recorded human speech for all message types.