The National Highway Traffic Safety Administration reported that the number of bicyclist fatalities has increased by more than 35% since 2010. One of the main reasons associated with cyclists’ crashes is the adverse effect of drivers’ cognitive distractions. However, very limited studies have evaluated the impact of cyclists’ cognitive distraction on their behaviors and safety. This study leverages an Immersive Virtual Environment (IVE) simulation environment to explore the effect of secondary tasks on cyclists’ cognitive distraction by evaluating their behavioral and physiological responses. Specifically, by recruiting 75 participants, this study explores the effect of listening to music versus talking on the phone as standardized secondary tasks on participants’ behavior (i.e., speed, lane position, input power, head movement) as well as, physiological responses including participants’ heart rate variability and skin conductance metrics. Our results show that listening to high-tempo music can lead to a significantly higher speed, a lower standard deviation of speed, and higher input power. Additionally, the trend is more significant for cyclists who have a strong habit of daily music listening ($>$4 hours/day). In the high cognitive workload situation (simulated hands-free phone talking), cyclists had a lower speed with less input power and less head movement variation. Our results indicate that participants’ heart rate variability (HRV) features (the power of high frequency, pnni-50) and electrodermal activity (EDA) features like numbers of skin conductance responses (SCR) peaks are sensitive to cyclists’ cognitive load changes in the IVE simulator. Gender differences are identified in cycling performance and physiological response.

Analyzing the impact of the environment on drivers’ stress level and workload is of high importance for designing human-centered driver-vehicle interaction systems and to ultimately help build a safer driving experience. However, driver’s state, including stress level and workload, are latent variables that cannot be measured on their own and should be estimated through sensor measurements such as psychophysiological measures. We propose using a latent-variable state-space modeling framework for driver state analysis. By using latent-variable state-space models, we model drivers’ workload and stress levels as latent variables estimated through multimodal human sensing data, under the perturbations of the environment in a state-space format and in a holistic manner. Through using a case study of multimodal driving data collected from 11 participants, we first estimate the latent stress level and workload of drivers from their heart rate, gaze measures, and intensity of facial action units. We then show that external contextual elements such as the number of vehicles as a proxy for traffic density and secondary task demands may be associated with changes in driver’s stress levels and workload. We also show that different drivers may be impacted differently by the aforementioned perturbations. We found out that drivers’ latent states at previous timesteps are highly associated with their current states. Additionally, we discuss the utility of state-space models in analyzing the possible lag between the two latent variables of stress level and workload, which might be indicative of information transmission between the different parts of the driver’s psychophysiology in the wild.

Integrating driver, in-cabin, and outside environ-ment's contextual cues into the vehicle's decision making is the centerpiece of semi-automated vehicle safety. Multiple systems have been developed for providing context to the vehicle, which often rely on video streams capturing drivers' physical and environmental states. While video streams are a rich source of information, their ability in providing context can be challenging in certain situations, such as low illuminance environments (e.g., night driving), and they are highly privacy-intrusive. In this study, we leverage passive sensing through smartwatches for classifying elements of driving context. Specifically, through using the data collected from 15 participants in a naturalistic driving study, and by using multiple machine learning algorithms such as random forest, we classify driver's activities (e.g., using phone and eating), outside events (e.g., passing intersection and changing lane), and outside road attributes (e.g., driving in a city versus a highway) with an average F1 score of 94.55, 98.27, and 97.86 % respectively, through 10-fold cross-validation. Our results show the applicability of multimodal data retrieved through smart wearable devices in providing context in real-world driving scenarios and pave the way for a better shared autonomy and privacy-aware driving data-collection, analysis, and feedback for future autonomous vehicles.

In an effort to better understand the different ways in which the discount factor affects the optimization process in reinforcement learning, we designed a set of experiments to study each effect in isolation. Our analysis reveals that the common perception that poor performance of low discount factors is caused by (too) small action-gaps requires revision. We propose an alternative hypothesis that identifies the size-difference of the action-gap across the state-space as the primary cause. We then introduce a new method that enables more homogeneous action-gaps by mapping value estimates to a logarithmic space. We prove convergence for this method under standard assumptions and demonstrate empirically that it indeed enables lower discount factors for approximate reinforcement-learning methods. This in turn allows tackling a class of reinforcement-learning problems that are challenging to solve with traditional methods.

Discrete-action algorithms have been central to numerous recent successes of deep reinforcement learning. However, applying these algorithms to high-dimensional action tasks requires tackling the combinatorial increase of the number of possible actions with the number of action dimensions. This problem is further exacerbated for continuous-action tasks that require fine control of actions via discretization. In this paper, we propose a novel neural architecture featuring a shared decision module followed by several network branches, one for each action dimension. This approach achieves a linear increase of the number of network outputs with the number of degrees of freedom by allowing a level of independence for each individual action dimension. To illustrate the approach, we present a novel agent, called Branching Dueling Q-Network (BDQ), as a branching variant of the Dueling Double Deep Q-Network (Dueling DDQN). We evaluate the performance of our agent on a set of challenging continuous control tasks. The empirical results show that the proposed agent scales gracefully to environments with increasing action dimensionality and indicate the significance of the shared decision module in coordination of the distributed action branches. Furthermore, we show that the proposed agent performs competitively against a state-of-the-art continuous control algorithm, Deep Deterministic Policy Gradient (DDPG).

In reinforcement learning, it is common to let an agent interact for a fixed amount of time with its environment before resetting it and repeating the process in a series of episodes. The task that the agent has to learn can either be to maximize its performance over (i) that fixed period, or (ii) an indefinite period where time limits are only used during training to diversify experience. In this paper, we provide a formal account for how time limits could effectively be handled in each of the two cases and explain why not doing so can cause state-aliasing and invalidation of experience replay, leading to suboptimal policies and training instability. In case (i), we argue that the terminations due to time limits are in fact part of the environment, and thus a notion of the remaining time should be included as part of the agent’s input to avoid violation of the Markov property. In case (ii), the time limits are not part of the environment and are only used to facilitate learning. We argue that this insight should be incorporated by bootstrapping from the value of the state at the end of each partial episode. For both cases, we illustrate empirically the significance of our considerations in improving the performance and stability of existing reinforcement learning algorithms, showing state-of-the-art results on several control tasks.

We have conducted two investigations on the ability of human participants to solve challenging collective coordination tasks in a distributed fashion with limited perception and communication capabilities similar to those of a simple ground robot. In these investigations, participants were gathered in a laboratory of networked workstations and were given a series of different collective tasks with varying communication and perception capabilities. Here, we focus on our latest investigation and describe our methodology, platform design considerations, and highlight some interesting observed behaviors. These investigations are the preliminary phase in designing a formal strategy for learning human-inspired behaviors for solving complex distributed multirobot problems, such as pattern formation.