Multi-robot teams can be very useful in a wide variety of search and rescue missions in challenging environments. In a mission with considerable uncertainty due to intermittent communications, degraded information flow, and failures, humans need to assess both the current and expected future states and update task assignments in human-robot teams as quickly as possible. We have developed an alert generation framework that can perform risk assessment and robot tasking suggestions to assist human supervisors. Our approach for task assignment suggestion generation combines heuristics-based task selection with forward simulation-based probabilistic assessment. As the characteristics of decision aids can largely vary human performance, an alert system may or may not improve decision-making. We aim to configure our framework with a goal to improve human decision-making performance. Towards that, we present some preliminary user studies and design reasoning, which informed our final comprehensive human subject study. We demonstrate in the study that supervisors can improve their decision-making abilities, make faster decisions, and increase mission performance by using our alert generation framework. Our empirical findings also show that our framework does not require significant training and that people with a higher level of trust in automation perform better when provided with alerts. We also find that people with certain personality traits such as high agreeableness and conscientiousness are the most benefited by alerts.

We explored how users perceive virtual characters that performed the role of a counseling interviewer, while presenting different levels of social class, as well as single or multi-tasking behavior. To investigate this subject, we designed a 2x2 experiment (tasking type and social class of the virtual counseling interviewer). In the experiment, participants experienced the counseling interview interactions over video conferencing on a smartphone. We measured user responses to and perceptions of the virtual human interviewer. The results demonstrate that the tasking types and social class of the virtual counselor affected user responses to and perceptions of the virtual counselor. The results offer insight into the design and development of effective, realistic, and believable virtual human counselors. Furthermore, the results also address current social questions about how smartphones might mediate social interactions, including human-agent interactions.

It is commonly known that a relationship exists between the human voice and various emotional states. Past studies have demonstrated changes in a number of vocal features, such as fundamental frequency f0 and peakSlope, as a result of varying emotional state. These voice characteristics have been shown to relate to emotional load, vocal tension, and, in particular, stress. Although much research exists in the domain of voice analysis, few studies have assessed the relationship between stress and changes in the voice during a dyadic team interaction. The aim of the present study was to investigate the multimodal interplay between speech and physiology during a high-workload, high-stress team task. Specifically, we studied task-induced effects on participants' vocal signals, specifically, the f0 and peakSlope features, as well as participants' physiology, through cardiovascular measures. Further, we assessed the relationship between physiological states related to stress and changes in the speaker's voice. We recruited participants with the specific goal of working together to diffuse a simulated bomb. Half of our sample participated in an "Ice Breaker" scenario, during which they were allowed to converse and familiarize themselves with their teammate prior to the task, while the other half of the sample served as our "Control". Fundamental frequency (f0), peakSlope, physiological state, and subjective stress were measured during the task. Results indicated that f0 and peakSlope significantly increased from the beginning to the end of each task trial, and were highest in the last trial, which indicates an increase in emotional load and vocal tension. Finally, cardiovascular measures of stress indicated that the vocal and emotional load of speakers towards the end of the task mirrored a physiological state of psychological "threat".

Here we describe the design and usability evaluation of a mixed reality prototype to simulate the role of a tank platoon leader, who is an individual who not only is a tank commander, but also directs a platoon of three other tanks with their own respective tank commanders. The domain of tank commander training has relied on physical simulators of the actual Abrams tank and encapsulates the whole crew. The TALK-ON system we describe here focuses on training communication skills of the leader in a simulated tank crew. We report results from a usability evaluation and discuss how they will inform our future work for collective tank training.

Abstract We explored the social influence of humor in a virtual human counselor's self‐disclosure while also varying the ethnicity of the virtual counselor. In a 2 × 3 experiment (humor and ethnicity of the virtual human counselor), participants experienced counseling interview interactions via Skype on a smartphone. We measured user responses to and perceptions of the virtual human counselor. The results demonstrate that humor positively affects user responses to and perceptions of a virtual counselor. The results further suggest that matching styles of humor with a virtual counselor's ethnicity influences user responses and perceptions. The results offer insight into the effective design and development of realistic and believable virtual human counselors. Furthermore, they illuminate the potential use of humor to enhance self‐disclosure in human–agent interactions.

Team cohesion has been suggested to be a critical factor in emotional resilience following periods of stress. Team cohesion may depend on several factors including emotional state, communication among team members and even psychophysiological response. The present study sought to employ several multimodal techniques designed to investigate team behavior as a means of understanding resilience to stress. We recruited 40 subjects to perform a cooperative-task in gender-matched, two-person teams. They were responsible for working together to meet a common goal, which was to successfully disarm a simulated bomb. This high-workload task requires successful cooperation and communication among members. We assessed several behaviors that relate to facial expression, word choice and physiological responses (i.e., heart rate variability) within this scenario. A manipulation of an â€oeice breakerâ€❝ condition was used to induce a level of comfort or familiarity within the team prior to the task. We found that individuals in the â€oeice breakerâ€❝ condition exhibited better resilience to subjective stress following the task. These individuals also exhibited more insight and cognitive speech, more positive facial expressions and were also able to better regulate their emotional expression during the task, compared to the control.

The Berkeley Foundation for Opportunities in Information Technology is a decade-old endeavor to expose pre-college young women and underrepresented racial and ethnic minorities to the fields of computer science and engineering, and prepare them for rigorous, university-level study. We have served more than 150 students, and graduated more than 65 seniors who have gone on to attend some of the top institutions in the country. Some of the lessons we have learned include the importance of sustained funding to support a continuing year-round program, world-class leaders and resources, and family and alumni involvement. In this article, we share the inner workings of our program, from its foundation during the dot-com heyday through today, in hopes that our best practices can be useful to others working toward the goal of broadening participation.

Three experiments examined how cognitive abilities and qualities of external visualizations affected performance of a mental visualization task; inferring the cross-section of a complex three-dimensional object. Experiment 1 investigated the effect of animations designed to provide different task-relevant views of the external object. Experiment 2 examined the effects of both stereoscopic and motion-based depth cues. Experiment 3 examined the effects of interactive animations, with and without stereoscopic viewing conditions. In all experiments, spatial and general reasoning abilities were measured. Effects of animation, stereopsis, and interactivity were relatively small and did not reach statistical significance. In contrast, spatial ability was significantly associated with superior performance in all experiments, and this remained true after controlling for general intelligence. The results indicate that difficulties in this task stem more from the cognitive ability to perform the relevant internal spatial transformations, than limited visual information about the three-dimensional structure of the object.

Computer-aided design (CAD) is a challenging domain of end-user programming because it involves complex spatial transformations of 3-dimensional (3-D) objects. Here we provide evidence that shows that the ability to perform spatial transformations of novel abstract 3-D objects can be augmented when additional information is presented in an external display.

We present a human-centered gestural system for musical improvisation, MIND. We demonstrate the relationship between music composition and programming. Preliminary field data from contextual inquiries in an elderly care setting suggest that the MIND system would be accessible to this population. A benefit of the system is the multimodal interaction technique through the invisible interface.