Spatial navigation is an important aspect of everyday life but may be negatively impacted by both cognitive and affective load. Cognitive and affective load may be measured via autonomic arousal and increased load may lead to reduced navigational abilities. 53 college students (64.0% female;Mage = 19.62) participated in the Virtual Reality Paced Serial Auditory Addition Task (VR-PASAT). Participants followed guides through different areas of the virtual environment (VE). In some areas participants completed the PASAT, high load, and in other areas they simply followed the guides, low load. Some participants were instructed beforehand they would perform a navigate task, increasing load. Results suggested that several psychophysiological measures including skin conductance and inter-beat intervals were impacted by increased load while others were related to the interactions between load and zone order. Awareness of the navigation task led to worse performance on the VR-PASAT, and high load decreased navigational performance. The VR-PASAT was used to implement a VE to manipulate cognitive load. This study may be useful for the creation of adaptive systems because it demonstrates that psychophysiological metrics can assess cognitive and affective load, which may impact navigation within a VE, and navigational task awareness may interact with load.

A growing body of literature has emerged that demonstrates the potential of neurogaming platforms for inter- facing with well-known video games. With the recent convergence of advances in consumer electronics, ubiquitous computing, and wearable sensor technologies real-time monitoring of neurocognitive and affective states can be studied in an objective manner. Whilst establishing the optimal relation among frequency bands, task engagement, and arousal states is a goal of neurogaming, a standardized method has yet to be established. Herein we aimed to test classifiers within the same context, group of participants, feature extraction methods, and protocol. Given the emphasis upon neurogaming, a commercial-grade electroencephalographic (EEG; Emotiv EPOC) headset was used to collect signals from 30 participants. The EEG data was then filtered to get separate frequency bands to train cognitive-affective classifiers with three classification techniques: Support Vector Machines (SVM), Naive Bayes (NB), and k-Nearest Neighbors (kNN). Results revealed that the NB classifier was the most robust classifier for identifying negative (e.g., character death) game-based events. The identification of general gameplay events is best identified using kNN and the Beta band. Results from this study suggest that a combination of classifiers is preferable over selection of a single classifier.

Understanding the ways in which persons rapidly transfer attention between tasks while still retaining ability to perform these tasks is an important area of study. Everyday activities commonly come in the form of emotional distractors. A recently developed Virtual Reality Stroop Task (VRST) allows for assessing neurocognitive and psychophysiological responding while traveling through simulated safe and ambush desert environments as Stroop stimuli appear on the windshield. We evaluated differences in psychophysiological response patterns associated with completion of an affective task alone versus completion of an affective task that also included a Stroop task. The VRST elicited increased heart rate, respiration rate, skin conductance level, and number of spontaneous fluctuations in electrodermal activity. Increased cognitive workload was found to be associated with the more cognitively challenging Stroop conditions which led to an increase in response level. This expands on previous findings and indicates that allocating attention away from the environment and toward Stroop stimuli likely requires greater inhibitory control. This is corroborated by behavioral findings from previous investigations with the VRST. The VRST revealed that the increased difficulty found in tasks like the Stroop interference task directly evoke autonomic changes in psychophysiological arousal beyond the threatening stimuli themselves.

The Stress Resilience In Virtual Environments (STRIVE) project aims to create a set of combat simulations (adapted from existing Virtual Iraq/Afghanistan exposure therapy system) that are part of a multi-episode narrative experience. Users can be immersed within challenging combat contexts and interact with virtual characters within these episodes as part of an experiential learning approach. Such approaches, often referred to as Stress Inoculation Training, have been found to reduce the neuroendocrine response, increase coping competence and reduce cortisol response. STRIVE is currently developing three combat modules with a narrative arc that includes increasingly intense combat situations, relational development, stress and loss, and emotional trauma. The goal is to raise the service member's threshold for the stress of combat.

Speech processing is an important element of affective computing. Most research in this direction has focused on classifying emotions into a small number of categories. However, numerical representations of emotions in a multi-dimensional space can be more appropriate to reflect the gradient nature of emotion expressions, and can be more convenient in the sense of dealing with a small set of emotion primitives. This paper presents three approaches (robust regression, support vector regression, and locally linear reconstruction) for emotion primitives estimation in 3D space (valence/activation/dominance), and two approaches (average fusion and locally weighted fusion) to fuse the three elementary estimators for better overall recognition accuracy. The three elementary estimators are diverse and complementary because they cover both linear and nonlinear models, and both global and local models. These five approaches are compared with the state-of-the-art estimator on the same spontaneously elicited emotion dataset. Our results show that all of our three elementary estimators are suitable for speech emotion estimation. Moreover, it is possible to boost the estimation performance by fusing them properly since they appear to leverage complementary speech features.

We recently proposed a family of robust linear and nonlinear estimation techniques for recognizing the three emotion primitives‐valence, activation, and dominance‐from speech. These were based on both local and global speech duration, energy, MFCC and pitch features. This paper aims to study the relative importance of these four categories of acoustic features in this emotion estimation context. Three measures are considered: the number of features from each category when all features are used in selection, the mean absolute error (MAE) when each category is used separately, and the MAE when a category is excluded from feature selection. We find that the relative importance is in the order of MFCC > Energy ! Pitch > Duration. Additionally, estimator fusion almost always improves performance, and locally weighted fusion always outperforms average fusion regardless of the number of features used. Index Terms: Emotion estimation, 3D emotion space, speech analysis, estimator fusion, support vector regression, robust regression, locally linear reconstruction, locally weighted fusion

A closed-loop system that offers real-time assessment and manipulation of a user's affective and cognitive states is very useful in developing adaptive environments which respond in a rational and strategic fashion to real-time changes in user affect, cognition, and motivation. The goal is to progress the user from suboptimal cognitive and affective states toward an optimal state that enhances user performance. In order to achieve this, there is need for assessment of both 1) the optimal affective/cognitive state and 2) the observed user state. This paper presents approaches for assessing these two states. Arousal, an important dimension of affect, is focused upon because of its close relation to a user's cognitive performance, as indicated by the Yerkes-Dodson Law. Herein, we make use of a Virtual Reality Stroop Task (VRST) from the Virtual Reality Cognitive Performance Assessment Test (VRCPAT) to identify the optimal arousal level that can serve as the affective/cognitive state goal. Three stimuli presentations (with distinct arousal levels) in the VRST are selected. We demonstrate that when reaction time is used as the performance measure, one of the three stimuli presentations can elicit the optimal level of arousal for most subjects. Further, results suggest that high classification rates can be achieved when a support vector machine is used to classify the psychophysiological responses (skin conductance level, respiration, ECG, and EEG) in these three stimuli presentations into three arousal levels. This research reflects progress toward the implementation of a closed-loop affective computing system.

Posttraumatic Stress Disorder (PTSD) is reported to be caused by traumatic events that are outside the range of usual human experience including (but not limited to) military combat, violent personal assault, being kidnapped or taken hostage and terrorist attacks. Initial data suggests that at least 1 out of 5 Iraq War veterans are exhibiting symptoms of depression, anxiety and PTSD. Virtual Reality (VR) delivered exposure therapy for PTSD has been previously used with reports of positive outcomes. The current paper is a follow-up to a paper presented at IEEE VR2006 and will present the rationale and description of a VR PTSD therapy application (Virtual Iraq) and present the findings from its use with active duty service members since the VR2006 presentation. Virtual Iraq consists of a series of customizable virtual scenarios designed to represent relevant Middle Eastern VR contexts for exposure therapy, including a city and desert road convoy environment. User-centered design feedback needed to iteratively evolve the system was gathered from returning Iraq War veterans in the USA and from a system deployed in Iraq and tested by an Army Combat Stress Control Team. Results from an open clinical trial using Virtual Iraq at the Naval Medical Center-San Diego with 20 treatment completers indicate that 16 no longer met PTSD diagnostic criteria at post-treatment, with only one not maintaining treatment gains at 3 month follow-up.

This paper describes interdisciplinary work on developing a virtual reality (VR) aided motor training task for post-stroke rehabilitation on functional deficits of the upper extremity: static reaching. Patient-specific and human-centered design of the VR system was addressed from the physical therapist's perspective. The two main features of the system were that it could actively drive the human kinetic behavior based on the therapist's rehabilitation goals and capture the patient's kinetic performance in an accurate way. A three-month clinical trial of this VR task was conducted with five post-stroke patients. To analyze the collected data, a methodology was proposed to visualize the patient's current status and progression over time based on three kinematics measures: performance time, movement efficiency, and moving speed. Results from the analysis clearly reveal the current status of the patient's hand and arm movement with respect to his/her range of motion, comprising pitch, yaw and arm length. Further, evidence of progress was found and visualized quantitatively over a series of practice sessions. Along with several conventional behavioral assessments at three points: pre-training, mid-training and post-training, the patient's progress was identified as well. Finally, human factors, such as perception of difficulty, confidence of movement, and system usability, were measured and studied.