Virtual Standardized Patients (VSPs) are computer-based training simulations that expand access to experiential, evidence-based mental health training at low cost and scale. This paper presents three phases of user-centered design studies to develop a VSP for Safety Planning Intervention (SPI) in the Department of Veterans Affairs (VA) and Department of Defense (DoD). VA and DoD mental health staff, trainees, and subject matter experts participated in qualitative interviews, prototype usability testing, and beta testing. Results identified key design features for effective training. Usability ratings of the prototype and beta were good, with higher ratings among trainees. Critical improvements were made to performance feedback, and users helped refine dialog clarity and training tasks. The resulting SPI VSP is now poised for a pilot randomized trial to evaluate training outcomes relative to routine training. If effective, it may provide a scalable, cost-effective way to improve dissemination of a promising suicide prevention intervention.

The 2023 IEEE VGTC Virtual Reality Lifetime Achievement Award goes to Albert “Skip” Rizzo of the USC Institute for Creative Technologies, in recognition of his lifetime contributions to the development of virtual reality and virtual human systems for the treatment of PTSD and other clinical conditions.

Automatic detection of psychological disorders has gained significant attention in recent years due to the rise in their prevalence. However, the majority of studies have overlooked the complexity of disorders in favor of a “present/not present” dichotomy in representing disorders. Recent psychological research challenges favors transdiagnostic approaches, moving beyond general disorder classifications to symptom level analysis, as symptoms are often not exclusive to individual disorder classes. In our study, we investigated the link between speech signals and psychological distress symptoms in a corpus of 333 screening interviews from the Distress Analysis Interview Corpus (DAIC). Given the semi-structured organization of interviews, we aggregated speech utterances from responses to shared questions across interviews. We employed deterministic sample selection in classification to rank salient questions for eliciting symptom-specific behaviors in order to predict symptom presence. Some questions include “Do you find therapy helpful?” and “When was the last time you felt happy?”. The prediction results align closely to the factor structure of psychological distress symptoms, linking speech behaviors primarily to somatic and affective alterations in both depression and PTSD. This lends support for the transdiagnostic validity of speech markers for detecting such symptoms. Surprisingly, we did not find a strong link between speech markers and cognitive or psychomotor alterations. This is surprising, given the complexity of motor and cognitive actions required in speech production. The results of our analysis highlight the importance of aligning affective computing research with psychological research to investigate the use of automatic behavioral sensing to assess psychiatric risk.

We present one of the first applications of virtual humans in Augmented Reality (AR), which allows young adults with Autism Spectrum Disorder (ASD) the opportunity to practice job interviews. It uses the Magic Leap's AR hardware sensors to provide users with immediate feedback on six different metrics, including eye gaze, blink rate and head orientation. The system provides two characters, with three conversational modes each. Ported from an existing desktop application, the main development lessons learned were: 1) provide users with navigation instructions in the user interface, 2) avoid dark colors as they are rendered transparently, 3) use dynamic gaze so characters maintain eye contact with the user, 4) use hardware sensors like eye gaze to provide user feedback, and 5) use surface detection to place characters dynamically in the world.

We present a versatile system for training job interviewing skills that focuses specifically on segments of the population facing increased challenges during the job application process. In particular, we target those with Autism Spectrum Disorder (ADS), veterans transitioning to civilian life, and former convicts integrating back into society. The system itself follows the SAIBA framework and contains several interviewer characters, who each represent a different type of vocational field, (e.g. service industry, retail, office, etc.) Each interviewer can be set to one of three conversational modes, which not only affects what they say and how they say it, but also their supporting body language. This approach offers varying difficulties, allowing users to start practicing with interviewers who are more encouraging and accommodating before moving on to personalities that are more direct and indifferent. Finally, the user can place the interviewers in different environmental settings (e.g. conference room, restaurant, executive office, etc.), allowing for many different combinations in which to practice.

Since the mid-1990s, a significant scientific literature has evolved regarding the outcomes from the use of what we now refer to as Clinical Virtual Reality (VR). This use of VR simulation technology has produced encouraging results when applied to address cognitive, psychological, motor, and functional impairments across a wide range of clinical health conditions. This talk addresses the question, “Is Clinical VR Ready for Primetime?”. After a brief description of the various forms of VR technology, I will discuss the trajectory of Clinical VR over the last 20 years and summarize the basic assets that VR offers for creating clinical applications. The discussion then addresses the question of readiness in terms of the theoretical basis for Clinical VR assets, the research to date, the pragmatic factors regarding availability, usability, and costs of Clinical VR content/systems, and the ethical issues for the safe use of VR with clinical populations. My key take home message is that when reviewing the theoretical underpinnings and research findings to date, it is clear that Clinical VR will have a significant impact on future research and practice. Pragmatic issues that can influence adoption across many areas of psychology also appear favorable, but professional guidelines will be needed to promote its safe and ethical use. While there is still much research needed to advance the science in this area, it is predicted that Clinical VR applications will become indispensable tools in the toolbox of psychological researchers and practitioners and will only grow in relevance and popularity in the future.

Augmented Reality (AR) is now more and more widespread in many application fields. Thanks to the new progresses in computer power, AR can now be used widely in education without any expensive additional devices. In this paper is presented the feedback of an experimental protocol using an AR application as an additional support for a History lesson in secondary schools. Even if the technical part has a lead role in the student experience, the most challenging issue is related to the choice of the teaching lesson as itself which must fit several, sometimes contradictory, requirements.

Youth suicide is a major public health problem. It is the third leading cause of death in the United States for ages 13 through 18. Many adolescents that face suicidal thoughts or make a suicide plan never seek professional care or help. Within this work, we evaluate both verbal and nonverbal responses to a five-item ubiquitous questionnaire to identify and assess suicidal risk of adolescents. We utilize a machine learning approach to identify suicidal from non-suicidal speech as well as characterize adolescents that repeatedly attempted suicide in the past. Our findings investigate both verbal and nonverbal behavior information of the face-to-face clinician-patient interaction. We investigate 60 audio-recorded dyadic clinician-patient interviews of 30 suicidal (13 repeaters and 17 non-repeaters) and 30 non-suicidal adolescents. The interaction between clinician and adolescents is statistically analyzed to reveal differences between suicidal versus non-suicidal adolescents and to investigate suicidal repeaters’ behaviors in comparison to suicidal non-repeaters. By using a hierarchical classifier we were able to show that the verbal responses to the ubiquitous questions sections of the interviews were useful to discriminate suicidal and non-suicidal patients. However, to additionally classify suicidal repeaters and suicidal non-repeaters more information especially nonverbal information is required.

Reduced frequency range in vowel production is a well documented speech characteristic of individuals with psychological and neurological disorders. Affective disorders such as depression and post-traumatic stress disorder (PTSD) are known to influence motor control and in particular speech production. The assessment and documentation of reduced vowel space and reduced expressivity often either rely on subjective assessments or on analysis of speech under constrained laboratory conditions (e.g. sustained vowel production, reading tasks). These constraints render the analysis of such measures expensive and impractical. Within this work, we investigate an automatic unsupervised machine learning based approach to assess a speaker's vowel space. Our experiments are based on recordings of 253 individuals. Symptoms of depression and PTSD are assessed using standard self-assessment questionnaires and their cut-off scores. The experiments show a significantly reduced vowel space in subjects that scored positively on the questionnaires. We show the measure's statistical robustness against varying demographics of individuals and articulation rate. The reduced vowel space for subjects with symptoms of depression can be explained by the common condition of psychomotor retardation influencing articulation and motor control. These findings could potentially support treatment of affective disorders, like depression and PTSD in the future.

We present the SimSensei system, a fully automatic virtual agent that conducts interviews to assess indicators of psychological distress. We emphasize on the perception part of the system, a multimodal framework which captures and analyzes user state for both behavioral understanding and interactional purposes.

We present the SimSensei system, a fully automatic virtual agent that conducts interviews to assess indicators of psychological distress. With this demo, we focus our attention on the perception part of the system, a multimodal framework which captures and analyzes user state behavior for both behavioral understanding and interactional purposes. We will demonstrate real-time user state sensing as a part of the SimSensei architecture and discuss how this technology enabled automatic analysis of behaviors related to psychological distress.

Suicide among adolescents is a major public health problem: it is the third leading cause of death in the US for ages 13-18. Up to now, there is no objective ways to assess the suicidal risk, i.e. whether a patient is non-suicidal, suicidal re-attempter (i.e. repeater) or suicidal non-repeater (i.e. individuals with one suicide attempt or showing signs of suicidal gestures or ideation). Therefore, features of the conversation including verbal information and nonverbal acoustic information were investigated from 60 audio-recorded interviews of 30 suicidal (13 repeaters and 17 non-repeaters) and 30 non-suicidal adolescents interviewed by a social worker. The interaction between clinician and patients was statistically analyzed to reveal differences between suicidal vs. non-suicidal adolescents and to investigate suicidal repeaters' behaviors in comparison to suicidal non-repeaters. By using a hierarchical ensemble classifier we were able to successfully discriminate non-suicidal patients, suicidal repeaters and suicidal non-repeaters.

This pilot study assessed the utility and acceptability of the VirtuSphere, a cutting edge navigation platform designed to enhance presence in virtual environments. The VirtuSphere includes a 12-ft hollow sphere within which the user stands, and it rolls within a wheeled platform, in any direction, according to the user's steps. The pilot was a within-subject crossover design comparing the VirtuSphere to standard game controller navigation. The comparison was based on locomotion in Virtual Iraq, a virtual world resembling Iraqi war zones. Participants were 10 active duty soldiers not suffering from posttraumatic stress disorder. Results indicated that there were negligible differences in sense of presence, simulator sickness, and satisfaction across the two navigation systems. Although the VirtuSphere may provide entertainment value, these results do not provide initial support for the use of the VirtuSphere to improve constructs thought to be important to behavioral health applications of virtual reality. Potential improvements to the design of the VirtuSphere are discussed.

The Stress Resilience In Virtual Environments (STRIVE) project aims to create a set of combat simulations (adapted from existing Virtual Iraq/Afghanistan exposure therapy system) that are part of a multi-episode narrative experience. Users can be immersed within challenging combat contexts and interact with virtual characters within these episodes as part of an experiential learning approach. Such approaches, often referred to as Stress Inoculation Training, have been found to reduce the neuroendocrine response, increase coping competence and reduce cortisol response. STRIVE is currently developing three combat modules with a narrative arc that includes increasingly intense combat situations, relational development, stress and loss, and emotional trauma. The goal is to raise the service member's threshold for the stress of combat.

Using video games in rehabilitation settings has the potential to provide patients with fun and motivating exercise tools. Within the Medical VR and MxR groups at the USC Institute for Creative Technologies, we have been leveraging the technology of the Microsoft Kinect 3D depth-sensing camera. Our Kinect-based rehabilitation game “JewelMine” consists of a set of static balance training exercises which encourage the players to reach out of their base of support. We plan to demonstrate a sophisticated post-session analysis tool and several content themes which can be changed dynamically during a therapy session.

Mental Rotation is the ability to rotate mental representations of two dimensional and three dimensional objects, and usually regarded as a predictor of math or science achievement. This study utilized a 6DOF-controlled method with stereoscopics in the training of mental rotation and investigated how these training methods may affect user's learning performance, perceived playfulness, and willingness to continue use. To evaluate the method, a video/animation-based training method was used in control group. A subject test with 30 participants was conducted. These participants were classified into two groups and trained with two different methods respectively. Results showed that the 6DOF-controlled method with stereoscopics gained higher improvement of the difference from pre-test to post-test in average, although the difference was not significant. While considering perceived playfulness and willingness to continue use, the 6DOF-controlled method with stereoscopics gained higher scores significantly then the control group.

The Flexible Action and Articulated Skeleton Toolkit (FAAST) is middleware to facilitate integration of full-body control with virtual reality applications and video games using OpenNI-compliant depth sensors (currently the PrimeSensor and the Microsoft Kinect). FAAST incorporates a VRPN server for streaming the user's skeleton joints over a network, which provides a convenient interface for custom virtual reality applications and games. This body pose information can be used for goals such as realistically puppeting a virtual avatar or controlling an on-screen mouse cursor. Additionally, the toolkit also provides a configurable input emulator that detects human actions and binds them to virtual mouse and keyboard commands, which are sent to the actively selected window. Thus, FAAST can enable natural interaction for existing off-the-shelf video games that were not explicitly developed to support input from motion sensors. The actions and input bindings are configurable at run-time, allowing the user to customize the controls and sensitivity to adjust for individual body types and preferences. In the future, we plan to substantially expand FAAST's action lexicon, provide support for recording and training custom gestures, and incorporate real-time head tracking using computer vision techniques.

Posttraumatic Stress Disorder (PTSD) is reported to be caused by traumatic events that are outside the range of usual human experience including (but not limited to) military combat, violent personal assault, being kidnapped or taken hostage and terrorist attacks. Initial data suggests that at least 1 out of 5 Iraq War veterans are exhibiting symptoms of depression, anxiety and PTSD. Virtual Reality (VR) delivered exposure therapy for PTSD has been previously used with reports of positive outcomes. The current paper is a follow-up to a paper presented at IEEE VR2006 and will present the rationale and description of a VR PTSD therapy application (Virtual Iraq) and present the findings from its use with active duty service members since the VR2006 presentation. Virtual Iraq consists of a series of customizable virtual scenarios designed to represent relevant Middle Eastern VR contexts for exposure therapy, including a city and desert road convoy environment. User-centered design feedback needed to iteratively evolve the system was gathered from returning Iraq War veterans in the USA and from a system deployed in Iraq and tested by an Army Combat Stress Control Team. Results from an open clinical trial using Virtual Iraq at the Naval Medical Center-San Diego with 20 treatment completers indicate that 16 no longer met PTSD diagnostic criteria at post-treatment, with only one not maintaining treatment gains at 3 month follow-up.

This paper describes interdisciplinary work on developing a virtual reality (VR) aided motor training task for post-stroke rehabilitation on functional deficits of the upper extremity: static reaching. Patient-specific and human-centered design of the VR system was addressed from the physical therapist's perspective. The two main features of the system were that it could actively drive the human kinetic behavior based on the therapist's rehabilitation goals and capture the patient's kinetic performance in an accurate way. A three-month clinical trial of this VR task was conducted with five post-stroke patients. To analyze the collected data, a methodology was proposed to visualize the patient's current status and progression over time based on three kinematics measures: performance time, movement efficiency, and moving speed. Results from the analysis clearly reveal the current status of the patient's hand and arm movement with respect to his/her range of motion, comprising pitch, yaw and arm length. Further, evidence of progress was found and visualized quantitatively over a series of practice sessions. Along with several conventional behavioral assessments at three points: pre-training, mid-training and post-training, the patient's progress was identified as well. Finally, human factors, such as perception of difficulty, confidence of movement, and system usability, were measured and studied.

Summary form only for tutorial. After an early period of inflated expectations and limited delivery, Virtual Reality (VR) technology has emerged as a viable tool for cognitive/behavioral mental health and rehabilitation applications. VR allows for the systematic presentation of stimuli that target human behavioral and cognitive processes relevant for assessment and intervention purposes. The capacity of VE technology to create dynamic interactive three-dimensional (3D) stimulus environments, within which all behavioral responding can be recorded and measured, offers clinical assessment and intervention options that are not available using traditional methods. These applications have shown promise for addressing: fear and anxiety with phobic and PTSD clients, distracting patients during painful and anxiety-provoking medical and dental procedures, treating eating disorders and obesity, navigation and spatial training in children with motor impairments, functional skills in persons with developmental disabilities and autism, and in the assessment (and in some cases, rehabilitation) of memory, attention, spatial skills, and executive cognitive functions in populations with CNS dysfunction. Functional VE training scenarios have also been designed to test and teach instrumental activities of daily living such as street-crossing, automobile driving, meal preparation, supermarket shopping, use of public transportation, and wheelchair navigation. The therapeutic targets chosen thus far for these clinical applications reflect an informed appreciation for the unique assets that are available using virtual technology and these initiatives have formed a foundation of work that provides support for the value of further development of VR Mental Health/ Rehabilitation applications.

Published in: IEEE Proceedings. VR 2005. Virtual Reality, 2005

In this paper, we introduce an interdisciplinary project, involving researchers from the fields of Physical Therapy, Computer Science, Psychology, Communication and Cell Neurobiology, to develop an integrated virtual reality, haptics and modern sensing technique system for post-stroke rehabilitation. The methodology to develop the system includes identification of movement pattern, development of simulated task and diagnostics. Each part of the methodology can be achieved through several sub-steps that are described in detail in this paper. The system is designed from Physical Therapy perspective that can address the motor rehabilitation needs of stroke patients. The system is implemented through stereoscopic displays, force feedback devices and modern sensing techniques that have game-like features and can capture accurate data for further analysis. Diagnostics and evaluation can be made through an Artificial Intelligence based model using collected data and clinical tests have been conducted.

No abstract available.

Panoramic video image acquisition is based on multiple overlapped sub-images. We will demonstrate high-resolution panoramic video by employing an array of five video cameras viewing the scene over a combined 360-degrees of horizontal arc and 50-degrees vertical. The five live video streams are digitized and processed in real time by a computer system. The camera lens distortions and colorimetric variations are corrected by the software application and a complete panoramic image is constructed in memory. Users can navigate the scene by wearing a head mounted display (HMD). A single window with a resolution of 800x600 is output to the HMD. A real-time (inertial-magnetic) orientation tracker is fixed to the HMD to sense the user’s head orientation. The orientation is reported to the viewing application through an IP socket, and the output display window is positioned (to mimic pan and tilt) within the full panoramic image in response to the user’s head orientation.

Virtual Reality (VR) technology continues to evolve from the realm of an “expensive toy” into that of a functional technology. Although media hype may have oversold VR’s potential during the early 90’s, data has now emerged that supports its use as a valuable tool for cognitive, behavioral, physical, mental health and medical applications. Much like an aircraft simulator serves to test and train pilots under a variety of controlled conditions, VE simulations can model processes that are vital in the assessment, rehabilitation and treatment of clinical populations. In this regard, a growing number of laboratories are developing research programs investigating the use of VEs for these patient purposes and some of these systems have now made the transition into clinical practice. The presenters for this workshop have all developed and/or implemented VR applications that have shown promise for addressing fear reduction with phobic clients, pain and stress reduction for cancer patients, post traumatic stress disorder, physical therapy, anger management, driving abilities with Multiple Sclerosis and brain injury patients, and in the assessment and rehabilitation of motor skills, attention, memory, and vis uospatial processing in persons with central nervous system dysfunction and the elderly. These therapeutic targets reflect an informed appreciation of the unique assets that are available when using VR technology and provide a foundation of work that is supportive of the further development of these “healing” applications. The presenters will detail their work and support the case that these areas represent the “Kinder Killer App”!

Virtual environment (VE) technology is increasingly being recognized as a useful medium for the study, assessment, and rehabilitation of cognitive processes and functional abilities. The capacity of VE technology to create dynamic three-dimensional (3D) stimulus environments, within which all behavioral responding can be recorded, offers clinical assessment and rehabilitation options that are not available using traditional neuropsychological methods. This work has the potential to advance the scientific study of normal cognitive and behavioral processes and to improve our capacity to understand, measure, and treat the impairments typically found in clinical populations with central nervous system (CNS) dysfunction. The paper provides a rationale for the application of VE technology in the areas of neuropsychological assessment and cognitive rehabilitation, presents a tabled summary of the VE literature targeting cognitive/functional processes in clinical CNS populations and briefly describes two of our VE applications targeting attention and visuospatial processing.