Virtual reality (VR) is increasingly being used to promote exercise among older adults. The data captured through VR may be useful indicator of the game user's experience as well as providing insight into functional ability of older adults. This paper presents classifiers to predict game user experience variables using VR data from community-dwelling older adults. Head-kinematic data of the VR headset was collected from 13 participants over a six-week period with three 20-minutes exergame sessions per week (e.g., 360 minutes per participant). Cognitive function was assessed using the Montreal Cognitive Assessment (MoCa) and multisensory response-time (RT). Game user experience was captured through perceived-levels of cybersickness, enjoyment, and exertion after each session. Data was used as references for discrete binary and ternary classification patterns. Combinations of kinematic features were used to train different classifiers: K-nearest-neighbors (KNN), linear discriminant analysis (LDA), support vector machines (SVM), and decision trees. Maximum classification accuracy of 70% was found for MoCa, 68% for perceived exertion, 60% for cybersickness, 59% for multisensory RT, and 53% for perceived enjoyment. Results suggest unobtrusive recording of head kinematics from VR headsets combined with machine learning classifiers could be used to predict cognition, exertion, and game user experience among older adults.

Dangers in modern human society are commonly attributed to the safety of online activities. In the domain of cybersecurity, Social Engineering (SE) relates to how attackers manipulate and coerce their targets into divulging sensitive information. One major problem in designing social engineering defenses is making users aware they are being targeted. In the context of fostering human empowerment and building an inclusive society, we explore the possibility of leveraging social robot companions to provide improved protection for individuals and companies against cybersecurity attacks, specifically focusing on the realm of social engineering (SE) tactics. We asked participants to play an immersive interactive storytelling game, challenging them with risky and social-engineering-related decisions and monitoring their explicit (i.e., decisions) and implicit (i.e., mouse trajectories and facial expressions) behavior. After each decision, the Furhat tabletop robot intervened, always suggesting the not-selected option. We compared two Compliance Gaining Behaviors (CGBs) the robot could use, either leveraging affection with the participants or logical thinking. Overall, Furhat’s interventions increased the acceptance of risky and SE proposals. However, comparing the situations in which the robot tried to convince participants to avoid a social engineering request to those in which it tried to persuade them to accept it, the former was significantly more successful. Also, participants struggled with ignoring Furhat’s advice, as shown by their more uncertain mouse trajectories and negative emotional valence. From the latter results, we trained a Decision Tree model, based on mouse trajectory features only, to predict if participants would change their minds with an accuracy of 64.9%. Such defense mechanisms could help better understand users’ decision-making process in cybersecurity and social engineering, designing more helpful and supportive robot companions.

Playing games is an inherent part of children’s lives as it impacts several aspects of their physical and mental development. Technological advances have been manifesting new and exciting avenues of interaction when children play games, ranging from board and card games, to videogames that are played on mobile devices, virtual and augmented reality (VR/AR) headsets, robotic systems, and social agents. These games encompass a wide range of applications aiming to provide educational benefits, promote development, enhance well-being, or simply enjoy leisure time. Along with the fun and excitement, these advancements also bring unique challenges in the game design process due to the inclusion of complex technology, the maximization of the players’ engagement and expectations and interests of the children. The player-centric approach of co-designing games with the target audience has a unique position as it involves creating the games for and with the children, allowing them to act as an equal stakeholder rather than simple users or informants. This half-day workshop aims to expose the researchers to collaborative techniques used in game design to create interactive and playful activities for children that involve contemporary technologies such as AR/VR and social agents.

Bullying in schools is a widespread problem with serious consequences. We are exploring the use of social robots and role-playing to foster anti-bullying peer-support. In this paper, we present results from a co-design study with 22 children (8-12 years old) to explore how they envision a “student robot”. To understand how they conceptualize bullying in this context (e.g. whether robots can be bullied), we also investigated how they envision this robot’s various social interactions. We prompted children to imagine a fictional robot about their age and follow a stepwise process to design the robot, and make stories about its interactions. Qualitative analysis of this study suggests themes of robots being described as highly-customizable characters with predominantly positive traits, that are imperfect. We also found that children can articulate various scenarios involving robots taking roles of bullies, victims, and bystanders. These findings contribute insights for designing pedagogical robots and anti-bullying interventions for children.

Bodily signals can complement subjective and behavioral measures to analyze human factors, such as user engagement or stress, when interacting with virtual reality (VR) environments. Enabling widespread use of (also the real-time analysis) of bodily signals in VR applications could be a powerful method to design more user-centric, personalized VR experiences. However, technical and scientific challenges (e.g., cost of research-grade sensing devices, required coding skills, expert knowledge needed to interpret the data) complicate the integration of bodily data in existing interactive applications. This paper presents the design, development, and evaluation of an open-source software framework named Excite-O-Meter. It allows existing VR applications to integrate, record, analyze, and visualize bodily signals from wearable sensors, with the example of cardiac activity (heart rate and its variability) from the chest strap Polar H10. Survey responses from 58 potential users determined the design requirements for the framework. Two tests evaluated the framework and setup in terms of data acquisition/analysis and data quality. Finally, we present an example experiment that shows how our tool can be an easy-to-use and scientifically validated tool for researchers, hobbyists, or game designers to integrate bodily signals in VR applications.

Technological advancements in creating and commercializing novel unobtrusive and wearable physiological sensors generate new opportunities to develop adaptive human-robot interaction (HRI) scenarios. Detecting complex human states such as engagement and stress when interacting with social agents could bring numerous advantages to create meaningful interactive experiences. Despite being widely used to explain human behaviors in post-interaction analysis with social agents, using bodily signals to create more adaptive and responsive systems remains an open challenge. This paper presents the development of an open-source, integrative, and modular library created to facilitate the design of physiologically adaptive HRI scenarios. The HRI Physio Lib streamlines the acquisition, analysis, and translation of human body signals to additional dimensions of perception in HRI applications using social robots. The software framework has four main components: signal acquisition, processing and analysis, social robot and communication, and scenario and adaptation. Information gathered from the sensors is synchronized and processed to allow designers to create adaptive systems that can respond to detected human states. This paper describes the library and presents a use case that uses a humanoid robot as a cardio-aware exercise coach that uses heartbeats to adapt the exercise intensity to maximize cardiovascular performance. The main challenges, lessons learned, scalability of the library, and implications of the physio-adaptive coach are discussed.

The use of games as vehicles to study human-robot interaction (HRI) has been established as a suitable solution to create more realistic and naturalistic opportunities to investigate human behavior. In particular, multiplayer games that involve at least two human players and one or more robots have raised the attention of the research community. This article proposes a scoping review to qualitatively examine the literature on the use of multiplayer games in HRI scenarios employing embodied robots aiming to find experimental patterns and common game design elements. We find that researchers have been using multiplayer games in a wide variety of applications in HRI, including training, entertainment and education, allowing robots to take different roles. Moreover, robots have included different capabilities and sensing technologies, and elements such as external screens or motion controllers were used to foster gameplay. Based on our findings, we propose a design taxonomy called Robo Ludens, which identifies HRI elements and game design fundamentals and classifies important components used in multiplayer HRI scenarios. The Robo Ludens taxonomy covers considerations from a robot-oriented perspective as well as game design aspects to provide a comprehensive list of elements that can foster gameplay and bring enjoyable experiences in HRI scenarios.

Virtual Reality (VR) is morphing into a ubiquitous technology by leveraging of smartphones and screenless cases in order to provide highly immersive experiences at a low price point. The result of this shift in paradigm is now known as mobile VR (mVR). Although mVR offers numerous advantages over conventional immersive VR methods, one of the biggest limitations is related with the interaction pathways available for the mVR experiences. Using physiological computing principles, we created the PhysioVR framework, an Open-Source software tool developed to facilitate the integration of physiological signals measured through wearable devices in mVR applications. PhysioVR includes heart rate (HR) signals from Android wearables, electroencephalography (EEG) signals from a low-cost brain computer interface and electromyography (EMG) signals from a wireless armband. The physiological sensors are connected with a smartphone via Bluetooth and the PhysioVR facilitates the streaming of the data using UDP communication protocol, thus allowing a multicast transmission for a third party application such as the Unity3D game engine. Furthermore, the framework provides a bidirectional communication with the VR content allowing an external event triggering using a real-time control as well as data recording options. We developed a demo game project called EmoCat Rescue which encourage players to modulate HR levels in order to successfully complete the in-game mission. EmoCat Rescue is included in the PhysioVR project which can be freely downloaded. This framework simplifies the acquisition, streaming and recording of multiple physiological signals and parameters from wearable consumer devices providing a single and efficient interface to create novel physiologically-responsive mVR applications.

The active monitoring of workload levels has been found to significantly reduce work-related stress. Heart rate and heart rate variability (HRV) measurements via photoplethysmography (PPG) sensors have shown a strong potential to accurately describe daily workload levels. However, due its complexity, HRV is commonly misunderstood and the associated measurements are rarely incorporated for workload monitoring in novel technological devices such as smartwatches and activity trackers. In this paper we explore the potential of consumer-grade smartwatches, equipped with PPG sensors, to assist in the active monitoring of workload during work hours. We develop a prototype that employs the SDNN index, a powerful HRV marker for cardiac resilience to differentiate between high and low workload levels along the work day, and presents feedback in glanceable form, by highlighting workload levels and physical activity over the past hour in 5-minutes blocks at the periphery of the smartwatch. A field study with 9 participants and 3 variations of our prototype attempts to quantify the impact of the HRV feedback over subjective and objective workload as well as users' engagement with the smartwatch. Results showed workload levels as inferred from the PPG sensor to positively correlate with self-reported workload and HRV feedback to result to lower levels of workload as compared to a conventional activity tracker. Moreover, users engaged more frequently with the smartwatch when HRV feedback was presented, than when only physical activity feedback was provided. The results suggest that HRV as inferred from PPG sensors in wearables can effectively be used to monitor workload levels during work hours.

This paper proposes the hybridization between two types of videogames (VG) that can be complementary in motor restoration tasks: the VG based on motion capture systems and the VG based on brain computer interfaces. This hybrid system allow the interaction inside of a VG created for the rehabilitation of standing balance in hemiparetic stroke patients through of movements and modulated mental intentions. The Brain Kinect Interface (BKI) is shown as a tool not only for improve the immersivity in serious VG for health, but as an instrumental arrangement for recording and analyzing of motion capture signals and electro-encephalographic (EEG) signals recorded from the low cost sensors in order to improve the objectivity in analyzing motor recovery process.