While developing a story, novices and published writers alike have had to look outside themselves for inspiration. Language models have recently been able to generate text fluently, producing new stochastic narratives upon request. However, effectively integrating such capabilities with human cognitive faculties and creative processes remains challenging. We propose to investigate this integration with a multimodal writing support interface that offers writing suggestions textually, visually, and aurally. We conduct an extensive study that combines elicitation of prior expectations before writing, observation and semi-structured interviews during writing, and outcome evaluations after writing. Our results illustrate the individual and situational variation in machine-in-the-loop writing approaches, suggestion acceptance, and ways the system is helpful. Centrally, we report how participants perform integrative leaps , by which they do cognitive work to integrate suggestions of varying semantic relevance into their developing stories. We interpret these findings, offering modeling and design recommendations for future creative writing support technologies.

The pairing of Virtual Reality technology with Physiological Sensing has gained much interest in clinical settings and beyond: from developing novel methods for diagnosis of perception and cognition impairments, biofeedback for anxiety treatment, to enhancing everyday practices such as self-guided meditation. However, conducting this type of research does not come without challenges. For example, accessing the equipment for recording data from the user and synchronizing physiological response data with the stimuli or interactive environment are not trivial tasks, and generating virtual content in response to the user’s real-time data is costly and complex. This paper presents Galea, a device for multi-modal signal acquisition able to measure the physiological response of a user when experiencing virtual content, enabling behavioral, affective computing , and human-computer interaction research and applications to access data from the Parasympathetic nervous system and Sympathetic nervous system simultaneously. We present a primer on detectable human physiology as an input source for Physiological Computing from the perspective of the signals available through our device. We describe the primary design considerations and circuit characterization results of in-vivo recordings from the wearer’s brain, eyes, heart, skin, and muscles. We also present an example to help contextualize how these signals can be used in a virtual reality setting. Galea makes working with physiological sensors in virtual reality more accessible and can offer a standard for inter and intra experiment data comparisons. Lastly, we discuss the importance and contributions of this work as well as future challenges that need to be considered.

Virtual and augmented reality headsets are unique as they have access to our facial area: an area that presents an excellent opportunity for always-available input and insight into the user's state. Their position on the face makes it possible to capture bio-signals as well as facial expressions. This paper introduces the PhysioHMD, a software and hardware modular interface built for collecting affect and physiological data from users wearing a head-mounted display. The PhysioHMD platform is a flexible architecture enables researchers and developers to aggregate and interprets signals in real-time, and use those to develop novel, personalized interactions and evaluate virtual experiences. Offering an interface that is not only easy to extend but also is complemented by a suite of tools for testing and analysis. We hope that PhysioHMD can become a universal, publicly available testbed for VR and AR researchers.

The sense of agency (SoA) describes the feeling of being the author and in control of one's movements. It is closely linked to automated aspects of sensorimotor control and understood to depend on one's ability to monitor the details of one's movements. As such SoA has been argued to be a critical component of self-awareness in general and contribute to presence in virtual reality environments in particular. A common approach to investigating SoA is to ask participants to perform goal-directed movements and introducing spatial or temporal visuomotor mismatches in the feedback. Feedback movements are traditionally either switched with someone else's movements using a 2D video-feed or modified by providing abstracted feedback about one's actions on a computer screen. The aim of the current study was to quantify conscious monitoring and the SoA for ecologically valid, three dimensional feedback of the participants' actual limb and movements. This was achieved by displaying an Infra-Red (IR) feed of the participants' upper limbs in an augmented virtuality environment (AVE) using a head-mounted display (HMD). Movements could be fed back in real-time (46ms system delay) or with an experimental delay of up to 570ms. As hypothesized, participant's SoA decreased with increasing temporal visuomotor mismatches (p<;.001), replicating previous findings and extending them to AVEs. In-line with this literature, we report temporal limits of 222±60ms (50% psychometric threshold) in N=28 participants. Our results demonstrate the validity of the experimental platform by replicating studies in SoA both qualitatively and quantitatively. We discuss our findings in relation to the use of virtual and mixed reality in research and implications for neurorehabilitation therapies.