Non-player characters (NPCs) are a vital part of many 3D and virtual reality (VR) experiences, but animating them can be a time-consuming and costly process, often requiring expensive motion-capture setups, especially when tracking multiple people simultaneously. Single-user record and replay in VR enables a single user to animate multiple NPCs on their own, offering a cheaper and more convenient solution as the devices are consumer-grade and already track user movements. A limitation of single-user record and replay is that an early recorded character has no information about a later recorded character and might not be able to interact with them believably. We therefore study interactivity between NPCs as an emergent property when actors can see other characters in later recordings as opposed to acting primarily on their own in the first recording. We conducted a user study (N=144) where participants compared first, second, and third recording runs of different social scenarios with multiple NPCs. Our results suggest that for scenarios with low and medium interactivity where timing is not crucial and characters are not in close contact, a single run can be sufficient, whereas for high-interactivity scenarios, with close-contact group interactions, two recordings are more appropriate.

We compared the body movements of five virtual reality (VR) avatar representations in a user study $(\mathrm{N}=53)$ to ascertain how well these representations could convey body motions associated with different emotions: one head-and-hands representation using only tracking data, one upper-body representation using inverse kinematics (IK), and three full-body representations using IK, motioncapture, and the state-of-the-art deep-learning model AGRoL. Participants’ emotion detection accuracies were similar for the IK and AGRoL representations, highest for the full-body motion-capture representation and lowest for the head-and-hands representation. Our findings suggest that from the perspective of emotion expressivity, connected upper-body parts that provide visual continuity improve clarity, and that current techniques for algorithmically animating the lower-body are ineffective. In particular, the deep-learning technique studied did not produce more expressive results, suggesting the need for training data specifically made for social VR applications.

Online social virtual worlds are now becoming widely available on consumer devices including virtual reality headsets. One goal of a virtual world could be to give a user an experience of a crowded environment with many virtual humans. However, gathering enough personnel to control the necessary number of avatars for creating a realistic scene is usually difficult. Additionally, current technology is not capable of fully simulating avatars with behaviours, especially when interaction with users is required. In this paper, we develop a system that enables an actor to take over control of one of a set of avatars. We built an immersive interface that allows an actor to select an avatar to take over and then segue into the currently playing animation. By allowing one person to take control of multiple avatars, we can enhance the plausibility of environments inhabited by simulated characters. In an experiment, we show that in a cafe scenario, one actor can take over the roles of a barista and two customers. Experiment participants reported experiencing the scene as if it were populated by more than one actor. This system and experiment demonstrate the feasibility of one actor controlling multiple avatars sequentially, thus enhancing users’ feelings of being in a populated environment.

Non-player characters are an essential element of many 3D and virtual reality experiences. They can make the experiences feel more lively and populated. Animation for non-player characters is often motion-captured using expensive hardware and the post-processing steps are time-consuming, especially when capturing multiple people at once. Using record and replay techniques in virtual reality can offer cheaper and easier ways of motion capture since the user is already tracked. We use immersive record and replay to enable a single user to create stacked recordings of themselves. We provide tools to help the user interact with their previous recorded self and in doing so allow them to create believable interactive scenarios with multiple characters that can be used to populate virtual environments. We create a small dialogue dataset with two amateur actors who used our tool to record dialogues alone and together in virtual reality. To evaluate whether stacked recordings are qualitatively comparable to conventional multi-user recordings and whether people could tell the difference between the two, we conducted two user studies, one online and one in virtual reality with 89 participants in total. We found that participants could not tell the difference and even slightly preferred stacked recordings.

While they have long been a subject of academic study, social virtual reality (SVR) systems are now attracting increasingly large audiences on current consumer virtual reality systems. The design space of SVR systems is very large, and relatively little is known about how these systems should be constructed in order to be usable and efficient. In this paper we present Ubiq, a toolkit that focuses on facilitating the construction of SVR systems. We argue for the design strategy of Ubiq and its scope. Ubiq is built on the Unity platform. It provides core functionality of many SVR systems such as connection management, voice, avatars, etc. However, its design remains easy to extend. We demonstrate examples built on Ubiq and how it has been successfully used in classroom teaching. Ubiq is open source (Apache License) and thus enables several use cases that commercial systems cannot.