CHI ’25, Yokohama, Japan

Robotic avatars can help disabled people extend their reach in interacting with the world. Technological advances make it possible for individuals to embody multiple avatars simultaneously. However, existing studies have been limited to laboratory conditions and did not involve disabled participants. In this paper, we present a real-world implementation of a parallel control system allowing disabled workers in a café to embody multiple robotic avatars at the same time to carry out different tasks. Our data corpus comprises semi-structured interviews with workers, customer surveys, and videos of café operations. Results indicate that the system increases workers’ agency, enabling them to better manage customer journeys. Parallel embodiment and transitions between avatars create multiple interaction loops where the links between disabled workers and customers remain consistent, but the intermediary avatar changes. Based on our observations, we theorize that disabled individuals possess specific competencies that increase their ability to manage multiple avatar bodies.

We explored continuous changes in self-other identity by designing an interpersonal facial morphing experience where the facial images of two users are blended and then swapped over time. Both users’ facial images are displayed side by side, with each user controlling their own morphing facial images, allowing us to create and investigate a multifaceted interpersonal experience. To explore this with diverse social relationships, we conducted qualitative and quantitative investigations through public exhibitions. We found that there is a window of self-identification as well as a variety of interpersonal experiences in the facial morphing process. From these insights, we synthesized a Self-Other Continuum represented by a sense of agency and facial identity. This continuum has implications in terms of the social and subjective aspects of interpersonal communication, which enables further scenario design and could complement findings from research on interactive devices for remote communication.

Generative adversarial networks (GANs) generate high-dimensional vector spaces (latent spaces) that can interchangeably represent vectors as images. Advancements have extended their ability to computationally generate images indistinguishable from real images such as faces, and more importantly, to manipulate images using their inherit vector values in the latent space. This interchangeability of latent vectors has the potential to calculate not only the distance in the latent space, but also the human perceptual and cognitive distance toward images, that is, how humans perceive and recognize images. However, it is still unclear how the distance in the latent space correlates with human perception and cognition. Our studies investigated the relationship between latent vectors and human perception or cognition through psycho-visual experiments that manipulates the latent vectors of face images. In the perception study, a change perception task was used to examine whether participants could perceive visual changes in face images before and after moving an arbitrary distance in the latent space. In the cognition study, a face recognition task was utilized to examine whether participants could recognize a face as the same, even after moving an arbitrary distance in the latent space. Our experiments show that the distance between face images in the latent space correlates with human perception and cognition for visual changes in face imagery, which can be modeled with a logistic function. By utilizing our methodology, it will be possible to interchangeably convert between the distance in the latent space and the metric of human perception and cognition, potentially leading to image processing that better reflects human perception and cognition.

Technological advancement has opened up opportunities for new sports and physical activities. We introduce a concept called machine-mediated teaming, in which a human and a surrogate machine form a team to participate in physical sports games. To understand the experience of machine-mediated teaming and the guidelines for designing the system to achieve the concept, we built a case study system based on tug-of-war. Our system is a sports game played by two against two. One team consists of a player who actually pulls the rope and another player who participates in the physical game by controlling the machine’s actuators. We conducted user studies using this system to investigate the sport experience in this form and to reveal insights to inform future research on machine-mediated teaming. Based on the data obtained from the user studies, we clarified three perspectives, machine stamina, action space, and explicit feedback, that should be considered when designing future machine-mediated teaming systems. The research presented in this paper offers a first step towards exploring how humans and machines can coexist in highly dynamic physical interactions.

Virtual Reality (VR) lets us experiment embodiment. Here we investigate dual embodiment under the prism of dual motor adaptation. We asked participants (N=21) to perform reaching motions in VR with opposite gradual visuomotor perturbations. The participants sequentially switched between 2 VR bodies and had to adapt to the VR body’s perturbation (up to +15° for the VR bodyA, and-15° for the VR bodyB). We then designed a 2$\times$2 within-subject study: 1 factor being the perspective (1stperson or 3rdperson), and 1 factor being the head rotation (without head rotation before the reaching motion or with head rotation before the reaching motion). We found that by providing strong visual cues between bodies (alternating symmetric perspective and/or symmetric head rotation), participants had little awareness of the perturbations, good adaptation, and large aftereffects in both VR bodies. Those elements are consistent with implicit dual adaptation. In contrast, a naive 1st person perspective resulted in little to no adaptation with a high cognitive load and no aftereffects.

Force-feedback enhances digital touch by enabling users to share non-verbal aspects such as rhythm, poses, and so on. To achieve this, interfaces actuate the user’s to touch involuntarily (using exoskeletons or electrical-muscle-stimulation); we refer to this as computer-driven touch. Unfortunately, forcing users to touch causes a loss of their sense of agency. While researchers found that delaying the timing of computer-driven touch preserves agency, they only considered the naïve case when user-driven touch is aligned with computer-driven touch. We argue this is unlikely as it assumes we can perfectly predict user-touches. But, what about all the remainder situations: when the haptics forces the user into an outcome they did not intend or assists the user in an outcome they would not achieve alone? We unveil, via an experiment, what happens in these novel situations. From our findings, we synthesize a framework that enables researchers of digital-touch systems to trade-off between haptic-assistance vs. sense-of-agency.

Abstract: Force feedback devices, such as motor-based exoskeletons or wearables based on electrical muscle stimulation (EMS), have the unique potential to accelerate users’ own reaction time (RT). However, this speedup has only been explored while the device is attached to the user. In fact, very little is known regarding whether this faster reaction time still occurs after the user removes the device from their bodies–this is precisely what we investigated by means of a simple reaction time (RT) experiment, in which participants were asked to tap as soon as they saw an LED flashing. Participants experienced this in three EMS conditions: (1) fast-EMS, the electrical impulses were synced with the LED; (2) agency-EMS, the electrical impulse was delivered 40ms faster than the participant’s own RT, which prior work has shown to preserve one’s sense of agency over this movement; and, (3) late-EMS: the impulse was delivered after the participant’s own RT. Our results revealed that the participants’ RT was significantly reduced by approximately 8ms (up to 20ms) only after training with the agency-EMS condition. This finding suggests that the prioritizing agency during EMS training is key to motor-adaptation, i.e., it enables a faster motor response even after the user has removed the EMS device from their body.

We present an approach for covert visual updates by leveraging change blindness with computationally generated morphed images. To clarify the design parameters for intentionally suppressing change detection with morphing visuals, we investigated the visual change detection in three temporal behaviors: visual blank, eye-blink, and step-sequential changes. The results showed a robust trend of change blindness with a blank of more than 33.3 ms and with eye blink. Our sequential change study revealed that participants did not recognize changes until an average of 57% morphing toward another face in small change steps. In addition, changes went unnoticed until the end of morphing in more than 10% of all trials. Our findings should contribute to the design of covert visual updates without consuming users’ attention by leveraging change blindness with computational visual morphing.

We enable preemptive force-feedback systems to speed up human reaction time without fully compromising the user's sense of agency. Typically these interfaces actuate by means of electrical muscle stimulation (EMS) or mechanical actuators; they preemptively move the user to perform a task, such as to improve movement performance (e.g., EMS-assisted drumming). Unfortunately, when using preemptive force-feedback users do not feel in control and loose their sense of agency. We address this by actuating the user's body, using EMS, within a particular time window (160 ms after visual stimulus), which we found to speed up reaction time by 80 ms in our first study. With this preemptive timing, when the user and system move congruently, the user feels that they initiated the motion, yet their reaction time is faster than usual. As our second study demonstrated, this particular timing significantly increased agency when compared to the current practice in EMS-based devices. We conclude by illustrating, using examples from the HCI literature, how to leverage our findings to provide more agency to automated haptic interfaces.

Growing evidence suggests that the gradual transformation of visuomotor association drives a distinct learning process from abrupt transformation in humans. In the current study, we developed a novel omnidirectional visuomotor transformation paradigm to study details of such difference in more realistic environment than conventional experimental systems. Participants were asked to perform a repetitive three-dimensional (3D) arm-reaching task to a target on a front touch panel, wearing a video see-through head-mounted device that displayed a rotating view of surrounding images. In the abrupt condition, the images were rotated by 20°; in the gradual condition, the rotation was increased in a stepwise-manner from 0° to 20°. In both conditions, pointing errors were decreased after adaptation. Further, although the aftereffect of adaptation was not different between conditions, the speed of decay of the aftereffect, which was quantified by an exponential fit, was slower in the gradual condition, suggesting longer-lasting aftereffects for the gradual shift.

StratoJump is an installation that allows participants to explore the experience of high altitude jumping toward the edge of space. In this peper, we presents an vision-based algorithm for generating hyperlapse of spherical videos captured in high altitude space. Unlike previous work, we consider high altitude videos that cannot perfectly be stabilized by 3D reconstruction-based algorithms. We proposed recursive intelligent sampling to simultaneously stabilize and shorten spherical videos. Our preliminary results show that the proposed method can reduce cumulative errors of stabilization compared to a frame by frame method in reasonable time.

We hypothesize that replacing the visual perception of one's body with a spatial-temporal deformed state would change sensations associated with the body. We developed a system that captures full-body movement and generates estimated past and future body movement by deformation. With a head mounted display, people could see their bodies as slightly deformed. We then investigated 1) how human movement is physically changed, and 2) how humans feel about the change in physical and emotional views of the body due to virtual body deformation. Our results show that spatial-temporal deformation of a virtual body actually changes the sense of body as well as physical movement. For instance, a body image generated at approximately 25-100 ms in the future induced a "lighter weight" sensation. On the basis of our findings, we discuss the design implication of computational control for the physical and emotional sense of body.

Positive social and collaborative effects are hailed as a major advantage of embodied and tangible approaches to interaction. This studio offers a hands-on exploration of potentially extreme versions of such benefits Systems and techniques that somehow share or transfer embodiment between two or more people. Through participatory demos, studio attendees will explore and compare a variety of approaches to experiencing the perspectives of another body, and controlling bodies other than their own. These comparisons will be a launch pad for collaboratively combining existing "body sharing" systems and mocking up new design concepts. By bringing together ideas and approaches in an actionable manner, this studio will share and develop imagination, theory, and skills relevant to the design and study of interactive systems in which the body plays a central role.

Sharing one's own immersive experience over the Internet is one of the ultimate goals of telepresence technology. In this paper, we present JackIn Head, a visual telepresence system featuring an omnidirectional wearable camera with image motion stabilization. Spherical omnidirectional video footage taken around the head of a local user is stabilized and then broadcast to others, allowing remote users to explore the immersive visual environment independently of the local user's head direction. We describe the system design of JackIn Head and report the evaluation results of real-time image stabilization and alleviation of cybersickness. Then, through an exploratory observation study, we investigate how individuals can remotely interact, communicate with, and assist each other with our system. We report our observation and analysis of inter-personal communication, demonstrating the effectiveness of our system in augmenting remote collaboration.

Our research explores how humans can understand and develop viewing behaviors with mutual paralleled first person view sharing in which a person can see others' first person video perspectives as well as their own perspective in realtime. We developed a paralleled first person view sharing system which consists of multiple video see-through head mounted displays and an embedded eye tracking system. With this system, four persons can see four shared first person videos of each other. We then conducted workshop based research with two activities, drawing pictures and playing a simple chasing game with our view sharing system. Our results show that 1) people can complement each other's memory and decisions and 2) people can develop their viewing behaviors to understand their own physical embodiment and spatial relationship with others in complex situations. Our findings about patterns of viewing behavior and design implications will contribute to building design experience in paralleled view sharing applications.

Sharing an immersive experience enhances situational awareness, enabling effective collaboration between persons in different places. The development of a head-worn camera enables us to capture the first-person view and share it as the personal experience. However, the limited viewing angle of such cameras prevents a remote viewer from obtaining a complete surrounding situation. In this paper, we propose a system called LiveSphere, which presents the entire surrounding visual environment using a head-worn system with multiple cameras. The image-stabilizing algorithm compensates image motion caused by the wearer's head movement. This enables the remote viewer to look around the environment independently from the wearer's head direction. We developed a prototype to examine how sharing the surrounding visual environment improves collaboration between persons in different places.

Remote collaboration to share abilities over the Internet is one of the ultimate goals of telepresence technology. In this paper, we present JackIn Head, a visual telepresence system with an omnidirectional wearable camera with image motion stabilization. Spherical omnidirectional video footage taken around the head of a local user is stabilized and then broadcast to others, allowing remote users to explore the scene independently of the local user's head direction. We describe the system design of JackIn Head and report the evaluation results of the system's motion decoupling. Then, through an exploratory observation study, we investigate how individuals can remotely interact, communicate with, and assist each other with our system. We report our observation and analysis of inter-personal communication, demonstrating the effectiveness of our system in augmenting remote collaboration.

As domestic robots and smart appliances become increasingly common, they require a simple, universal interface to control their motion. Such an interface must support a simple selection of a connected device, highlight its capabilities and allow for an intuitive manipulation. We propose "exTouch", an embodied spatially-aware approach to touch and control devices through an augmented reality mediated mobile interface. The "exTouch" system extends the users touchscreen interactions into the real world by enabling spatial control over the actuated object. When users touch a device shown in live video on the screen, they can change its position and orientation through multi-touch gestures or by physically moving the screen in relation to the controlled object. We demonstrate that the system can be used for applications such as an omnidirectional vehicle, a drone, and moving furniture for reconfigurable room.