integrating system-led optimization methods with Large Language Models (LLMs), allowing designers to intervene in the optimization process and better understand the system's reasoning.Experimental results show that our method provides higher user agency than a system-led method and shows promising optimization performance compared to manual design.It also matches the performance of an existing cooperative method with lower cognitive load.

The replication of object stiffness is essential for enhancing haptic feedback in virtual environments. However, existing research has overlooked how stylus stiffness influences the perception of virtual object stiffness during tool-mediated interactions. To address this, we conducted a psychophysical experiment demonstrating that changing stylus stiffness combined with visual stimuli altered users’ perception of virtual object stiffness. Based on these insights, we developed Transtiff, a stylus-shaped interface capable of on-demand stiffness control using a McKibben artificial muscle mechanism. Unlike previous approaches, our method manipulates the perceived stiffness of virtual objects via the stylus by controlling the stiffness of the stylus without altering the properties of the real object being touched, creating the illusion of a hard object feeing soft. Our user study confirmed that Transtiff effectively simulates a range of material properties, such as sponge, plastic, and tennis balls, providing haptic rendering that is closely aligned with the perceived material characteristics. By addressing the challenge of delivering realistic haptic feedback through tool-based interactions, Transtiff represents a significant advancement in the haptic interface design for VR applications.

Interpersonal communication often involves navigating social challenges like managing expectations and conveying intentions. In practical contexts like productivity and navigation of social boundaries, agent- and AI-mediated communication (AIMC) has served as an effective social intermediary. Communication within personal relationships, especially between intercultural friends from different backgrounds, can also face significant challenges, such as miscommunication and expressive suppression. However, AIMC remains underutilized in the context of established personal relationships due to ethical concerns about agency and authenticity. We propose designing openly-interpretable AIMC output as an augmented context cue to reduce its active social involvement, balancing AI support with preservation of relational agency. We developed and evaluated Poet-Weaver, a Discord text chat plugin that presents AI-generated insights on user conversations in a stylized, openly-interpretable way to encourage reflection on communication challenges. We conducted a mixed-methods study with 30 intercultural friend pairs to assess Poet-Weaver's impact. Findings showed Poet-Weaver effectively helped participants address communication failures, although AIMC still influenced users' behavior even with openly-interpretable output. We recommend future uses of AIMC that support transformative, positive relationship changes while preserving individual responsibility and identity within relationships.

The human brain’s plasticity allows for the integration of artificial body parts into the human body. Leveraging this, embodied systems realize intuitive interactions with the environment. We introduce a novel concept: embodied swarm robots. Swarm robots constitute a collective of robots working in harmony to achieve a common objective, in our case, serving as functional body parts. Embodied swarm robots can dynamically alter their shape, density, and the correspondences between body parts and individual robots. We contribute an investigation of the influence on embodiment of swarm robot-specific factors derived from these characteristics, focusing on a hand. Our paper is the first to examine these factors through virtual reality (VR) and real-world robot studies to provide essential design considerations and applications of embodied swarm robots. Through quantitative and qualitative analysis, we identified a system configuration to achieve the embodiment of swarm robots.

Robot swarms hold immense potential for performing complex tasks far beyond the capabilities of individual robots. However, the challenge in unleashing this potential is the robots’ limited sensory capabilities, which hinder their ability to detect and adapt to unknown obstacles in real-time. To overcome this limitation, we introduce a novel robot swarm control method with an indirect obstacle detector using a smoothed particle hydrodynamics (SPH) model. The indirect obstacle detector can predict the collision with an obstacle and its collision point solely from the robot’s velocity information. This approach enables the swarm to effectively and accurately navigate environments without the need for explicit obstacle detection, significantly enhancing their operational robustness and efficiency. Our method’s superiority is quantitatively validated through a comparative analysis, showcasing its significant navigation and pattern formation improvements under obstacle-unaware conditions.

Half a century since the concept of a cyborg was introduced, digital cyborgs, enabled by the spread of wearable robotics, are the focus of much research in recent times. We introduce JIZAI ARMS, a supernumerary robotic limb system consisting of a wearable base unit with six terminals and detachable robot arms controllable by the wearer. The system was designed to enable social interaction between multiple wearers, such as an exchange of arm(s), and explore possible interactions between digital cyborgs in a cyborg society. This paper describes the JIZAI ARMS’ design process, an interdisciplinary collaboration between human augmentation researchers, product designers, a system architect, and manufacturers, to realize a technically complex system while considering the aesthetics of a digital cyborg. We also provide an autobiographical report of our first impressions of using the JIZAI ARMS and use our findings to speculate on a model of potential social interactions between digital cyborgs.

When receiving text messages from unacquainted colleagues in fully remote workplaces, insufficient mutual understanding and limited social cues can lead people to misinterpret the tone of the message and further influence their impression of remote colleagues. Emojis have been commonly used for supporting expressive communication; however, people seldom use emojis before they become acquainted with each other. Hence, we explored how changing facial expressions in profile pictures could be an alternative channel to communicate socio-emotional cues. By conducting an online controlled experiment with 186 participants, we established that changing facial expressions of profile pictures can influence the impression of the message receivers toward the sender and the message valence when receiving neutral messages. Furthermore, presenting incongruent profile pictures to positive messages negatively affected the interpretation of the message valence, but did not have much effect on negative messages. We discuss the implications of affective profile pictures in supporting text-based communication.

Timbre is high-dimensional and sensuous, making it difficult for musical-instrument learners to improve their timbre. Although some systems exist to improve timbre, they require expert labeling for timbre evaluation; however, solely visualizing the results of unsupervised learning lacks the intuitiveness of feedback because human perception is not considered. Therefore, we employ crossmodal correspondences for intuitive visualization of the timbre. We designed TimToShape, a system that visualizes timbre with 2D shapes based on the user’s input of timbre–shape correspondences. TimToShape generates a shape morphed by linear interpolation according to the timbre’s position in the latent space, which is obtained by unsupervised learning with a variational autoencoder (VAE). We confirmed that people perceived shapes generated by TimToShape to correspond more to timbre than randomly generated shapes. Furthermore, a user study of six violin players revealed that TimToShape was well-received in terms of visual clarity and interpretability.

Apparent color and pixel color are known to be different due to color constancy and contexts of images. An apparent color dataset for photographs and images will be a basis for developing perception-based color extraction and transfer system. In this study, we collected the apparent color dataset for a given image and extracted position through an online experiment that mimics the situation of using a color extraction system. Additionally, we analyzed 1110 data and verified the difference between apparent color and pixel color. As a result, approximately 80% data has $\Delta E_{\mathrm{NBS}} \gt 6.0$ color difference that is defined “Much” or “Very Much” difference by the national bureau of standards (NBS). Luminance (L*) of the apparent color was on average $\triangle E_{00}=[9.90,11.9]$ (95% confidence interval) greater than that of the pixel color. These results indicate that the color deviations cannot be ignored when apparent colors are extracted in images.

Humans can estimate the properties of wielded objects (e.g., inertia and viscosity) using the force applied to the hand. We focused on this mechanism and aimed to represent the properties of wielded objects by dynamically changing the force applied to the hand. We propose MetamorphX, which uses control moment gyroscopes (CMGs) to generate ungrounded, 3-degrees of freedom moment feedback. The high-response moments obtained CMGs allow the inertia and viscosity of motion to be set to the desired values via impedance control. A technical evaluation indicated that our device can generate a moment with a 60-ms delay. The inertia and viscosity of motion were varied by 0.01 kgm2 and 0.1 Ns, respectively. Additionally, we demonstrated that our device can dynamically change the inertia and viscosity of motion through virtual reality applications.

Emotional contagion is a phenomenon in which one’s emotions are transmitted among individuals unconsciously by observing others’ emotional expressions. In this paper, we propose a method for mediating people’s emotions by triggering emotional contagion through artificial bodily changes such as pseudo tears. We focused on shedding tears because of the link to several emotions besides sadness. In addition, it is expected that shedding tears would induce emotional contagion because it is observable by others. We designed an eyeglasses-style wearable device, Teardrop glasses, that release water drops near the wearer’s eyes. The drops flow down the cheeks and emulate real tears. The study revealed that artificial crying with pseudo tears increased sadness among both wearers and those observing them. Moreover, artificial crying attenuated happiness and positive feelings in observers. Our findings show that actual bodily changes are not necessary for inducing emotional contagion as artificial bodily changes are also sufficient.

Several virtual reality (VR) proxies have been developed that can emulate impact sensations by generating actual forces on the hand. Although these proxies contribute to increasing the reality of VR, they still have some limitations, such as high latency, high power consumption, and low frequency to provide impact sensations. To overcome these limitations, we first propose a method to provide an impact sensation without actual force generation by quickly changing the rotational inertia of a handheld proxy while users are swinging it. Then, we developed Unident, a handheld proxy capable of changing its rotational inertia by moving a weight along one axis at a high speed. Two experiments were conducted to evaluate the ability of Unident to provide users with impact sensations. In the first experiment, we demonstrate that Unident can physically provide an impact sensation applied to a handheld object by analyzing the pressure on the user's palm. The second experiment shows that Unident can provide an impact sensation with various magnitudes depending on the amount of rotational inertia to be changed. Finally, we present an application that can be enabled by Unident.

Timbre is a crucial element in playing musical instruments, and it is difficult for beginners to learn it independently. Therefore, external feedback (FB) is required. However, conventional FB methods lack intuitiveness in visualization. In this study, we propose a novel FB method that adopts crossmodal correspondence to enhance the intuitive visualization of timbre with visual shapes. Based on the experiments, it was inferred that the FB based on crossmodal correspondence prevents dependence on FB and promotes learning.

We introduce PoCoPo, the first handheld pin-based shape display that can render various 2.5D shapes in hand in realtime. We designed the display small enough for a user to hold it in hand and carry it around, thereby enhancing the haptic experiences in a virtual environment. PoCoPo has 18 motor-driven pins on both sides of a cuboid, providing the sensation of skin contact on the user's palm and fingers. We conducted two user studies to understand the capability of PoCoPo. The first study showed that the participants were generally successful in distinguishing the shapes rendered by PoCoPo with an average success rate of 88.5%. In the second study, we investigated the acceptable visual size of a virtual object when PoCoPo rendered a physical object of a certain size. The result led to a better understanding of the acceptable differences between the perceptions of visual size and haptic size.

Humans can estimate the shape of a wielded object through the illusory feeling of the mass properties of the object obtained using their hands. Even though the shape of hand-held objects influences immersion and realism in virtual reality (VR), it is difficult to design VR controllers for rendering desired shapes according to the perceptions derived from the illusory effects of mass properties and shape perception. We propose Transcalibur, which is a hand-held VR controller that can render a 2D shape by changing its mass properties on a 2D planar area. We built a computational perception model using a data-driven approach from the collected data pairs of mass properties and perceived shapes. This enables Transcalibur to easily and effectively provide convincing shape perception based on complex illusory effects. Our user study showed that the system succeeded in providing the perception of various desired shapes in a virtual environment.

We present PaCaPa, a handheld device that renders haptics on a user's palm when the user interacts with virtual objects using virtual tools such as a stick. PaCaPa is a cuboid device with two wings that open and close. As the user's stick makes contact with a virtual object, the wings open by a specific degree to dynamically change the pressure on the palm and fingers. The open angle of the wings is calculated from the angle between the virtual stick and hand direction. As the stick bites into the target object, a large force is generated. Our device enables three kinds of renderings: size, shape, and stiffness. We conducted user studies to evaluate the performance of our device. We also evaluated our device in two application scenarios. User feedback and qualitative ratings indicated that our device can make indirect interaction with handheld tools more realistic.

During group decision-making, a group with an imbalanced minority and majority typically suffers normative bias. This bias creates socio-emotional conflict and decreases group member consent and decision power. Thus, we propose a novel video-chat system, "Divided Presence," which aims to reduce this bias by equalizing the apparent number of majority and minority participants using a pseudo-population increase. Members of a discussion use computer graphics avatars on monitors instead of their actual video appearances during communication. The apparent number of discussion members increases when two avatars are assigned to an arbitrary minority member; the avatars then speak on behalf of him or her. We evaluate the system with a three-person consensus game, where a minority member is assigned two avatars. The results show that our system increases the degree of consent among the majority participants when they finally agree to the minority opinion.

PaCaPa is a handheld device that presents haptic stimuli on a user's palm when the user interacts with virtual objects using virtual tools. It is a box-shaped device and has two wings. This device can present the contact force and contact angle by opening and closing the wings based on the angle between the direction of the virtual tool and hand. By changing these haptic sensations on the palm and fingers, it enables the users to perceive different size, shape, and stiffness of virtual objects.

"Mirroring" refers to the unconscious mimicry of another person's behaviors, such as their facial expressions. Mirroring has many positive effects, such as enhancing closeness and improving the flow of a conversation, which enriches the quality of communication. Our study set out to devise a means of evoking these positive effects in a video chat without any conscious effort of participants. We constructed a videophone system, called FaceShare, which can deform the user's face into a smile in response to their partner's smiling. That is, our system generates mirroring by producing a pseudo-smile through image processing. We conducted an experiment in which pairs of participants had brief conversations via FaceShare. The results implied that mirroring using the pseudo-smile lets the mimicker, whose face is deformed according to the expressions of their partner, feel a closeness, and improves the flow of the conversation for both the mimicker and the mimickee, who sees the mimicker's deformed face.

To effectively utilize gaze in one-to-many communication, robots must direct and avert their gaze with appropriate timing. However, few studies have focused on this aspect of communication because of the difficulty in detecting the position of many gaze recipients and directing appropriate gazes to each. This paper proposes a system of robotic eyes that can send averted or direct gazes simultaneously to multiple recipients by switching between convex and hollow eye shapes, which we call Transgazer.

Humans are capable of haptically perceiving the shape of an object by simply wielding it, even without seeing it. On the other hand, typical hand-held controllers for virtual reality (VR) applications are pre-designed for general applications, and thus not capable of providing appropriate haptic shape perception when wielding specific virtual objects. Contradiction between haptic and visual shape perception causes a lack of immersion and leads to inappropriate object handling in VR. To solve this problem, we propose a novel method for designing hand-held VR controllers which illusorily represent haptic equivalent of visual shape in VR. In ecological psychology, it has been suggested that the perceived shape can be modeled using the limited mass properties of wielded objects. Based on this suggestion, we built a shape perception model using a data-driven approach; we aggregated data of perceived shapes against various hand-held VR controllers with different mass properties, and derived the model using regression techniques. We implemented a design system which enables automatic design of hand-held VR controllers whose actual shapes are smaller than target shapes while maintaining their haptic shape perception. We verified that controllers designed with our system can present aimed shape perception irrespective of their actual shapes.

Surface and object recognition is of significant importance in ubiquitous and wearable computing. While various techniques exist to infer context from material properties and appearance, they are typically neither designed for real-time applications nor for optically complex surfaces that may be specular, textureless, and even transparent. These materials are, however, becoming increasingly relevant in HCI for transparent displays, interactive surfaces, and ubiquitous computing. We present SpecTrans, a new sensing technology for surface classification of exotic materials, such as glass, transparent plastic, and metal. The proposed technique extracts optical features by employing laser and multi-directional, multi-spectral LED illumination that leverages the material's optical properties. The sensor hardware is small in size, and the proposed classification method requires significantly lower computational cost than conventional image-based methods, which use texture features or reflectance analysis, thereby providing real-time performance for ubiquitous computing. Our evaluation of the sensing technique for nine different transparent materials, including air, shows a promising recognition rate of 99.0%. We demonstrate a variety of possible applications using SpecTrans' capabilities.

This study develops a method for improving creativity by changing the facial appearance of people during video conferences. We focus on enhancing creativity during human interactions, especially cooperative work situations such as video conferences. Psychological studies have revealed that the facial appearance of people can affect creativity, i.e., "emotion can affect creativity" and "the appearance of others can affect the emotions." Based on the knowledge, we hypothesized that the media technology which modifies the facial appearance of people appropriately can enhance our creativity during cooperative work. Therefore, we develop a method to affect creativity by changing the facial appearance in real-time, such as facial expressions and facial resemblance. We test the effectiveness of this method by building a system for collaborative video conferences. The user study show that the proposed method could enhance creativity.

Audio/video (AV) devices capable of IP communications over Ethernet are evolving into ubiquitous network home appliances. However, cabling an Ethernet network in existing dwellings is a difficult challenge that hinders their popularization. To overcome this obstacle, we have focused our attention on high-speed power line communications technology, which enables 100Base-T class IP networking using existing power lines as the communication medium. However, power line channel characteristics are fairly unstable and it is very difficult to provide stable transport of mixed streams (such as video contents, voice-over-IP, audio and data transfers) requiring different communication quality. We propose a real time error handling technique to alleviate the effects of errors by quickly responding to changes in power line channel characteristics and a MAC (media access control) technique to guarantee the transmission of buffered data using intelligent TDMA (time division multiple access). Through simulation results, we verify that the proposed techniques achieve stable AV stream transport.

We introduce a robust method for detecting evolutionary trends of gene expression from a temporal sequence of microarray data. In this method we perform gene clustering via multi-objective optimization to reveal genes with interesting and statistically significant temporal patterns. We illustrate this gene filtering methodology in the context of exploring the time trajectories of mouse retinal genes acquired at different points over the lifetimes of a population of mice. For 6 time points sampled over 24 mouse subjects, our method can reliably reveal genes whose expression level increases or decreases monotonically, hits a peak or valley at birth, or exhibits other trends.