There is growing interest in communication artifacts that allow users to connect with others while supporting the emotional well-being that often arises from social interactions. Social presence is central in digital interactions, as it fosters a sense of “being together,” which is essential for affective communication, collaboration, and relationship-building in computer-mediated environments. This study explores the impact of behavioral synchronization through HugBits, a cushion -shaped device designed for remote hug-based interaction, on both perceived social presence and users' prosociality. We conducted two experiments: (1) testing whether hug synchronization increased perceived social presence, and (2) assessing whether synchronization influenced cooperative decision-making in a Prisoner's Dilemma game. In both experiments, the occurrence of behavioral synchronization (i.e., hugging) was controlled by having participants interact with a virtual agent that was designed to probabilistically promote or avoid synchronized hugs. Results show that more synchronized interactions significantly enhanced perceived social presence and promoted cooperative behavior. These findings highlight behavioral synchronization as a promising design strategy for affective communication technologies and provide concrete implications for integrating emotional and behavioral outcomes in computer-mediated communication research.

Single-sided deafness (SSD) significantly restricts social participation in hearing/speaking cultures due to the person’s difficulty hearing conversations on their deaf side. Although hearing aids for SSD are effective in social situations, the acceptance rate remains low at 4%. To address this problem, we designed and developed a bone conduction-based device to be worn with eyeglasses, involving 53 individuals with SSD including two authors. We conducted a four-week diary study comparing our proposed device with traditional Contralateral Routing of Signals (CROS) hearing aids and explored the factors that might affect the acceptance rate of assistive devices for SSD. The findings indicated that our design was more acceptable for users with SSD due to its effectiveness, social acceptability, and the ability for wearers to use other devices simultaneously, such as earbuds. Based on our results, we discuss implications for designing wearable assistive devices to promote greater acceptance among the target population.

Text chat applications are an integral part of daily social and professional communication. However, messages sent over text chat applications do not convey vocal or nonverbal information from the sender, and detecting the emotional tone in text-only messages is challenging. In this paper, we explore the effects of speech balloon shapes on the sender-receiver agreement regarding the emotionality of a text message. We first investigated the relationship between the shape of a speech balloon and the emotionality of speech text in Japanese manga. Based on these results, we created a system that automatically generates speech balloons matching linear emotional arousal intensity by Auxiliary Classifier Generative Adversarial Networks (ACGAN). Our evaluation results from a controlled experiment suggested that the use of emotional speech balloons outperforms the use of emoticons in decreasing the differences between message senders’ and receivers’ perceptions about the level of emotional arousal in text messages.

Group exercise is more effective for gaining motivation than exercising alone, but it can be difficult to always find such partners. In this paper, we explore the experiences that joggers have with a virtual partner instead of a human partner and report on the results of two controlled experiments evaluating our approach. In Study 1, we investigated how participants felt and how their behav-ior changed when they jogged indoors with a human partner or with a virtual partner compared to solitary jogging. The virtual partner was represented either as a full-body, limb-only, or a point-light avatar displayed on smartglasses. In Study 2, we investigated the differences between the three representations as virtual partners for casual joggers in an outdoor setting. Based on our results, we propose implications for the design of virtual runners as casual jogging partners and speculate on their relationship with human users.

Augmented reality (AR) optical-see-through (OST) head-mounted displays (HMD) have developed to a point where browsing information on the go is possible. In this paper, we investigate the readability of text on an AR HMD while the user is walking. There are two common methods of displaying text on a HMD: anchoring the text on the screen coordinate system or the world coordinate system. We report on the results of two laboratory experiments comparing text readability when the text is displayed in these two coordinate systems, and while the participants walked on a treadmill. In the first experiment, the participants read letter strings comprising Sloane letters, whereas the second experiment used English words. In addition to evaluating the text readability and workload experienced by participants, we employed IMU sensors to compare the effects of the text display method on the participants' head movement and gait. In both experiments, the reading speed and head movement were significantly higher and mental workload significantly lower for the world coordinate system than for the screen coordinate system. These results suggest that text readability while walking is better on the world coordinate system, and displaying text with the screen coordinate system results in an unnatural gait owing to the user trying to keep their head still in an effort to stabilize the HMD screen.

The meaning we attribute to another's actions significantly impact our subsequent behaviors and interactions towards that person. Distributed teams often combine native speakers (NS) and non-native speakers (NNS) and are particularly prone to making attribution errors. Language difficulties place NNS under a higher cognitive load, potentially leading NS to make inaccurate attributions of NNS. We conducted an exploratory laboratory study to investigate the attributions NS and NNS form about each other in multiparty videoconferencing. Our findings revealed significant mismatches in NS' attributions of NNS behavior, but no significant mismatch in NNS' attributions of NS behavior. Due to cognitive overload stemming from language challenges, NNS were only able to engage in "compromised" impression management during the task. Yet, NS were relatively unaware of how profoundly language difficulties impacted NNS' behaviors. Our findings identify opportunities for technology support for NS-NNS interactions, particularly with regards to impression construction and impression management.

Conversational grounding, or establishing mutual knowledge that messages have been understood as intended, can be difficult to achieve when some conversational participants are using a non-native language. These difficulties in grounding can be challenging for native speakers to detect. In this paper, we examine the value of signaling potential grounding problems to native speakers (NS) by displaying how non-native speakers (NNS) use automated transcripts and bilingual dictionaries. We conducted a laboratory experiment in which NS and NNS of English collaborated via audio conferencing on a map navigation task. Triads of one NS guider, one NS follower, and one NNS follower performed the task using one of three awareness displays: (a) a no awareness display that showed only the automated transcripts, (b) a general awareness display that showed whether each follower was reading the automated transcripts and/or translating a word; or (c) a detailed awareness display that showed which line of the transcripts a follower was reading and/or which words he/she was translating. NS guiders and NNS followers collaborated most successfully with the detailed awareness display, while NS guiders and NS followers performed equally across conditions. Our findings suggest several ways to improve systems to support multilingual collaboration.

Real-time transcripts generated by automated speech recognition (ASR) technologies have the potential to facilitate communication between native speakers (NS) and non-native speakers (NNS). Previous studies of ASR have focused on how transcripts aid NNS speech comprehension. In this study, we examine whether transcripts benefit multiparty real-time conversation between NS and NNS. We hypothesized that ASR transcripts would be more beneficial when the transcripts were publicly shared by all group members as opposed to when they were seen only by the NNS. To test our hypothesis, we conducted a lab experiment in which 14 groups of native and non-native speakers engaged in a story-telling task. Half of the groups received private transcripts that were available only to the NNS; the other half received publicly shared transcripts that were available to all group members. NS spoke more clearly, and both NS and NNS rated the quality of communication higher, when transcripts were publicly shared. These findings inform the design of future tools to support multilingual group communication.

Previous studies have shown that people can effectively detect emotions in text-only messages written in their native languages. But is this the same for non-native speakers' In this paper, we conduct an experiment where native English speakers (NS) and Japanese non-native English speakers (NNS) rate the emotional valence in text-only messages written by native English-speaking authors. They also annotate all emotional cues (words, symbols and emoticons) that affected their rating. Accuracy of NS and NNS ratings and annotations are calculated by comparing their average correlations with author ratings and annotations used as a gold standard. Our results conclude that NNS are significantly less accurate at detecting the emotional valence of messages, especially when the messages include highly negative words. Although NNS are as accurate as NS at detecting emotional cues, they are not able to make use of symbols (exclamation marks) and emoticons to detect the emotional valence of text-only messages.

Wikipedia translation activities aim to improve the quality of the multilingual Wikipedia through article translation. We performed an activity analysis of the translation work done by individual English to Chinese non-expert translators, who translated linguistically complex Wikipedia articles in a laboratory setting. From the analysis, which was based on Activity Theory, and which examined both information search and translation activities, we derived three translation strategies that were used to inform the design of a support system for human translation activities in Wikipedia.

Previous research has shown that audio communication is particularly difficult for non-native speakers (NNS) during multilingual collaborations. Especially when audio signals become distorted, NNS are overburdened by not only having to communicate with imperfect language skills, but also compensating for the deteriorations. Under these faulty audio conditions, NNS need to pay extra time and effort to understand the conversation. In order to give NNS more time to process conversations, we tested the insertion of silent gaps (from 0.2 to 0.4 seconds) between conversational turns. First, gaps were inserted into a previously taped conversation, resulting in a significant improvement of NNS's understanding of the conversation. Second, gaps were inserted during a real-time audio conference by adding artificial delay between native speakers. The results show that the added delays have a combination of beneficial and detrimental effects for both native and non-native speakers. The findings have implications towards how audio conferencing can be improved for NNS.

In this demonstration video, we introduce the Language Grid Toolbox, an open source multilingual communication tool, and two community sites based on the Language Grid Toolbox. The G30 Community Site aims to create a multilingual and multicultural community to accommodate the needs of Japanese and international students. The Pangaea Community Site is used by facilitators of NPO Pangaea located in different countries around the world to communicate in their native languages through machine translation supported multilingual BBS.