The emotional expressions of social robots are increasingly important for improving human–robot interaction. However, few studies have explored the emotional expressions of robots in auditory and multimodal modalities within specific interaction contexts. Three experiments were conducted to investigate the impact of emotion categories and emotional intensity in robots’ facial and auditory expressions on user perception and visual behavior under unimodal and multimodal conditions. Results showed that: (1) in unimodal conditions, higher emotional intensity in facial expressions and voice led to higher perceived trustworthiness than low emotional intensity; (2) in multimodal conditions, positive emotion and higher emotional intensity in both modalities enhanced trustworthiness and likability; (3) visual behavior showed that the eye region attracted the most attention. Additionally, participants’ experience and gender significantly influenced visual behavior. This study offers insights into the impact of robots’ emotional expressions on user perception and visual behavior while providing important guidance for the design of empathetic robots.

Facial paralysis, as a common nerve system disease, seriously affects the patients' facial muscle function and appearance. Accurate facial paralysis grading is of great significance for the formulation of personalized treatment. Existing artificial intelligence based grading methods extensively focus on static image classification, which fails to capture the dynamic facial movements. Additionally, due to private concerns, building comprehensive facial paralysis datasets is challenging, making it impractical to fully train a robust model from scratch. Finally, maintaining precision and inference speed on edge devices remains a key challenge. To address these shortcomings, we propose MLST-Net, a novel and explainable three-stage deep-learning method based on multi-task learning. In the first stage, the pre-trained model is used to extract the facial static appearance structure and dynamic texture changes. The second stage fuses the proxy task results to construct a unified face semantic expression and outputs the "with or without facial paralysis" simple task results. In the third stage, we use spatial-temporal disentanglement to capture the spatial-temporal combinatorial-dependencies in video sequences. Finally, we input the classifier to get the results of complex tasks of facial paralysis classification. Compared with all advanced methods, MLST-Net is computationally inexpensive and achieves state-of-the-art results on the 1241 public dataset videos. It significantly benefits the digital diagnosis of facial palsy and offers innovative and explainable ideas for video-based digital medical treatment.

Knee osteoarthritis (KOA) as a prevalent chronic disease, detrimentally impacts the quality of life among affected individuals. The knee adduction moment (KAM) during the stance phase has been identified as a potential biomechanical measure for assessing the severity of KOA. Traditional KAM assessment relies on expensive equipment, which limits its popularization. In contrast, current KAM estimation methods based on wearables and deep-learning technology offer the advantage of lower costs. However, it still suffers challenges in achieving accurate estimation. To address this challenge, a novel deep-learning framework is proposed in this work, which estimates the KAM from Inertial Measurement Units (IMU) and Electromyography (EMG) data by a well-designed self-aware fusion model. Walking data from 18 effective subjects were recorded with 4 IMUs and 6 EMGs. Results show that the model significantly improves KAM estimation accuracy. The relative root-mean-square error of the proposed model is 9.15% BW $ \cdot $ BH lower than counterpart estimation methods.

During the design, it is important to evaluate the user experience of representative users in many human product interactions. But, in some cases, it is difficult or even impossible to recruit representative users because they have disabilities that do not allow them to take part in such investigations. Thus, alternative populations are widely studied. The most common way to replace real blind people is to use sighted but blindfolded users when studying design solutions. To test whether such alternative or proxy users can be used to represent blind people in social interactions, we examined the communication quality of 20 blind-sighted pairs and 20 blindfolded-sighted pairs in two different experiments. A prototype named E-Gaze glasses was evaluated as the testing tool. Results clearly show that the blindfolded participants achieved significantly higher communication quality than the blind participants. In qualitative data analysis, the blindfolded participants also reported their user experience of being blindfolded in conversations. Our qualitative results strengthen the conclusion that blindfolded users’ behaviour is different from real blind users’ behaviour. We recommend that blind users should not be substituted for blindfolded users in human product evaluations when communication quality is measured.

Researcher has been trying to optimize the method for evaluating presence in virtual reality (VR) to address variability and uncertainty in quick evaluations using questionnaires. We recommend using the Technique for Order Performance by Similarity to Ideal Solution (TOPSIS) model to calculated the iGroup Presence Questionnaire to measure users’ overall presence in VR prototyping, instead of the weighted sum approach. The effects of two presence factors (scene density and motion trajectory technology) on a self-developed VR swimming virtual system were tested using the TOPSIS model with 20 participants each 12 tasks in a user experiment. The results were compared using two different weighting methods, fuzzy hierarchical analysis and uniform weighting methods. TOPSIS had a narrower range of data within the 95% confidence interval and a significantly lower coefficient of variation (CV). This indicates enhanced precision in evaluating presence and can be used to compare different technique setting of virtual systems.

As the population ages rapidly, there is a strong focus on the healthy aging of older adults. A central part of healthy aging is keeping people connected in later social life. Exergames are recommended as one of the coping strategies to help improve health and quality of life in older adults. In our study, we developed an exergame called Social Balance Ball to engage older and younger people to play together, encouraging social interaction between generations. From May to July 2021, we evaluated this exergame in Shanghai, China, performing a user experiment with 18 unfamiliar young-old pairs under three test conditions (virtual player, mediated human player, and co-located human player). To evaluate the exergame, our main findings demonstrated that participants felt significantly perceived social interaction in mediated play and co-located play than in virtual play. Overall, older participants perceived significantly higher social interaction than younger participants. In this study, we contribute (1) empirical research findings on how the Social Balance Ball exergame enhances social interaction in generations; (2) design implications for informing future design and development of social exergames.

Ovarian cancer presents significant well-being challenges for middle-aged and older women. Recent research underscores the vital role of recovery identity in predicting wellbeing. However, a research gap exists regarding the influence of online support groups (OSPs) on identity synthesis for middle-aged and older cancer patients. This study introduces "Rehab-Diary," a mobile age-friendly OSP grounded in The Social Identity Model of Identity Change, aimed at helping ovarian cancer patients foster recovery identity. A four-week randomized controlled trial involving 68 participants assessed the OSP's impact. The interface was tailored for ease of use by older individuals. The findings demonstrate the feasibility of utilizing Rehab- Diary among older individuals. The intervention effectively enhanced recovery identity. This study offers evidence-based insights for developing future age-friendly online support interventions, ultimately enhancing ovarian cancer patients' quality of care.

The increasing demand for medical services in hospitals has sparked interest in exploring alternative methods to support children with functional articulation disorders. Online speech games have emerged as a promising avenue to motivate children to engage in speech therapy. This study investigates the impact of gamification strategies on children's motivation within the context of speech rehabilitation games. Four distinct game prototypes were developed, with a specific emphasis on stimulating children's motivation to speak. Two sets of experiments involving 48 participants were conducted to assess the influence of (1) time-limitation and (2) interactive imitation objects on children's motivation. The results revealed that time limitation significantly increased motivation, while the effect of imitation objects on motivation was not statistically significant. These findings offer valuable insights into designing effective speech games for children. By leveraging gamification strategies in online speech games, we can address the motivation challenges faced by children with functional articulation disorders and potentially enhance the efficiency of speech therapy interventions.

Picture passwords, which require users to draw selections on images as their secret password, typically provide globalized solutions without taking into consideration that people across diverse cultures exhibit differences within interactive systems. Aiming to shed light on the effects of culture towards users' interactions within picture password schemes, we conducted a between-subjects cross-cultural (Eastern vs. Western) study (n=67). Users created a password on a picture illustrating content highly related to their daily-life experiences (culture-internal) vs. a picture illustrating the same daily-life experiences, but in a different cultural context (culture-external). Results revealed that people across cultures exhibited differences in visual processing, comprehension, and exploration of the picture content prior to making their password selections. The observed differences can be accounted by considering sociocultural theories highlighting the holistic preference of Eastern populations compared to the analytic preference of Western populations. Qualitative data also triangulate the findings by exposing the likeability and users' engagement towards the picture content familiar to individual's culture. Findings underpin the necessity to consider cultural differences in the design of personalized picture passwords.

Eye contact is crucial in social interactions, linking with sincerity and friendliness. However, blind people cannot see and make eye contact when they communicate with sighted people. It influences the involvement of blind people in blind-sighted conversations. Based on this context, we implemented Social glasses with an eye-tracking system, aiming to improve the communication quality between blind and sighted people in face-to-face conversations. Social glasses attempts to simulate the appropriate gaze for blind people, especially establishing the “eye contact” in blind-sighted conversations. To evaluate the impact of the interactive gaze displayed on the Social glasses, we performed dyadic-conversation tests under four experimental conditions (No Gaze, Constant Gaze, Random Gaze, and Interactive Gaze) for 40 participants. Quantitative results showed that the Interactive gaze has a positive impact on improving the communication quality between blind and sighted people, which were consistent with a qualitative analysis of the participants’ comments.