Recent advances in generative AI have accelerated the deployment of virtual celebrity agents for commercial endorsements. However, little is known about how their visual style and vocal type, especially when these characteristics are generated via advanced AI reconstruction or voice cloning techniques, affect user trust, perceived realism, familiarity, and social presence. We extracted a representative clip of Sheldon Cooper from The Big Bang Theory as a baseline and generated multiple Sheldon virtual agents that varied in visual style (hand-sculpted, AI-reconstructed, or non-stylized) and vocal type (cloned or synthesized). A 3 × 2 within-subjects experiment (N = 30) revealed that visual style significantly affected user trust, perceived realism, familiarity, and social presence, while vocal type affected only perceived realism and familiarity. Comparisons between the virtual agents and the video baseline confirmed that even cutting-edge 3D modeling still differs significantly from authentic video representations. Behavioral data indicate that the relationship between interpersonal-distance variations and trust levels is not a simple linear one. These findings provide actionable guidance for designers leveraging generative AI to create trustworthy virtual avatars or agents and delineate new research avenues for virtual celebrity agents.

With the continuous advancement of artificial intelligence technology, data-driven methods for reconstructing and animating virtual agents have achieved increasing levels of realism. However, there is limited research on how these novel data-driven methods, combined with voice cues, affect user perceptions. We use advanced data-driven methods to reconstruct stylized agents and combine them with synthesized voices to study their effects on users' trust and other perceptions (e.g. social presence and empathy). Through an experiment with 27 participants, our findings reveal that stylized virtual agents enhance user trust to a degree comparable to real style, while voice has a negligible effect on trust. Additionally, elder agents are more likely to be trusted. The style of the agents also plays a key role in participants' perceived realism, and audio-visual matching significantly enhances perceived empathy. These results provide new insights into designing trustworthy virtual agents and further support and validate the audio-visual integration theory.

Telepresence robots are increasingly used to support remote social interaction. Telerobots allow the user to move a camera and a microphone at a remote location in real time – often with a display of the user’s face at the robot. These robots can increase the quality of life for people with special needs, who are, for instance, bed bound. However, interface accessibility barriers have made them difficult to use for some people. Still, no state-of-the-art literature review has been made of research on telerobots for people with disabilities. We used Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines for a review. Web of Science (WoS), ACM Digital Library, IEEE Xplore, PubMed, and Scopus were searched, and a supplemental by hand examination of reference lists was done. The search includes studies published between 2009 and 2019.A total of 871 articles were included in this review, 42 of which were eligible for the analysis. These articles were further characterized in terms of problems addressed, objectives, types of special needs considered, features of the devices, features of solutions, and the evaluation methods applied. Based on the review, future research directions are being proposed, addressing issues like use-cases; user conditions; universal accessibility; safety; privacy and security; independence and autonomy; evaluation methods; and user training programs. The review provides an overview of existing research, a summary of common research directions, and a summary of issues, which need to be considered in future research.

Gaze may be a good alternative input modality for people with limited hand mobility. This accessible control based on eye tracking can be implemented into telepresence robots, which are widely used to promote remote social interaction and providing the feeling of presence. This extended abstract introduces a Ph.D. research project, which takes a two-phase approach towards investigating gaze-controlled telepresence robots. A system supporting gaze-controlled telepresence has been implemented. However, our current findings indicate that there were still serious challenges with regard to gaze-based driving. Potential improvements are discussed, and plans for future study are also presented.

Mobile robotic telepresence systems are increasingly used to promote social interaction between geographically dispersed people. People with severe motor disabilities may use eye-gaze to control a telepresence robots. However, use of gaze control for navigation of robots needs to be explored. This paper presents an experimental comparison between gaze-controlled and hand-controlled telepresence robots with a head-mounted display. Participants (n = 16) had similar experience of presence and self-assessment, but gaze control was 31% slower than hand control. Gaze-controlled robots had more collisions and higher deviations from optimal paths. Moreover, with gaze control, participants reported a higher workload, a reduced feeling of dominance, and their situation awareness was significantly degraded. The accuracy of their post-trial reproduction of the maze layout and the trial duration were also significantly lower.

Eye-gaze interaction is a common control mode for people with limited mobility of their hands. Mobile robotic telepresence systems are increasingly used to promote social interaction between geographically dispersed people. We are interested in how gaze interaction can be applied to such robotic systems, in order to provide new opportunities for people with physical challenges. However, few studies have implemented gaze-interaction into a telepresence robot and it is still unclear how gaze-interaction within these robotic systems impacts users and how to improve the systems. This paper introduces our research project, which takes a two-phase approach towards investigating a novel interaction-system we developed. Results of these two studies are discussed and future plans are described.

People who cannot use their hands may use eye-gaze to interact with robots. Emerging virtual reality head-mounted displays (HMD) have built-in eye-tracking sensors. Previous studies suggest that users need substantial practice for gaze steering of wheeled robots with an HMD. In this paper, we propose to apply a VR-based simulator for training of gaze-controlled robot steering. The simulator and preliminary test results are presented.

Gaze interaction with telerobots is a new opportunity for wheelchair users with severe motor disabilities. We present a video showing how head-mounted displays (HMD) with gaze tracking can be used to monitor a robot that carries a 360° video camera and a microphone. Our interface supports autonomous driving via way-points on a map, along with gaze-controlled steering and gaze typing. It is implemented with Unity, which communicates with the Robot Operating System (ROS).