Smartphone Augmented Reality (AR) has already introduced numerous social applications. However, extending smartphone AR to support social experiences akin to VRChat or AltspaceVR requires robust user tracking to animate avatars accurately. While Virtual Reality (VR) benefits from devices like hand-tracking systems, controllers, and Head-mounted displays (HMDs) for precise avatar tracking, smartphone AR is limited to back and front cameras alongside inertial measurement units (IMUs). Previous work has explored animating smartphone users' avatars using singlepoint Inverse Kinematics (IK) derived from smartphone tracking data. In this paper, we propose an enhanced approach that combines face tracking with single-point IK to improve avatar animation. Our system leverages ARKit to track hand and face positions and orientations to estimate user poses through IK. We evaluate our system's performance against a commercial motion tracking system and traditional single-point IK in terms of tracking accuracy, user embodiment, and user experience. Our findings suggests that our dual-point IK (DPIK) method demonstrated significantly lower mean tracking errors and more consistent performance across joints compared to traditional single-point IK. Although statistical differences in embodiment measures were not significant, users reported a stronger sense of control and agency, particularly in head-tracked movements. Participants also favored DPIK in terms of both Pragmatic and Hedonic qualities, highlighting its potential for improving avatar representation in social AR experiences.

Abstract Behavioral realism plays a crucial role in virtual and augmented reality (VR/AR). Various avatar animation techniques, ranging from full-body motion capture to single-point inverse kinematics (IK), offer different levels of realism. While the animation of a user’s own avatar influences embodiment, the perceived realism of others’ avatars is equally important for immersion. This study ( N = 53) examines how users in smartphone AR, head-mounted display (HMD) AR, and VR perceive the behavioral realism of avatars animated with single-point IK compared to those driven by full-body motion capture. In addition, we explore whether the congruence between visual fidelity of an avatar and tracking accuracy affects perception. Our findings indicate that full-body motion capture produces significantly higher perceived realism than single-point IK, but the type of device does not have measurable impact. Furthermore, while congruence between visual realism and tracking fidelity was expected to play a role, our results suggest that its influence is limited. Despite lower realism than motion capture, modern IK techniques are still perceived positively, highlighting their viability for multi-user AR and VR applications.

Behavioral realism is a key element in social virtual and augmented reality (VR/AR). Different techniques, from full-body to single-point IK, are used to animate virtual humans, providing varying levels of behavioral realism. While the animation method of the user’s own avatar is important, e.g., considering embodiment, the animation of others is equally important to display a convincing immersive experience. This study ($\mathrm{N}=36$) investigates how users (in AR or VR) rate the behavioral realism of others when the other person’s avatar is animated using a single point Inverse Kinematics (IK) and compares results to a full-body motion-capture-driven avatar. In addition, we investigate whether congruence between visual realism and tracking quality matters. Our results reveal that animations based on full-body motion capture are found to result in higher perceived behavioral realism compared to those using single-point inverse kinematics (IK), that device type has no influence, and that congruence seems to matter less. Despite not reaching the same level of realism as motion-captured animations, our results suggest the applicability of state-of-the-art IK techniques for multi-user applications in AR and VR as, in general, participants rated both positively.

The combination of smartphone Augmented Reality (AR) and Virtual Reality (VR) makes it possible for on-site and remote users to simultaneously explore a physical space and its digital twin through an asymmetric Collaborative Virtual Environment (CVE). In this paper, we investigate two spatial awareness visualizations to enable joint exploration of a space for dyads consisting of a smartphone AR user and a head-mounted display VR user. Our study revealed that both, a mini-map-based method and an egocentric compass method with a path visualization, enabled the on-site visitors to locate and follow a virtual companion reliably and quickly. Furthermore, the embodiment of the AR user by an inverse kinematics avatar allowed the use of natural gestures such as pointing and waving which was preferred over text messages by the participants of our study. In an expert review in a museum and its digital twin we observed an overall high social presence for on-site AR and remote VR visitors and found that the visualizations and the avatar embodiment successfully facilitated their communication and collaboration.

Smartphone Augmented Reality (AR) has already provided us with a plethora of social applications such as Pokemon Go or Harry Potter Wizards Unite. However, to enable smartphone AR for social applications similar to VRChat or AltspaceVR, proper user tracking is necessary to accurately animate the avatars. In Virtual Reality (VR), avatar tracking is rather easy due to the availability of hand-tracking, controllers, and HMD whereas smartphone AR has only the back-(and front) camera and IMUs available for this task. In this paper we propose ARIKA, a tracking solution for avatars in smartphone AR. ARIKA uses tracking information from ARCore to track the users hand position and to calculate a pose using Inverse Kinematics (IK). We compare the accuracy of our system against a commercial motion tracking system and compare both systems with respect to sense of agency, self-location, and body-ownership. For this, 20 participants observed their avatars in an augmented virtual mirror and executed a navigation and a pointing task. Our results show that participants felt a higher sense of agency and self location when using the full body tracked avatar as opposed to IK avatars. Interestingly and in favor of ARIKA, there were no significant differences in body-ownership between our solution and the full-body tracked avatars. Thus, ARIKA and it’s single-camera approach is valid solution for smartphone AR applications where body-ownership is essential.

With the increasing popularity of e-commerce and the coronavirus situation, an increasing number of shoppers are opting for online shopping because of store closures and their fear of contracting the coronavirus in public. While conventional retail provides consumers with a full spectrum of interaction, online shopping has been deficient in these types of experiences. Therefore, the virtual reality technology is used to bridge the gap between the two shopping techniques and create a more natural and intuitive shopping environment. The aim of this research is to investigate how, in a virtual store environment, consumers’ interaction affect their shopping experience. A virtual supermarket with the interaction facility was designed. An experiment was conducted to track the effect of the social interaction on the consumers using different metrics. The results showed that consumers’ social interaction in the form of avatar- mediated communication has a beneficial impact on their social presence. It also demonstrates that consumers felt more immersed and socially engaged to the shopping environment via the social interaction among avatars.