Prior research has highlighted users’ preferences for embodiment when interacting with virtual agents in augmented reality headsets. However, open questions remain regarding users’ preferences towards agent placement and gaze direction. In our study, we asked 48 adults to wear the Microsoft HoloLens 2 and find objects in a hidden object game with the help of embodied agents. We examined four distinct agent configurations for both male and female agents: a human-size agent standing beside participants, a human-size agent sitting beside participants, a small desk agent facing the screen, and a small desk agent facing the participant. Overall, participants preferred male over female virtual agents when receiving assistance, and no consistent preference emerged regarding the agents’ position or gaze direction. From our results, we build upon existing guidelines for designing better virtual agents for AR with headsets.

Although companion virtual agents (CVAs) are increasingly adopted to support well-being, interaction patterns between adults and CVAs remain underexplored. This study examines how users engage with CVAs over time, exploring conversation patterns, interaction frequency, attitudes, and emotional responses over a seven-day period. Twenty-four adults engaged with a GPT-4-powered embodied CVA daily, discussing topics from personal interests to emotional reflections. Quantitative measures, including loneliness and affect scales, revealed no significant reduction in loneliness but noted decreases in positive affect and nervousness. Qualitative analysis highlighted evolving conversational dynamics, with participants shifting from exploratory questions to more reflective and personal discussions. Participants appreciated the agent’s ability to engage in fluid and meaningful conversations. However, participants also noted shortcomings, including limited recall and occasional conversational unnaturalness. These findings inform the design of CVAs, emphasizing the need for adaptive conversational strategies, enhanced emotional responsiveness, and improved memory systems to foster meaningful connections.

Shared-gaze visualizations (SGVs) in augmented reality enable collaborators to share focus and intentions through gaze interactions. Most prior research has examined bi-directional visualizations, where both users see their own and their partner's gaze, to provide feedback on how their gaze is communicated to their partner. However, bi-directional SGV approaches are largely based on research for remote collaboration. In collocated settings, bi-directional SGVs can obstruct views and cause distractions. Additionally, collocated applications differ from remote ones. We propose that if eye-tracking is well-calibrated, bi-directional visualizations may be unnecessary in collocated settings. To explore this, we conducted a user study comparing perceptions of uni- and bi-directional gaze visualizations in a virtual collaborative sorting task. Our results suggest that self-gaze may not always be necessary for users; however, there are cases in which self-gaze helps them feel more confident in the task. We offer a deeper understanding for future collaborative gaze interaction systems.

To design intuitive and effective context-aware task guidance systems, we must understand users’ thought processes and the obstacles they experience when they perform tasks. Though task guidance systems have proven beneficial in many domains for improving task performance and reducing user frustration, there is a lack of general guidelines and design principles for their development. Prior work has shown that recipe-based cooking is a strong medium for studying task planning and execution. In response, we conducted a contextual inquiry study in home kitchens, observing eight different participants’ cooking sessions. We used affinity diagramming of our notes and transcripts to identify common obstacles faced by participants and establish user needs in the areas of object interaction, safety, knowledge base, and task coordination. We discuss how these findings can inform the design of technology-driven solutions for task guidance systems beyond cooking.

Motivational agents are virtual agents that seek to motivate users by providing feedback and guidance. Prior work has shown how certain factors of an agent, such as the type of feedback given or the agent's appearance, can influence user motivation when completing tasks. However, it is not known how nonverbal mirroring affects an agent's ability to motivate users. Specifically, would an agent that mirrors be more motivating than an agent that does not? Would an agent trained on real human behaviors be better? We conducted a within-subjects study asking 30 participants to play a "find-the-hidden-object" game while interacting with a motivational agent that would provide hints and feedback on the user's performance. We created three agents: a Control agent that did not respond to the user's movements, a simple Mimic agent that mirrored the user's movements on a delay, and a Complex agent that used a machine-learned behavior model. We asked participants to complete a questionnaire asking them to rate their levels of motivation and perceptions of the agent and its feedback. Our results showed that the Mimic agent was more motivating than the Control agent and more helpful than the Complex agent. We also found that when participants became aware of the mimicking behavior, it can feel weird or creepy; therefore, it is important to consider the detection of mimicry when designing virtual agents.