Recent advancements in the conversational and social capabilities of generative AI (GenAI) have sparked interest in its role as an agent capable of actively participating in human-AI group discussions. Despite this momentum, we don’t fully understand how GenAI shapes conversational dynamics or how the interface design impacts its influence on the group. In this paper, we introduce interface-driven social prominence as a design lens for collaborative GenAI systems. We then present a GenAI-based conversational agent that can actively engage in spoken dialogue during video calls and design three distinct collaboration modes that vary the social prominence of the agent by manipulating its presence in the shared space and the degree of control users have over its participation. A mixed-methods within-subjects study, in which 18 dyads engaged in realistic discussions with a GenAI agent, offers empirical insights into how communication patterns and the collective negotiation of GenAI’s influence shift based on how it is embedded into the collaborative experience. Based on these findings, we outline design implications for supporting the coordination and critical engagement required in human-AI groups.

Peer Reviewed

While generative artificial intelligence (GenAI) is finding increased adoption in workplaces, current tools are primarily designed for individual use. Prior work established the potential for these tools to enhance personal creativity and productivity towards shared goals; however, we don't know yet how to best take into account the nuances of group work and team dynamics when deploying GenAI in work settings. In this paper, we investigate the potential of collaborative GenAI agents to augment teamwork in synchronous group settings through an exploratory study that engaged 25 professionals across 6 teams in speculative design workshops and individual follow-up interviews. Our workshops included a mixed reality provotype to simulate embodied collaborative GenAI agents capable of actively participating in group discussions. Our findings suggest that, if designed well, collaborative GenAI agents offer valuable opportunities to enhance team problem-solving by challenging groupthink, bridging communication gaps, and reducing social friction. However, teams' willingness to integrate GenAI agents depended on its perceived fit across a number of individual, team, and organizational factors. We outline the key design tensions around agent representation, social prominence, and engagement and highlight the opportunities spatial and immersive technologies could offer to modulate GenAI influence on team outcomes and strike a balance between augmentation and agency.

Collaborative Mixed Reality (MR) systems that help extend expertise for physical tasks to remote environments often situate experts in an immersive view of the task environment to bring the collaboration closer to collocated settings. In this paper, we design UnMapped, an alternative interface for remote experts that combines a live 3D view of the active space within the novice’s environment with a static 3D recreation of the expert’s own workspace to leverage their existing spatial memories within it. We evaluate the impact of this approach on single and repeated use of collaborative MR systems for remote guidance through a comparative study. Our results indicate that despite having a limited understanding of the novice’s environment, using an UnMapped interface increased performance and communication efficiency while reducing experts’ task load. We also outline the various affordances of providing remote experts with a familiar and spatially-stable environment to assist novices.

Voice-based Intelligent Virtual Assistants (IVAs) promise to improve healthcare management and Quality of Life (QOL) by introducing the paradigm of hands-free and eye-free interactions. However, there has been little understanding regarding the challenges for designing such systems for older adults, especially when it comes to healthcare related tasks. To tackle this, we consider the processes of care delivery and QOL enhancements for older adults as a collaborative task between patients and providers. By interviewing 16 older adults living independently or semi-independently and 5 providers, we identified 12 barriers that older adults might encounter during daily routine and while managing health. We ultimately highlighted key design challenges and opportunities that might be introduced when integrating voice-based IVAs into the life of older adults. Our work will benefit practitioners who study and attempt to create full-fledged IVA-powered smart devices to deliver better care and support an increased QOL for aging populations.

Traumatic injuries require timely intervention, but medical expertise is not always available at the patient’s location. Despite recent advances in telecommunications, surgeons still have limited tools to remotely help inexperienced surgeons. Mixed Reality hints at a future where remote collaborators work side-by-side as if co-located; however, we still do not know how current technology can improve remote surgical collaboration. Through role-playing and iterative-prototyping, we identify collaboration practices used by expert surgeons to aid novice surgeons as well as technical requirements to facilitate these practices. We then introduce ARTEMIS, an AR-VR collaboration system that supports these key practices. Through an observational study with two expert surgeons and five novice surgeons operating on cadavers, we find that ARTEMIS supports remote surgical mentoring of novices through synchronous point, draw, and look affordances and asynchronous video clips. Most participants found that ARTEMIS facilitates collaboration despite existing technology limitations explored in this paper.

Mixed Reality has been shown to enhance remote guidance and is especially well-suited for physical tasks. Conversations during these tasks are heavily anchored around task objects and their spatial relationships in the real world, making referencing - the ability to refer to an object in a way that is understood by others - a crucial process that warrants explicit support in collaborative Mixed Reality systems. This paper presents a 2x2 mixed factorial experiment that explores the effects of providing spatial information and system-generated guidance to task objects. It also investigates the effects of such guidance on the remote collaborator’s need for spatial information. Our results show that guidance increases performance and communication efficiency while reducing the need for spatial information, especially in unfamiliar environments. Our results also demonstrate a reduced need for remote experts to be in immersive environments, making guidance more scalable, and expertise more accessible.