Augmented Reality (AR) presents new opportunities for immersive storytelling. However, this immersiveness faces two main hurdles. First, AR’s immersive quality is often confined to visual elements, such as pixels on a screen. Second, crafting immersive narratives is complex and generally beyond the reach of amateurs due to the need for advanced technical skills. We introduce Jigsaw, a system that empowers beginners to both experience and craft immersive stories, blending virtual and physical elements. Jigsaw uniquely combines mobile AR with readily available Internet-of-things (IoT) devices. We conducted a qualitative study with 20 participants to assess Jigsaw’s effectiveness in both consuming and creating immersive narratives. The results were promising: participants not only successfully created their own immersive stories but also found the playback of three such stories deeply engaging. However, sensory overload emerged as a significant challenge in these experiences. We discuss design trade-offs and considerations for future endeavors in immersive storytelling involving AR and IoT.

Technology, especially the smartphone, is villainized for taking meaning and time away from in-person interactions and secluding people into “digital bubbles”. We believe this is not an intrinsic property of digital gadgets, but evidence of a lack of imagination in technology design. Leveraging augmented reality (AR) toward this end allows us to create experiences for multiple people, their pets, and their environments. In this work, we explore the design of AR technology that “piggybacks” on everyday leisure to foster co-located interactions among close ties (with other people and pets). We designed, developed, and deployed three such AR applications, and evaluated them through a 41-participant and 19-pet user study. We gained key insights about the ability of AR to spur and enrich interaction in new channels, the importance of customization, and the challenges of designing for the physical aspects of AR devices (e.g., holding smartphones). These insights guide design implications for the novel research space of co-located AR.

A central challenge of social computing research is to enable people to communicate expressively with each other remotely. Augmented reality has great promise for expressive communication since it enables communication beyond texts and photos and towards immersive experiences rendered in recipients' physical environments. Little research, however, has explored AR's potential for everyday interpersonal communication. In this work, we prototype an AR messaging system, ARwand, to understand people's behaviors and perceptions around communicating with friends via AR messaging. We present our findings under four themes observed from a user study with 24 participants, including the types of immersive messages people choose to send to each other, which factors contribute to a sense of immersiveness, and what concerns arise over this new form of messaging. We discuss important implications of our findings on the design of future immersive communication systems.

Ubiquitous computing encapsulates the idea for technology to be interwoven into the fabric of everyday life. As computing blends into everyday physical artifacts, powerful opportunities open up for social connection. Prior connected media objects span a broad spectrum of design combinations. Such diversity suggests that people have varying needs and preferences for staying connected to one another. However, since these designs have largely been studied in isolation, we do not have a holistic understanding around how people would configure and behave within a ubiquitous social ecosystem of physically-grounded artifacts. In this paper, we create a technology probe called Social Wormholes, that lets people configure their own home ecosystem of connected artifacts. Through a field study with 24 participants, we report on patterns of behaviors that emerged naturally in the context of their daily lives and shine a light on how ubiquitous computing could be leveraged for social computing.

We present Project IRL (In Real Life), a suite of five mobile apps we created to explore novel ways of supporting in-person social interactions with augmented reality. In recent years, the tone of public discourse surrounding digital technology has become increasingly critical, and technology's influence on the way people relate to each other has been blamed for making people feel "alone together," diverting their attention from truly engaging with one another when they interact in person. Motivated by this challenge, we focus on an under-explored design space: playful co-located interactions. We evaluated the apps through a deployment study that involved interviews and participant observations with 101 people. We synthesized the results into a series of design guidelines that focus on four themes: (1) device arrangement (e.g., are people sharing one phone, or does each person have their own?), (2) enablers (e.g., should the activity focus on an object, body part, or pet?), (3) affordances of modifying reality (i.e., features of the technology that enhance its potential to encourage various aspects of social interaction), and (4) co-located play (i.e., using technology to make in-person play engaging and inviting). We conclude by presenting our design guidelines for future work on embodied social AR.

Digital communication is often brisk and automated. From auto-completed messages to "likes," research has shown that such lightweight interactions can affect perceptions of authenticity and closeness. On the other hand, effort in relationships can forge emotional bonds by conveying a sense of caring and is essential in building and maintaining relationships. To explore effortful communication, we designed and evaluated Auggie, an iOS app that encourages partners to create digitally handcrafted Augmented Reality (AR) experiences for each other. Auggie is centered around crafting a 3D character with photos, animated movements, drawings, and audio for someone else. We conducted a two-week-long field study with 30 participants (15 pairs), who used Auggie with their partners remotely. Our qualitative findings show that Auggie participants engaged in meaningful effort through the handcrafting process, and felt closer to their partners, although the tool may not be appropriate in all situations. We discuss design implications and future directions for systems that encourage effortful communication.

With the growing ubiquity of wearable devices, sensed physiological responses provide new means to connect with others. While recent research demonstrates the expressive potential for biosignals, the value of sharing these personal data remains unclear. To understand their role in communication, we created Significant Otter, an Apple Watch/iPhone app that enables romantic partners to share and respond to each other’s biosignals in the form of animated otter avatars. In a one-month study with 20 couples, participants used Significant Otter with biosignals sensing OFF and ON. We found that while sensing OFF enabled couples to keep in touch, sensing ON enabled easier and more authentic communication that fostered social connection. However, the addition of biosignals introduced concerns about autonomy and agency over the messages they sent. We discuss design implications and future directions for communication systems that recommend messages based on biosignals.