When families live in the same home, they feel a sense of connection through the subtle, passive aspects of family life. Over distance, these passive aspects are hard to experience as most communication technologies support sharing conversations or activities. Through a research-through-design methodology, we explored the design of smart home technologies for passive co-presence over distance. Based on our design explorations, we arrived at an interaction space which includes the dimensions of Activity, Solitude, Synchronicity, and Spontaneity. Our research-through-design process additionally resulted in the design of two smart home systems. The There Chair employs the senses of touch and sight to passively display when a remote family member is sitting at the dining room table. The Fragrance Frame is a paired picture frame that detects when a remote family member is passing by their frame, and emits a scent reminiscent of togetherness. We reflect on our design decisions and propose considerations for future design.

Young children (3-5 years old) benefit from social connections with distance-separated grandparents and other family, but can struggle to connect in common digital settings like Zoom calls due to asymmetric needs of different generations. Additionally, they cannot currently engage independently with asynchronous family communication, such as texting or sharing media, because existing technologies rely on literacy and technical skills beyond their development. In this paper, we present the results of a co-design probe with 12 young children, collecting and analyzing their ideas for independent communications technologies. Our probe is based on age-appropriate pedagogy, using make-believe play and an original story about astronauts to create a design space that supported their agency as designers. Our young designers’ detailed ideas incorporated broader systems of communication. From those ideas, we present design considerations, including making connection and transmission across devices transparent and integrating learning about wireless communication into their everyday use.

Emergency response to large-scale disasters is often supported with multimedia from social media. However, while these features are common in everyday video calls, the complex needs of 911 and other systems make it difficult to directly incorporate these features. We assess an ME911 (Multimedia-Enabled 911) app to understand how the design will need to deviate from common norms and how callers will respond to those non-standard choices. We expand the role of 911 call taker control over emergency situations to the calling interface while incorporating key features like map-based location finding. Participants’ experiences in mock emergencies show the non-standard design helps callers in the unfamiliar setting of emergency calling yet it also causes confusion and delays. We find the need for emergency-specific deviations from design norms is supported by participant feedback. We discuss how broader system changes will support callers to use these non-standard designs during emergencies.

Playing remote tabletop games is a fun way to connect with distant friends. Yet most systems for remote tabletop gaming lack support for tangible and social interaction, two important aspects of gameplay for most players. We are interested in how to better design systems for remote tangible gameplay that support social connection. We investigate this topic through the design and evaluation of a prototype system for playing the board game Wavelength across two locations. First, we describe the design goals that informed our prototype: “Remote Wavelength”. Then, we discuss the results of a qualitative user study in which ten friend groups played Remote Wavelength. Our findings indicate that a synchronized, tangible gameboard benefits player engagement, communication, and awareness. Our results also illustrate the value of integration across communication and gameplay systems. We conclude by offering considerations for the design of both digital and remote tangible gameplay systems.

Picture interactions are key to daily and long-term social connections between families and communities, especially through reminiscence. Across the nearly 200-year history of domestic photography, this social reminiscence has been accomplished largely through photo albums. However, in the now common digital setting, albums are pushed aside for the endless film roll metaphor. In this article, we explore this metaphor through the 20-year history of proposed digital picture interactions from Human–Computer Interaction research, and compare this to ongoing interactions with modern picture tools. Through these, we reveal that this prominent design metaphor does not create space for social reminiscence, but does fit the novel use of immediate sharing seen across social networking. Furthermore, the endless and ever-growing nature of digital film rolls are not meaningfully browsable for either intended use. We close by reconnecting to the past works that explore the broad potential of digital pictures.

Interactions with our personal and family pictures are essential to continued social reminiscence, leading to long-term benefits, including reduced social isolation. Previous research has identified how designs of digital picture tools fall short of physical options specifically in terms of reminiscence. However, the relative prompting abilities of different digital interactions, including the types of memories prompted like external facts or person-centred memories, have not yet been explored. To investigate this, we present a controlled study of the memories prompted by three digital picture interactions (slideshow, gallery, and tabletop) on personal touchscreen devices. We find differences in how these tools and the interactions they support prompt reminiscence. In particular, gallery views prompt significantly fewer memories than either the tabletop or slideshow. Slideshows prompt significantly more external, factual memories, but not more person-centred memories, which are key to reminiscence. This has implications for the overall social usability of digital picture interactions.

The timeless social activity of passing down oral stories preserves family memory, identity, values, and culture. Existing tools for family memories often take a techno-determinist approach by focusing on the mechanics of connecting families and the resulting documentation, rather than the social process of sharing stories and morals, and largely without considering the specific needs of immigrant families. For immigrant families, cultural exchange, particularly crucial across grandparent and grandchild generations, is threatened by the language and cultural barriers emerging from displacement and migration. As a result, immigrant grandparents and their young grandchildren struggle with fostering social kinship, leading to social disconnect and loss of cultural heritage. In our research, we collaborate with multi-generational and culturally-at-risk immigrant families through Participatory Design activities towards the design of reminiscence tools that support their needs focusing on language and cultural connection. We report on the designs created by families and propose design guidelines supporting cultural resilience, focusing on flexible, visual storytelling.

Speech is now common in daily interactions with our devices, thanks to voice user interfaces (VUIs) like Alexa. Despite their seeming ubiquity, designs often do not match users’ expectations. Science fiction, which is known to influence design of new technologies, has included VUIs for decades. Star Trek: The Next Generation is a prime example of how people envisioned ideal VUIs. Understanding how current VUIs live up to Star Trek’s utopian technologies reveals mismatches between current designs and user expectations, as informed by popular fiction. Combining conversational analysis and VUI user analysis, we study voice interactions with the Enterprise’s computer and compare them to current interactions. Independent of futuristic computing power, we find key design-based differences: Star Trek interactions are brief and functional, not conversational, they are highly multimodal and context-driven, and there is often no spoken computer response. From this, we suggest paths to better align VUIs with user expectations.

Language and cultural barriers critically threaten the social relationships between grandparents and grandchildren in immigrant families. Cultural exchange activities, like shared storytelling, can foster these crucial connections. However, existing barriers make these seemingly routine interactions challenging for families to navigate. The resulting intergenerational drift places grandparents at high risk of sustained social isolation from their families. Past works have presented technology-mediated supports for grandparent-grandchild social interactions in non-immigrant families and have found that these interventions do foster stronger connections in both physically close and distant multigenerational families. We explore how to support the specific needs of immigrant families through Magic Thing participatory design workshops with grandchildren and grandparents together in order to reveal the social interactions that would support their cultural exchange. We use the Magic Thing to move the standard dialogic grandparent-grandchild relationship into a trialogic one, creating space for comfortable social connection and storytelling through the shared creation of the design. We find that technology-mediated support of intergenerational immigrant cultural exchange must be designed for this trialogic process, consider the role of expressing values as a form of meta-commentary on a story, and shift the perspective on existing "barriers" to consider how they might foster further engagement.

Reading a book together with a family member who has impaired vision or other difficulties reading is an important social bonding activity. However, for the person being read to, there is little support in making these experiences repeatable. While audio can easily be recorded, synchronizing it with the text for later playback requires the use of forced alignment algorithms, which do not perform well on amateur read-aloud speech. We propose a human-in-the-loop approach to augmenting such algorithms, in the form of touch metaphors during collocated read-aloud sessions using tablet e-readers. The metaphor is implemented as a finger-follows-text tracker. We explore how this could better handle the variability of amateur reading, which poses accuracy challenges for existing forced alignment techniques. Data collected from users reading aloud as assisted by touch metaphors show increases in the accuracy of forced alignment algorithms and reveal opportunities for how to better support reading aloud.

Mobile technologies have made family photo collections extremely portable. People can now carry all their pictures with them wherever they go and show them to others in any setting with smartphones or tablets. However, current options for portable photo viewing are not intended for in-person sharing and reminiscence. Frame of Mind presents a new way to interact with digital pictures on a touch screen that encourages storytelling through its free-flowing interaction using the metaphor of looking at pictures on a table top. This allows family reminiscence to be lightweight, portable, and more accessible by supporting photo viewing on tablets that can have access to complete picture collections. So Frame of Mind moves towards digital tools that support our current photo viewing and sharing activities.

Limited literacy and visual impairment reduce the ability of many to read on their own. Current e-reader solutions rely on either unnatural synthetic voices or professionally produced audio e-books. Neither provide the same enjoyment as having a family member read to a user, especially when the user requires assistive reading (following printed text while listening to it being read). Unfortunately, the support for non-commercial production of such e-books is limited and requires significant effort. We evaluate a novel, assistive mobile interaction technique that facilitates the recording of audio e-books and their synchronization with the read text. We show that a technique based on a finger tracking metaphor provides optimal support with respect to reading speed. These human-in-the-loop, adaptive techniques can now be used to reduce the content-creation burden that is associated with supporting those who cannot read on their own.