Augmentative and Alternative Communication (AAC) users often face barriers when communicating with people unfamiliar with their systems, especially in unpredictable or dynamic contexts.We present Pectogram, a generative AI-augmented Picture Exchange Communication System (PECS) designed to support nonspeaking individuals by transforming natural language text into PECS-style image cards in real time.Pectogram accepts input from any text-generating assistive technology and supports personalization through custom uploads, PECS databases, and AI-generated content.The system is WCAG 2.2-compliant and tested for screen reader compatibility.By dynamically bridging vocabulary gaps and enabling inclusive, on-demand visual communication, Pectogram opens new opportunities for accessible, culturally responsive AAC technologies.We demonstrate its potential and discuss opportunities for co-design and user-centered evaluation with AAC users and their communication partners.

Sharing data with collaborators is a complicated task that is nonetheless fundamental to academic research. We present the results of two studies investigating data sharing within academic scientific collaborations, as well as a system called DriveGroups designed to facilitate data sharing. First, we observed and interviewed 38 academic researchers engaged in collaborative research about their data sharing practices. We found that these researchers struggle to manage access to data, especially when different types of collaborators (e.g., students, co-principal investigators) require different access settings. In response, we built DriveGroups, a Google add-on designed to alleviate participant challenges with access control, and compared its usability to unmodified Google Drive. DriveGroups allows users to manage file access from two separate perspectives: 1) the traditional file perspective and 2) a role-based group perspective, which simplifies the data sharing process. DriveGroups matched or outperformed unmodified Google Drive in terms of usability, access control, and transparency, and will help scientists advance high-impact academic research.

Artificial Intelligence (AI) has been enhancing data analysis efficiency and accuracy during plant phenotyping, which is vital for tackling global agricultural and environmental challenges. Designing a reliable AI system to assist precise plant phenotyping begins with high-quality phenotypic feature annotation, which usually involves collaboration between plant scientists and AI specialists. However, due to the high level of diversity in these researchers’ backgrounds, it is likely that they have differing user needs from a fine-grained plant feature annotation system. We conducted semi-structured interviews with eight experienced annotators from diverse backgrounds, and observed how they interact with their preferred annotation system, to elucidate the challenges faced when annotating plant features and identify user needs. We collected qualitative responses to the interview questions, and conducted a quantitative evaluation of the agreement of their annotations on the given images. By analyzing the participants’ behaviors and the collected data, we identified common user needs and derived implications for the design of an AI-assisted annotation system, including providing a range of annotation options, the flexibility to adapt annotations, and functions to help addressing uncertainty. Our research contributes to the design of systems that make annotations efficient and reliable, not only benefiting plant phenotyping, but also other interdisciplinary fields that rely on user-driven annotations.

Due to the COVID-19 pandemic, research is increasingly conducted remotely without the benefit of informal interactions that help maintain awareness of each collaborator's work progress. We developed AmbiTeam, an ambient display that shows activity related to the files of a team project, to help collaborations preserve a sense of the team's involvement while working remotely. We found that using AmbiTeam did have a quantifiable effect on researchers' perceptions of their collaborators' project prioritization. We also found that the use of the system motivated researchers to work on their collaborative projects. This effect is known as "the motivational presences of others," one of the key challenges that make distance work difficult. We discuss how ambient displays can support remote collaborative work by recreating the motivational presence of others.

Interactive spherical displays offer numerous opportunities for engagement and education in public settings. Prior work established that users' touch-gesture patterns on spherical displays differ from those on flatscreen tabletops, and speculated that these differences stem from dissimilarity in how users conceptualize interactions with these two form factors. We analyzed think-aloud data collected during a gesture elicitation study to understand adults' and children's (ages 7 to 11) conceptual models of interaction with spherical displays and compared them to conceptual models of interaction with tabletop displays from prior work. Our findings confirm that the form factor strongly influenced users' mental models of interaction with the sphere. For example, participants conceptualized that the spherical display would respond to gestures in a similar way as real-world spherical objects like physical globes. Our work contributes new understanding of how users draw upon the perceived affordances of the sphere as well as prior touchscreen experience during their interactions.

The future of smart environments is likely to involve both passive and active interactions on the part of users. Depending on what sensors are available in the space, users may make use of multimodal interaction modalities such as hand gestures or voice commands. There is a shortage of robust yet controlled multimodal interaction datasets for smart environment applications. One application domain of interest based on current state-of-the-art is authentication for sensitive or private tasks, such as banking and email. We present a novel, large multimodal dataset for authentication interactions in both gesture and voice, collected from 106 volunteers who each performed 10 examples of each of a set of hand gesture and spoken voice commands chosen from prior literature (10,600 gesture samples and 13,780 voice samples). We present the data collection method, raw data and common features extracted, and a case study illustrating how this dataset could be useful to researchers. Our goal is to provide a benchmark dataset for testing future multimodal authentication solutions, enabling comparison across approaches.

Although smartwatches are gaining popularity among mainstream consumers, the input space is limited due to their small form factor. The goal of this work is to explore how to design non-touchscreen gestures to extend the input space of smartwatches. We conducted an elicitation study eliciting gestures for 31 smartwatch tasks. From this study, we demonstrate that a consensus exists among the participants on the mapping of gesture to command and use this consensus to specify a user-defined gesture set. Using gestures collected during our study, we define a taxonomy describing the mapping and physical characteristics of the gestures. Lastly, we provide insights to inform the design of non-touchscreen gestures for smartwatch interaction.

Motion gestures are an underutilized input modality for mobile interaction despite numerous potential advantages. Negulescu et al. found that the lack of feedback on attempted motion gestures made it difficult for participants to diagnose and correct errors, resulting in poor recognition performance and user frustration. In this article, we describe and evaluate a training and feedback technique, Glissando , which uses audio characteristics to provide feedback on the system’s interpretation of user input. This technique enables feedback by verbally confirming correct gestures and notifying users of errors in addition to providing continuous feedback by manipulating the pitch of distinct musical notes mapped to each of three dimensional axes in order to provide both spatial and temporal information.

Many studies have highlighted the advantages of expanding the input space of mobile devices by utilizing the back of the device. We extend this work by performing an elicitation study to explore users' mapping of gestures to smartphone commands and identify their criteria for using back-of-device gestures. Using the data collected from our study, we present elicited gestures and highlight common user motivations, both of which inform the design of back-of-device gestures for mobile interaction.