Aphasia, which is a language disorder that affects a person’s ability to communicate, can present profound challenges in one’s daily life. Augmentative and Alternative Communication (AAC) technology can support people with aphasia (PWA) in navigating such challenges, but is often difficult to customize and can overlook personal communication needs. End-user programming, which allows a user to develop and use their own custom programs, presents a promising solution where PWA can create their own personalized solutions. With recent research highlighting how generative AI can be helpful for AAC users, we propose a visual user-programming method that enables PWA to combine multiple AI functions to create personalized communication tools. We conducted semi-structured interviews and co-design workshops with eight participants with aphasia to understand how user-programmed AI systems can address personal communication needs. We present PWA’s perspectives on user-programmable communication tools and discuss the accessibility of visual programming methods for creating aphasia-friendly AI systems.

Generic AI auto-complete for message composition often fails to capture the nuance of personal identity, requiring editing. While harmless in low-stakes settings, for users of Augmentative and Alternative Communication (AAC) devices, who rely on such systems to communicate, this burden is severe. Intuitively, the need for edits would be lower if language models were personalized to the specific user’s communication. While personalization is technically feasible, it raises questions about how such systems affect AAC users’ agency, identity, and privacy. We conducted an autoethnographic study in three phases: (1) seven months of collecting all the lead author’s AAC communication data, (2) fine-tuning a model on this dataset, and (3) three months of daily use of personalized AI suggestions. We observed that: logging everyday conversations reshaped the author’s sense of agency, model training selectively amplified or muted aspects of his identity, and suggestions occasionally resurfaced private details outside their original context. Our findings show that ultra-personalized AAC reshapes communication by continually renegotiating agency, identity, and privacy between user and model. We highlight design directions for building personalized AAC technology that supports expressive, authentic communication.

AI offers key advantages such as instant generation, multi-modal support, and personalized adaptability-potential that can address the highly heterogeneous communication barriers faced by people with aphasia (PwA).We designed AI-enhanced communication tools and used them as design probes to explore how AI's realtime processing and generation capabilities-across text, image, and audio-can align with PwA's needs in real-time communication and preparation for future conversations.Through a two-phase "Research through Design" approach, 11 PwA contributed design insights and evaluated four AI-enhanced prototypes.These prototypes aimed to improve communication grounding and conversational agency through visual verification, grammar construction support, error correction, and reduced language processing load.Despite some challenges, such as occasional mismatches with user intent, findings demonstrate how AI's specific capabilities can be advantageous in addressing PwA's complex needs.Our work contributes design insights for future Augmentative and Alternative Communication (AAC) systems.

Figure 1: A conversation between two AAC users.We find that backchanneling (active-listening) plays a crucial role in communication.However, here to enter text, both users engage with their devices, missing out on non-verbal cues from their interlocutor.We identify a need for better support of backchanneling for AAC, while respecting their established micro-culture.

Group conversations often shift quickly from topic to topic, leaving a small window of time for participants to contribute. AAC users often miss this window due to the speed asymmetry between using speech and using AAC devices. AAC users may take over a minute longer to contribute, and this speed difference can cause mismatches between the ongoing conversation and the AAC user’s response. This results in misunderstandings and missed opportunities to participate. We present COMPA, an add-on tool for online group conversations that seeks to support conversation partners in achieving common ground. COMPA uses a conversation’s live transcription to enable AAC users to mark conversation segments they intend to address (Context Marking) and generate contextual starter phrases related to the marked conversation segment (Phrase Assistance) and a selected user intent. We study COMPA in 5 different triadic group conversations, each composed by a researcher, an AAC user and a conversation partner (n=10) and share findings on how conversational context supports conversation partners in achieving common ground.

Users of augmentative and alternative communication (AAC) devices sometimes find it difficult to communicate in real time with others due to the time it takes to compose messages. AI technologies such as large language models (LLMs) provide an opportunity to support AAC users by improving the quality and variety of text suggestions. However, these technologies may fundamentally change how users interact with AAC devices as users transition from typing their own phrases to prompting and selecting AI-generated phrases. We conducted a study in which 12 AAC users tested live suggestions from a language model across three usage scenarios: extending short replies, answering biographical questions, and requesting assistance. Our study participants believed that AI-generated phrases could save time, physical and cognitive effort when communicating, but felt it was important that these phrases reflect their own communication style and preferences. This work identifies opportunities and challenges for future AI-enhanced AAC devices.

Augmentative and alternative communication (AAC) devices enable speech-based communication, but generating speech is not the only resource needed to have a successful conversation. Being able to signal one wishes to take a turn by raising a hand or providing some other cue is critical in securing a turn to speak. Experienced conversation partners know how to recognize the nonverbal communication an augmented communicator (AC) displays, but these same nonverbal gestures can be hard to interpret by people who meet an AC for the first time. Prior work has identified motion-based AAC as a viable and underexplored modality for increasing ACs’ agency in conversation. We build on this prior work to dig deeper into a particular case study on motion-based AAC by co-designing a physical expressive object to support ACs during conversations. We found that our physical expressive object could support communication with unfamiliar partners. As such, we present our process and resulting lessons on the designed object itself and the co-design process.

Augmentative and alternative communication (AAC) devices enable speech-based communication. However, AAC devices do not support nonverbal communication, which allows people to take turns, regulate conversation dynamics, and express intentions. Nonverbal communication requires motion, which is often challenging for AAC users to produce due to motor constraints. In this work, we explore how socially assistive robots, framed as ''sidekicks,'' might provide augmented communicators (ACs) with a nonverbal channel of communication to support their conversational goals. We developed and conducted an accessible co-design workshop that involved two ACs, their caregivers, and three motion experts. We identified goals for conversational support, co-designed prototypes depicting possible sidekick forms, and enacted different sidekick motions and behaviors to achieve speakers' goals. We contribute guidelines for designing sidekicks that support ACs according to three key parameters: attention, precision, and timing. We show how these parameters manifest in appearance and behavior and how they can guide future designs for augmented nonverbal communication.

The COVID-19 pandemic has forced institutions to rapidly alter their behavior, which typically has disproportionate negative effects on people with disabilities as accessibility is overlooked. To investigate these issues, we analyzed Twitter data to examine accessibility problems surfaced by the crisis. We identified three key domains at the intersection of accessibility and technology: (i) the allocation of product delivery services, (ii) the transition to remote education, and (iii) the dissemination of public health information. We found that essential retailers expanded their high-risk customer shopping hours and pick-up and delivery services, but individuals with disabilities still lacked necessary access to goods and services. Long-experienced access barriers to online education were exacerbated by the abrupt transition of in-person to remote instruction. Finally, public health messaging has been inconsistent and inaccessible, which is unacceptable during a rapidly-evolving crisis. We argue that organizations should create flexible, accessible technology and policies in calm times to be adaptable in times of crisis to serve individuals with diverse needs.

Augmented communicators (ACs) use augmentative and alternative communication (AAC) technologies to speak. Prior work in AAC research has looked to improve efficiency and expressivity of AAC via device improvements and user training. However, ACs also face constraints in communication beyond their device and individual abilities such as when they can speak, what they can say, and who they can address. In this work, we recast and broaden this prior work using conversational agency as a new frame to study AC communication. We investigate AC conversational agency with a study examining different conversational tasks between four triads of expert ACs, their close conversation partners (paid aide or parent), and a third party (experimenter). We define metrics to analyze AAC conversational agency quantitatively and qualitatively. We conclude with implications for future research to enable ACs to easily exercise conversational agency.

Gaze-tracking technologies can enable computer access for users who are unable to use standard input devices. However, using gaze as input poses challenges for interactions that require visual planning, like playing a digital instrument. We explore how multimodality can support eye-controlled musical expression by designing different multi-modal gaze interactions around a digital instrument we call Dueto. We tackle three design goals: creating an instrument that is explorable, easy to learn, and allows feature controllability. We showcase three different multimodal interactions for music playing such as eye gaze only, gaze + switch, and gaze + partner mode.

Consumer-fabrication technologies potentially improve the effectiveness and adoption of assistive technology (AT) by engaging AT users in AT creation. However, little is known about the role of clinicians in this revolution. We investigate clinical AT fabrication by working as expert fabricators for clinicians over a four-month period. We observed and co-designed AT with four occupational therapists at two clinics: a free clinic for uninsured clients, and a Veteran's Affairs Hospital. We find that existing fabrication processes, particularly with respect to rapid prototyping, do not align with clinical practice and itsdo-no-harm ethos. We recommend software solutions that would integrate into client care by: amplifying clinicians' expertise, revealing appropriate fabrication opportunities, and supporting adaptable fabrication.

Maker culture is on the rise in healthcare with the adoption of consumer-grade fabrication technologies. However, little is known about the activities and resources involved in prototyping medical devices to improve patient care. In this paper, we refer to such activity asmedical making to report findings based on a qualitative study of stakeholder engagement in physical prototyping (making) experiences. We examine perspectives from diverse stakeholders including clinicians, engineers, administrators, and medical researchers. Through 18 semi-structured interviews with medical-makers in the US and Canada, we analyze making activity in medical settings. We find that medical makers share strategies to address risks, adopt labor roles, and acquire resources within traditional medical practice. Our findings outline how medical-makers mitigate risks for patient safety, collaborate with local and global stakeholder networks, and overcome constraints of co-location and material practices. We recommend a clinician-aided software system, partially-open repositories, and a collaborative skill-sharing social network to extend their strategies in support of medical making.