Playing video games can enhance older adults’ well-being and social connections. However, most mainstream games rely on button-based controls that require fine motor skills, limiting accessibility. We present ElderPlay, a real-time game input translation system that enables older adults to play unmodified commercial games using intuitive, motion-based interaction. We first conducted a gesture elicitation study to derive user-defined gestures grounded in everyday experiences, which informed the design of a proof-of-concept system translating gestures into controller inputs. We then evaluated ElderPlay with two commercial Nintendo Switch games. Results show that gesture-based interaction improves enjoyment, perceived physical engagement, and performance. Rather than replacing controllers, our findings highlight the effectiveness of hybrid interaction, where gesture and controller inputs support different gameplay actions. We discuss implications for context-dependent and inclusive game interaction design.

Family co-play of video games is common and linked to family closeness and positive intergenerational interaction. Mixed-age co-play exposes controller-skill gaps that cause frustration and exclusion, yet most party games lack ability-inclusive assists. We explore ability-inclusive co-play for unmodified party and family games: We contribute (1) an open-source controller-skills benchmark; (2) the first lifespan study combining both broad age and deep skill coverage (ages 6–90; n=80; six skills), showing that gaps between children and older adults are skill-specific rather than uniform; (3) we present PartyAssist, a real-time, computer-vision-to-input system that detects on-screen state and injects micro controller-input assistance without altering game code; (4) in a feasibility study with 16 mixed-skill dyads (n=32; assisted player <10 or >50), assistance improved children’s survival time and success rate while remaining largely unnoticed and was viewed positively, while older adults detected assistance and reported mixed views. Interviews surfaced socially-considerate design nuances and implications to inform future designs to support ability-inclusive co-play across the lifespan.

Deaf and hard-of-hearing (DHH) individuals encounter difficulties when engaged in group conversations with hearing individuals, due to factors such as simultaneous utterances from multiple speakers and speakers whom may be potentially out of view. We interviewed and co-designed with eight DHH participants to address the following challenges: 1) associating utterances with speakers, 2) ordering utterances from different speakers, 3) displaying optimal content length, and 4) visualizing utterances from out-of-view speakers. We evaluated multiple designs for each of the four challenges through a user study with twelve DHH participants. Our study results showed that participants significantly preferred speechbubble visualizations over traditional captions. These design preferences guided our development of SpeechBubbles, a real-time speech recognition interface prototype on an augmented reality head-mounted display. From our evaluations, we further demonstrated that DHH participants preferred our prototype over traditional captions for group conversations.