The vast majority of deaf or hard-of-hearing (DHH) children are born to hearing parents. Due to a lack of immersive exposure to their natural language - sign language - they are at severe risk of language deprivation. In response to this challenge, this paper presents a novel computer-mediated communication platform named Tabletop Interactive Play System (TIPS). It serves as a test-bed to investigate technical and ethical solutions that enable hearing parents to use American Sign Language (ASL) during face-to-face play with their with their DHH children. The TIPS platform offers a variety of user options in three key aspects: (1) ASL recommendation in alignment with hearing parents’ real-time speech; (2) ASL display through different form-factors (Augmented Reality (AR) projection, tablet, and smart glasses); and (3) autonomy support to enhance users’ sense of agency and trust in the system. In this paper, we will describe the system’s design, implementation, and preliminary evaluation results.

Over 90 percent of Deaf and Hard of Hearing (DHH) children in the United States are born to hearing parents, who have little to no command of American Sign Language (ASL). This leaves the majority of DHH children at risk of language deprivation in early childhood. This study investigates the design space of Augmented Reality (AR) and wearable technologies in supporting hearing parents to offer sign language environments for young DHH children. We conducted an online survey with 65 participants (hearing/DHH parents and teachers of DHH children aged 6 months to 5 years) to gather preferences and interests of technologies that support hearing parents to deliver ASL on-the-fly, and stay attentive to the DHH child’s visual attention during joint toy play. We found that Near-Object Projection is most preferred for real-time ASL delivery, and haptic feedback is most preferred for raising the parent’s awareness of a child’s attention. Results also show a strong interest in using the proposed technologies in interacting with and maintaining joint attention with DHH children on a daily basis. We discuss key design recommendations that inform the design of future technologies that support just-in-time and contextual-aware communication in ASL, with minimal obtrusion to face-to-face interaction.

Parental language input in early childhood plays a critical role in lifelong neuro-cognitive and social development. Deaf and Hard of Hearing (DHH) children are often at risk of language deprivation due to hearing parents’ limited knowledge of sign language - the natural language for DHH children at birth. To offer an immersive sign language environment for DHH children, we designed a novel computer-mediated communication technology named Table Top Interactive System (TIPS). It aims to provide context-responsive recommendation of American Sign Language (ASL) in real-time for hearing parents during face-to-face joint play with their DHH children. The system emphasizes supporting parent autonomy by adapting ASL recommendations using parent’s speech during play, and minimizes obtrusion for face-to-face interaction through an Augmented Reality (AR) display. This paper describes the design and development of an initial working prototype of TIPS and preliminary results of the system’s efficiency regarding system latency and accuracy for ASL recommendation and visualization. Next, we plan to conduct a user study to gather expert and parent feedback about the system design and ASL recommendation strategies for long-term and personalized usage.

This paper explores gender differences in the evaluation of male and female speakers' affective features in public speaking. We analyzed 260 two-minute behavioral videos (200 of females and 60 of males), collected from an online public speaking practice tool. We adopted a linear regression model that utilized facial and prosodic features, including facial action units (AU), word count, pitch, and volume, to automatically assess speaker performance. The model was evaluated against ratings from 2 expert speakers from Toastmasters, an international public speaking club, on speaker performance. Our feature analysis suggests that certain combinations of features are correlated with higher ratings only in males, such as the combined increase of speech rate and vocal pitch variation. Moreover, our clustering analysis suggests that exhibiting certain negative emotions correlates with higher ratings for males but not for females, illustrating the impact of gender in generating effective feedback on public speaking.