The vast majority of deaf or hard-of-hearing (DHH) children are born to hearing parents. Due to a lack of immersive exposure to their natural language - sign language - they are at severe risk of language deprivation. In response to this challenge, this paper presents a novel computer-mediated communication platform named Tabletop Interactive Play System (TIPS). It serves as a test-bed to investigate technical and ethical solutions that enable hearing parents to use American Sign Language (ASL) during face-to-face play with their with their DHH children. The TIPS platform offers a variety of user options in three key aspects: (1) ASL recommendation in alignment with hearing parents’ real-time speech; (2) ASL display through different form-factors (Augmented Reality (AR) projection, tablet, and smart glasses); and (3) autonomy support to enhance users’ sense of agency and trust in the system. In this paper, we will describe the system’s design, implementation, and preliminary evaluation results.

Due to usage of self-reported data which may contain biasness, the existing studies may not unveil the exact relation between academic grades and app categories such as Video. Additionally, the existing systems' requirement for data of prolonged period to predict grades may not facilitate early intervention to improve it. Thus, we presented an app that retrieves past 7 days' actual app usage data within a second (Mean=0.31s, SD=1.1s). Our analysis on 124 Bangladeshi students' real-time data demonstrates app usage sessions have a significant (p<0.05) negative association with CGPA. However, the Productivity and Books categories have a significant positive association whereas Video has a significant negative association. Moreover, the high and low CGPA holders have significantly different app usage behavior. Leveraging only the instantly accessed data, our machine learning model predicts CGPA within ±0.36 of the actual CGPA. We discuss the design implications that can be potential for students to improve grades.

Parental language input in early childhood plays a critical role in lifelong neuro-cognitive and social development. Deaf and Hard of Hearing (DHH) children are often at risk of language deprivation due to hearing parents’ limited knowledge of sign language - the natural language for DHH children at birth. To offer an immersive sign language environment for DHH children, we designed a novel computer-mediated communication technology named Table Top Interactive System (TIPS). It aims to provide context-responsive recommendation of American Sign Language (ASL) in real-time for hearing parents during face-to-face joint play with their DHH children. The system emphasizes supporting parent autonomy by adapting ASL recommendations using parent’s speech during play, and minimizes obtrusion for face-to-face interaction through an Augmented Reality (AR) display. This paper describes the design and development of an initial working prototype of TIPS and preliminary results of the system’s efficiency regarding system latency and accuracy for ASL recommendation and visualization. Next, we plan to conduct a user study to gather expert and parent feedback about the system design and ASL recommendation strategies for long-term and personalized usage.