Educational videos are widely used, but accessibility guidelines beyond captions for d/Deaf and Hard-of-Hearing (DHH) learners remain limited. Mayer’s multimedia learning theory assumes visual-auditory dual-channel processing, yet DHH learners with limited access to the auditory channel have distinct visual abilities and cognitive demands. This paper introduces motion-driven design ideas to support cognitive processing and improve video-based learning for DHH learners. Through a three-phase study, we identified four key challenges—such as misaligned content and visual overload—and proposed four design ideas that extend multimedia learning theory. We then evaluated these ideas with 16 DHH learners and 6 experts in Deaf education. The results show that motion-driven approaches reduce misalignment, ease visual attention switching, and improve the integration of visual and textual information across video types. For example, guiding visual attention switching minimizes confusion in complex visual contexts, such as programming demonstrations, while using relevant visuals enriches talking-head videos with graphics to clarify abstract ideas in captions. More research is needed to develop these promising ideas into well-defined principles.

Generative AI is increasingly present in children’s learning environments, yet little is known about how families navigate this technology in middle childhood (ages 7–13), when parental guidance remains strong but children seek independence. Drawing on self-directed learning (SDL), we explore how parents in our exploratory sample perceived children’s emerging self-directness and agency. Through focus groups with 13 parent–child pairs, we examine parents’ views on children’s AI literacy development, readiness factors, and mediation strategies. Parents described emergent pathways shaped by screen time, self-directness, and knowledge growth. They often confined AI to learning-only contexts, positioning it as a tutor while overlooking non-learning uses and risks such as privacy and infrastructural embedding. Many acknowledged limited AI literacy and turned to joint engagement as opportunities for co-learning. Our findings surface possible parental pathways of children’s AI literacy, highlight gaps between pragmatic expectations and critical literacies, and offer situated design considerations for AI systems that scaffold SDL while balancing oversight with autonomy.

This study investigates innovative interaction designs for communication and collaborative learning between learners of mixed hearing and signing abilities, leveraging advancements in mixed reality technologies like Apple Vision Pro and generative AI for animated avatars. Adopting a participatory design approach, we engaged 15 d/Deaf and hard of hearing (DHH) students to brainstorm ideas for an AI avatar with interpreting ability (sign language to English and English to sign language) that would facilitate their face-to-face communication with hearing peers. Participants envisioned the AI avatars to address some issues with human interpreters, such as lack of availability, and provide affordable options to expensive personalized interpreting services. Our findings indicate a range of preferences for integrating the AI avatars with actual human figures of both DHH and hearing communication partners. The participants highlighted the importance of having control over customizing the AI avatar, such as AI-generated signs, voices, facial expressions, and their synchronization for enhanced emotional display in communication. Based on our findings, we propose a suite of design recommendations that balance respecting sign language norms with adherence to hearing social norms. Our study offers insights into improving the authenticity of generative AI in scenarios involving specific and sometimes unfamiliar social norms.

Accessibility efforts for d/Deaf and hard of hearing (DHH) learners in video-based learning have mainly focused on captions and interpreters, with limited attention to learners' emotional awareness--an important yet challenging skill for effective learning. Current emotion technologies are designed to support learners' emotional awareness and social needs; however, little is known about whether and how DHH learners could benefit from these technologies. Our study explores how DHH learners perceive and use emotion data from two collection approaches, self-reported and automatic emotion recognition (AER), in video-based learning. By comparing the use of these technologies between DHH (N=20) and hearing learners (N=20), we identified key differences in their usage and perceptions: 1) DHH learners enhanced their emotional awareness by rewatching the video to self-report their emotions and called for alternative methods for self-reporting emotion, such as using sign language or expressive emoji designs; and 2) while the AER technology could be useful for detecting emotional patterns in learning experiences, DHH learners expressed more concerns about the accuracy and intrusiveness of the AER data. Our findings provide novel design implications for improving the inclusiveness of emotion technologies to support DHH learners, such as leveraging DHH peer learners' emotions to elicit reflections.

Previous research underscored the potential of danmaku–a text-based commenting feature on videos–in engaging hearing audiences. Yet, for many Deaf and hard-of-hearing (DHH) individuals, American Sign Language (ASL) takes precedence over English. To improve inclusivity, we introduce “Signmaku,” a new commenting mechanism that uses ASL, serving as a sign language counterpart to danmaku. Through a need-finding study (N=12) and a within-subject experiment (N=20), we evaluated three design styles: real human faces, cartoon-like figures, and robotic representations. The results showed that cartoon-like signmaku not only entertained but also encouraged participants to create and share ASL comments, with fewer privacy concerns compared to the other designs. Conversely, the robotic representations faced challenges in accurately depicting hand movements and facial expressions, resulting in higher cognitive demands on users. Signmaku featuring real human faces elicited the lowest cognitive load and was the most comprehensible among all three types. Our findings offered novel design implications for leveraging generative AI to create signmaku comments, enriching co-learning experiences for DHH individuals.

We present a group autoethnography detailing a hearing student’s journey in adopting communication technologies at a mixed-hearing ability summer research camp. Our study focuses on how this student, a research assistant with emerging American Sign Language (ASL) skills, (in)effectively communicates with deaf and hard-of-hearing (DHH) peers and faculty during the ten-week program. The DHH members also reflected on their communication with the hearing student. We depict scenarios and analyze the (in)effectiveness of how emerging technologies like live automatic speech recognition (ASR) and typing are utilized to facilitate communication. We outline communication strategies to engage everyone with diverse signing skills in conversations - directing visual attention, pause-for-attention-and-proceed, and back-channeling via expressive body. These strategies promote inclusive collaboration and leverage technology advancements. Furthermore, we delve into the factors that have motivated individuals to embrace more inclusive communication practices and provide design implications for accessible communication technologies within the mixed-hearing ability context.

Developing novel research questions (RQs) often requires extensive literature reviews, especially in interdisciplinary fields. To support RQ development through human-AI co-creation, we leveraged Large Language Models (LLMs) to build an LLM-based agent system named CoQuest. We conducted an experiment with 20 HCI researchers to examine the impact of two interaction designs: breadth-first and depth-first RQ generation. The findings revealed that participants perceived the breadth-first approach as more creative and trustworthy upon task completion. Conversely, during the task, participants considered the depth-first generated RQs as more creative. Additionally, we discovered that AI processing delays allowed users to reflect on multiple RQs simultaneously, leading to a higher quantity of generated RQs and an enhanced sense of control. Our work makes both theoretical and practical contributions by proposing and evaluating a mental model for human-AI co-creation of RQs. We also address potential ethical issues, such as biases and over-reliance on AI, advocating for using the system to improve human research creativity rather than automating scientific inquiry. The system’s source is available at: https://github.com/yiren-liu/coquest.

The think-aloud protocol is an effective method frequently used by designers and researchers to understand how users interact with computing systems. However, there is limited research on the use of this method with deaf and hard of hearing (DHH) populations, especially in virtual settings. In this paper, we investigate the behaviors of DHH participants in virtual think-aloud sessions to better understand the challenges of conducting this type of research with this population. We conducted twelve virtual think-aloud sessions with DHH participants using Zoom, and we gathered feedback from surveys, interviews, and observations. Our results identified DHH behaviors leading to a lack of clarity in think-aloud data, such as asynchrony between signing and navigating the interfaces, as well as the use of visual descriptive signs instead of explicit terminology to ambiguously refer to interface components. Based on our findings, we provide methodological and design implications to help researchers effectively carry out virtual think-aloud studies with DHH participants (e.g., when and how to prompt for clarification).

Ethnic minorities face challenges in sustaining their culture in regions dominated by ethnic majorities. With the growing popularity of video blogging (vlogging) in China, many ethnic minority vloggers are using vlogs to present and promote their ethnic culture online. In this study, we interviewed 16 vloggers onDouyin to understand why and how vlogs can be used to sustain ethnic culture. We found that both ethnic cultural experts and non-experts were involved in ethnic vlog making and sharing activities onDouyin, and cultural experts took more initiative in preserving and promoting ethnic culture while non-experts were more motivated by getting more traffic and income. Vloggers' imagined audiences included both intra-ethnic and mainstream viewers, impacting their vlog-making strategies, the utilized platform features, and the created vlog content. For example, vloggers taught ethnic language and built an identity for intra-ethnic viewers. Both ethnic minority vloggers and viewers protected their culture from misinterpretation by mainstream viewers. Our findings suggest the potential of using video blogging to address the challenges of cultural sustainability, providing design implications for future ICTs to support the cultural sustainability of ethnic minorities.

Learners' awareness of their own affective states (emotions) can improve their meta-cognition, which is a critical skill of being aware of and controlling one's cognitive, motivational, and affect, and adjusting their learning strategies and behaviors accordingly. To investigate the effect of peers' affects on learners' meta-cognition, we proposed two types of cues that aggregated peers' affects that were recognized via facial expression recognition:Locative cues (displaying the spikes of peers' emotions along a video timeline) andTemporal cues (showing the positivities of peers' emotions at different segments of a video). We conducted a between-subject experiment with 42 college students through the use of think-aloud protocols, interviews, and surveys. Our results showed that the two types of cues improved participants' meta-cognition differently. For example, interacting with theTemporal cues triggered the participants to compare their own affective responses with their peers and reflect more on why and how they had different emotions with the same video content. While the participants perceived the benefits of using AI-generated peers' cues to improve their awareness of their own learning affects, they also sought more explanations from their peers to understand the AI-generated results. Our findings not only provide novel design implications for promoting learners' meta-cognition with privacy-preserved social cues of peers' learning affects, but also suggest an expanded design framework for Explainable AI (XAI).

Prior research suggests that affective states of self-regulated learning can be used to improve learners’ cognitive processes and their learning outcomes. However, little research explored the effect of using facial movements to detect learners’ affective states on self-regulated learning. In this work, we designed, implemented, and evaluated Mirror: a self-regulated learning tool that applies facial expression recognition to support learners’ reflections in video-based learning. We conducted two studies to identify user needs (with 12 participants) and to evaluate the tool (with 16 participants). The results show that, after watching a video, participants benefited from using Mirror through different reflection processes, e.g., gaining a deeper understanding of their learning experiences through self-observation and attributing causes for their learning affects through self-judgment. Meanwhile, we also identified several ethical concerns, e.g., users’ agency of handling the uncertainty of AI, reactivity towards outcome-based AI, over-reliance on “positive” AI results, and fairness of AI informed decision-making.

The outbreak of COVID-19 has led to a sharp transition from offline to online education in many countries and areas. This transition heightens the intensity of existing challenges of online education, such as student attendance and education equality. During this time of uncertainty, the vast disparities in teachers? online experience and technical backgrounds, students' education level and their families' economic status, and schools' support, further pose new challenges to teachers and students. In this work, we study how Chinese teachers and students addressed challenges during this transition. We interviewed 15 teachers and 18 students from diverse backgrounds at varying education levels (K-12 and college). Our work makes timely and new contributions to the literature of online education. For example, our results showed that teachers applied Live Video Streaming (LVS) on multiple social media platforms and re-purposed different entertainment features to deliver online teaching for better student engagement; some teachers came to enjoy this new form of instruction after being resistant to it in the beginning, and students developed a better sense of intimacy with their teachers after experiencing certain online interactions. Our work also reveals the remaining challenges and prospects of LVS-based online education and sheds light on the future design of collaborative technologies for online education.