Although students need to self-monitor and manage their learning process for effective metacognition, it can be particularly challenging in solitary e-learning environments that rely on pre-recorded videos. Unlike interactive e-learning or physical classrooms, typical e-learning environments prevent students from interacting with their teachers and peers, thereby hindering metacognitive support. To address this challenge, we introduce GPTalk, a system designed to support students’ learning experiences by facilitating interactions with LLM-based virtual companions. Through interviews with students and teachers, we identified design recommendations and implemented them in GPTalk. A user study involving 32 high-school students demonstrated that, compared to a baseline system, GPTalk fostered richer metacognitive engagement and self-regulated learning processes during video-based study (e.g., more monitoring questions and in-situ reflections), while short-term content understanding accuracy remained comparable across conditions. Overall, our findings suggest that students’ interactions with a virtual teacher and peer can support key aspects of their metacognition and self-regulated e-learning processes.

Integrating mixed-hearing groups in musical collaboration presents unique challenges and opportunities for their communication and equal contribution. This observational study aims to explore their collaborative work, focusing on the way for equitable music-making. We observed two music-making workshops to identify the potential and dynamics of their musical collaboration. While the first workshop proceeded in a traditional manner of music-making, the second workshop used an assistive tool with multimodality. Our findings highlight the dynamics in musical collaboration that foster engagement and bridge interaction gaps. In turn, sensory inclusion with multimodal music-making promoted role transition in mixed-hearing groups and their equal contributions, leading to the embracing of diverse cultural perspectives. Based on the insights derived from the observations, we propose a design guideline and future research directions for harnessing group dynamics and building equitable musical collaborations for an inclusive environment for mixed-hearing groups.

Deaf and hard of hearing (DHH) people enjoy music and access it using a music-sensory substitution system that delivers sound together with the corresponding visual and tactile feedback. However, it is often challenging for them to comprehend the colorful visuals and strong vibrations that are designed to represent music. We confirmed that it is necessary to conceptualize cross-modal mapping before experiencing music sensory substitution through focus group interviews with 24 DHH people. To improve the music appreciation experience, a cross-modal music conceptualization system was implemented herein, which is a prototype that allows DHH people to explore the visuals and vibrations associated with music to perceive and appreciate. An evaluation with 28 DHH individuals demonstrated the capability of the system to improve subjective music appreciation experience via music-sensory substitution. Eventually, DHH people with negative attitudes toward music became positive in the exploration and customization process with our system.