Although students need to self-monitor and manage their learning process for effective metacognition, it can be particularly challenging in solitary e-learning environments that rely on pre-recorded videos. Unlike interactive e-learning or physical classrooms, typical e-learning environments prevent students from interacting with their teachers and peers, thereby hindering metacognitive support. To address this challenge, we introduce GPTalk, a system designed to support students’ learning experiences by facilitating interactions with LLM-based virtual companions. Through interviews with students and teachers, we identified design recommendations and implemented them in GPTalk. A user study involving 32 high-school students demonstrated that, compared to a baseline system, GPTalk fostered richer metacognitive engagement and self-regulated learning processes during video-based study (e.g., more monitoring questions and in-situ reflections), while short-term content understanding accuracy remained comparable across conditions. Overall, our findings suggest that students’ interactions with a virtual teacher and peer can support key aspects of their metacognition and self-regulated e-learning processes.

Gesture data collection in a controlled lab environment often restricts participants to performing gestures in a uniform or biased manner, resulting in gesture data which may not sufficiently reflect gesture variability to build robust gesture recognition models. Crowdsourcing has been widely accepted as an efficient high-sample-size method for collecting more representative and variable data. In this study, we evaluated the effectiveness of crowdsourcing for gesture data collection, specifically for gesture variability. When compared to a controlled lab environment, crowdsourcing resulted in improved recognition performance of 8.98% and increased variability for various gesture features, eg, a 142% variation increase for Quantity of Movement. Integrating a supplemental gesture data collection methodology known as Styling Words increased recognition performance by an additional 2.94%. The study also investigated the efficacy of gesture collection methodologies and gesture memorization paradigms.

Due to advances in deep learning, gestures have become a more common tool for human-computer interaction. When implementing a large amount of training data, deep learning models show remarkable performance in gesture recognition. Since it is expensive and time consuming to collect gesture data from people, we are often confronted with a practicality issue when managing the quantity and quality of training data. It is a well-known fact that increasing training data variability can help to improve the generalization performance of machine learning models. Thus, we directly intervene in the collection of gesture data to increase human gesture variability by adding some words (called styling words) into the data collection instructions, e.g., giving the instruction "perform gesture #1 faster" as opposed to "perform gesture #1." Through an in-depth analysis of gesture features and video-based gesture recognition, we have confirmed the advantageous use of styling words in gesture training data collection.