Informal learners of computational skills often find it difficult to self-direct their learning pursuits, which may be spread across different mediums and study sessions. Inspired by self-monitoring interventions from domains such as health and productivity, we investigate key requirements for helping informal learners better self-reflect on their learning experiences. We carried out two elicitation studies with article-based and interactive probes to explore a range of manual, automatic, and semi-automatic design approaches for capturing and presenting a learner's data. We found that although automatically generated visual overviews of learning histories are initially promising for increasing awareness, learners prefer having controls to manipulate overviews through personally relevant filtering options to better reflect on their past, plan for future sessions, and communicate with others for feedback. To validate our findings and expand our understanding of designing self-monitoring tools for use in real settings, we gathered further insights from experts, who shed light on factors to consider in terms of data collection techniques, designing for reflections, and carrying out field studies. Our findings have several implications for designing learner-centered self-monitoring interventions that can be both useful and engaging for informal learners.

The rapid advancement of technology and increased access to computing knowledge have led to a rise in the adoption of computing skills, such as programming and complex software. As a result, online platforms such as Coursera, EdX, and Udacity have recently seen some of their highest enrollments in Machine Learning (ML) and other advanced courses. Platforms such as Medium and YouTube have also witnessed an explosion of informational resources on computing-related topics [1]. Although there is an abundance of educational materials on computing and tools that enable individuals to self-learn, learners may not necessarily be able to effectively self-direct their learning [2].

In recent years, more people from different backgrounds are trying to informally learn Machine Learning (ML) using a plethora of online resources, yet we know little about their motivations and learning strategies. We carried out interviews with 22 informal learners of ML from diverse job roles and backgrounds, including Computer Science, Medicine, Finance, and others, to understand their approaches, preferences, and challenges in locating and interacting with different resources to manage their learning. We analyzed our findings using the framework of self-directed learning and found that these informal learners struggled in all stages of self-direction, including identifying learning goals and selecting resources, and that their challenges were most acute in the last stage of gauging progress and evaluating outcomes. We identify several opportunities for future research to better understand and support informal learners of ML (and other complex technical skills). In particular, there is a need to foster more self-monitoring and self-reflection techniques that can help informal learners become more self-aware and effective in directing their learning.

Learners increasingly refer to online videos for learning new technical concepts, but often overlook or forget key details. We investigated how retrieval practice, a learning strategy commonly used in education, could be designed to reinforce key concepts in online videos. We began with a formative study to understand users' perceptions of cued and free-recall retrieval techniques. We next designed a new in-context flashcard-based technique that provides expert-curated retrieval exercises in context of a video's playback. We evaluated this technique with 14 learners and investigated how learners engage with flashcards that are prompted automatically at predefined intervals or flashcards that appear on-demand. Our results overall showed that learners perceived automatically prompted flashcards to be less effortful and made the learners feel more confident about grasping key concepts in the video. However, learners found that on-demand flashcards gave them more control over their learning and allowed them to personalize their review of content. We discuss the implications of these findings for designing hybrid automatic and on-demand in-context retrieval exercises for online videos.