Debugging is a crucial skill for programmers, yet it can be challenging for novices to learn. The introduction of large language models (LLMs) has opened up new possibilities for providing personalized debugging support to students. However, concerns have been raised about potential student over-reliance on LLM-based tools. This mixed-methods study investigates how a pedagogically-designed LLM-based chatbot supports students’ debugging efforts in an introductory programming course. We conducted interviews and debugging think-aloud tasks with 20 students at three points throughout the semester. We specifically focused on characterizing when students initiate help from the chatbot during debugging, how they engage with the chatbot’s responses, and how they describe their learning experiences with the chatbot. By analyzing data from the debugging tasks, we identified varying help-seeking behaviors and levels of engagement with the chatbot’s responses, depending on students’ familiarity with the suggested strategies. Interviews revealed that students appreciated the content and experiential knowledge provided by the chatbot, but did not view it as a primary source for learning debugging strategies. Additionally, students self-identified certain chatbot usage behaviors as negative, “non-ideal” engagement and others as positive, “learning-oriented” usage. Based on our findings, we discuss pedagogical implications and future directions for designing pedagogical chatbots to support debugging.

One-on-one tutoring is effective for learning computer science since a tutor can work alongside a student and provide tailored feedback on their code. However, translating this type of instruction to a remote setting is challenging. Traditional methods such as screensharing lack key pedagogical functionality and most available tools are designed for collaboration rather than instruction. To identify tools that can support remote tutoring, we conducted an experiment to assess two resources: synchronous editing and awareness tools. In our study, an instructor teaches a learner introductory programming concepts remotely, collaborating through screensharing alone, a shared notebook with real-time collaborative editing, or a shared notebook with additional awareness tools overlaid. To embed the awareness tools, we designed a Chrome extension that enables real-time sharing of gaze and cursor data. Our results show that synchronous editing combined with awareness tools significantly improved learning. The awareness tools also helped tutors better communicate with the student, track their understanding, and establish a sense of presence. As a final contribution, we also assessed the efficacy of gaze-sharing using a webcam eye-tracker. While the accuracy was not as precise as a dedicated sensor, instructors described instances when the gaze was useful for gauging student attention and establishing presence. We discuss implications for remote tutoring in computer science and scaling awareness technology.

Learning physics is often difficult for students because concepts such as electricity, magnetism and sound, cannot be seen with the naked eye. Emerging technologies such as Augmented Reality (AR) can transform education by making challenging concepts visible and accessible to novices. We present a Hololens-based augmented reality system where collaborators learn about the invisible electromagnetism phenomena involved in audio speakers, and we measure the benefits of AR technology through quantitative and qualitative methods. Specifically, we measure learning (knowledge gains and transfer) and collaborative knowledge exchange behaviors. Our results indicate that, while AR generally provides a novelty effect, specific educational AR visualizations can be both beneficial and detrimental to learning - they helped students to learn spatial content and structural relationships, but hindered their understanding of kinesthetic content. Furthermore, AR facilitated learning in collaborations by providing representational common ground, which improved communication and peer teaching. We discuss these effects, as well as identify factors that have positive impact (e.g., co-located representations, easier access to resources, better grounding) or negative impact (e.g., tunnel vision, overlooking kinesthetic feedback) on student collaborative learning with augmented reality applications.

Augmented Reality (AR) technology is gaining popularity and adoption by educational pioneers. While there is great excitement about this technology, there is also a lack of systematic understanding of what makes this technology effective or ineffective for education, and the practical and systemic challenges of implementing it at scale in classroom settings. In this research we provide a synthesis of empirical scientific findings about the factual benefits and detriments of this technology (from a dataset of 2023 academic papers, of which 39 were analyzed in depth), and of public opinions about augmented reality in education practice (gathered from a dataset of 86 websites and blog posts, of which 53 were analyzed in depth). We contribute a list of specific cognitive, motivational and social processes that are enhanced by AR technology. We also identify popular curriculum topics and popular AR applications, as well as summarize several factors that are important for educators to consider for integrating AR into classrooms.

Research indicates that makerspaces equip students with the practical skills needed to build their own projects and thrive in the twenty-first-century workforce. While the appeal of makerspaces lies in their spirit of tinkering and community-driven ethos, these same attributes make it difficult to monitor and facilitate learning. Makerspaces also attract students from diverse backgrounds and skills, further challenging facilitators to accommodate the needs of each student and their self-directed projects. We propose a dashboard interface that visualizes Kinect sensor data to aid facilitators in monitoring student collaboration. The tool was designed with an iterative and participatory approach. Five facilitators were involved at each phase of the design process, from need-finding to prototyping to implementation and evaluation. Insights derived from interviews were used to inform the design decisions of the final interface. The final evaluation suggests that the use of normalized summary scores and an interactive network graph can successfully support facilitators in tasks related to improving collaboration. Moreover, the use of a red-green color scheme and the inclusion of student photos improved the usability for facilitators, but issues of trustworthiness need to be further examined.

While Multimodal Learning Analytics (MMLA) is becoming a popular methodology in the LAK community, most educational researchers still rely on traditional instruments for capturing learning processes (e.g., click-stream, log data, self-reports, qualitative observations). MMLA has the potential to complement and enrich traditional measures of learning by providing high frequency data on learners’ behavior, cognition and affects. However, there is currently no easy-to-use toolkit for recording multimodal data streams. Existing methodologies rely on the use of physical sensors and custom-written code for accessing sensor data. In this paper, we present the EZ-MMLA toolkit. This toolkit was implemented as a website that provides easy access to the latest machine learning algorithms for collecting a variety of data streams from webcams: attention (eye-tracking), physiological states (heart rate), body posture (skeletal data), hand gestures, emotions (from facial expressions and speech), and lower-level computer vision algorithms (e.g., fiducial / color tracking). This toolkit can run from any browser and does not require special hardware or programming experience. We compare this toolkit with traditional methods and describe a case study where the EZ-MMLA toolkit was used in a classroom context. We conclude by discussing other applications of this toolkit, potential limitations, and future steps.

Abstract The past decade has witnessed a growing interest for using dual eye tracking to understand and support remote collaboration, especially with studies that have established the benefits of displaying gaze information for small groups. While this line of work is promising, we lack a consistent framework that researchers can use to organize and categorize studies on the effect of shared gaze on social interactions. There exists a wide variety of terminology and methods for describing attentional alignment; researchers have used diverse techniques for designing gaze visualizations. The settings studied range from real-time peer collaboration to asynchronous viewing of eye-tracking video of an expert providing explanations. There has not been a conscious effort to synthesize and understand how these different approaches, techniques and applications impact the effectiveness of shared gaze visualizations (SGVs). In this paper, we summarize the related literature and the benefits of SGVs for collaboration, describe important terminology as well as appropriate measures for the dual eye-tracking space and discuss promising directions for future research. As eye-tracking technology becomes more ubiquitous, there is pressing need to develop a consistent approach to evaluation and design of SGVs. The present paper makes a first and significant step in this direction.

In this paper we contribute a literature review and organization framework for classifying the collaboration needs and features that should be considered when designing headset-based augmented reality (AR) experiences for collocated settings. In recent years augmented reality technology has been experiencing significant growth through the emergence of headsets that allow gestural interaction, and AR designers are increasingly interested in using this technology to enhance collaborative activities in a variety of physical environments. However, collaborative AR applications need to contain features that enhance collaboration and satisfy needs that are present during the group activities. When AR designers lack an understanding of what collaborators need during an interaction, or what features have already been designed to solve those needs, then AR creators will spend time redesigning features that have already been created, or worse, create applications that do not contain necessary features. While much work has been done on designing virtual reality (VR) collaborative environments, AR environments are a relatively newer design space, and designers are lacking a comprehensive framework for describing needs that arise during collaborative activities and the features that could be designed into AR applications to satisfy those needs. In this paper we contribute a literature review of 92 papers in the areas of augmented reality and virtual reality, and we contribute a list of design features and needs that are helpful to consider when designing for headset-based collaborative AR experiences.

Shared gaze visualizations, where the real-time gaze location of group members is shared with one another, have become increasingly studied over the last decade by HCI researchers. Shared gaze studies so far have found improved outcomes and better collaboration for peer collaborators. Less is known, however, about how gaze sharing may aid learners and instructors. In our study, an instructor teaches a learner how to assemble and program a simple microcontroller, communicating either through a webcam feed (webcam condition), a field-of-view video feed (HMC condition), or a field-of-view video feed with a gaze location pointer (gaze condition). We find that learning gain is highest in the gaze condition, especially for low achievers. Moreover, instructors predict learner post-test scores more accurately with gaze visualizations, suggesting gaze sharing can help instructors track the cognitive state of the learner. This effect was also most salient for low achievers. We find that in the HMC condition that only lacked this single dot, many of the benefits for both learning and teaching were lost. The paper concludes with discussions on how gaze visualization may support learning and teaching, and on the tool's limitations and conditions for usefulness.

Augmented reality (AR) applications are growing in popularity in educational settings. While the effects of AR experiences on learning have been widely studied, there is relatively less research on understanding the impact of AR on the dynamics of co-located collaborative learning, specifically in the context of novices programming robots. Educational robotics are a powerful learning context because they engage students with problem solving, critical thinking, STEM (Science, Technology, Engineering, Mathematics) concepts, and collaboration skills. However, such collaborations can suffer due to students having unequal access to resources or dominant peers. In this research we investigate how augmented reality impacts learning and collaboration while peers engage in robot programming activities. We use a mixed methods approach to measure how participants are learning, manipulating resources, and engaging in problem solving activities with peers. We investigate how these behaviors are impacted by the presence of augmented reality visualizations, and by participants? proximity to resources. We find that augmented reality improved overall group learning and collaboration. Detailed analysis shows that AR strongly helps one participant more than the other, by improving their ability to learn and contribute while remaining engaged with the robot. Furthermore, augmented reality helps both participants maintain a common ground and balance contributions during problem solving activities. We discuss the implications of these results for designing AR and non-AR collaborative interfaces.

Emerging technologies such as Augmented Reality (AR), have the potential to radically transform education by making challenging concepts visible and accessible to novices. In this project, we have designed a Hololens-based system in which collaborators are exposed to an unstructured learning activity in which they learned about the invisible physics involved in audio speakers. They learned topics ranging from spatial knowledge, such as shape of magnetic fields, to abstract conceptual knowledge, such as relationships between electricity and magnetism. We compared participants' learning, attitudes and collaboration with a tangible interface through multiple experimental conditions containing varying layers of AR information. We found that educational AR representations were beneficial for learning specific knowledge and increasing participants' self-efficacy (i.e., their ability to learn concepts in physics). However, we also found that participants in conditions that did not contain AR educational content, learned some concepts better than other groups and became more curious about physics. We discuss learning and collaboration differences, as well as benefits and detriments of implementing augmented reality for unstructured learning activities.

The proliferation of varied types of multi-user interactive surfaces (such as digital whiteboards, tabletops and tangible interfaces) is opening a new range of applications in face-to-face (f2f) contexts. They offer unique opportunities for Learning Analytics (LA) by facilitating multi-user sensemaking of automatically captured digital footprints of students' f2f interactions. This paper presents an analysis of current research exploring learning analytics associated with the use of surface devices. We use a framework to analyse our first-hand experiences, and the small number of related deployments according to four dimensions: the orchestration aspects involved; the phases of the pedagogical practice that are supported; the target actors; and the levels of iteration of the LA process. The contribution of the paper is twofold: 1) a synthesis of conclusions that identify the degree of maturity, challenges and pedagogical opportunities of the existing applications of learning analytics and interactive surfaces; and 2) an analysis framework that can be used to characterise the design space of similar areas and LA applications.

In this study, we investigated the way users memorize, analyze, collaborate, and learn new concepts on a Tangible User Interface (TUI). Twenty-seven pairs of apprentices in logistics ( N = 54) interacted with an interactive simulation of a warehouse. Their task was to discover efficient design principles for building storehouses. In a between-subjects experimental design, half of the participants used 3D physical shelves, whereas the other half used 2D paper shelves. This manipulation allowed us to control for the “representational effect” of 3D tangibles: the first group saw the warehouse as a small-scale model with realistic shelves, whereas the second group had access to a more abstract layout with rectangular pieces of paper. Both groups interacted with the system in the same way. We found that participants in the first group (i.e., who used 3D realistic shelves) better memorized a warehouse layout, built a more efficient model, and scored higher on a learning test. Additionally, students wore eye-tracking goggles while completing those tasks; preliminary results suggest that 3D interfaces increased joint visual attention, which was found to be a significant predictor for participants’ task performance and learning gains. Implications for designing TUIs in collaborative settings are discussed.

In this paper we describe preliminary applications of network analysis techniques to eye-tracking data. In a previous study, the first author conducted a collaborative learning experiment in which subjects had access (or not) to a gaze-awareness tool: their task was to learn from neuroscience diagrams in a remote collaboration. In the treatment group, they could see the gaze of their partner displayed on the screen in real-time. In the control group, they could not. Dyads in the treatment group achieved a higher quality of collaboration and a higher learning gain. In this paper, we describe how network analysis techniques can further illuminate these results, and contribute to the development of 'collaboration sensing'. More specifically, we describe two contributions: first, one can use networks to visualize and explore eye-tracking data. Second, network metrics can be computed to interpret the properties of the graph. We conclude with comments on implementing this approach for formal learning environments.

Phylogenetic trees are representations of evolutionary relationships amongst species. Interviews of instructors and students have revealed that novice biologists have difficulty understanding phylogenetics. Moreover, misinterpretations of phylogenetics are common among college-level students. In this paper we present Phylo-Genie, a tabletop interface for fostering collaborative learning of phylogenetics. We conducted an experimental study with 56 participants, comparing students' conceptual learning and engagement using Phylo-Genie as: 1) a multi-touch tabletop interface and 2) a pen and paper activity. Our findings show that the tabletop implementation fosters collaborative learning by engaging users in the activity. We also shed light on the way in which our design principles facilitated engagement and collaborative learning in a tabletop environment.