Collegiate student-athletes train and compete in a dense data ecology where information about their bodies and performances circulates among coaches, staff, and fans. To understand how student-athletes themselves engage with this data, we conducted interviews with 20 student-athletes, identifying four modes of engagement: 1) performance-directive, executing training and targeting improvement; 2) reflective-monitoring, assessing the body’s reaction to training and daily load; 3) coach-mediated, receiving insights through staff expertise; and 4) selective-disengagement, intentionally stepping back to protect confidence or avoid overload. These findings fill a gap left open by three related areas of research: SportsHCI, collegiate athletics, and personal data engagement. Each mode entails reasons, practices, and trade-offs. Student-athletes draw on different combinations of these modes as they respond to training demands, coaching oversight, and their own well-being. Our findings highlight how an evolving data ecology creates opportunities and pressures, requiring student-athletes to balance performance with protecting their state of mind.

Simulating learner actions helps stress-test open-ended interactive learning environments and prototype new adaptations before deployment. While recent studies show the promise of using large language models (LLMs) for simulating human behavior, such approaches have not gone beyond rudimentary proof-of-concept stages due to key limitations. First, LLMs are highly sensitive to minor prompt variations, raising doubts about their ability to generalize to new scenarios without extensive prompt engineering. Moreover, apparently successful outcomes can often be unreliable, either because domain experts unintentionally guide LLMs to produce expected results, leading to self-fulfilling prophecies; or because the LLM has encountered highly similar scenarios in its training data, meaning that models may not be simulating behavior so much as regurgitating memorized content. To address these challenges, we propose Hyp-Mix, a simulation authoring framework that allows experts to develop and evaluate simulations by combining testable hypotheses about learner behavior. Testing this framework in a physics learning environment, we found that GPT-4 Turbo maintains calibrated behavior even as the underlying learner model changes, providing the first evidence that LLMs can be used to simulate realistic behaviors in open-ended interactive learning environments, a necessary prerequisite for useful LLM behavioral simulation.

Indirect User Requests (IURs), such as “It’s cold in here” instead of “Could you please increase the temperature?” are common in human-human task-oriented dialogue and require world knowledge and pragmatic reasoning from the listener. While large language models (LLMs) can handle these requests effectively, smaller models deployed on virtual assistants often struggle due to resource constraints. Moreover, existing task-oriented dialogue benchmarks lack sufficient examples of complex discourse phenomena such as indirectness. To address this, we propose a set of linguistic criteria along with an LLM-based pipeline for generating realistic IURs to test natural language understanding (NLU) and dialogue state tracking (DST) models before deployment in a new domain. We also release IndirectRequests, a dataset of IURs based on the Schema-Guided Dialogue (SGD) corpus, as a comparative testbed for evaluating the performance of smaller models in handling indirect requests.

Question-asking is a crucial learning and teaching approach. It reveals different levels of students' understanding, application, and potential misconceptions. Previous studies have categorized question types into higher and lower orders, finding positive and significant associations between higher-order questions and students' critical thinking ability and their learning outcomes in different learning contexts. However, the diversity of higher-order questions, especially in collaborative learning environments. has left open the question of how they may be different from other types of dialogue that emerge from students' conversations, To address these questions, our study utilized natural language processing techniques to build a model and investigate the characteristics of students' higher-order questions. We interpreted these questions using Bloom's taxonomy, and our results reveal three types of higher-order questions during collaborative problem-solving. Students often use "Why", "How" and "What If' questions to I) understand the reason and thought process behind their partners' actions: 2) explore and analyze the project by pinpointing the problem: and 3) propose and evaluate ideas or alternative solutions. In addition. we found dialogue labeled 'Social'. 'Question - other', 'Directed at Agent', and 'Confusion/Help Seeking' shows similar underlying patterns to higher-order questions, Our findings provide insight into the different scenarios driving students' higher-order questions and inform the design of adaptive systems to deliver personalized feedback based on students' questions.

With the increasing prevalence of large language models (LLMs) such as ChatGPT, there is a growing need to integrate natural language processing (NLP) into K-12 education to better prepare young learners for the future AI landscape. NLP, a sub-field of AI that serves as the foundation of LLMs and many advanced AI applications, holds the potential to enrich learning in core subjects in K-12 classrooms. In this experience report, we present our efforts to integrate NLP into science classrooms with 98 middle school students across two US states, aiming to increase students' experience and engagement with NLP models through textual data analyses and visualizations. We designed learning activities, developed an NLP-based interactive visualization platform, and facilitated classroom learning in close collaboration with middle school science teachers. This experience report aims to contribute to the growing body of work on integrating NLP into K-12 education by providing insights and practical guidelines for practitioners, researchers, and curriculum designers.

As conversational AI apps such as Siri and Alexa become ubiquitous among children, the CS education community has begun leveraging this popularity as a potential opportunity to attract young learners to AI, CS, and STEM learning. However, teaching conversational AI to K-12 learners remains challenging and unexplored due in part to the abstract and complex nature of some conversational AI concepts, such as intents and training phrases. One promising approach to teaching complex topics in engaging ways is through unplugged activities, which have been shown to be highly effective in fostering CS conceptual understanding without using computers. Research efforts are underway toward developing unplugged activities for teaching AI, but few thus far have focused on conversational AI. This experience report describes the design and iterative refinement of a series of novel unplugged activities for a conversational AI summer camp for middle school learners. We discuss learner responses and lessons learned through our implementation of these unplugged activities. Our hope is that these insights support CS education researchers in making conversational AI learning more engaging and accessible to all learners.

As artificial intelligence permeates our lives through various tools and services, there is an increasing need to consider how to teach young learners about AI in a relevant and engaging way. One way to do so is to leverage familiar and pervasive technologies such as conversational AIs. By learning about conversational AIs, learners are introduced to AI concepts such as computers’ perception of natural language, the need for training datasets, and the design of AI-human interactions. In this experience report, we describe a summer camp curriculum designed for middle school learners composed of general AI lessons, unplugged activities, conversational AI lessons, and project activities in which the campers develop their own conversational agents. The results show that this summer camp experience fostered significant increases in learners’ ability beliefs, willingness to share their learning experience, and intent to persist in AI learning. We conclude with a discussion of how conversational AI can be used as an entry point to K-12 AI education.

The ubiquity of AI-based conversational apps such as Siri, Alexa and Google Assistant means more young users are interacting with these apps. The increasing popularity of these conversational applications brings a potential opportunity to attract learners to AI, CS and STEM fields. CS Education researchers need to explore how to leverage this opportunity, in particular to serve learners who are underrepresented in CS and STEM. This experience report describes the design and iterative refinement of a series of two-week summer camps in which 62 predominantly Black students participated in hands-on AI-based learning experiences to design and develop their own conversational AI apps. We discuss the organization of this summer camp experience, including strategies for recruiting from and building trust within the target community, designing professional development for camp facilitators, structuring the camp activities, and encouraging projects that are personally and socially relevant. We share challenges and lessons learned from this AI summer camp in the hopes that they will inform other researchers and practitioners who are interested in designing and deploying similar experiences.

Artificial Intelligence (AI) and Natural Language Processing (NLP) have become increasingly relevant across multiple fields, creating a necessity for young learners to understand these concepts. However, resources enabling learners to apply AI and NLP, particularly in middle school science, remain limited. To address this gap, we present the early development of NLP4Science, an interactive visualization application facilitating the integration of NLP concepts such as sentiment analysis and keyword extraction into middle school science. We adopted an iterative co-design process starting with a professional development workshop with four teachers, followed by a 2-day pilot study with 48 eighth graders, and concluding with a 5-day study involving 50 sixth graders. This poster presents an overview of NLP4Science, highlighting its key features, and sharing insights gained from the iterative design process, demonstrating the potential of NLP4Science to transform AI and NLP learning within middle school science classrooms.

Intelligent virtual agents have tremendous potential for facilitating collaborative learning by modeling and reinforcing desirable collaborative practices. Despite recent work in this area, the extent to which intelligent virtual agents can facilitate improvements in the collaborative behavior of children is largely unknown. This study employed a wizard-of-oz study design and investigated elementary children's collaborative behavior after interacting with virtual agents. These agents model exploratory talk for upper elementary school dyads, such as asking higher-order questions and listening to their partners. The findings uncover associations between elementary learner dyads' positive changes in collaboration after agent interventions, the dyads' affective reactions to interventions, and their attentiveness to the agents. Our results also reveal associations between positive changes in collaboration and the timing of interventions: for example, earlier interventions had a higher occurrence of positive changes, and positive changes in collaboration typically happened within five seconds of interventions. The results suggest ways in which intelligent virtual agents may be used to promote effective collaborative learning practices for children.

Collaborative problem solving has numerous benefits for learners, such as improving higher-level reasoning and developing critical thinking. While learners engage in collaborative activities, they often experience impasse, a potentially brief encounter with differing opinions or insufficient ideas to progress. Impasses provide valuable opportunities for learners to critically discuss the problem and re-evaluate their existing knowledge. Yet, despite the increasing research efforts on developing multimodal modeling techniques to analyze collaborative problem solving, there is limited research on detecting impasse in collaboration. This paper investigates multimodal detection of impasse by analyzing 46 middle school learners’ collaborative dialogue—including speech and facial behaviors—during a coding task. We found that the semantics and speaker information in the linguistic modality, the pitch variation in the audio modality, and the facial muscle movements in the video modality are the most significant unimodal indicators of impasse. We also trained several multimodal models and found that combining indicators from these three modalities provided the best impasse detection performance. To the best of our knowledge, this work is the first to explore multimodal modeling of impasse during the collaborative problem solving process. This line of research contributes to the development of real-time adaptive support for collaboration.

The COVID-19 pandemic has demonstrated that learning remotely is a crucial skill for K-12 students. However, remote instruction and collaboration bring a new set of challenges for these students, especially in the context of pair programming. An important goal for the CS education community is to understand these younger learners' experiences during remote programming activities. This experience report describes a three-day learning experience in which 18 middle school students engaged in remote pair programming activities by modeling scientific processes in a block-based programming language. After three remote pair programming sessions, we conducted individual interviews to understand middle school students' experiences during remote pair programming activities as well as comparing these new experiences to their previous co-located pair programming experiences. The results from these interviews suggest that the majority of the students (72%) enjoyed the remote activities despite many (55%) experiencing some form of technical difficulty. The interviews revealed important opportunities and challenges that being remote brought to pair programming within themes of changes in communication and focus, pair programming dynamics, and available resources. Students also identified issues with remote collaboration such as technical difficulties from software that impaired their ability to work and to communicate. These observations inform new efforts to adapt CS education to the increased demand for remote collaborative work and reveal patterns that may increase success in this new work style.

The COVID-19 pandemic shifted many U.S. schools from in-person to remote instruction. While collaborative CS activities had become increasingly common in classrooms prior to the pandemic, the sudden shift to remote learning presented challenges for both teachers and students in implementing and supporting collaborative learning. Though some research on remote collaborative CS learning has been conducted with adult learners, less has been done with younger learners such as elementary school students. This experience report describes lessons learned from a remote after-school camp with 24 elementary school students who participated in a series of individual and paired learning activities over three weeks. We describe the design of the learning activities, participant recruitment, group formation, and data collection process. We also provide practical implications for implementation such as how to guide facilitators, pair students, and calibrate task difficulty to foster collaboration. This experience report contributes to the understanding of remote CS learning practices, particularly for elementary school students, and we hope it will provoke methodological advancement in this important area.

Discussion forums are invaluable resources when scaling up undergraduate CS courses to larger class sizes. However, passive incorporation of discussion forums is not a silver bullet, as these platforms tend to devolve into places of shallow engagement. To aid our understanding of the factors that influence the nature of these interactions, we collected data from three CS1/CS2 forums. We obtained survey responses from the course instructors and performed a content analysis of the question-response pairs across all the courses. The results suggest that students' help-seeking patterns are influenced by the course curriculum, mode of delivery, and the existence of other help-seeking avenues. The findings also shed light on common strategies used by instructors to incentivize productive student-teaching staff and student-student interactions (e.g., instructing students to describe their debugging questions in detail, asking teaching staff to respond with hints/questions instead of direct answers). This poster presents a series of takeaways that can inform CS educators' choices around discussion forums.

The computer science education community has created many adaptive feedback tools and intelligent tutoring systems to improve students' experience in computing-related courses. However, the extent to which these systems-which we collectively refer to as adaptive pedagogical systems-support equitable outcomes for learners of all genders and racial identities is not known. We conducted a systematic literature review of SIGCSE, ITiCSE, and ICER publications on adaptive pedagogical systems in computing courses from the last five years. The results reveal that not only is there little to no data on the effectiveness of adaptive pedagogical systems for CS education by gender or race, the vast majority of published papers reporting on these systems do not even include the demographics of their users. Based on these findings, this position paper makes a call to action: we must include the voices of historically marginalized students in the design and evaluation of our software, lest we continue to perpetuate that marginalization. We highlight key ideas that every CS education researcher should consider when designing and evaluating technologies to support learners. We argue that this community must hold ourselves and each other accountable to create technologies that support learners equitably.

More young people are interacting with smart conversational agents such as Alexa and Google Assistant. These platforms are extensible, providing, in principle, a compelling opportunity for young users to create and tinker with their own conversational agents. However, to date the interfaces for conversational app development are adult-focused. This paper presents the early design process for AMBY (AI Made by You), which we are building to empower young learners to create their own conversational agents. We first conducted a contextual inquiry with 14 middle school students (aged 11-13) in an AI summer camp, followed by two other usability studies. The system design has been refined after each study. Key features of AMBY include a visual dialogue management panel, testing panel with a diverse avatar, and a voice input modality. AMBY is designed to serve as a pedagogically-robust resource for K-12 AI education and as an engaging and creative way for middle schoolers to explore AI.

Co-creative proccesses between people can be characterized by rich dialogue that carries each person's ideas into the collaborative space. When people co-create an artifact that is both technical and aesthetic, their dialogue reflects the interplay between these two dimensions. However, the dialogue mechanisms that express this interplay and the extent to which they are related to outcomes, such as peer satisfaction, are not well understood. This paper reports on a study of 68 high school learner dyads' textual dialogues as they create music by writing code together in a digital learning environment for musical remixing. We report on a novel dialogue taxonomy built to capture the technical and aesthetic dimensions of learners' collaborative dialogues. We identified dialogue act n-grams (sequences of length 1, 2, or 3) that are present within the corpus and discovered five significant n-gram predictors for whether a learner felt satisfied with their partner during the collaboration. The learner was more likely to report higher satisfaction with their partner when the learner frequently acknowledges their partner, exchanges positive feedback with their partner, and their partner proposes an idea and elaborates on the idea. In contrast, the learner is more likely to report lower satisfaction with their partner when the learner frequently accepts back-to-back proposals from their partner and when the partner responds to the learner's statements with positive feedback. This work advances understanding of collaborative dialogue within co-creative domains and suggests dialogue strategies that may be helpful to foster co-creativity as learners collaborate to produce a creative artifact. The findings also suggest important areas of focus for intelligent or adaptive systems that aim to support learners during the co-creative process.

Use-Modify-Create (UMC) has gained recognition as a viable scaffolding approach for student programming activities, but little is known about how UMC could support CS learning in game-based learning environments. We designed and developed a game to teach middle grade students (ages 11-13) CS through block-based programming challenges. The game integrates a UMC pedagogical framework to promote successful student outcomes for a wide variety of student abilities, including those without prior programming experience. Utilizing a mixed-methods research design, we investigated how the game influenced student learning of CS concepts and the role of UMC on the problem-solving strategies students applied to complete the game. In particular, we were interested in how prior experience would moderate these outcomes. Results from a multilevel model of students' pre-and post-assessment scores (N = 77) on a CS concepts assessment indicated that all students, regardless of prior programming experience, showed significant learning gains from pre to post after playing the game. Qualitative results revealed that the UMC scaffolding progression provided students, particularly those with little to no prior programming experience, with the foundational knowledge needed to progress through the game levels and challenges. Specifically, we found that the Use phases of the game reduced novice students' cognitive load and facilitated the necessary CS conceptual understanding to solve the open-ended programming tasks encountered in the game's Modify and Create phases. Our findings demonstrate the efficacy of UMC to support the learning of novice programmers in a game-based learning environment while not to the detriment of those more experienced.

Pair programming is a popular strategy in computer science education to teach programming to novices. In this study, we examined the effect of three different pair programming conditions on upper elementary school students' CS conceptual understanding. The three conditions were one-computer with roles (1C with roles), two computers without roles (2C no roles), and two computers with roles (2C with roles). These students were engaged in four days of computer programming activities and took the CS concept assessment, CS attitudes, and collaboration perceptions before and after the activities. We used the validated E-CSCA (Elementary Computer Science Concepts Assessment) to measure elementary students' understanding of CS concepts. We tested the relationship of different pair programming conditions on the students' CS conceptual understanding and found that different conditions impacted students' CS conceptual understanding, wherein students in 2C roles demonstrated better CS learning than the other two conditions. The results also showed no changes in students' CS attitudes and perceptions of collaboration before and after the activities. Furthermore, the results indicated no significant impact of these attitudinal factors on students' learning CS concepts in pair programming settings. Our study highlights the importance of the roles and number of computers in pair programming settings, especially for elementary students.

The computer science education community strives to improve equity and representation within the field, yet the proportion of women earning CS bachelor's degrees in countries such as the US remains low. In addition to recruitment and retention initiatives that support women, we need to better understand women's experiences within CS. This paper makes a novel contribution toward this effort by examining women's self-reported stress during remote collaborative programming with a peer. Women reported significantly more stress than men, so we analyzed the women's collaborative dialogues and identified the most common dialogue acts and sequences of dialogue acts. We used these dialogue acts to predict women's stress and found six significant patterns of dialogue. Women reported less stress with higher frequencies of offering suggestions, having their partner provide explanations, and having their own rapport-building messages reciprocated by their partner. In contrast, women reported more stress with higher frequencies of their own explanations, having their partner answer their questions, and having their partner send a rapport-building message that they reciprocated. Understanding the nuances of these experiences allows us to make better predictions of when women might be feeling stressed and what we might be able to do to relieve these feelings. Improving women's CS experiences holds the potential to, in turn, improve gender equity within CS.

In successful collaborative paradigms such as pair programming, students engage in productive dialogue and work to resolve conflicts as they arise. However, little is known about how elementary students engage in collaborative dialogue for computer science learning. Early findings indicate that these younger students may struggle to manage conflicts that arise during pair programming. To investigate collaborative dialogue that elementary learners use and the conflicts that they encounter, we analyzed videos of twelve pairs of fifth grade students completing pair programming activities. We developed a novel annotation scheme with a focus on collaborative dialogue and conflicts. We found that student pairs used best-practice dialogue moves such as self-explanation, question generation, uptake, and praise in less than 23% of their dialogue. High-conflict pairs antagonized their partner, whereas this behavior was not observed with low-conflict pairs. We also observed more praise (e.g., "We did it!") and uptake (e.g., "Yeah and...") in low-conflict pairs than high-conflict pairs. All pairs exhibited some conflicts about the task, but high-conflict pairs also engaged in conflicts about control of the computer and their partner's contributions. The results presented here provide insights into the collaborative process of young learners in CS problem solving, and also hold implications for educators as we move toward building learning environments that support students in this context.

Block-based programming environments are widely used by novices who are learning computer science. However, even in block-based coding environments that have been carefully developed to serve novices, students frequently struggle and require additional support. A promising avenue to provide this support is the use of intelligent tutoring systems, which offer adaptive hints to assist learners. In order to provide students with the adaptive hints they need, we must investigate their help-seeking behaviors and identify patterns surrounding their need for support. In this experience report, we examine data collected from 174 college students in an introductory engineering course, who used an intelligent block-based coding environment to learn computer science. These students made more than 1,000 hint requests, which we represent in two-dimensional space along axes of elapsed time and code completeness. Analysis revealed five major clusters of hint requests, which we further characterized through qualitative examination of the coding trajectories that preceded each hint request. We also analyzed how students' incoming knowledge and perceived computer skill were related to their help-seeking behaviors. Students with higher incoming knowledge requested hints when their code was more complete than students with lower incoming knowledge. Students with high perceived computer skill asked for hints when their code was less complete than those with low perceived computer skill. The results presented here provide insight into student help-seeking behavior in computer science education, informing CS educators and system designers on how best to develop support strategies.

The computer science education community has long strived to create more equitable opportunities for students, such as initiatives to foster inclusion of women and other people from historically marginalized groups in CS. Despite these efforts, the gender gap has persisted, with less than a quarter of CS Bachelor's degrees awarded to women in the United States in 2019. As a community, we must strive to improve women's experiences in CS. This paper describes work conducted at a large research university which has traditionally offered CS1 through lecture sections ranging in size from 400-650 students. In Fall 2019, we offered an alternative small all-women's class (35 students) in addition to the traditional lecture class (601 students; 149 women). Both classes covered the same CS concepts but were led by different instructors. Students reported on their experience through a survey administered at the end of the semester. Students in the all-women's class reported significantly greater social connections and comfort collaborating with their peers compared to women in the traditional class. They also reported significantly greater feelings of support within their class, more confidence in their CS knowledge, and a more welcoming classroom environment compared to women in the traditional class. Additionally, the drop rate for students in the all-women's class was significantly lower (5.7%) than the drop rate for women in the traditional class (24.8%). In light of these positive results, we provide actionable insights for CS educators and discuss how to better support women in their CS endeavors.

Block-based programming environments are widely used in computer science education. However, these environments pose significant challenges for student modeling. Given a series of problem-solving actions taken by students in block-based programming environments, student models need to accurately infer problem-solving students’ programming abilities in real time to enable adaptive feedback and hints that are tailored to students’ abilities. While student models for block-based programming offer the potential to support student-adaptivity, creating student models for these environments is challenging because students can develop a broad range of solutions to a given programming activity. To address these challenges, we introduce a progression trajectory-based student modeling framework for modeling novice student block-based programming across multiple learning activities. Student trajectories utilize a time series representation that employs code analysis to incrementally compare student programs to expert solutions as students undertake block-based programming activities. This paper reports on a study in which progression trajectories were collected from more than 100 undergraduate students engaging in a series of block-based programming activities in an introductory computer science course. Using progression trajectory-based student modeling, we identified three distinct trajectory classes: Early Quitting, High Persistence, and Efficient Completion. Analysis of these trajectories revealed that they exhibit significantly different characteristics with respect to students’ actions and can be used to accurately predict students’ programming behaviors on future programming activities compared to competing baseline models. The findings suggest that progression trajectory-based student models can accurately model students’ block-based programming problem solving and hold potential for informing adaptive support in block-based programming environments.

Recent years have seen increasing awareness of the need for all students in primary and secondary education to learn computer science (CS) concepts and skills. Educational games hold significant potential to serve as a platform for CS education because they integrate engaging problem solving with effective pedagogical strategies. This potential is especially high for narrative-centered educational games that embed learning activities within rich interactive stories. In this paper, we present an educational game featuring block-based programming challenges contextualized within an engaging narrative, designed to promote CS learning for middle school students (ages 11 to 13). In the game, students undertake problem-solving challenges that are aligned with the K-12 Computer Science Framework. Results from a classroom implementation of the game with middle grade students suggest that their perceived game control ratings are positively correlated with their progress in the game, which suggests the need for adaptively supporting students' game-based learning activities. Building on these findings, we discuss design implications for creating student-adaptive CS learning experiences in educational games that incorporate block-based programming enriched narrative-centered gameplay.

Collaborative learning has demonstrated benefits for girls in computer science [7] and this may be a way to help address the gender gap in CS. Research indicates that while collaborating, boys often express more individualistic ideas whereas girls tend to be more supportive [1]. It is important for students to regulate their learning in collaborative learning environments because they need to negotiate group goals and diverse approaches to the task [3], and the use of open-ended tasks with multiple solution paths are common [4]. There is minimal research in CS education on regulation of learning (e.g., [5,6,]). Co-regulated learning is the process in which an other helps regulate the learning of a student [2] self such as by asking questions that prompt the student to monitor and evaluate (e.g., "What do you already know about 'if' blocks that would help here?''). In this way, thinking and reasoning through the problem is shared by the group members [2] self.

Computer science (CS) is widely recognized as a field with a significant gender gap despite the growing prevalence of computing. Several factors including CS attitudes, exposure to CS, experience with computer programming, and confidence in using computers are understood to be correlated with the low participation of women in CS. These factors also play an important role in students' interest in CS careers and are particularly crucial during secondary school. However, there is a dearth of research that examines differences in how these factors are inter-correlated for younger students (ages 11-13). The purpose of this study was to generate and test a statistical model that demonstrates the inter-correlation amongst these factors with respect to gender. A total of 260 middle school students participated in this study. Four instruments measuring students' CS attitudes, confidence in using computers, CS conceptual understanding, and prior experience with CS-related activities were used. Structural equation modeling was utilized to test the hypothesized model. The findings showed that previous participation in CS-related activities had a significant direct effect on CS attitudes and confidence in using computers, but the effect on students' CS conceptual understanding was indirect. We also found that in a female specific model, previous participation had a significantly stronger direct effect on CS attitudes compared to its effect in a male specific model. The importance of providing more CS-related experience, especially to female students, as well as suggestions on activities that promote gender equity in the field are discussed.

Learning computer science presents many challenges to students, and providing resources for meaningful practice is recognized as a way to support rigorous learning and diverse student participation. At the same time, mobile phones are increasingly ubiquitous, creating an underutilized opportunity for practice outside of traditional methods. This experience report presents a user-centered approach to designing a practice app for introductory Java. We investigated user preferences through a series of small studies, first conducting think-aloud sessions and focus groups, and finally conducting a usability study comparing two prototype versions. The initial studies suggested how to leverage the affordances of small screens, ruling out free-response practice problems in favor of either fill-in-the-blank (FB) or multiple-choice (MC) questions. The comparison study revealed statistically significant differences in students' survey responses: (1) usability scores were significantly higher for the MC version than the FB version; (2) students reported significantly greater satisfaction and desire to learn for the MC version; and (3) students reported enjoying and being more comfortable with the MC version compared to the FB version. We contextualize this observation within related research on question formats. Takeaways from this experience report can provide guidance on designing mobile applications that give students opportunities for meaningful practice.

The lack of effective guidelines for assessing students' computer science (CS) competencies is creating significant demand by K-12 teachers for CS assessments to evaluate students' learning. We propose a conceptual assessment framework that guides teachers through designing appropriate assessments for computer science (CS) activities in their classrooms. The framework addresses the critical problem of incorporating CS into K-12 curricula without corresponding assessments. We illustrate its use with the design of a rubric for a bubble sort algorithm situated in a game-based learning environment for middle-grade students. We also apply a preliminary and a revised version of this assessment on two datasets collected from students' interactions with the learning environment. We found consistency among results identified through applying the preliminary and the revised rubric. The results reveal distinctive patterns in students' approaches to CS problem solving and coherency with respect to different aspects of the rubric.*

In recent years, there has been a dramatic increase in teaching CS in the context of other disciplines such as science. However, learning CS in an interdisciplinary context may be particularly challenging for students. An important goal for CS education researchers is to develop a deep understanding of the student experience when integrating CS into science classrooms in K-12. This paper presents the results of a mixed-methods study in which 75 middle school students engaged in a series of computationally rich science activities by creating simulations and models in a block-based programming language. After two semesters, students reported their experiences on in-class computer science activities through reflection essays. The quantitative results show that both experienced and novice students increased their CS knowledge significantly after several weeks, and a majority of students (72%) had positive sentiment toward the integration of CS into their science class. Deeper qualitative analysis of students' reflections revealed positive themes centered around the visualization and gamification of science concepts, the hands-on nature of the coding activities, and showing science from a different angle. On the other hand, students expressed negative sentiments on weaknesses in the activity design, lack of CS/science background/interest, and failing to make connections between CS and science concepts. These findings inform efforts to infuse CS education into different disciplines and reveal patterns that may foster success of K-12 classroom implementations.

As efforts to integrate computer science into K-8 teaching in the US are dramatically rising, professional development workshops for teachers are becoming widespread. An open challenge for the CS education community is to understand teachers' needs and develop empirically grounded best practices for professional development. This experience report describes a five-day professional development workshop in which 22 third through eighth grade teachers learned about fundamental CS concepts, practiced coding, and created lesson plans for integrating what they learned into their classroom. We describe the professional development workshop including its modules and sequencing, and present teachers' perception of CS and how to integrate it into their classrooms. Teachers achieved significant gains in technical knowledge and improvements in attitude toward computer science. In initial focus groups, teachers reported that limited exposure to CS, time constraints, and lack of understanding of CS are barriers to integrating it into their classrooms. After the workshop, focus group feedback indicated that the workshop provided teachers a clearer sense of the potential of CS to enhance their classroom plans. Teachers noted that they felt able to use CS to help students learn critical thinking, prepare them for their futures, and address their individual needs. The results of this experience can inform future workshops that address the needs of teachers and students.

Recent years have seen an increasing interest in identifying common student misconceptions during introductory programming. In a parallel development, block-based programming environments for novice programmers have grown in popularity, especially in introductory courses. While these environments eliminate many syntax-related errors faced by novice programmers, there has been limited work that investigates the types of misconceptions students might exhibit in these environments. Developing a better understanding of these misconceptions will enable these programming environments and instructors to more effectively tailor feedback to students, such as prompts and hints, when they face challenges. In this paper, we present results from a cluster analysis of student programs from interactions with programming activities in a block-based programming environment for introductory computer science education. Using the interaction data from students' programming activities, we identify three families of student misconceptions and discuss their implications for refinement of the activities as well as design of future activities. We then examine the value of block counts, block sequence counts, and system interaction counts as programming features for clustering block-based programs. These clusters can help researchers identify which students would benefit from feedback or interventions and what kind of feedback provides the most benefit to that particular student.

Debugging is an important skill for novice programmers to master, but many students struggle to learn how to debug due in part to difficulty with program syntax. Block-based environments provide an alternative to traditional textual programming that reduces syntax errors, and recently hybrid block-based/textual environments have become more common. This poster presents preliminary research to understand how novice debugging strategies differ between block- based and hybrid environments. We assigned seven participants to debug four programs within one of the two environments and conducted interviews about their debugging approaches. Thematic analysis of interview responses suggest that students adjusted their strategies based on their prior experience with textual environments. By understanding novice programmers' strategies in these environments, the field can move toward more effectively support- ing productive strategies.

As computer science education opportunities for elementary students (grades K-5) are expanding, there is growing interest in using pair programming with these students. However, previous research findings do not fully support its use with younger learners, and some researchers have begun to examine whether introducing a second computer with a shared coding workspace can provide important benefits. This experience report describes a series of classroom activities in the 4th and 5th grades (ages 9-11 years old) with two different pair programming configurations: one-computer pair programming, in which both students share a keyboard, mouse, and monitor; and two-computer pair programming, in which each student has a separate computer but coding workspaces are synchronized over the web. In both cases the students sat next to each other and engaged in face-to-face conversation. We found that students largely preferred two-computer pair programming over one-computer pair programming. We conducted focus groups and transcribed collaborative dialogues to gain more insight into this preference. We learned that students felt more independence in two-computer pair programming, although they struggled with coordinating their edits with their partner. In one-computer pair programming, students reported not wanting to wait for their turn to drive, but feeling as though they communicated more with their partner. Both configurations can be productive for students, but the tradeoffs described in this experience report are important for CS educators and researchers to consider when determining which collaborative configuration to use in each K-5 classroom context.

Recent years have seen a growing interest in block-based programming environments for computer science education. Although block-based programming offers a gentle introduction to coding for novice programmers, introductory computer science still presents significant challenges, so there is a great need for block-based programming environments to provide students with adaptive support. Predictive student modeling holds significant potential for adaptive support in block-based programming environments because it can identify early on when a student is struggling. However, predictive student models often make a number of simplifying assumptions, such as assuming a normal response distribution or homogeneous student characteristics, which can limit the predictive performance of models. These assumptions, when invalid, can significantly reduce the predictive accuracy of student models.

The language and concepts used by curriculum designers are not always interpreted by children as designers intended. This can be problematic when researchers use self-reported survey instruments in concert with curricula, which often rely on the implicit belief that students’ understanding aligns with their own. We report on our refinement of a validated survey to measure upper elementary students’ attitudes and perspectives about computer science (CS), using an iterative, design-based research approach informed by educational and psychological cognitive interview processes. We interviewed six groups of students over three iterations of the instrument on their understanding of CS concepts and attitudes toward coding. Our findings indicated that students could not explain the terms computer programs nor computer science as expected. Furthermore, they struggled to understand how coding may support their learning in other domains. These results may guide the development of appropriate CS-related survey instruments and curricular materials for K–6 students.

In recent years, remote collaboration has become increasingly common both in the workplace and in the classroom. It is imperative that we understand and support remote collaborative problem solving, particularly understanding the experiences of people from historically marginalized groups whose intellectual contributions are essential for addressing the pressing needs society faces. This paper reports on a study in which 58 introductory computer science students constructed code remotely with a partner following either predefined structured roles (driver and navigator in pair programming) or without predefined structured roles. Between the structured-role and unstructured-role conditions, participants? normalized learning gain, Intrinsic Motivation Inventory scores, and system usability scores were not significantly different. However, regardless of the collaboration condition, women reported significantly higher levels of stress, lower levels of perceived competence, and less perceived choice compared to men. Because computer science is a context in which women have been historically marginalized, we next examined the relationship between student gender and collaborative dialogues by extracting lexical and sentiment features from the textual messages partners exchanged. Results reveal that dialogue features, such as number of utterances, utterance length, and partner sentiment, significantly correlated with women's reports of stress, perceived competence, or perceived choice. These findings provide insight on women's experiences in remote programming, suggest that dialogue features can predict their collaborative experiences, and hold implications for designing systems that help provide collaborative experiences in which everyone can thrive.

Participatory design practices create informed designs by bringing stakeholders into the design process early and often. This approach is a powerful tool, especially when the designer and the intended user are very different. This paper reports on work in which researchers co-design pedagogical agents to support collaborative computer science learning with elementary school students using an iterative drawing methodology. In the open drawing phase, students drew what they believe good collaboration looked like. Next, researchers analyzed those drawings under the requirements of the broader project and created a drawing scaffold (similar to a coloring book page). In the scaffolded drawing phase, students ideated within the more focused context. This process resulted in actionable design guidelines for the appearance of pedagogical agents.

Educational games offer significant potential for supporting personalized learning in engaging virtual worlds. However, many educational games do not provide adaptive gameplay to meet the needs of individual students. To address this issue, educational games should include game levels that can self-adjust to the specific needs of individual students. However, creating a large number of adaptable game levels requires considerable effort by game developers. A promising solution to this problem is to leverage procedural content generation to automatically generate levels for educational games that incorporate the desired learning objectives. In this paper, we propose a multistep deep convolutional generative adversarial network for generating new levels within a game for middle school computer science education. The model operates in two phases: (1) train a generator with a small set of human-authored example levels and generate a much larger set of synthetic levels to augment the training data for a second generator, and (2) train a second generator using the augmented training data and use it to generate novel educational game levels with enhanced solvability. We evaluate the performance of the model by comparing the novelty and solvability of generated levels between the two generators. Results suggest that the proposed multistep model significantly enhances the solvability of the generated levels with only minor degradation in the novelty of the generated levels.

With national K-12 education initiatives such as "CSForAll," block-based programming environments have emerged as widely used tools for teaching novice programming. A key challenge presented by block-based programming environments is assessing students' computational thinking (CT) and programming competencies. Developing assessment methods that can evaluate students' use of CT practices such as testing and refining, and developing and using appropriate algorithms, can help teachers evaluate students learning and provide appropriate scaffolding. In this work, we utilize an evidence-centered assessment design approach to devise a three-dimensional assessment to evaluate students' CT competencies based on evidence extracted from their programming trajectories in a block-based programming environment. In this assessment, the first dimension assesses students' knowledge of essential CT concepts, the second dimension assesses students' dynamic testing and refining strategies, and the third dimension assesses their overall problem-solving efficiency. We apply the assessment framework to data collected from students' interactions with a game-based learning environment designed to develop middle-grade students' CT competencies and programming skills. The results demonstrate that students' knowledge of basic CT constructs, such as appropriate use and combination of control structures, serves as the foundation for designing and implementing effective algorithms. Further, we assessed students testing and refining strategies over the three dimensions of novelty, positivity, and scale. The results demonstrate that students with higher algorithmic capabilities tend to make more novel, positive, and small-scale changes. The results reveal distinctive patterns in students' approaches to computational thinking problem solving and make a step toward identifying and assessing productive computational thinking practices.

As efforts to integrate computer science into K-8 teaching in the US are dramatically rising, professional development workshops for teachers are becoming widespread. An open challenge for the CS education research community is to understand teachers' needs and develop empirically grounded best practices for professional development. This poster displays our findings from a five-day professional development workshop in which 22 third through eighth grade teachers from various subjects learned about fundamental CS concepts, practiced coding, and created lesson plans for integrating what they learned into their classroom. We present quantitative and qualitative outcomes, with emphasis on teachers' perception of CS and how to integrate CS into their classrooms. The quantitative results, based on pre- and post-tests, reveal that the teachers gained significant technical knowledge and improved their attitudes toward CS. The qualitative results, based on the focus groups conducted before and after the workshop, indicated that teachers' limited exposure to CS can lead to misconceptions and some negative attitudes. Moreover, limitations such as time constraints and lack of understanding of CS are barriers to integrating CS into classrooms. On the other hand, the workshop provided teachers a clearer sense of the potential of CS to enhance their classroom plans. Teachers noted that they felt able to use CS to help students learn critical thinking, prepare them for their futures, and address their individual needs. These results can inform future workshops that address the needs of teachers and students.

The computer science education community has made great strides in promoting diversity and inclusion in computing fields and bringing K-12 CS learning opportunities to broader groups of learners. However, one area that has not been investigated fully is how assessments can influence a learner's confidence and attitudes towards CS. Stereotype threat and test modality have been shown to affect the performance of CS test takers. This experience report examines how novice CS learners respond to a CS assessment by investigating an increasingly important group of novice CS learners: K-8 classroom teachers. We conducted focus groups with elementary and middle school teachers as part of a week-long CS professional development workshop. The focus groups were held after teachers completed pre- and post-assessments. The assessment instrument featured multiple-choice and short-answer questions with block-based programming snippets. Many teachers reported a positive disposition towards learning CS after completing the pre-assessment, which they attributed to having a growth mindset. Themes related to their confidence involved the difficulty and format of the assessment, with comments about difficulty reducing after the post-assessment. When asked about their thoughts on the assessment from the perspective of their students, they provided suggestions with particular attention to its format. These findings provide insight for CS assessment design and implementation, as well as support further research on the impact of assessments on CS learners.

Extensive prior research suggests that pair programming holds many benefits for novices. Pair programming has been well studied at the undergraduate level, and recently, the CS education research community has started to realize that younger learners may also benefit from pair programming. However, an important factor in pair programming success for young learners is the ability to resolve conflicts during the process. Little is known about what types of conflicts occur while elementary students pair program or how those conflicts are, or are not, resolved. To investigate this phenomenon, we analyzed the videos of six pairs of students completing a programming activity. We found that conflicts evolve in four general stages, which may not all be present in each conflict: initiation, escalation, de-escalation, and conclusion. Some conflicts are resolved when the students come to an agreement, others end passively. The analysis revealed that the pairs' conflicts began around disagreements about code, who should have control of the keyboard and mouse, and other interpersonal events. This research indicates that conflicts are a significant concern for young students, and supporting young learners in developing improved collaboration skills is a key direction for CS education research.

Demand for K-6 computer science (CS) curricula is growing considerably. Many of the existing curricula have been developed by domain experts who are comfortable with specific and technical terminology, which they expect students to master. However, children are not always comfortable with these terms nor do they understand general concepts like 'coding' in the way that the curriculum designers intend. This is a problem because many researchers use self-report and attitudinal survey instruments with the implicit belief that the students' understanding of the terms and concepts resemble their own. This mismatch may invalidate results. For this project, we report on our modification of a validated survey to measure upper elementary students' attitudes about and perspectives on CS by attempting to understand the appropriate language to use when querying children about these topics. We use an iterative, design-based research approach that is informed by educational and psychological cognitive interview processes. We interviewed two groups (N=64) of upper elementary students on their understanding of computer science concepts and attitudes toward coding. Our findings indicate that 4th and 5th grade students could not explain the terms computer programs nor computer science as we had expected and that they struggled to understand how coding may connect with or support their learning in other domains. These results will help to guide the development of appropriate survey instruments and course materials for K-6 students, which both match their use of broad domain concepts and therefore inform their understanding and improve their outcomes.

The recognition of middle grades as a critical juncture in CS education has led to the widespread development of CS curricula and integration efforts. The goal of many of these interventions is to develop a set of underlying abilities that has been termed computational thinking (CT). This goal presents a key challenge for assessing student learning: we must identify assessment items associated with an emergent understanding of key cognitive abilities underlying CT that avoid specialized knowledge of specific programming languages. In this work we explore the psychometric properties of assessment items appropriate for use with middle grades (US grades 6-8; ages 11-13) students. We also investigate whether these items measure a single ability dimension. Finally, we strive to recommend a "lean" set of items that can be completed in a single 50-minute class period and have high face validity. The paper makes the following contributions: 1) adds to the literature related to the emerging construct of CT, and its relationship to the existing CTt and Bebras instruments, and 2) offers a research-based CT assessment instrument for use by both researchers and educators in the field.

Women continue to be underrepresented in computer science. Previous research has identified factors that contribute to women's decisions to pursue computing-related majors, but in order to truly address the problem of underrepresentation, we need to develop a deeper understanding of women's experiences within computer science courses. Pair programming is demonstrably beneficial in many ways, and we hypothesize that there are gender differences in student perceptions of this widely used collaboration framework. To explore these differences and move toward a thorough understanding of students' experiences, this paper investigates students' written responses about their experiences with pair programming in a university-level introductory computer science course. Using thematic analysis, we identified overarching themes and distinguished between what men and women reported. Both women and men wrote about their overwhelmingly positive perceptions of pair programming. Women often mentioned that pair programming helps with engagement, feeling less frustrated, building confidence, and making friends. Women also noted that it is easier to learn from peers. These findings shed light on how pair programming may lower barriers to women's participation and retention in computing and inform ongoing efforts to create more inclusive spaces in computing education.

The K-12 Computer Science Framework has established that students should be learning about the computer science concept of variables as early as middle school, although the field has not yet determined how this and other related concepts should be introduced. Secondary school computer science curricula such as Exploring CS and AP CS Principles often teach the concept of variables in the context of algebra, which most students have already encountered in their mathematics courses. However, when strategizing how to introduce the concept at the middle school level, we confront the reality that many middle schoolers have not yet learned algebra. With that challenge in mind, this position paper makes a case for introducing the concept of variables in the context of middle school science. In addition to an analysis of existing curricula, the paper includes discussion of a day-long pilot study and the consequent teacher feedback that further supports the approach. The CS For All initiative has increased interest in bringing computer science to middle school classrooms; this paper makes an argument for doing so in a way that can benefit students' learning of both computer science and core science content.

Collaborative learning frameworks such as pair programming have been shown to be highly effective for computer science learning. Skeptics of this approach often refer to the risk of one student relying on a stronger partner to solve the problem. Lending weight to this skepticism, many theories emphasize the importance of learner autonomy. Therefore, it is reasonable to hypothesize that a hybrid pair programming paradigm-one in which partners work together side-by-side at two separate computers and produce their own versions of the code-may be even more effective than traditional pair programming. To investigate this hypothesis, we conducted a study in which 200 introductory programming students were paired and then placed in either a pair-programming condition (two students at one computer) or a hybrid condition (two students at two computers). The results show that traditional pair programming fostered comparable learning gains as measured on an individual post-test, and significantly higher student satisfaction, than the hybrid approach. These findings highlight the importance of not just collaborating, but working together on shared code, for novice computer science learners.

Pair programming is a successful approach for improving student performance, retention, and motivation toward computer science. However, not all students benefit equally from this approach. An open challenge for researchers is to develop a deep understanding of the student experience in pair programming, particularly for novices. This paper reports on a study of the cognitive, affective, and social experiences of students in an introductory programming course in which pair programming was utilized throughout the term. Students reported their experience through reflection essays written at the end of the semester. We analyzed 137 student reflection papers in a mixed-methods study. The quantitative results show that overall, students have a positive attitude toward pair programming. Looking more deeply at the reflection essays, thematic analysis revealed themes centered around cognitive, affective, and social dimensions. In the cognitive dimension, students expressed the importance of exposure to different ideas and developing deeper understanding. Affectively, students reported that working with a partner reduced their frustration and increased their confidence. Students also pointed out the social benefits of forming friendships and helpful connections. These results highlight the powerful benefits of pair programming and point to ways in which this collaborative approach could be adapted to better meet student needs.

Collaboration plays an essential role in computer science. While there is growing recognition that learners of all ages can benefit from collaborative learning, little is known about how elementary-age children engage in collaborative problem solving in computer science. This paper reports on the analysis of a dataset of elementary students collaborating on a programming project. We found that children tend to make several different types of suggestions. In turn, their partners address those suggestions in different ways such as by implementing them directly in code or by replying through dialogue. We observe that students regularly accept or reject suggestions without explanation or explicit acknowledgement and that it is often unclear whether they understand the substance of the suggestion. These behaviors may inhibit the development of a shared understanding between the partners and limit the value of the collaborative process. These results can inform instructional practice and the development of new adaptive tools that facilitate productive collaborative problem solving in computer science.

Pair programming is a collaboration paradigm that has been increasingly adopted in computer science education. Research has established that pair programming can hold benefits for students' learning and attitudes, but comparatively little is known about the ways in which the collaborative process benefits students' CS learning. This paper examines the collaboration process, comparing important outcomes with how students' dialogue and problem-solving approaches unfolded. The results show that the collaboration is more effective when both partners make substantive dialogue contributions, express uncertainty, and resolve it. In particular, driver dialogue expressivity is associated with improved outcomes. The findings provide insight into the ways in which pair programming dialogue benefits student learning during CS problem solving.

Increased enrollments in computer science programs presents a new challenge of quickly accommodating higher enrollment in computer science introductory courses. Because peer teaching scales with enrollment size, it is a promising solution for supporting computer science students in this setting. However, pedagogical and logistical challenges can arise when implementing a large peer teaching program. To study these challenges, we developed a transparent online tool, My Digital Hand, for tracking one-to-one peer teaching interactions. We deployed the tool across three universities in large CS2 computer science courses. The data gathered confirms the pedagogical and logistical challenges that exist at scale and gives insight into ways we might address them. Using this information, we developed the second iteration of My Digital Hand to better support peer teaching. This paper presents the modified tool for use by the computer science education community.

Online discussion forums are widely used and hold great promise for supporting students in learning computer science. Understanding how we can best support students in learning computer science through online discussion forums is an important open question for the CS Ed community. This paper analyzes discussion forum posts from 395 students enrolled in CS2 across two different universities. The results demonstrate that students use the discussion forums often for logistical and relatively shallow questions. However, the largest portion of questions reflect some level of constructive problem-solving activity, and are positively correlated with course grades. Questions that neither describe students' reasoning nor their attempts to solve the problem constitute the smallest percentage of questions, but these questions may be particularly important to attend to because of their relationship to students' prior experience.

In recent years, block-based programming languages have been employed as learning tools to help students starting out with programming. How we design the layout of the available blocks likely impacts the success of the student. In this study, we compare student performance in three conditions consisting of different layouts of block categories in a block-based language: a grouping based on computer science (CS) concepts, a grouping based on block functionality, and one with no groupings. We measured task completion time, quality of the final code product, and perceived system usability. We found that although time and quality did not differ across conditions, the students in the functionality condition reported higher usability scores than the students in the CS concepts condition. These results can inform how we design block-based interfaces to improve learner satisfaction without affecting their performance.

The important goal of broadening participation in computing has inspired many successful outreach initiatives. Yet many of these initiatives, such as out-of-school activities or innovative new computer science courses for secondary school students, may disproportionately attract students who already have prior interest and experience in computing. How, then, do we engage the silent majority of students who do not self-select computer science? This paper examines this question in the context of ENGAGE, an in-school outreach initiative for middle school students. ENGAGE's learning activities center on a game-based learning environment for computer science. Results reveal that the initiative improved the computer science attitudes of students who were not already predisposed to study computer science, in a way that a corresponding after-school program could not. The results illustrate how an in-school initiative can empower young students who might not otherwise consider studying computer science.

Elementary computer science has gained increasing attention within the computer science education research community. We have only recently begun to explore the many unanswered questions about how young students learn computer science, how they interact with each other, and how their skill levels and backgrounds vary. One set of unanswered questions focuses on gender equality for young computer science learners. This paper examines how the gender composition of collaborative groups in elementary computer science relates to student achievement. We report on data collected from an in-school 5th grade computer science elective offered over four quarters in 2014-2015. We found a significant difference in the quality of artifacts produced by learner groups depending upon their gender composition, with groups of all female students performing significantly lower than other groups. Our analyses suggest important factors that are influential as these learners begin to solve computer science problems. This new evidence of gender disparities in computer science achievement as young as ten years of age highlights the importance of future study of these factors in order to provide effective, equitable computer science education to learners of all ages.

Understanding situated dialogue requires identifying referents in the environment to which the dialogue participants refer.This reference resolution problem, often in a complex environment with high ambiguity, is very challenging.We propose an approach that addresses those challenges by combining learned semantic structure of referring expressions with dialogue history into a ranking-based model.We evaluate the new technique on a corpus of human-human tutorial dialogues for computer programming.The experimental results show a substantial performance improvement over two recent state-of-the-art approaches.The proposed work makes a stride toward automated dialogue in complex problem-solving environments.

Affective support is crucial during learning, with recent evidence suggesting it is particularly important for female students. Facial expression is a rich channel for affect detection, but a key open question is how facial displays of affect differ by gender during learning. This paper presents an analysis suggesting that facial expressions for women and men differ systematically during learning. Using facial video automatically tagged with facial action units, we find that despite no differences between genders in incoming knowledge, self-efficacy, or personality profile, women displayed one lower facial action unit significantly more than men, while men displayed brow lowering and lip fidgeting more than women. However, numerous facial actions including brow raising and nose wrinkling were strongly correlated with learning in women, whereas only one facial action unit, eyelid raiser, was associated with learning for men. These results suggest that the entire affect adaptation pipeline, from detection to response, may benefit from gender-specific models in order to support students more effectively.

Massively Open Online Courses (MOOCs) have gained attention recently because of their great potential to reach learners. Substantial empirical study has focused on student persistence and their interactions with the course materials. However, most MOOCs include a rich textual dialogue forum, and these textual interactions are largely unexplored. Automatically understanding the nature of discussion forum posts holds great promise for providing adaptive support to individual students and to collaborative groups. This paper presents a study that applies unsupervised student understanding models originally developed for synchronous tutorial dialogue to MOOC forums. We use a clustering approach to group similar posts, compare the clusters with manual annotations by MOOC researchers, and further investigate clusters qualitatively. This paper constitutes a step toward applying unsupervised models to asynchronous communication, which can enable massive-scale automated discourse analysis and mining to better support students' learning.

Supporting learning with rich natural language dialogue has been the focus of increasing attention in recent years. Many adaptive learning environments model students' natural language input, and there is growing recognition that these systems can be improved by leveraging multimodal cues to understand learners better. This paper investigates multimodal features related to posture and gesture for the task of classifying students' dialogue acts within tutorial dialogue. In order to accelerate the modeling process by eliminating the manual annotation bottleneck, a fully unsupervised machine learning approach is utilized for this task. The results indicate that these unsupervised models are significantly improved with the addition of automatically extracted posture and gesture information. Further, even in the absence of any linguistic features, a model that utilizes posture and gesture features alone performed significantly better than a majority class baseline. This work represents a step toward achieving better understanding of student utterances by incorporating multimodal features within adaptive learning environments. Additionally, the technique presented here is scalable to very large student datasets.

Dialogue systems that support users in complex problem solving must interpret user utterances within the context of a dynamically changing, user-created problem solving artifact.This paper presents a novel approach to semantic grounding of noun phrases within tutorial dialogue for computer programming.Our approach performs joint segmentation and labeling of the noun phrases to link them to attributes of entities within the problem-solving environment.Evaluation results on a corpus of tutorial dialogue for Java programming demonstrate that a Conditional Random Field model performs well, achieving an accuracy of 89.3% for linking semantic segments to the correct entity attributes.This work is a step toward enabling dialogue systems to support users in increasingly complex problem-solving tasks.

We present ENGAGE, a game-based learning environment for teaching computational thinking to middle school students. This project has dual aims: introducing computational thinking practices to students at a young age, and improving computational thinking attitudes among underrepresented students. In pursuit of these two goals, the ENGAGE team has mapped the learning objectives of the AP CS Principles course to the middle school level, and then built an immersive game experience upon that foundation. Students choose computer scientist avatars to represent themselves, and then play in pairs as they investigate a data-related mystery in an underwater research station, solving computational thinking challenges along the way. ENGAGE is currently being implemented as part of a quarterly elective in four middle schools in North Carolina. During the elective, students spend a total of ten classroom sessions playing the game, supplemented by "unplugged" activities that reinforce concepts learned in the game environment. We plan to expand to more middle schools in the 2015-2016 school year. In this demo, members of the SIGCSE community will be able to experience the ENGAGE game for themselves and learn more about its development and future directions. We will also discuss our success in recruiting and teaching the ENGAGE curriculum to middle school teachers who had no prior computer science experience, and the success of those middle school teachers in implementing ENGAGE within their classrooms.

Knowledge assessment instruments, or tests, are commonly created by faculty in classroom settings to measure student knowledge and skill. Another crucial role for assessment instruments is in gauging student learning in response to a computer science education research project, or intervention. In an increasingly interdisciplinary landscape, it is crucial to validate knowledge assessment instruments, yet developing and validating these tests for computer science poses substantial challenges. This paper presents a seven-step approach to designing, iteratively refining, and validating knowledge assessment instruments designed not to assign grades but to measure the efficacy or promise of novel interventions. We also detail how this seven-step process is being instantiated within a three-year project to implement a game-based learning environment for middle school computer science. This paper serves as a practical guide for adapting widely accepted psychometric practices to the development and validation of computer science knowledge assessments to support research.

Introductory computer science courses cultivate the next generation of computer scientists. The impressions students take away from these courses are crucial, setting the tone for the rest of the students' computer science education. It is known that students struggle with many concepts central to computer science, struggles that could be alleviated in part through hands-on practice and individualized instruction. However, even the best existing instructional practices do not facilitate individualized hands-on support for students at large. We have built JavaTutor, an intelligent tutoring system for introductory computer science, which works alongside students to support them through both cognitive (skills and knowledge) and affective (emotion-based) feedback. JavaTutor aims to make advances in interactive, scalable student support. JavaTutor's behaviors were developed within a novel framework that leverages machine learning to acquire tutorial strategies from data collected within tutorial sessions between novice students and experienced human tutors. This demo presents an overview of the data-driven development of JavaTutor and shows how JavaTutor assesses and responds to students' contextualized needs. It is hoped that JavaTutor will help to usher in a new generation of tutorial systems for computer science education that adapt to individual students based not only on incoming student knowledge, but on a broad range of other student characteristics.

Detecting learning-centered affective states is difficult, yet crucial for adapting most effectively to users. Within tutoring in particular, the combined context of student task actions and tutorial dialogue shape the student's affective experience. As we move toward detecting affect, we may also supplement the task and dialogue streams with rich sensor data. In a study of introductory computer programming tutoring, human tutors communicated with students through a text-based interface. Automated approaches were leveraged to annotate dialogue, task actions, facial movements, postural positions, and hand-to-face gestures. These dialogue, nonverbal behavior, and task action input streams were then used to predict retrospective student self-reports of engagement and frustration, as well as pretest/posttest learning gains. The results show that the combined set of multimodal features is most predictive, indicating an additive effect. Additionally, the findings demonstrate that the role of nonverbal behavior may depend on the dialogue and task context in which it occurs. This line of research identifies contextual and behavioral cues that may be leveraged in future adaptive multimodal systems.

A variety of studies have established that users with different personality profiles exhibit different patterns of behavior when interacting with a system. Although patterns of behavior have been successfully used to predict cognitive and affective outcomes of an interaction, little work has been done to identify the variations in these patterns based on user personality profile. In this paper, we model sequences of facial expressions, postural shifts, hand-to-face gestures, system interaction events, and textual dialogue messages of a user interacting with a human tutor in a computer-mediated tutorial session. We use these models to predict the user's learning gain, frustration, and engagement at the end of the session. In particular, we examine the behavior of users based on their Extraversion trait score of a Big Five Factor personality survey. The analysis reveals a variety of personality-specific sequences of behavior that are significantly indicative of cognitive and affective outcomes. These results could impact user experience design of future interactive systems.

Spurred by evidence that students' future studies are highly influenced during middle school, recent efforts have seen a growing emphasis on introducing computer science to middle school learners. This paper reports on the in-progress development of a new middle school curricular module for Big Data, situated as part of a new CS Principles-based middle school curriculum. Big Data is of widespread societal importance and holds increasing implications for the computer science workforce. It also has appeal as a focus for middle school computer science because of its rich interplay with other important computer science principles. This paper examines three key aspects of a Big Data unit for middle school: its alignment with emerging curricular standards; the perspectives of middle school classroom teachers in mathematics, science, and language arts; and student feedback as explored during a middle school pilot study with a small subset of the planned curriculum. The results indicate that a Big Data unit holds great promise as part of a middle school computer science curriculum.

Exposing students early to computer science may influence their choice of career, and there is increasing recognition that even for students who do not pursue computer science careers, computational literacy is important. This poster reports on a project targeting the development of a new middle school computer science curriculum. This research aims to highlight the role of computation in Big Data in the context of middle school computer science education, which serves as a catalyst to keep students engaged in computer science through middle school via the ENGAGE narrative game-based learning environment. This poster discusses steps taken to validate one activity meant to highlight the role of computation in the context of Big Data: skip list manipulation. While we found that most of the middle school students performed poorly in assessments after the skip list activities, several students showed they were capable of completing the activity successfully, implying that a repetition of the revised skip list study and additional pilot studies for other Big Data activities are needed to pave the way for the development of this Big Data curriculum. This activity will be just one part of a broader curriculum designed to showcase the social relevance and power of Big Data.

Emotion, or affect, plays a central role in learning. In particular, promoting positive emotions throughout the learning process is important for students' motivation to pursue computer science and for retaining computer science students. Positive emotions, such as engagement or enjoyment, may be fostered by timely individualized help. Especially promising are interventions if the student is having difficulty completing a task. Recognizing when a student is facing a complex task may better inform teachers or adaptive learning environments about the students' affective states, which in turn can inform instructional adaptations. We approach this research goal by analyzing a data set of student facial videos from computer-mediated human tutorial sessions in Java programming. Students and tutors interacted with a synchronized web-based development environment. The tutorial sessions were divided into six lessons each with subtasks, and featured corresponding learning objectives for the students. In post-hoc analysis, we identified "difficult" tasks by comparing the frequencies of student-tutor interaction and task behaviors such as running the program and the time to complete tasks. Nonverbal behaviors, such as gesturing or postural shifting, were then compared with task difficulty. Understanding such nonverbal behavior can inform individualized interventions, which may keep students engaged and foster greater learning gains.

Unsupervised machine learning ap-proaches hold great promise for recog-nizing dialogue acts, but the performance of these models tends to be much lower than the accuracies reached by supervised models. However, some dialogues, such as task-oriented dialogues with parallel task streams, hold rich information that has not yet been leveraged within unsu-pervised dialogue act models. This paper investigates incorporating task features into an unsupervised dialogue act model trained on a corpus of human tutoring in introductory computer science. Exper-imental results show that incorporating task features and dialogue history fea-tures significantly improve unsupervised dialogue act classification, particularly within a hierarchical framework that gives prominence to dialogue history. This work constitutes a step toward building high-performing unsupervised dialogue act models that will be used in the next generation of task-oriented dialogue systems.

This paper explores dialogue adaptation over repeated interactions within a task-oriented human tutorial dialogue corpus. We hypothesize that over the course of four tutorial dialogue sessions, tutors adapt their strategies based on the person-ality of the student, and in particular to student introversion or extraversion. We model changes in strategy over time and use them to predict how effectively the tutorial interactions support student learn-ing. The results suggest that students lean-ing toward introversion learn more effec-tively with a minimal amount of inter-ruption during task activity, but occasion-ally require a tutor prompt before voicing uncertainty; on the other hand, students tending toward extraversion benefit signif-icantly from increased interaction, partic-ularly through tutor prompts for reflection on task activity. This line of investiga-tion will inform the development of future user-adaptive dialogue systems. 1

Affective and cognitive processes form a rich substrate on which learning plays out. Affective states often influence progress on learning tasks, resulting in positive or negative cycles of affect that impact learning outcomes. Developing a detailed account of the occurrence and timing of cognitive-affective states during learning can inform the design of affective tutorial interventions. In order to advance understanding of learning-centered affect, this paper reports on a study to analyze a video corpus of computer-mediated human tutoring using an automated facial expression recognition tool that detects fine-grained facial movements. The results reveal three significant relationships between facial expression, frustration, and learning: (1) Action Unit 2 (outer brow raise) was negatively correlated with learning gain, (2) Action Unit 4 (brow lowering) was positively correlated with frustration, and (3) Action Unit 14 (mouth dimpling) was positively correlated with both frustration and learning gain. Additionally, early prediction models demonstrated that facial actions during the first five minutes were significantly predictive of frustration and learning at the end of the tutoring session. The results represent a step toward a deeper understanding of learning-centered affective states, which will form the foundation for data-driven design of affective tutoring systems.

Understanding how students solve computational problems is central to computer science education research. This goal is facilitated by recent advances in the availability and analysis of detailed multimodal data collected during student learning. Drawing on research into student problem-solving processes and findings on human posture and gesture, this poster utilizes a multimodal learning analytics framework that links automatically identified posture and gesture features with student problem-solving and dialogue events during one-on-one human tutoring of introductory computer science. The findings provide new insight into how bodily movements occur during computer science tutoring, and lay the foundation for programming feedback tools and deep analyses of student learning processes.

Computer-mediated textual communication has become ubiquitous in recent years. Compared to face-to-face interactions, there is decreased bandwidth in affective information, yet studies show that interactions in this medium still produce rich and fulfilling affective outcomes. While overt communication (e.g., emoticons or explicit discussion of emotion) can explain some aspects of affect conveyed through textual dialogue, there may also be an underlying implicit affective channel through which participants perceive additional emotional information. To investigate this phenomenon, computer-mediated tutoring sessions were recorded with Kinect video and depth images and processed with novel tracking techniques for posture and hand-to-face gestures. Analyses demonstrated that tutors implicitly perceived students' focused attention, physical demand, and frustration. Additionally, bodily expressions of posture and gesture correlated with student cognitive-affective states that were perceived by tutors through the implicit affective channel. Finally, posture and gesture complement each other in multimodal predictive models of student cognitive-affective states, explaining greater variance than either modality alone. This approach of empirically studying the implicit affective channel may identify details of human behavior that can inform the design of future textual dialogue systems modeled on naturalistic interaction.

Using effective teaching practices is a high priority for educators. One important pedagogical skill for computer science instructors is asking effective questions. This paper presents a set of instructional principles for effective question asking during guided problem solving. We illustrate these principles with results from classifying the questions that untrained human tutors asked while working with students solving an introductory programming problem. We contextualize the findings from the question classification study with principles found within the relevant literature. The results highlight ways that instructors can ask questions to 1) facilitate students' comprehension and decomposition of a problem, 2) encourage planning a solution before implementation, 3) promote self-explanations, and 4) reveal gaps or misconceptions in knowledge. These principles can help computer science educators ask more effective questions in a variety of instructional settings.

Recent years have seen a growing awareness in the computing education community that initiatives outside the classroom are vital for retaining students and preparing them for a collaborative and dynamic professional environment. Particularly important are programs that develop rich technical skills while increasing students' interest in computing disciplines. We present Computing Identity Mentoring, an intervention designed to increase commitment to computing while enhancing students' technical and leadership skills. This program was implemented at seven universities during 2008-2009. Preliminary results suggest that Computing Identity Mentoring contributes to students' self-efficacy regarding computing and leadership, and solidifies students' commitment to a career in computing. This paper presents early findings on the effectiveness of the approach and illustrates Computing Identity Mentoring in the context of three of the seven institutions where it has been implemented.

Identifying effective tutorial strategies is a key problem for tutorial dialogue systems research. Ongoing work in human-human tutorial dialogue continues to reveal the complex phenomena that characterize these interactions, but we have not yet seen the emergence of an automated approach to discovering tutorial dialogue strategies. This paper presents a first step toward establishing a methodology for such an approach. In this methodology, a corpus is first annotated with dialogue acts that are grounded in theories of tutoring and natural language dialogue. Hidden Markov modeling is then applied to discover tutorial strategies inherent in the structure of the sequenced dialogue acts. The methodology is illustrated by demonstrating how hidden Markov models can be learned from a corpus of human-human tutoring in the domain of introductory computer science.

In the quest to find instructional approaches that benefit student learning, engagement, and retention, evidence suggests providing students with hands-on practice is a worthwhile use of class time. This paper presents results from an exploratory study of two different instructional approaches that were encountered in a study of experienced human tutors working with novice computing students engaged in a programming exercise. No difference in average learning gains was found between a moderate approach, in which students were given control of problem solving nearly half the time, and a proactive approach in which the tutor took initiative nearly three-fourths of the time. Implications of this finding for fine-grained instructional strategy, as well as for broader classroom management decisions, are discussed. This paper also makes the case for the value of one-on-one tutoring studies as an exploratory research methodology for the comparative evaluation of computer science teaching strategies.

While collaborative approaches in the classroom have been shown to be highly beneficial for students of computer science, obstacles inherent in today's academic environment often prevent collocated collaborative approaches from being implemented. One solution to the collocation problem may lie with tools that facilitate distributed collaboration. This paper presents RIPPLE (Remote Interactive Pair Programming and Learning Environment), a development environment for distributed synchronous collaborative programming. RIPPLE is an open source software tool. Initial user tests demonstrate positive responses from students, and the potential for long term learning, motivation, and retention benefits is significant. In addition to its benefits for students, RIPPLE is a tool for computing education researchers who wish to collect data on collaborative programming.

Prompted by changes in the numbers and demographics of students enrolled and being retained in computer science, the Department of Computer Science at NC State University is revising its undergraduate curriculum to better meet the needs of its students, and increase student attraction and retention. One set of changes concerns introductory computer science courses (CS1). This paper reports on a study conducted to assess the impact of class size and active learning in our CS1 courses. We find that smaller classes with integrated laboratories improve both learning and retention, as well as satisfaction of the students. Among other benefits, we found retention rates in small classes to be about 20% better than large classes.