Question-asking is a crucial learning and teaching approach. It reveals different levels of students' understanding, application, and potential misconceptions. Previous studies have categorized question types into higher and lower orders, finding positive and significant associations between higher-order questions and students' critical thinking ability and their learning outcomes in different learning contexts. However, the diversity of higher-order questions, especially in collaborative learning environments. has left open the question of how they may be different from other types of dialogue that emerge from students' conversations, To address these questions, our study utilized natural language processing techniques to build a model and investigate the characteristics of students' higher-order questions. We interpreted these questions using Bloom's taxonomy, and our results reveal three types of higher-order questions during collaborative problem-solving. Students often use "Why", "How" and "What If' questions to I) understand the reason and thought process behind their partners' actions: 2) explore and analyze the project by pinpointing the problem: and 3) propose and evaluate ideas or alternative solutions. In addition. we found dialogue labeled 'Social'. 'Question - other', 'Directed at Agent', and 'Confusion/Help Seeking' shows similar underlying patterns to higher-order questions, Our findings provide insight into the different scenarios driving students' higher-order questions and inform the design of adaptive systems to deliver personalized feedback based on students' questions.

The current study compared American, Korean, and Indonesian middle and high school students’ CS attitudes. Concurrently, this study also examined whether the items in the CS attitudes scale exhibit country and gender measurement biases. We gathered data on CS attitudes from middle and high school students in the US, Korea, and Indonesia. The participating students took the same (translated) previously validated CS attitudes scale. We ran a unidimensional IRT, differential item functioning (DIF), a two-way ANOVA, and the Kruskal-Wallis H test. Despite the valid instrument, we found it inappropriate as is for international comparison studies because students from different countries interpreted some items differently. We then compared gender-based differences in CS attitudes across countries. The results revealed no significant differences between males and females in the Indonesian middle school data, whereas male students had significantly higher CS attitudes than female students in both American and Korean student data. Furthermore, we found the same pattern in gender differences in Korean and Indonesian high school students’ CS attitudes scores as in the middle school study. These findings underscore the importance of a country’s sociocultural context in influencing gap and diversity in secondary school students’ CS attitudes.

The lack of gender diversity in the computer science (CS) field and workforce is a well-documented challenge that many, but not all, countries face. Such a challenge may tie to socio-cultural issues that have impacted K-12 CS education, eventually creating a gender gap in CS performance and attitudes. The current study compared American and Indonesian middle school students' computational thinking (CT) skills and CS attitudes. Concurrently, this study also examined whether the items in the instruments we used exhibit country, gender, or prior CS experience measurement biases. A total of 592 American n = 242 and Indonesian n = 350 middle school students took a CT assessment and CS attitudes scale. Differential item functioning (DIF) was used to detect biased items, and a two-way ANOVA was utilized to examine the interaction effects of country and gender in the two constructs. The results showed some items were flagged as having country-specific DIF. The results also indicated that the American students had higher CT scores than Indonesian students. However, Indonesian students obtained higher CS attitudes scores compared to American students. Further results showed a significant gender difference in CS attitudes in the American samples; however, such a significant difference was not found in the Indonesian sample. These findings underscore the importance of a country's socio-cultural context in influencing gender diversity in the CS field.

Intelligent virtual agents have tremendous potential for facilitating collaborative learning by modeling and reinforcing desirable collaborative practices. Despite recent work in this area, the extent to which intelligent virtual agents can facilitate improvements in the collaborative behavior of children is largely unknown. This study employed a wizard-of-oz study design and investigated elementary children's collaborative behavior after interacting with virtual agents. These agents model exploratory talk for upper elementary school dyads, such as asking higher-order questions and listening to their partners. The findings uncover associations between elementary learner dyads' positive changes in collaboration after agent interventions, the dyads' affective reactions to interventions, and their attentiveness to the agents. Our results also reveal associations between positive changes in collaboration and the timing of interventions: for example, earlier interventions had a higher occurrence of positive changes, and positive changes in collaboration typically happened within five seconds of interventions. The results suggest ways in which intelligent virtual agents may be used to promote effective collaborative learning practices for children.

The COVID-19 pandemic shifted many U.S. schools from in-person to remote instruction. While collaborative CS activities had become increasingly common in classrooms prior to the pandemic, the sudden shift to remote learning presented challenges for both teachers and students in implementing and supporting collaborative learning. Though some research on remote collaborative CS learning has been conducted with adult learners, less has been done with younger learners such as elementary school students. This experience report describes lessons learned from a remote after-school camp with 24 elementary school students who participated in a series of individual and paired learning activities over three weeks. We describe the design of the learning activities, participant recruitment, group formation, and data collection process. We also provide practical implications for implementation such as how to guide facilitators, pair students, and calibrate task difficulty to foster collaboration. This experience report contributes to the understanding of remote CS learning practices, particularly for elementary school students, and we hope it will provoke methodological advancement in this important area.

Use-Modify-Create (UMC) has gained recognition as a viable scaffolding approach for student programming activities, but little is known about how UMC could support CS learning in game-based learning environments. We designed and developed a game to teach middle grade students (ages 11-13) CS through block-based programming challenges. The game integrates a UMC pedagogical framework to promote successful student outcomes for a wide variety of student abilities, including those without prior programming experience. Utilizing a mixed-methods research design, we investigated how the game influenced student learning of CS concepts and the role of UMC on the problem-solving strategies students applied to complete the game. In particular, we were interested in how prior experience would moderate these outcomes. Results from a multilevel model of students' pre-and post-assessment scores (N = 77) on a CS concepts assessment indicated that all students, regardless of prior programming experience, showed significant learning gains from pre to post after playing the game. Qualitative results revealed that the UMC scaffolding progression provided students, particularly those with little to no prior programming experience, with the foundational knowledge needed to progress through the game levels and challenges. Specifically, we found that the Use phases of the game reduced novice students' cognitive load and facilitated the necessary CS conceptual understanding to solve the open-ended programming tasks encountered in the game's Modify and Create phases. Our findings demonstrate the efficacy of UMC to support the learning of novice programmers in a game-based learning environment while not to the detriment of those more experienced.

Pair programming is a popular strategy in computer science education to teach programming to novices. In this study, we examined the effect of three different pair programming conditions on upper elementary school students' CS conceptual understanding. The three conditions were one-computer with roles (1C with roles), two computers without roles (2C no roles), and two computers with roles (2C with roles). These students were engaged in four days of computer programming activities and took the CS concept assessment, CS attitudes, and collaboration perceptions before and after the activities. We used the validated E-CSCA (Elementary Computer Science Concepts Assessment) to measure elementary students' understanding of CS concepts. We tested the relationship of different pair programming conditions on the students' CS conceptual understanding and found that different conditions impacted students' CS conceptual understanding, wherein students in 2C roles demonstrated better CS learning than the other two conditions. The results also showed no changes in students' CS attitudes and perceptions of collaboration before and after the activities. Furthermore, the results indicated no significant impact of these attitudinal factors on students' learning CS concepts in pair programming settings. Our study highlights the importance of the roles and number of computers in pair programming settings, especially for elementary students.

Computational modeling provides an excellent vehicle for raising scientific awareness of emergent and topical phenomena such as COVID-19. Now more than ever, it is crucial to provide students with factual information about how diseases spread and how their own actions can impact that spread. In order to both encourage computational thinking skills and build scientific knowledge of the COVID-19 pandemic, we have created a series of programming activities through which students construct their own computational models based on the emerging scientific consensus around COVID-19. Students are able to model everyday situations such as being in a crowded area or going to stores while unknowingly infected, and immediately see the consequences of those actions. By including accurate scientific variables such as the reproductive number of the virus, incubation period, and period of communicability, students are able to create their own epi-curves that demonstrate the severity of the disease and provide students with visual representation of how quickly COVID-19 spreads. We also use the scientific model and associated modeling activities to reinforce best practices at home and in the community. Finally, this curriculum development effort demonstrates how block-based computational modeling activities lend themselves to agile curricular re-design around emerging and topics of local interest

We have established a research-practice partnership (RPP) to build a computer science (CS) and computational thinking (CT)-focused STEM ecosystem at two middle schools. Creating such an ecosystem to broaden student participation in computing through an RPP approach involves all stakeholders in the research process. Borrowing upon visual participatory research methods, we developed a graphic research instrument to engage teachers in the research process and elicit their perspectives on strategies for building the ecosystem. This experience report describes our research methodology across two distinct cases to demonstrate the utility of this drawing activity as an investigative and partnership development tool. The contribution is in offering a flexible approach to other university-based RPP teams that enables a synergistic partnership development tool and data collection instrument that can be tailored to a variety of RPP contexts, facilitating more productive and equitable ways of engaging stakeholders in the research process. We describe our project contexts and share results from the pilot study with practitioner-members of our RPP teams. We discuss two cases to highlight the contribution this approach made to the development of our partnerships.

In successful collaborative paradigms such as pair programming, students engage in productive dialogue and work to resolve conflicts as they arise. However, little is known about how elementary students engage in collaborative dialogue for computer science learning. Early findings indicate that these younger students may struggle to manage conflicts that arise during pair programming. To investigate collaborative dialogue that elementary learners use and the conflicts that they encounter, we analyzed videos of twelve pairs of fifth grade students completing pair programming activities. We developed a novel annotation scheme with a focus on collaborative dialogue and conflicts. We found that student pairs used best-practice dialogue moves such as self-explanation, question generation, uptake, and praise in less than 23% of their dialogue. High-conflict pairs antagonized their partner, whereas this behavior was not observed with low-conflict pairs. We also observed more praise (e.g., "We did it!") and uptake (e.g., "Yeah and...") in low-conflict pairs than high-conflict pairs. All pairs exhibited some conflicts about the task, but high-conflict pairs also engaged in conflicts about control of the computer and their partner's contributions. The results presented here provide insights into the collaborative process of young learners in CS problem solving, and also hold implications for educators as we move toward building learning environments that support students in this context.

Block-based programming environments are widely used by novices who are learning computer science. However, even in block-based coding environments that have been carefully developed to serve novices, students frequently struggle and require additional support. A promising avenue to provide this support is the use of intelligent tutoring systems, which offer adaptive hints to assist learners. In order to provide students with the adaptive hints they need, we must investigate their help-seeking behaviors and identify patterns surrounding their need for support. In this experience report, we examine data collected from 174 college students in an introductory engineering course, who used an intelligent block-based coding environment to learn computer science. These students made more than 1,000 hint requests, which we represent in two-dimensional space along axes of elapsed time and code completeness. Analysis revealed five major clusters of hint requests, which we further characterized through qualitative examination of the coding trajectories that preceded each hint request. We also analyzed how students' incoming knowledge and perceived computer skill were related to their help-seeking behaviors. Students with higher incoming knowledge requested hints when their code was more complete than students with lower incoming knowledge. Students with high perceived computer skill asked for hints when their code was less complete than those with low perceived computer skill. The results presented here provide insight into student help-seeking behavior in computer science education, informing CS educators and system designers on how best to develop support strategies.

Block-based programming environments are widely used in computer science education. However, these environments pose significant challenges for student modeling. Given a series of problem-solving actions taken by students in block-based programming environments, student models need to accurately infer problem-solving students’ programming abilities in real time to enable adaptive feedback and hints that are tailored to students’ abilities. While student models for block-based programming offer the potential to support student-adaptivity, creating student models for these environments is challenging because students can develop a broad range of solutions to a given programming activity. To address these challenges, we introduce a progression trajectory-based student modeling framework for modeling novice student block-based programming across multiple learning activities. Student trajectories utilize a time series representation that employs code analysis to incrementally compare student programs to expert solutions as students undertake block-based programming activities. This paper reports on a study in which progression trajectories were collected from more than 100 undergraduate students engaging in a series of block-based programming activities in an introductory computer science course. Using progression trajectory-based student modeling, we identified three distinct trajectory classes: Early Quitting, High Persistence, and Efficient Completion. Analysis of these trajectories revealed that they exhibit significantly different characteristics with respect to students’ actions and can be used to accurately predict students’ programming behaviors on future programming activities compared to competing baseline models. The findings suggest that progression trajectory-based student models can accurately model students’ block-based programming problem solving and hold potential for informing adaptive support in block-based programming environments.

Recent years have seen increasing awareness of the need for all students in primary and secondary education to learn computer science (CS) concepts and skills. Educational games hold significant potential to serve as a platform for CS education because they integrate engaging problem solving with effective pedagogical strategies. This potential is especially high for narrative-centered educational games that embed learning activities within rich interactive stories. In this paper, we present an educational game featuring block-based programming challenges contextualized within an engaging narrative, designed to promote CS learning for middle school students (ages 11 to 13). In the game, students undertake problem-solving challenges that are aligned with the K-12 Computer Science Framework. Results from a classroom implementation of the game with middle grade students suggest that their perceived game control ratings are positively correlated with their progress in the game, which suggests the need for adaptively supporting students' game-based learning activities. Building on these findings, we discuss design implications for creating student-adaptive CS learning experiences in educational games that incorporate block-based programming enriched narrative-centered gameplay.

Collaborative learning has demonstrated benefits for girls in computer science [7] and this may be a way to help address the gender gap in CS. Research indicates that while collaborating, boys often express more individualistic ideas whereas girls tend to be more supportive [1]. It is important for students to regulate their learning in collaborative learning environments because they need to negotiate group goals and diverse approaches to the task [3], and the use of open-ended tasks with multiple solution paths are common [4]. There is minimal research in CS education on regulation of learning (e.g., [5,6,]). Co-regulated learning is the process in which an other helps regulate the learning of a student [2] self such as by asking questions that prompt the student to monitor and evaluate (e.g., "What do you already know about 'if' blocks that would help here?''). In this way, thinking and reasoning through the problem is shared by the group members [2] self.

Computer science (CS) is widely recognized as a field with a significant gender gap despite the growing prevalence of computing. Several factors including CS attitudes, exposure to CS, experience with computer programming, and confidence in using computers are understood to be correlated with the low participation of women in CS. These factors also play an important role in students' interest in CS careers and are particularly crucial during secondary school. However, there is a dearth of research that examines differences in how these factors are inter-correlated for younger students (ages 11-13). The purpose of this study was to generate and test a statistical model that demonstrates the inter-correlation amongst these factors with respect to gender. A total of 260 middle school students participated in this study. Four instruments measuring students' CS attitudes, confidence in using computers, CS conceptual understanding, and prior experience with CS-related activities were used. Structural equation modeling was utilized to test the hypothesized model. The findings showed that previous participation in CS-related activities had a significant direct effect on CS attitudes and confidence in using computers, but the effect on students' CS conceptual understanding was indirect. We also found that in a female specific model, previous participation had a significantly stronger direct effect on CS attitudes compared to its effect in a male specific model. The importance of providing more CS-related experience, especially to female students, as well as suggestions on activities that promote gender equity in the field are discussed.

The lack of effective guidelines for assessing students' computer science (CS) competencies is creating significant demand by K-12 teachers for CS assessments to evaluate students' learning. We propose a conceptual assessment framework that guides teachers through designing appropriate assessments for computer science (CS) activities in their classrooms. The framework addresses the critical problem of incorporating CS into K-12 curricula without corresponding assessments. We illustrate its use with the design of a rubric for a bubble sort algorithm situated in a game-based learning environment for middle-grade students. We also apply a preliminary and a revised version of this assessment on two datasets collected from students' interactions with the learning environment. We found consistency among results identified through applying the preliminary and the revised rubric. The results reveal distinctive patterns in students' approaches to CS problem solving and coherency with respect to different aspects of the rubric.*

This project is a Research to Practice Partnership (RPP) between two middle schools and two universities. It focuses on investigating problems and on identifying solutions around increasing participation and interest in computer science (CS). We aim to do this by identifying, experimenting with, and fine-tuning methods to help students develop computational thinking (CT) skills. The research employs a STEM ecosystem model, which facilitates a support structure that aims to mitigate barriers and impact students as they progress in STEM areas. While this RPP is still a work in progress, we present data from the first year of our collaboration with one of the middle schools. While the research questions guiding this RPP are intended to be iterative and revised annually, year one data provides perspectives on (1) barriers to developing a STEM ecosystem that supports CS/CT for every student through integration into science, math, and language arts courses, (2) the factors or interventions needed for the development of a CS/CT focused ecosystem that supports everyone in the school, (3) the indicators of success for a CS/CT focused STEM ecosystem in a school, and (4) how the ecosystem prepares and engages all students for CS/CT work in high school. Year one data is discussed in terms of the STEM ecosystem framework and in how it will guide the next steps in this partnership. This project contributes to the understanding of how to prepare future generations for participation in a workforce where knowledge of the foundations of CS/CT is integral to success.

In recent years, there has been a dramatic increase in teaching CS in the context of other disciplines such as science. However, learning CS in an interdisciplinary context may be particularly challenging for students. An important goal for CS education researchers is to develop a deep understanding of the student experience when integrating CS into science classrooms in K-12. This paper presents the results of a mixed-methods study in which 75 middle school students engaged in a series of computationally rich science activities by creating simulations and models in a block-based programming language. After two semesters, students reported their experiences on in-class computer science activities through reflection essays. The quantitative results show that both experienced and novice students increased their CS knowledge significantly after several weeks, and a majority of students (72%) had positive sentiment toward the integration of CS into their science class. Deeper qualitative analysis of students' reflections revealed positive themes centered around the visualization and gamification of science concepts, the hands-on nature of the coding activities, and showing science from a different angle. On the other hand, students expressed negative sentiments on weaknesses in the activity design, lack of CS/science background/interest, and failing to make connections between CS and science concepts. These findings inform efforts to infuse CS education into different disciplines and reveal patterns that may foster success of K-12 classroom implementations.

Recent years have seen an increasing interest in identifying common student misconceptions during introductory programming. In a parallel development, block-based programming environments for novice programmers have grown in popularity, especially in introductory courses. While these environments eliminate many syntax-related errors faced by novice programmers, there has been limited work that investigates the types of misconceptions students might exhibit in these environments. Developing a better understanding of these misconceptions will enable these programming environments and instructors to more effectively tailor feedback to students, such as prompts and hints, when they face challenges. In this paper, we present results from a cluster analysis of student programs from interactions with programming activities in a block-based programming environment for introductory computer science education. Using the interaction data from students' programming activities, we identify three families of student misconceptions and discuss their implications for refinement of the activities as well as design of future activities. We then examine the value of block counts, block sequence counts, and system interaction counts as programming features for clustering block-based programs. These clusters can help researchers identify which students would benefit from feedback or interventions and what kind of feedback provides the most benefit to that particular student.

As computer science education opportunities for elementary students (grades K-5) are expanding, there is growing interest in using pair programming with these students. However, previous research findings do not fully support its use with younger learners, and some researchers have begun to examine whether introducing a second computer with a shared coding workspace can provide important benefits. This experience report describes a series of classroom activities in the 4th and 5th grades (ages 9-11 years old) with two different pair programming configurations: one-computer pair programming, in which both students share a keyboard, mouse, and monitor; and two-computer pair programming, in which each student has a separate computer but coding workspaces are synchronized over the web. In both cases the students sat next to each other and engaged in face-to-face conversation. We found that students largely preferred two-computer pair programming over one-computer pair programming. We conducted focus groups and transcribed collaborative dialogues to gain more insight into this preference. We learned that students felt more independence in two-computer pair programming, although they struggled with coordinating their edits with their partner. In one-computer pair programming, students reported not wanting to wait for their turn to drive, but feeling as though they communicated more with their partner. Both configurations can be productive for students, but the tradeoffs described in this experience report are important for CS educators and researchers to consider when determining which collaborative configuration to use in each K-5 classroom context.

Recent years have seen a growing interest in block-based programming environments for computer science education. Although block-based programming offers a gentle introduction to coding for novice programmers, introductory computer science still presents significant challenges, so there is a great need for block-based programming environments to provide students with adaptive support. Predictive student modeling holds significant potential for adaptive support in block-based programming environments because it can identify early on when a student is struggling. However, predictive student models often make a number of simplifying assumptions, such as assuming a normal response distribution or homogeneous student characteristics, which can limit the predictive performance of models. These assumptions, when invalid, can significantly reduce the predictive accuracy of student models.

The language and concepts used by curriculum designers are not always interpreted by children as designers intended. This can be problematic when researchers use self-reported survey instruments in concert with curricula, which often rely on the implicit belief that students’ understanding aligns with their own. We report on our refinement of a validated survey to measure upper elementary students’ attitudes and perspectives about computer science (CS), using an iterative, design-based research approach informed by educational and psychological cognitive interview processes. We interviewed six groups of students over three iterations of the instrument on their understanding of CS concepts and attitudes toward coding. Our findings indicated that students could not explain the terms computer programs nor computer science as expected. Furthermore, they struggled to understand how coding may support their learning in other domains. These results may guide the development of appropriate CS-related survey instruments and curricular materials for K–6 students.

Participatory design practices create informed designs by bringing stakeholders into the design process early and often. This approach is a powerful tool, especially when the designer and the intended user are very different. This paper reports on work in which researchers co-design pedagogical agents to support collaborative computer science learning with elementary school students using an iterative drawing methodology. In the open drawing phase, students drew what they believe good collaboration looked like. Next, researchers analyzed those drawings under the requirements of the broader project and created a drawing scaffold (similar to a coloring book page). In the scaffolded drawing phase, students ideated within the more focused context. This process resulted in actionable design guidelines for the appearance of pedagogical agents.

Educational games offer significant potential for supporting personalized learning in engaging virtual worlds. However, many educational games do not provide adaptive gameplay to meet the needs of individual students. To address this issue, educational games should include game levels that can self-adjust to the specific needs of individual students. However, creating a large number of adaptable game levels requires considerable effort by game developers. A promising solution to this problem is to leverage procedural content generation to automatically generate levels for educational games that incorporate the desired learning objectives. In this paper, we propose a multistep deep convolutional generative adversarial network for generating new levels within a game for middle school computer science education. The model operates in two phases: (1) train a generator with a small set of human-authored example levels and generate a much larger set of synthetic levels to augment the training data for a second generator, and (2) train a second generator using the augmented training data and use it to generate novel educational game levels with enhanced solvability. We evaluate the performance of the model by comparing the novelty and solvability of generated levels between the two generators. Results suggest that the proposed multistep model significantly enhances the solvability of the generated levels with only minor degradation in the novelty of the generated levels.

Computational Thinking (CT) is being infused into curricula in a variety of core K-12 STEM courses. As these topics are being introduced to students without prior programming experience and are potentially taught by instructors unfamiliar with programming and CT, appropriate lesson design might help support both students and teachers. "Use-Modify-Create" (UMC), a CT lesson progression, has students ease into CT topics by first "Using" a given artifact, "Modifying" an existing one, and then eventually "Creating" new ones. While studies have presented lessons adopting and adapting this progression and advocating for its use, few have focused on evaluating UMC's pedagogical effectiveness and claims. We present a comparison study between two CT lesson progressions for middle school science classes. Students participated in a 4-day activity focused on developing an agent-based simulation in a block-based programming environment. While some classrooms had students develop code on days 2-4, others used a scaffolded lesson plan modeled after the UMC framework. Through analyzing student's exit tickets, classroom observations, and teacher interviews, we illustrate differences in perception of assignment difficulty from both the students and teachers, as well as student perception of artifact "ownership" between conditions.

With national K-12 education initiatives such as "CSForAll," block-based programming environments have emerged as widely used tools for teaching novice programming. A key challenge presented by block-based programming environments is assessing students' computational thinking (CT) and programming competencies. Developing assessment methods that can evaluate students' use of CT practices such as testing and refining, and developing and using appropriate algorithms, can help teachers evaluate students learning and provide appropriate scaffolding. In this work, we utilize an evidence-centered assessment design approach to devise a three-dimensional assessment to evaluate students' CT competencies based on evidence extracted from their programming trajectories in a block-based programming environment. In this assessment, the first dimension assesses students' knowledge of essential CT concepts, the second dimension assesses students' dynamic testing and refining strategies, and the third dimension assesses their overall problem-solving efficiency. We apply the assessment framework to data collected from students' interactions with a game-based learning environment designed to develop middle-grade students' CT competencies and programming skills. The results demonstrate that students' knowledge of basic CT constructs, such as appropriate use and combination of control structures, serves as the foundation for designing and implementing effective algorithms. Further, we assessed students testing and refining strategies over the three dimensions of novelty, positivity, and scale. The results demonstrate that students with higher algorithmic capabilities tend to make more novel, positive, and small-scale changes. The results reveal distinctive patterns in students' approaches to computational thinking problem solving and make a step toward identifying and assessing productive computational thinking practices.

In 1962, Thomas Kuhn changed our understanding of scientific progress through his explanation of scientific paradigms and attribution of scientific advances to paradigm shifts. According to Kuhn, a discipline's paradigm drives research, provides explanations, and directs the accumulation of discipline-specific knowledge. In 2006 and 2008 Jeanette Wing authored articles that reignited interest in computational thinking (CT) education and CT education research. Wing's articles, arguably, set in place the conceptual foundations of the paradigm currently guiding CT education and paved the way for a fruitful decade of CT education research. We are concerned, however, that the present direction of CT education and research will not support CT as an integral part of K-12 education. In its current form, CT instruction focuses almost exclusively on teaching students to program and isolates CT from disciplinary content; perpetuating the persistent misconception that programming = CT. This approach to CT education may deprive students of the opportunity to adequately develop foundational CT skills (e.g. systems thinking, abstraction and generalization, data collection and utilization, solution evaluation) and may prevent teachers from persistently and meaningfully integrating CT into their curriculum. Through this talk, we identify features of the current CT paradigm, question their alignment with Wing's ideal of "computational thinking for everyone," and propose specific recommendations for expanding the current CT paradigm.

Extensive prior research suggests that pair programming holds many benefits for novices. Pair programming has been well studied at the undergraduate level, and recently, the CS education research community has started to realize that younger learners may also benefit from pair programming. However, an important factor in pair programming success for young learners is the ability to resolve conflicts during the process. Little is known about what types of conflicts occur while elementary students pair program or how those conflicts are, or are not, resolved. To investigate this phenomenon, we analyzed the videos of six pairs of students completing a programming activity. We found that conflicts evolve in four general stages, which may not all be present in each conflict: initiation, escalation, de-escalation, and conclusion. Some conflicts are resolved when the students come to an agreement, others end passively. The analysis revealed that the pairs' conflicts began around disagreements about code, who should have control of the keyboard and mouse, and other interpersonal events. This research indicates that conflicts are a significant concern for young students, and supporting young learners in developing improved collaboration skills is a key direction for CS education research.

Demand for K-6 computer science (CS) curricula is growing considerably. Many of the existing curricula have been developed by domain experts who are comfortable with specific and technical terminology, which they expect students to master. However, children are not always comfortable with these terms nor do they understand general concepts like 'coding' in the way that the curriculum designers intend. This is a problem because many researchers use self-report and attitudinal survey instruments with the implicit belief that the students' understanding of the terms and concepts resemble their own. This mismatch may invalidate results. For this project, we report on our modification of a validated survey to measure upper elementary students' attitudes about and perspectives on CS by attempting to understand the appropriate language to use when querying children about these topics. We use an iterative, design-based research approach that is informed by educational and psychological cognitive interview processes. We interviewed two groups (N=64) of upper elementary students on their understanding of computer science concepts and attitudes toward coding. Our findings indicate that 4th and 5th grade students could not explain the terms computer programs nor computer science as we had expected and that they struggled to understand how coding may connect with or support their learning in other domains. These results will help to guide the development of appropriate survey instruments and course materials for K-6 students, which both match their use of broad domain concepts and therefore inform their understanding and improve their outcomes.

The recognition of middle grades as a critical juncture in CS education has led to the widespread development of CS curricula and integration efforts. The goal of many of these interventions is to develop a set of underlying abilities that has been termed computational thinking (CT). This goal presents a key challenge for assessing student learning: we must identify assessment items associated with an emergent understanding of key cognitive abilities underlying CT that avoid specialized knowledge of specific programming languages. In this work we explore the psychometric properties of assessment items appropriate for use with middle grades (US grades 6-8; ages 11-13) students. We also investigate whether these items measure a single ability dimension. Finally, we strive to recommend a "lean" set of items that can be completed in a single 50-minute class period and have high face validity. The paper makes the following contributions: 1) adds to the literature related to the emerging construct of CT, and its relationship to the existing CTt and Bebras instruments, and 2) offers a research-based CT assessment instrument for use by both researchers and educators in the field.

The K-12 Computer Science Framework has established that students should be learning about the computer science concept of variables as early as middle school, although the field has not yet determined how this and other related concepts should be introduced. Secondary school computer science curricula such as Exploring CS and AP CS Principles often teach the concept of variables in the context of algebra, which most students have already encountered in their mathematics courses. However, when strategizing how to introduce the concept at the middle school level, we confront the reality that many middle schoolers have not yet learned algebra. With that challenge in mind, this position paper makes a case for introducing the concept of variables in the context of middle school science. In addition to an analysis of existing curricula, the paper includes discussion of a day-long pilot study and the consequent teacher feedback that further supports the approach. The CS For All initiative has increased interest in bringing computer science to middle school classrooms; this paper makes an argument for doing so in a way that can benefit students' learning of both computer science and core science content.

The important goal of broadening participation in computing has inspired many successful outreach initiatives. Yet many of these initiatives, such as out-of-school activities or innovative new computer science courses for secondary school students, may disproportionately attract students who already have prior interest and experience in computing. How, then, do we engage the silent majority of students who do not self-select computer science? This paper examines this question in the context of ENGAGE, an in-school outreach initiative for middle school students. ENGAGE's learning activities center on a game-based learning environment for computer science. Results reveal that the initiative improved the computer science attitudes of students who were not already predisposed to study computer science, in a way that a corresponding after-school program could not. The results illustrate how an in-school initiative can empower young students who might not otherwise consider studying computer science.

Affective support is crucial during learning, with recent evidence suggesting it is particularly important for female students. Facial expression is a rich channel for affect detection, but a key open question is how facial displays of affect differ by gender during learning. This paper presents an analysis suggesting that facial expressions for women and men differ systematically during learning. Using facial video automatically tagged with facial action units, we find that despite no differences between genders in incoming knowledge, self-efficacy, or personality profile, women displayed one lower facial action unit significantly more than men, while men displayed brow lowering and lip fidgeting more than women. However, numerous facial actions including brow raising and nose wrinkling were strongly correlated with learning in women, whereas only one facial action unit, eyelid raiser, was associated with learning for men. These results suggest that the entire affect adaptation pipeline, from detection to response, may benefit from gender-specific models in order to support students more effectively.

We present ENGAGE, a game-based learning environment for teaching computational thinking to middle school students. This project has dual aims: introducing computational thinking practices to students at a young age, and improving computational thinking attitudes among underrepresented students. In pursuit of these two goals, the ENGAGE team has mapped the learning objectives of the AP CS Principles course to the middle school level, and then built an immersive game experience upon that foundation. Students choose computer scientist avatars to represent themselves, and then play in pairs as they investigate a data-related mystery in an underwater research station, solving computational thinking challenges along the way. ENGAGE is currently being implemented as part of a quarterly elective in four middle schools in North Carolina. During the elective, students spend a total of ten classroom sessions playing the game, supplemented by "unplugged" activities that reinforce concepts learned in the game environment. We plan to expand to more middle schools in the 2015-2016 school year. In this demo, members of the SIGCSE community will be able to experience the ENGAGE game for themselves and learn more about its development and future directions. We will also discuss our success in recruiting and teaching the ENGAGE curriculum to middle school teachers who had no prior computer science experience, and the success of those middle school teachers in implementing ENGAGE within their classrooms.

Knowledge assessment instruments, or tests, are commonly created by faculty in classroom settings to measure student knowledge and skill. Another crucial role for assessment instruments is in gauging student learning in response to a computer science education research project, or intervention. In an increasingly interdisciplinary landscape, it is crucial to validate knowledge assessment instruments, yet developing and validating these tests for computer science poses substantial challenges. This paper presents a seven-step approach to designing, iteratively refining, and validating knowledge assessment instruments designed not to assign grades but to measure the efficacy or promise of novel interventions. We also detail how this seven-step process is being instantiated within a three-year project to implement a game-based learning environment for middle school computer science. This paper serves as a practical guide for adapting widely accepted psychometric practices to the development and validation of computer science knowledge assessments to support research.

Introductory computer science courses cultivate the next generation of computer scientists. The impressions students take away from these courses are crucial, setting the tone for the rest of the students' computer science education. It is known that students struggle with many concepts central to computer science, struggles that could be alleviated in part through hands-on practice and individualized instruction. However, even the best existing instructional practices do not facilitate individualized hands-on support for students at large. We have built JavaTutor, an intelligent tutoring system for introductory computer science, which works alongside students to support them through both cognitive (skills and knowledge) and affective (emotion-based) feedback. JavaTutor aims to make advances in interactive, scalable student support. JavaTutor's behaviors were developed within a novel framework that leverages machine learning to acquire tutorial strategies from data collected within tutorial sessions between novice students and experienced human tutors. This demo presents an overview of the data-driven development of JavaTutor and shows how JavaTutor assesses and responds to students' contextualized needs. It is hoped that JavaTutor will help to usher in a new generation of tutorial systems for computer science education that adapt to individual students based not only on incoming student knowledge, but on a broad range of other student characteristics.

Detecting learning-centered affective states is difficult, yet crucial for adapting most effectively to users. Within tutoring in particular, the combined context of student task actions and tutorial dialogue shape the student's affective experience. As we move toward detecting affect, we may also supplement the task and dialogue streams with rich sensor data. In a study of introductory computer programming tutoring, human tutors communicated with students through a text-based interface. Automated approaches were leveraged to annotate dialogue, task actions, facial movements, postural positions, and hand-to-face gestures. These dialogue, nonverbal behavior, and task action input streams were then used to predict retrospective student self-reports of engagement and frustration, as well as pretest/posttest learning gains. The results show that the combined set of multimodal features is most predictive, indicating an additive effect. Additionally, the findings demonstrate that the role of nonverbal behavior may depend on the dialogue and task context in which it occurs. This line of research identifies contextual and behavioral cues that may be leveraged in future adaptive multimodal systems.

Real-time formative assessment of student learning has become the subject of increasing attention. Students' textual responses to short answer questions offer a rich source of data for formative assessment. However, automatically analyzing textual constructed responses poses significant computational challenges, and the difficulty of generating accurate assessments is exacerbated by the disfluencies that occur prominently in elementary students' writing. With robust text analytics, there is the potential to accurately analyze students' text responses and predict students' future success. In this paper, we present WriteEval, a hybrid text analytics method for analyzing student-composed text written in response to constructed response questions. Based on a model integrating a text similarity technique with a semantic analysis technique, WriteEval performs well on responses written by fourth graders in response to short-text science questions. Further, it was found that WriteEval's assessments correlate with summative analyses of student performance.

Spurred by evidence that students' future studies are highly influenced during middle school, recent efforts have seen a growing emphasis on introducing computer science to middle school learners. This paper reports on the in-progress development of a new middle school curricular module for Big Data, situated as part of a new CS Principles-based middle school curriculum. Big Data is of widespread societal importance and holds increasing implications for the computer science workforce. It also has appeal as a focus for middle school computer science because of its rich interplay with other important computer science principles. This paper examines three key aspects of a Big Data unit for middle school: its alignment with emerging curricular standards; the perspectives of middle school classroom teachers in mathematics, science, and language arts; and student feedback as explored during a middle school pilot study with a small subset of the planned curriculum. The results indicate that a Big Data unit holds great promise as part of a middle school computer science curriculum.

Emotion, or affect, plays a central role in learning. In particular, promoting positive emotions throughout the learning process is important for students' motivation to pursue computer science and for retaining computer science students. Positive emotions, such as engagement or enjoyment, may be fostered by timely individualized help. Especially promising are interventions if the student is having difficulty completing a task. Recognizing when a student is facing a complex task may better inform teachers or adaptive learning environments about the students' affective states, which in turn can inform instructional adaptations. We approach this research goal by analyzing a data set of student facial videos from computer-mediated human tutorial sessions in Java programming. Students and tutors interacted with a synchronized web-based development environment. The tutorial sessions were divided into six lessons each with subtasks, and featured corresponding learning objectives for the students. In post-hoc analysis, we identified "difficult" tasks by comparing the frequencies of student-tutor interaction and task behaviors such as running the program and the time to complete tasks. Nonverbal behaviors, such as gesturing or postural shifting, were then compared with task difficulty. Understanding such nonverbal behavior can inform individualized interventions, which may keep students engaged and foster greater learning gains.

Affective and cognitive processes form a rich substrate on which learning plays out. Affective states often influence progress on learning tasks, resulting in positive or negative cycles of affect that impact learning outcomes. Developing a detailed account of the occurrence and timing of cognitive-affective states during learning can inform the design of affective tutorial interventions. In order to advance understanding of learning-centered affect, this paper reports on a study to analyze a video corpus of computer-mediated human tutoring using an automated facial expression recognition tool that detects fine-grained facial movements. The results reveal three significant relationships between facial expression, frustration, and learning: (1) Action Unit 2 (outer brow raise) was negatively correlated with learning gain, (2) Action Unit 4 (brow lowering) was positively correlated with frustration, and (3) Action Unit 14 (mouth dimpling) was positively correlated with both frustration and learning gain. Additionally, early prediction models demonstrated that facial actions during the first five minutes were significantly predictive of frustration and learning at the end of the tutoring session. The results represent a step toward a deeper understanding of learning-centered affective states, which will form the foundation for data-driven design of affective tutoring systems.

Understanding how students solve computational problems is central to computer science education research. This goal is facilitated by recent advances in the availability and analysis of detailed multimodal data collected during student learning. Drawing on research into student problem-solving processes and findings on human posture and gesture, this poster utilizes a multimodal learning analytics framework that links automatically identified posture and gesture features with student problem-solving and dialogue events during one-on-one human tutoring of introductory computer science. The findings provide new insight into how bodily movements occur during computer science tutoring, and lay the foundation for programming feedback tools and deep analyses of student learning processes.

Computer-mediated textual communication has become ubiquitous in recent years. Compared to face-to-face interactions, there is decreased bandwidth in affective information, yet studies show that interactions in this medium still produce rich and fulfilling affective outcomes. While overt communication (e.g., emoticons or explicit discussion of emotion) can explain some aspects of affect conveyed through textual dialogue, there may also be an underlying implicit affective channel through which participants perceive additional emotional information. To investigate this phenomenon, computer-mediated tutoring sessions were recorded with Kinect video and depth images and processed with novel tracking techniques for posture and hand-to-face gestures. Analyses demonstrated that tutors implicitly perceived students' focused attention, physical demand, and frustration. Additionally, bodily expressions of posture and gesture correlated with student cognitive-affective states that were perceived by tutors through the implicit affective channel. Finally, posture and gesture complement each other in multimodal predictive models of student cognitive-affective states, explaining greater variance than either modality alone. This approach of empirically studying the implicit affective channel may identify details of human behavior that can inform the design of future textual dialogue systems modeled on naturalistic interaction.

Cloud computing is gaining popularity in higher education settings, but the costs and benefits of this tool have gone largely unexplored. The purpose of this study was to examine the factors that lead to technology adoption in a higher education setting. Specifically, we examined a range of predictors and outcomes relating to the acceptance of a cloud computing platform in rural and urban community colleges. Drawing from the Technology Acceptance Model 3 (TAM3) (Venkatesh, V. and Bala, H., 2008. Technology Acceptance Model 3 and a research agenda on interventions. Decision Sciences, 39 (2), 273–315), we build on the literature by examining both the actual usage and future intentions; further, we test the direct and indirect effects of a range of predictors on these outcomes. Approximately 750 community college students enrolled in basic computing skills courses participated in this study; findings demonstrated that background characteristics such as the student's ability to travel to campus had influenced the usefulness perceptions, while ease of use was largely determined by first-hand experiences with the platform, and instructor support. We offer recommendations for community college administrators and others who seek to incorporate cloud computing in higher education settings.

We describe the results from a two-year study with fourteen universities on presenting formal languages in a more visual, interactive and applied manner using JFLAP. In our results the majority of students felt that having access to JFLAP made learning course concepts easier, made them feel more engaged in the course and made the course more enjoyable. We also describe changes and additions to JFLAP we have made based on feedback from users. These changes include new algorithms such as a CYK parser and a user-controlled parser, and new resources that include a JFLAP online tutorial, a wiki and a listserv.

In a course environment, pairing a student with one partner for the entire semester is beneficial, but may not be optimal. We conduct a study in two undergraduate level courses to observe the advantages and disadvantages of pair rotation whereby a student pairs with several different students throughout the semester. We summarize teaching staff and student perceptions on the viability of pair rotation. Teachers find pair rotation valuable because the teaching staff can obtain multiple peer evaluations on each student and because dysfunctional pairs are regularly disbanded. However, pair rotation adds to the burden of assigning pairs multiple times per semester. The majority of students in the study perceived pair rotation to be a desirable approach. Additionally, most students considered peer evaluation to be an effective means of providing feedback to teaching staff. However, they did not significantly believe that peer evaluation was an effective means for motivating students.

Pair programming is a practice in which two programmers work collaboratively at one computer, on the same design, algorithm, or code. Prior research indicates that pair programmers produce higher quality code in essentially half the time taken by solo programmers. An experiment was run to assess the efficacy of pair programming in an introductory Computer Science course. Student pair programmers were more self-sufficient, generally perform better on projects and exams, and were more likely to complete the class with a grade of C or better than their solo counterparts. Results indicate that pair programming creates a laboratory environment conducive to more advanced, active learning than traditional labs; students and lab instructors report labs to be more productive and less frustrating.