In 2024 the ACM Education Board began a project to develop a ''Living curriculum'' (referred to as LiveC) where all current curricula will be updated on a regular basis. The new Living Curriculum model is based on three major principles - that it is (1) community-sourced, (2) moderated, and (3) on-going. The panel will provide a brief context of LiveC, present the various aspects of LiveC and invite the audience to provide comments and suggestions about its structure and function.

Educational terminology in computing remains fragmented across geographic regions and educational traditions. Identical terms may refer to different concepts in different regions, while equivalent concepts are often termed differently in different traditions, creating barriers to the exchange of pedagogical practices. A globally shared terminology resource will help bridge this inconsistency in usage. This working group will explore the use of generative AI (GenAI) to support the aggregation, validation, and curation of terminology from diverse sources. Generative AI will serve in a directed co-development role while domain experts will still retain responsibility for the final product. The working group will evaluate the benefits and limitations of this GenAI-in-the-loop approach with particular attention to accuracy, bias, and coverage.

Professional dispositions are important in numerous fields, including computing. However, strategies to promote dispositions are not yet well understood. In this experience report, we share what students suggest to promote five dispositions: being collaborative, meticulous, persistent, responsive, and self-directed. To get a broad perspective of students' voices, we gathered feedback over two terms in four courses at three institutions.

Computing Curricula 2020 raises several challenges associated with making the design of computing curricula accessible for implementation across internationally diverse educational settings. Among these are limitations created by local languages, customs, and politics. As examples, a North American Computer Science degree generally translates to a European Informatics degree despite the definitions of these terms differing in English. Within English, for example, there are no standard definitions for references such as earning a certificate or via an accelerated degree. The ACM Globalization of Computing Curricula task force is charged with understanding factors that limit the global diversity of curricular recommendations and with developing mechanisms to facilitate dialog among international stakeholders. Following a brief introduction summarizing the task force's progress, BoF attendees will be divided into facilitated breakout groups. These groups will be asked to suggest and discuss ideas that will advance mechanisms and dialog to promote the globalization of computing curricula. Facilitators will report back to the larger group, items considered to be of significance with ample time for additional feedback from other participants. As the objective of this BoF is to capture as much input as possible from a broadly diverse group of SIGCSE attendees, each BoF activity will be designed to encourage participation among all attendees. Any SIGCSE attendee with an interest in making curricular computing recommendations accessible to a more globally diverse community is invited to attend this BoF.

In this panel, the experience adopting and adapting computer science education to the availability of generative AI will be examined from multiple international perspectives spanning varied educational and cultural contexts: the Arab world, India, Latin America, and the US. This will help computer science educators compare notes on what they teach, how they teach, and how students engage with the emerging tools. It will provide an opportunity to discuss how institutions are updating curricula, supporting faculty, and preparing students to use generative AI effectively and responsibly. The premise of the panel is that the similarities among these contexts outweigh the educational and cultural differences. Given how generative AI is dominating the discussion of computer science curricula, the global conversation will be of interest to all computer science educators.

The latest in the series of computer science curricular guidelines produced by the ACM in collaboration with IEEE-Computer Society and the AAAI is CS2023 (csed.acm.org). Given that generative AI had only been available for a short while at the time of publication of CS2023, the task force that drafted CS2023 could at best speculate on its impact on the curricular guidelines. Two years later, this special session explores how the design of some of the knowledge areas of CS2023 most affected by generative AI could/should change in terms of content, pedagogy, assessment and policies. Given the ubiquity of generative AI in computer science education and its rapid evolution, re-examination of its impacts on computer science curriculum on an ongoing basis is both warranted and needed.

Dispositions are valued by employers and promoted by recent computing curricular recommendations. Yet, fostering and assessing dispositions are not well understood. In a multi-institutional study, students were asked to assess their dispositions in terms of behaviors that were identified in prior literature for those dispositions, both at the start and the end of a term. During the term, instruments were used to have students reflect on their dispositions. The research questions of the study are: 1) Do students associate behaviors with the dispositions for which they were identified in prior work? 2) Does reflecting on dispositions change how students assess themselves in terms of the behaviors? and 3) Is there a difference between introductory and upper-level students in how they assess themselves in terms of the behaviors? The findings of the study are that 1) at least 60% of the students associated the behavior statements with the dispositions for which they were identified; 2) students lowered their self-assessment of some behaviors after reflecting on dispositions; and 3) upper-level students assessed themselves more positively on some behaviors than introductory students. These results support a model of development of dispositions in which self-assessment of behaviors associated with dispositions improves with academic level, but at each level, gets revised lower after reflection.

Lifelong learning is essential in computing, given the dynamic nature of the field. Employers and curricular reviewers recognize the value of being self-directed in support of becoming a lifelong learner. The ACM/IEEE-CS Computing Curricula 2020 report identifies self-directed as having elements of self-motivation, determination, and independence. Little is known, however, about how to cultivate this disposition in computing courses. The motivation of this study is to better understand what behaviors computing students believe are self-directed. This study's research questions are: 1) What do students describe as their self-directed practices in computing? and 2) What do students report are factors that prevent them from being self-directed? Assignments in five undergraduate computing courses from four institutions included prompts to elicit student's reflections on how they were self-directed (or not). Thematic content analysis using the constant comparative method produced eight categories of self-directed behaviors (utilizing external resources, learning necessary material, working independently, assessing oneself, planning ahead, applying useful techniques, completing the assigned work, and reviewing against expectations). Thematic analysis also resulted in five categories of factors that impeded the self-directed behavior (assignment structure, unsuccessful effort, self-sufficiency, insufficient motivation, and insufficient time). Understanding how students describe self-directedness can help educators design pedagogical and assessment approaches that facilitate self-directed student behaviors in the classroom.

This full research paper contributes to current work on fostering professional dispositions in computing and engineering education by identifying the categories of behaviors that students associate with dispositions while doing course work. Professional dispositions, demonstrated through desirable behaviors in the workplace, such as being persistent or self-directed, are explicitly sought by employers. Fostering dispositions among students has been identified in various curricular recommendations as an important goal. In prior work, the authors used reflection exercises, in which students were presented with the definition of a disposition and asked to answer an open-ended reflection prompt on how they applied the disposition in their own work. Thematic analysis of student responses to reflection exercises resulted in categories of behaviors that students associated with dispositions. In the work discussed in this paper, the authors used vignette exercises to collect and analyze similar data and gain further insight into behavioral categories and students' perceptions of dispositions. Vignettes include short scenarios that demonstrate the application of dispositions in real life. A vignette exercise involves students reading a vignette scenario, identifying the disposition demonstrated by the scenario, and answering the same open-ended reflection prompt as in the reflection exercises from the earlier studies. The research question for this study is: Which behavioral categories obtained from analyzing student responses to reflection exercises were confirmed using vignette exercises (and which were not confirmed), and which behavioral categories were refined? To answer this question, researchers from four different institutions of higher education collected data in multiple courses over two semesters. The student open-ended responses to vignettes were thematically analyzed to identify behavioral categories for four dispositions: collaborative, meticulous, persistent and self-directed. The ultimate goal of this work is to create classroom inter-ventions and learning activities that foster dispositions among students based on behavioral categories. This study supports this goal in two ways. It provides another iteration of behavioral category analysis and introduces vignettes to encourage students to reflect candidly and communicate clearly how they apply dispositions in terms of behaviors. The study results and their implications for fostering dispositions in a classroom setting are presented and discussed.

Model curricula for baccalaureate computer science (CS) have been published regularly from 1968 through 2013. In early 2021, the ACM, IEEE-Computer Society, and the Association for the Advancement of Artificial Intelligence (AAAI) constituted a task force to revise these curricula, which have now been released as Computer Science 2023 Curricula (CS2023). The CS2023 curricular guidelines inform educators and administrators on the what, why, and how to cover undergraduate CS over the next decade. Like past guidelines, CS2023 provides curricular content - a knowledge model largely backward compatible with CS2013, supplemented by a competency framework influenced by Computing Curricula 2020 (CC2020) - and complementary curricular practices, which include articles by international experts on program design and delivery. Ongoing drafts of CS2023 were disseminated via the CS2023 website, along with regular publications or presentations at various computing education venues.

Characterising code quality is a challenge that was addressed by Börstler et al. 's working group in 2017. As emerged from their study, educators, developers and students have different perceptions of the manifold aspects involved, and a major conclusion of that WG was that "code quality should be discussed more thoroughly in educational programs" [2, p. 70]. However, the lack of materials and the time constraints have slowed down progress in that regard.

Dispositions, skills, and knowledge form the three components of competency-based education. Moreover, dispositions are considered crucial for students to succeed in the workplace. Few studies investigate how dispositions manifest in the form of observable behaviors, which causes challenges for both students and educators. Computing students, for example, may not understand what is expected of them, and how to achieve dispositions. This paper presents the results of a qualitative, multi-institutional study on students' understanding of the dispositions adaptable, persistent, self-directed, meticulous, and professional. Perceptions were gathered by asking for exemplary situations of students applying each of the five dispositions in the context of assignments within computing courses. Students who indicated they did not apply the disposition were asked to describe the hindering circumstances. The data was evaluated by using Mayring's content analysis technique, resulting in the development of deductive-inductive categories of observable behaviors reflecting the student's perspective. For meticulous and professional, new categories representing observable behaviors were developed. For adaptable, persistent, and self-directed, the authors confirmed and extended prior work. Moreover, factors hindering students in applying the investigated dispositions are identified. The resulting categories with observable student behaviors are an important step toward the operationalization of competency-based learning outcomes including dispositions. A common understanding of dispositions will also help with the design of new forms of instruction and measures to foster the application of dispositions in the context of computing education.

A joint task force of the ACM, IEEE-Computer Society, and AAAI has updated the computer science curricular guidelines last published in 2013. Included in the updated guidelines, referred to as CS2023, is a revised knowledge model and a new framework for building a customized competency model of the computer science curriculum. Acknowledging that one size does not fit all, the guidelines provide flexibility for computer science programs to structure their curriculum around the competency area(s) they wish to target. Given the pervasiveness of computing in everyday life, the guidelines emphasize the importance of society, ethics, and issues of the computing profession throughout the curriculum. The role of mathematics has been expanded in the guidelines, particularly in response to the latest developments in artificial intelligence. The curricular guidelines will be accompanied by articles written by experts on curricular issues such as ethics, computing for the social good, and accessibility. In the special session, the CS2023 curricular guidelines will be presented, and feedback solicited on facilitating their adoption and adaptation. The session is aimed at computer science educators, administrators, and professionals interested in computer science education.

Vignettes are short stories along with a set of questions that engage the reader to comment on the story. Vignettes have been used in professional academic programs (e.g., teacher preparation and medical education), for professional development in various fields (e.g., teaching ethics in psychology and medicine), and in various research fields for data collection. In this work, vignettes are used to elicit students' understanding of dispositions in computing education. Professional dispositions enable behaviors that are valued in the workplace, such as adaptability or self-directedness. They are often explicitly stated in computing job postings. While the relevance of dispositions is widely recognized in the workplace, only recently have curricular guidelines for computing programs recognized professional dispositions as an integral part of competencies and as complementary to knowledge and skills. There is scarce literature on the use of vignettes in teaching undergraduate computing, or on how best to foster dispositions in students. In this project, four faculty from four diverse institutions in the U.S., along with three consulting experts, have collaborated to design and evaluate the use of vignettes in the classroom. This paper documents researchers' efforts to gain insights into students' perceptions of dispositions through the use of vignettes. Such insights may guide educators to identify pedagogical strategies for fostering dispositions among students. This paper presents an iterative process for vignette design with continuous review by researchers and focus group members. The vignettes in this study use stories of situations which demonstrate the application of a disposition, drawn from various fields and walks of life to represent diverse groups and experiences. Students are presented with the vignette story and asked to identify the disposition illustrated. To elicit students' understanding of dispositions in terms of their personal behaviors, students are asked to describe a situation in which they have experienced the disposition. Lessons learned in the design and use of vignettes are discussed.

Dispositions, along with skills and knowledge, form the three components of competency-based education. Moreover, studies have shown dispositions to be necessary for a successful career. However, unlike evidence-based teaching and learning approaches for knowledge acquisition and skill development, few studies focus on translating dispositions into observable behavioral patterns. An operationalization of dispositions, however, is crucial for students to understand and achieve respective learning outcomes in computing courses. This paper describes a multi-institutional study investigating students' understanding of dispositions in terms of their behaviors while completing coursework. Students in six computing courses at four different institutions filled out a survey describing an instance of applying each of the five surveyed dispositions (adaptable, collaborative, persistent, responsible, and self-directed) in the courses' assignments. The authors evaluated data by using Mayring's qualitative content analysis. The result was a coding scheme with categories summarizing students' concepts of dispositions and how they see themselves applying dispositions in the context of computing. These results are a first step in understanding dispositions in computing education and how they manifest in student behavior. This research has implications for educators developing new pedagogical approaches to promote and facilitate dispositions. Moreover, the operationalized behaviors constitute a starting point for new assessment strategies of dispositions.

In Fall 2021, a preliminary study was conducted to gain insight into students' perceptions of the importance of professional dispositions to their computing courses and career. Students filled out a pre-survey, post-assignment reflection exercises, and a post-survey. We found that 1) students rated dispositions as being maximally important for the course and their career on the pre- and post-surveys; 2) students rated dispositions not relevant to the course lower than those that were relevant; and 3) students rated their application of dispositions in course assignments lower than they had rated the importance of the dispositions for success in the course in the pre-survey.

A Joint Task Force of the ACM, IEEE-Computer Society, and AAAI commenced work in 2021 to revise the Computer Science curricular guidelines that were last updated in 2013. Planned for publication in 2023, the revised guidelines (CS2023) cover curricular content and curricular practices. Curricular content includes updates to the CS2013 knowledge areas, a new sunflower model of what constitutes core computer science topics, a proposal for packaging knowledge areas into courses, and a competency model of the curriculum. Curricular practices cover computer science program design and delivery issues, including social aspects, professional practices, and programmatic considerations. In the special session, the latest CS2023 draft will be presented and feedback solicited. The session is targeted toward educators, administrators, and professionals interested in computer science curricular issues.

Dispositions are cultivable behaviors desirable in the workplace. Examples of dispositions are being adaptable, meticulous, and self-directed. The eleven dispositions described in the CC2020 report should not be confused with the professional knowledge of computing topics, or with skills, including technical skills, along with cross-disciplinary skills such as critical thinking, problem-solving, teamwork, or communication. Dispositions, more inherent to human characteristics, identify personal qualities and behavioral patterns important for successful professional careers.

Which code-tracing concepts are introductory programming students likely to learn from classroom instruction and which ones need additional problem-solving practice to master? Are there relationships among programming concepts that can be used to build adaptive assessment instruments? To answer these questions, we analyzed the data collected over several semesters by a suite of code-tracing tutors called problets, that administered pre-test, practice, post-test protocol. Each tutor covered a single programming topic, which consisted of 9-25 concepts. For each concept, we used the pretest data to calculate the probability that students knew the concept before using the tutor. Using a weighted average of the concept probabilities, we found that students had learned some topics more than others: if/if-else (0.85), function behavior (0.76), arrays (0.73), while (0.7), for (0.69), switch (0.67), and debugging functions (0.55). Some of the concepts on which students needed additional practice included bugs, nested loops and back-to-back loops. Expressions, even when used in novel contexts, were not challenging for students. We built a Bayesian network for each topic based on conditional probabilities to discover the concepts that must be covered, and those whose coverage is redundant in the presence of other concepts. A strength of this empirical study is that it uses a large dataset collected from multiple institutions over multiple semesters. We also list threats to the validity of the study.

No abstract available.

We developed a suite of tutors called solvelets to help students learn the process of programming. They are based on the hypothesis that problem-solving is asking the right questions in the right sequence. The tutors use a series of questions to step the student through all three stages of program development: algorithm formulation, program design and writing code. So, the tutors help students start from a problem statement and end with a complete and correct program for the problem. At each intermediate step, the tutors provide immediate feedback at multiple progressively specific levels until they bottom out with the correct answer. So, the proficiency of a student is determined not by whether the student solves the problem correctly, but by the number of attempts the student takes to complete each step: the more the attempts, the less proficient the student. The tutors reify the steps for writing each control statement and scaffold the student through writing it one step at a time. We describe the tutors and the process of programming scaffolded by them. The tutors are currently available for C++ and Java and cover expressions, selection statements and logic-controlled loops. They are available for free for educational use at solvelets.org.

A Joint Task Force of the Association for Computing Machinery (ACM), the Institute of Electrical and Electronics Engineers~-~Computer Society (IEEE-CS), and Association for the Advancement of Artificial Intelligence (AAAI) was constituted in early 2021 to begin the decennial process of revising the Computer Science curricular guidelines, which were last released as Computer Science Curricula 2013 (CS2013). This special session will present the first draft of the revised curricular guidelines, currently referred to as CS202X, and solicit feedback. The CS202X draft will include revisions to CS2013 Knowledge Areas, a proposed competency model being incorporated into the curricular guidelines, and other updates. Targeted towards educators, administrators and others interested in Computer Science curricular issues, this session will be led by the co-chairs and members of the CS202X Steering Committee as part of their process to engage the community and solicit feedback.

Since the early 21st century, ABET's accreditation criteria have focused on learning outcomes (what students learn) rather than what professors teach. Such accreditation criteria bring to bear the need for programs to establish clear learning objectives and assessment processes that ensure that program graduates have the requisite technical and professional preparation. To this end, ABET defines student outcomes as "what students are expected to know and be able to do by the time of graduation," further noting that these outcomes "relate to the knowledge, skills, and behaviors that students acquire as they progress through the program." With the recent release of Computing Curricula 2020 (CC2020), the competencies of computing program graduates have received additional attention. CC2020 describes competency as "comprising knowledge, skills, and dispositions that are observable in accomplishing a task within a work context."

In this article, we leverage ideas from the theory of coevolutionary computation to analyze interactions of students with problems. We introduce the idea of informatively easy or hard concepts. Our approach is different from more traditional analyses of problem difficulty such as item analysis in the sense that we consider Pareto dominance relationships within the multidimensional structure of student–problem performance data rather than average performance measures. This method allows us to uncover not just the problems on which students are struggling but also the variety of difficulties different students face. Our approach is to apply methods from the Dimension Extraction Coevolutionary Algorithm to analyze problem-solving logs of students generated when they use an online software tutoring suite for introductory computer programming called problets . The results of our analysis not only have implications for how to scale up and improve adaptive tutoring software but also have the promise of contributing to the identification of common misconceptions held by students and thus, eventually, to the construction of a concept inventory for introductory programming.

This Research to Practice Full Paper presents the experiences and lessons learned from five programs that provide financial awards and a holistic student support structure to low-income, academically talented students in Science, Technology, Engineering, and Mathematics (STEM). This report synthesizes the experiences of a diverse set of institutions, both public and private, that vary in size and geographic location. We have experience supporting students from a range of disciplines with an emphasis on students studying Computer Science. The goals of this work are to (1) outline the decisions that must be considered when designing a financial award program; (2) describe the interventions we have implemented and underline the institutional contexts that have led to their success; (3) describe the unique challenges posed by the COVID pandemic; and (4) highlight key elements necessary for successful program implementation. We specifically discuss the challenges we have encountered when implementing existing best practices. We report observations and results, some of which buttress those reported in the literature. Our work is intended to serve as a guide for educators who wish to implement programs to support students from financially disadvantaged and/or historically marginalized groups. By sharing our experiences and pain points, we hope to make it easier for them to design and implement effective programs adapted to their institutional needs and contexts.

Competency-based learning has been a successful pedagogical approach for centuries, but only recently has it gained traction within computing education. Building on recent developments in the field, this working group will explore competency-based learning from practical considerations and show how it benefits computing. In particular, the group will identify existing computing competencies and provide a pathway to generate competencies usable in the field. The working group will also investigate appropriate assessment approaches, provide guidelines for evaluating student attainment, and show how accrediting agencies can use these techniques to assess the level of competence reflected in their standards and criteria. Recommendations from the working group report are intended to help practical computing education writ large.

This Birds-of-a-Feather session is for anyone interested in the NSF Scholarships in STEM (S-STEM) program, including current and former Principal Investigators (PIs) and those planning to apply. The S-STEM program funds scholarships and activities to support low-income, academically talented students in STEM. Any institution of higher education may apply, and the program supports a variety of projects. Designing and implementing a successful S-STEM project is challenging. The goal of this session is to catalyze a community of practice for S-STEM PIs. It will provide an opportunity to discuss lessons learned and best practices for proposal writing, project implementation, and providing student support. Specific topics to be discussed include the following: (1) Understanding the solicitation requirements and common proposal mistakes; (2) Scholar recruitment and data-driven approaches for selection; (3) Cohort building including activities for students from different majors or class years and integration of new students into existing cohorts; and (4) Remediation strategies including proactive interventions and peer support. Session leaders will introduce each topic; participants will then join a breakout group discussion of one topic. Lastly, participants will be invited to join a Slack workspace dedicated to S-STEM best practices and lessons.

Research Category/Full paper) - We analyzed the data collected in a Computer Science I course to quantify the relationship between programming projects, code-tracing assignments and course grade when online tests and closed lab instruction were used in the course. We found that completion of programming projects was positively and moderately correlated with course grade; each completed project contributed nearly one sign grade to the course grade; the grade of students who had completed at least a given number of projects was four sign grades better than of those who had not and the difference was statistically significant; the mean course grade ranged from F for those who had completed 1 or fewer projects to A- for those who had completed 9 or more projects; and completion of later projects was indicative of higher grade in the course. Similarly, completion of code-tracing assignments was positively, but weakly correlated with course grade; the grade of students who had completed at least a given number of assignments was one letter grade better than of those who had not, and the difference was statistically significant; and the mean course grade ranged from C- for those who had completed 6 or fewer assignments to B+ for those who had completed 11 or more assignments. Concurrence among the course objectives, classroom instruction, assessment techniques, programming projects and assignments may be a pre-requisite for obtaining the results of this study.

In a Parsons puzzle, the student must re-assemble the lines of a program that are provided in scrambled order, and eliminate any distracters included within the scrambled lines of code. We investigated two issues in terms of whether they helped students solve the puzzles better: 1) Would it be better to present distracters and the lines of code of which they are a variant paired together or randomly separated apart? 2) Would telling students that they would not be penalized for any mistakes they make before submitting a complete solution for the first time enable them to get closer to the correct solution when they first submit it? We conducted a controlled study over five semesters and used ANOVA to analyze the data collected from introductory programming students who solved Parsons puzzles on if-else statements. We found that when students were told that their puzzle-solving actions before the first submission would not be penalized, they took significantly more exploratory actions. But, their solution was not significantly closer to being correct. So, trial-and-error exploration was no better for solving Parsons puzzles than deliberate approach. Presenting distracters paired together with the original line of code was found to be beneficial only on the longer puzzle.

A recent study conducted at the University of Oklahoma, a large research university attempted to use the grades on initial Computer Science courses to predict the success of Computer Science majors. We attempted to reproduce this study in a mid-sized liberal arts institution. We analyzed 15 years of data of students (majors as well as non-Computer Science majors) who had taken introductory Computer Science courses. We found that the better the grade on Computer Science I, the introductory course in the major, the better the cumulative GPA of the student upon graduation, and this applied to Computer Science majors as well as non-majors. All the students who had successfully graduated with a Computer Science degree had earned at least a C grade in the first three required courses: Computer Science I, Computer Science II and Data Structures. When we considered grades on six of the required courses in the Computer Science sequence, we found that students generally earned the same or lower grade on each subsequent course. Therefore, the performance of Computer Science majors on the first three courses in the required course sequence can reasonably be used as predictors of their success in the major. Finally, we found that Math SAT score was a good predictor of student success in Computer Science I as well as obtaining an undergraduate degree regardless of the major. Our study generalizes the results of the previous study and strengthens the results by finding that they are statistically significant.

We wanted to cover mobile computing in our curriculum without incurring the costs of adding a new course to the curriculum or hiring a new instructor to the faculty roster. We did so through collateral learning, by incorporating mobile computing into the projects of two existing upper-level Computer Science courses: Organization of Programming Languages and Artificial Intelligence. We recount our experience using collateral learning of mobile computing over the last four years: motivation, logistics, course-specific details, reception by students, results of course evaluations, challenges faced and solutions devised. Our experience affirms that collateral learning is an excellent option for incorporating emerging topics such as drone programming, cybersecurity and parallel computing into the curriculum at resource-strapped Computer Science departments.

A broad review of research on the teaching and learning of programming was conducted by Robins et al. in 2003. Since this work there have been several reviews of research concerned with the teaching and learning of programming, in particular introductory programming. However, these reviews have focused on highly specific aspects, such as student misconceptions, teaching approaches, program comprehension, potentially seminal papers, research methods applied, automated feedback for exercises, competency-enhancing games, and program visualisation. While these aspects encompass a wide range of issues, they do not cover the full scope of research into novice programming. Some notable areas that have not been reviewed are assessment, academic integrity, and novice student attitudes to programming. There does not appear to have been a comprehensive review of research into introductory programming since that of Robins et al. It is therefore timely to conduct and present such a review in order to gain an understanding of the research focuses, to highlight advances in knowledge since 2003, and to indicate possible future directions for research. The working group will conduct a systematic literature review based on the guidelines proposed by Kitchenham et al. This research project is well suited to an ITiCSE working group as the synthesis and discussion of the literature will benefit from input from a variety of researchers drawn from different backgrounds and countries.

Performance on Parsons puzzles has been found to correlate with that on code-writing exercises. Parsons puzzles are preferred by students over alternative programming tasks. In order to make Parsons puzzles widely available to students in the introductory programming course, we developed a tool that administers the puzzles in C++, Java and C#, called epplets. Our design of the tool improves upon the work done by earlier researchers in several ways: students rearrange lines of code rather than program fragments; they get credit based on the number of actions they take to reassemble the code; they get feedback that helps them fix their incorrect answer; and the tool adapts to the needs of the student. The tool runs as a Java Web application. We describe our experience using the tool for two years: how it benefited the students; the revisions made to address the feedback provided by the users; and our plans for future work. We found that practicing with the tool helped reduce the time and actions with which students solved successive puzzles.

We conducted a study to investigate how learning styles related to the introductory Computer Science I course: the learning styles of the student population that took the course; course completion rates as they related to learning styles; relationship between the final grade in the course and learning styles; and the correlation between project and assignment completion rates and learning styles. We found that students in the course preferred active, sensing, visual and sequential learning styles. There was no significant difference in the course completion rates along any dimension that could not be explained based on prior preparation. Reflective and intuitive learners earned better grades in the course than active and sensing learners. While male students earned better grades than female students and traditionally represented students earned better grades than underrepresented students, these differences may be attributable to differences in prior preparation. In this context, SAT scores were found to be better predictors of student grades in Computer Science I than high school GPA. Programming projects in the course favored reflective students whereas online assignments favored sensing and sequential learners - unsurprising, since the design of these course instruments were congruent with the definition of these learning styles. It is hoped that this study provides insight into the types of course activities that might be incorporated into Computer Science I to accommodate the different learning styles of students.

This paper identifies design guidelines for the application of evolutionary techniques to the task of generating practice problems for learners in an Intelligent Tutoring System. To this end, we designed experiments that progressively incorporated an increasing number of the characteristics we expect to find in our target application. These features included noisy evaluations, overspecialization, and the need to mitigate user fatigue resulting from interactive evaluations of practice problems. As we did so, we evaluated the potential of recent breakthroughs in coevolutionary learning theory and identified the tradeoff specific to educational applications.

A study was conducted to reproduce the results of an earlier study on the effectiveness of visualization for learning expression evaluation in a problem-solving software tutor on arithmetic expressions. In the current reproducibility study, data was collected from a software tutor on assignment expressions over six semesters. ANOVA analysis of the amount and speed of learning was conducted with treatment, sex and racial groups as fixed factors. Results include that visualization helped the students learn significantly more concepts, whether the students needed to use the tutor or benefited from using the tutor. However, it only benefited the less-prepared students. It did not help the students learn faster. It benefited both the sexes and traditionally represented as well as underrepresented groups. The current study confirmed almost all the results from the previous study, albeit for a harder topic. One reason why visualization was found to be effective in both these studies may be that the same visualization scheme was used by the students to both view feedback and construct their answers.

What are the concepts in introductory programming that are easy/hard for students? We propose to use Dimension Extraction algorithm (DECA) inspired by coevolution and co-optimization theory to answer this question. We propose and use the metrics of informatively easy/hard concepts to identify programming concepts that are solved correctly by the most "dominated student" versus solved incorrectly by the most "dominant student". As a proof of concept, we applied DECA to analyze the data collected by software tutors called problets used by introductory programming students in Spring 2014. We present the results, i.e., informatively easy/hard concepts on a dozen different topics covered in a typical introductory programming course. It is hoped that these results will inform programming instructors on the concepts they should (de)/emphasize in class. They will also contribute towards creating a concept inventory for introductory programming.

We conducted a study to evaluate the effect of using software tutors on the self-efficacy of students - in particular, whether the type of activity covered by the software tutor correlated with any improvement in self-efficacy after using the tutor along the levels of Bloom's taxonomy; and whether any differential effects could be observed in the improvement of self-efficacy among the sexes and racial groups. The study was conducted over four semesters using two different software tutors. The collected data was analyzed using paired sample t-test and 2 × 2 ANOVA for three different sets of students - those who used the tutor, those who needed to use the tutor and those who learned one or more concepts by using the tutor. We found that if a significant difference was found among taxonomic levels of self-efficacy after using a software tutor, the improvement was statistically significantly greater on the taxonomic level directly relevant to the topic/activity of the tutor than any other level on Bloom's taxonomy except comprehension. Such a difference was found at least among those who actually learned one or more concepts using the tutor, if not everyone who used the tutor. In most cases, the improvement in self-efficacy resulting from the use of the software tutor was indistinguishable across sexes and racial groups.

The understanding and transfer of mathematical expressions in a programming language is essential for studying engineering. Our experience of recent years shows that students increasingly struggle in reading, understanding and eventually transferring mathematical expressions to a programming expression so we need to provide many relevant tasks to help students overcoming this challenge. From educational psychology as well as our application-oriented research we know that the success of learning can be operationalized with objective, reliable and valid tasks only. Thus, we perceive tasks not only to be a training but also to be an empirical measuring instrument. However, tasks that are not theory driven are likely to be not suitable to measure the learning outcomes of students in an objective, reliable and valid manner. We'll present two different successful approaches that were developed individually at two different locations. Both propose a formal task description as well as a process model for task development that will facilitate the transfer of mathematical expressions to programming expressions. The tasks generated are valid and allow objective and reliable measurements of the students' learning outcomes. Our proposed tasks can also be used to iteratively improve and individualize introductory programing courses.

Educational performance indicators are being considered or implemented in different ways by institutions and governments in different countries. What impact is this likely to have on computing education?

An earlier study had found that solving code tracing problems helped improve code writing skills of students. But, given the instruments used in the earlier study, the improvement in code writing pertained primarily to language syntax. A follow-up within-subjects controlled study was conducted to investigate whether solving code tracing problems could help improve code writing skills pertaining to the semantics of a program. In the study, students were asked to write code for a control and a test problem both before and after a problem-solving session on code tracing. Increase in the score from pre-quiz to post-quiz was treated as improvement in code writing attributable to code tracing. Repeated measures ANOVA was used to analyze the data collected over four semesters. A statistically significant improvement in code writing skills pertaining to program semantics was observed on the test problems, but not on control concepts in the control problems. The improvement in code-writing skills as they pertain to program semantics accrued to the students who scored 90% or more on code-tracing problems in this study. Finally, the transfer in learning from code-tracing activities to code-writing skills may be near as well as far.

Interactive activities in textbooks and online courses are no longer just decorative, but have become compelling tools for engaging students. This BOF, lead by professors and authors with experience in designing complex interactive tutoring materials, invites interested instructors and authors to discuss best practices in designing activities, integrating them into courses, and measuring outcomes.

A controlled study was conducted to evaluate the effectiveness of providing visualization as part of feedback in a problem-solving software tutor on arithmetic expression evaluation. Data was collected over six semesters from multiple institutions. ANOVA analysis of the collected data was conducted in three stages. Statistically significant results include that visualization helped students learn more concepts; visualization did not improve the speed of learning; the benefits of visualization accrued primarily to less-prepared students; and visualization may affect different demographic subgroups differently. Incidental results include that there was no difference among demographic groups (male/female, traditional/underrepresented, Computer Science/non-CS) in the number of concepts learned using the software tutor, although some groups (female, underrepresented) were less-prepared before using the tutor than their counterparts, and some groups learned concepts with fewer practice problems (male, traditionally represented) than their counterparts. Concurrence of the results obtained whether the analysis was conducted based on pre-condition (need) or post-condition (benefit) of using the tutor strengthens the claims made as a result of this study.

Test anxiety is known to negatively affect test performance. Having students write about their testing worries before taking a test was recently shown to improve test performance by reducing test anxiety. We conducted a controlled study to replicate this result in the context of students using online Computer Science tutors unsupervised and on their own time. Instead of using open-ended expressive writing exercise, we used a multiple-choice questionnaire that addressed student anxiety. During the study, we collected data from two tutors on advanced programming concepts over three semesters. We did ANOVA analysis of the number of problems solved, score per problem and time spent per problem with treatment as the between-subjects factor. The test group solved significantly more problems and scored more points per problem than the control group on one of the two tutors, but not the other. We discuss a possible explanation for the result and the significance of addressing test anxiety for broadening participation in Computer Science.

Short, automatically-assessed programming exercises, and other types of short practice problems, are a useful way to introduce and reinforce concepts and techniques in introductory programming courses. When delivered over the web, they allow students to learn and practice, with immediate feedback, at any time and place where they have access to a web browser. However, such exercises do not seem to be as widely used as they could be. Similarly, there is not a lot of literature on the effectiveness of these types of problems. The purpose of this BOF is to bring together users (and potential users) of programming exercises with developers of programming exercise systems to discuss how exercises could be used more widely and effectively. Possible discussion topics include: What features are absolutely essential for faculty to consider adoption? What are the major obstacles preventing more widespread adoption? Are faculty willing to share their exercises under an open/non-commercial license? Should exercises best used for extra practice, as graded assignments, or both?

For over 40 years, the ACM and IEEE-Computer Society have sponsored international curricular guidelines for undergraduate programs in computing. The rapid evolution and expansion of the computing field and the growing number of topics in computer science have made regular revision of curricular recommendations necessary. Thus, the Computing Curricula volumes are updated on an approximately 10-year cycle, with the aim of keeping curricula modern and relevant. The latest volume in the series, Computer Science Curricula 2013 (CS2013), is due for release in the Fall of 2013. This panel seeks to inform the SIGCSE community about the final version of the report, provide insight on interpreting the CS2013 guidelines, and give guidance regarding how the guidelines may be implemented at different institutions.

This workshop will help participants introduce problem-solving exercises into their introductory C++/Java/C# programming courses. The purpose of problem-solving exercises is two-fold: they supplement classroom instruction and complement the programming projects traditionally assigned in the course. The benefits of problem-solving exercises are many: they improve students' comprehension of programming constructs, their self-confidence, especially that of female students, and their coding skills. In this workshop, problets (www.problets.org) will be introduced as a tool for problem-solving exercises. They parameterize problems to deter plagiarism; provide step-by-step explanation of the correct solution to each problem, which helps students learn; and adapt to the learner's needs. They are a web-based service freely available for educational use. Problets have been rigorously evaluated, and have been adopted and used by dozens of instructors every semester since 2004. The workshop is appropriate for instructors of introductory C++/Java/C# programming courses in Computer Science or engineering. Participants are asked to bring a WiFi-enabled laptop to the workshop for hands-on experience.

A study was conducted to find out who needed online problem-solving tutors and who benefited from using them. In particular, the study focused on whether there were any significant differences between male and female students and between traditionally represented and under-represented racial groups. Data collected by two Computer Science tutors over multiple semesters was analyzed. The only significant differences found between sexes and racial groups were when female students practiced significantly more concepts because they had solved significantly fewer problems during pre-test, or when they demonstrated greater pre-post increase in score because they had scored significantly less on the pre-test. In both the cases, the tutors helped female students overcome differences in prior preparation vis-a-vis male students. No difference was found between the sexes or racial groups on the number of practice problems solved per practiced concept. Finally, students needed and benefited from the tutors in the same proportion, regardless of sex or racial group.

A study was conducted to find out whether solving code tracing problems helps improve code writing skills of students. Students were asked to answer a code writing quiz before and after a problem-solving session on code tracing. Increase in the score from pre-quiz to post-quiz was treated as improvement in code writing attributable to code tracing. Repeated measures ANOVA was used to analyze the data collected over three semesters. The results of the study are that whenever a statistically significant difference was observed between pre-quiz and post-quiz scores, post-quiz mean score was higher and standard deviation was smaller, confirming better student performance on post-quiz than on pre-quiz. The improvement in code writing pertained primarily to language syntax. The improvement was observed even among the students who scored 100% on code tracing problems. Finally, the students whose code writing skills improved due to code tracing had spent 10% more time on code tracing than the others.

A mid-career review is presented, of how the teaching of Computer Science I has changed for this instructor over the last two decades. The content of the course has evolved to include algorithm development and program design. Assessment in the course has gone online and moved away from testing how clever the student is, to how much the student has learned in the course. Professional practices are now covered that help students understand and incorporate preferred practices of the discipline. Changes incorporated into the pedagogy include going from using anthropomorphic and ad-hoc to discipline-specific and consistent vocabulary, and from writing code in the class like an experienced programmer to writing it to suit a beginning learner. It is hoped that this review will help new Computer Science I instructors avoid some misconceptions with which this instructor started out.

The effect of interleaving an alternate task during tutoring and testing was studied in the context of an online problem-solving software tutor. A controlled study was used and ANOVA was used for data analysis. It was found that introduction of the alternate task did not result in any difference in the scores of control and test groups during testing. It did not promote greater cognitive or affective learning during tutoring. Some possible reasons for the negative results are discussed.

A controlled study was conducted to detect stereotype threat on harder topics in introductory Computer Science. Students in the control group were asked to identify their demographic information before taking the test whereas students in the experimental group were asked to do so after completing the test. So, the control group was indirectly reminded of stereotypes before taking the test, when it could affect performance on the test, whereas the experimental group was reminded after the test when it could not affect test performance.

No abstract available.

We analyzed the data collected over 7 semesters by a single Computer Science software tutor to study the differences between the sexes and races on their prior self-confidence, prior preparedness and their assessment of the tutor. We found that when there was a statistically significant difference in the prior self-confidence of male and female students, female students had lower prior self-confidence than male students, in spite of the fact that there was no significant difference in the prior preparedness of male and female students. The prior self-confidence of female students in Computer Science may be improving with increasing enrollment. Whenever there was a statistically significant difference among racial groups, positively stereotyped racial groups were better prepared and had higher prior self-confidence than the traditionally under-represented racial groups. Whenever there was a statistically significant difference between the sexes in the assessment of the tutor, female students assessed the tutor more favorably than male students. When there was a statistically significant difference between racial groups, under-represented racial groups assessed the tutor more favorably than positively stereotyped racial groups. When there was a statistically significant difference in how developer's students assessed the tutor versus how other adopters' students assessed it, assessment by developer's students was more positive than that by students of other adopters.

No abstract available.

The majority of earlier studies have found no positive effect of closed labs on student performance or retention in Computer Science I. Since these studies used written tests to assess student performance, their results may have been affected by a mismatch between what was taught in closed labs and what was assessed in the written test. On the other hand, online tests that involve writing and debugging programs assess the very knowledge and skills taught in closed labs. So, we conducted a study to evaluate the effect of closed labs on student performance and retention when they are combined with online testing in the course.

This panel will consider the issues related to undergraduate research in computer science from a global perspective. Panelists from different countries and varied backgrounds will relate their experiences in conducting such research.

Traditionally, educational interventions in Computer Science have been studied for their effect on entire classes, or specific demographic groups. But, in our studies, we have found that often, significant interactions exist among demographic groups. Treating demographic groups as homogeneous groups when evaluating educational interventions in Computer Science could miss subtle interactions among the groups.

We incorporated animation of the data space into a web-based tutor for solving problems on C++ pointers and made the tutor available to students. In evaluation of the tutor, we found that data space animation indeed helps students learn the semantics of pointers. But, it is no more effective at this than text explanation of the step-by-step execution of the program.

Our goal in this study was to compare the effectiveness of displaying the open student model as a set of skillometers versus concept maps. The data suggests that concept maps are significantly more effective than a set of skillometers when answering questions that require synthesizing an overview of the topic.

We examined whether using problem-solving software tutors in Computer Science I can help improve the self-confidence of female students. We analyzed the data collected by five software tutors in spring 2006. We found that 1) the self-confidence of female Computer Science I students before using the software tutors was in many cases lower than that of male students, as has been stated in prior literature; 2) Using problem-solving software tutors improved the self-confidence of female students to be on par with that of male students when female students started with lower prior self-confidence. Since researchers have suggested that self-confidence is one of the factors contributing to the shrinking pipeline, problem-solving software tutors can be used to improve the retention of female students in Computer Science.

We used megavariate analysis techniques to analyze the post-practice reflection data collected by our programming tutors. We found that correctly solving the problem is closely associated with the number of attempts needed by the learner to identify the underlying concept during reflection.

No abstract available.

No abstract available.

No abstract available.

This tutorial will present how instructors can incorporate LEGO robots into their AI course with minimal time, effort, and resource commitment. The tutorial will: 1) cover the principles behind using robots for knowledge-based, open-laboratory projects; 2) share the design and details of several projects; 3) work the participants through sample solutions to a few of the projects, preferably hands-on; and finally, 4) discuss alternatives to LEGO and knowledge-based projects in AI. Participants will be able to apply the tutorial materials immediately to their AI course.

Researchers have been developing online tutors for various disciplines, including Computer Science. Educators are increasingly using online tutors to supplement their courses. Are online tutors effective? Can they help students learn? If so, what features contribute to their effectiveness? We will examine these questions in the context of an online tutor that we developed for introductory Computer Science. The tutor is designed to help students learn expression evaluation in C++/Java.We evaluated the tutor over several years, in multiple sections of Computer Science I each year. We used controlled tests with differential treatments, and used pre and post-tests to evaluate the effectiveness of the tutor. Our results show that online tutors indeed help students learn. Students who use the tutor for practice learn better than those who use a printed workbook. Students who receive both graphic visualization and text explanation learn better than those who receive only graphic visualization. Students who use graphic visualization learn better than those who receive no explanation. These results will be of interest to both developers and users of online tutors.

Researchers and educators have been developing tutors to help students learn by solving problems. The tutors vary in their ability to generate problems, generate answers, grade student answers, and provide feedback. At one end of the spectrum are tutors that depend on hand-coded problems, answers, and feedback. These tutors can be expected to be pedagogically effective, since all the problem-solving content is carefully hand-crafted by a teacher. However, their repertoire is limited. At the other end of the spectrum are tutors that can automatically generate problems, answers, and feedback. They have an unlimited repertoire, but it is not clear that they are effective in helping students learn. Most extant tutors lie somewhere along this spectrum.In this article we examine the feasibility of developing a tutor that can automatically generate problems, generate answers, grade student answers, and provide feedback. We investigate whether such a tutor can help students learn. For our study, we considered a tutor for our Programming Languages course, which covers static and dynamic scope (i.e., static scope of variables and procedures, dynamic scope of variables, and static and dynamic referencing environment of procedures in the context of a language that permits nested procedure definitions). The tutor generates simple and complex problems on each of these five topics, solves the problems, grades the students' answers, and provides feedback about incorrect and missed answers. Our evaluation over two semesters shows that the feedback provided by the tutor helps improve student learning.

No abstract available.

In order to facilitate problem-based learning in our Programming Languages course, we developed a tutor on static and dynamic scope. Static scope includes the scope of variables, the referencing environment of procedures and the scope of procedure names in a language that permits nesting of procedure definitions (e.g., Pascal, Ada). Dynamic scope includes the scope of variables, and the referencing environment of procedures. In this paper, we will describe the design of our tutor, and present the results of evaluating it for two semesters in our Programming Languages course.

We have been using robots in our artificial intelligence course since fall 2000. We have been using the robots for open-laboratory projects. The projects are designed to emphasize high-level knowledge-based AI algorithms. After three offerings of the course, we paused to analyze the collected data and to see if we could answer the following questions: (i) Are robot projects effective at helping students learn AI concepts? (ii) What advantages, if any, can be attributed to using robots for AI projects? (iii) What are the downsides of using robots for traditional projects in AI? In this article we discuss the results of our evaluation and list the lessons learned.

Here, we present two user interfaces we developed for a tutor on debugging programs. The second interface is reified with respect to the first, and is hence, better at capturing student misconceptions and promoting the development of an accurate mental model in the learner. We discuss the rationale behind the interface, its significance, and its impact on the maintenance of the student model. We also describe the design and implementation of the reified interface.

We have developed a tutoring system for the parameter passing mechanisms discussed in a typical Comparative Programming Languages course, viz., value, result, value-result, reference and name. The tutor helps students better understand these parameter passing mechanisms by administering problems for them to solve and providing instant feedback on their solution. In this paper, we will describe the design and features of the tutor. We will also discuss a test that we conducted to evaluate the effectiveness of using the tutor, and present its results. The test confirmed our hypothesis that using the tutor would result in a systematic improvement in the learning of our students. This tutor may be used in the Comparative Programming Languages course as well as Computer Science I.

Computer Science as an academic discipline should be guided not only by the "state of the art", but also by the "state of the practice"[1]. Over the last few years, Internet/Web has been undeniably the most "high profile" practice of computing. Yet, Computer Science curricula across the country have not kept up with this development - not many schools are offering courses, concentrations and/or majors that identify the Internet/Web as the central principle, and address its issues and needs.In this panel, the panelists will share their experience designing courses and concentrations to address this need, and present their vision for what an Internet-related Curriculum should include: the courses, the technologies, and the overarching themes. The viewpoints presented here are quite diverse: arguing in favor of Internet-related coursework for majors versus non-majors, as a course/minor/major, as an across-the-curriculum theme, as an interdisciplinary endeavor, as an introductory course versus a capstone course, and from the points of view of a community college, four-year institutions and a graduate institution. We hope that these diverse viewpoints will foster vigorous discussion at the panel about the place of Internet-Computing in the Computer Science curriculum, and its design.

Traditionally, pointers, and their interaction with scope in C++ have been a source of frustration and confusion for students in our Computer Science II course. Since problem-solving is known to improve learning [6], we set out to develop software that would help our students better understand these concepts by repeatedly solving problems based on them.In this paper, we will first describe the design and features of this software. We conducted tests in two sections of our Computer Science II course this fall to evaluate the effectiveness of using this software. The results have been very encouraging: the class average in both the sections increased by 100% from the pretest to the post-test. We will also present the design and results of these tests.

Many curricular guidelines, such as the Recommended Curriculum for Computer Science at Liberal Arts Colleges [4], suggest that students be exposed to many different programming paradigms (e.g., imperative, functional, object-oriented, declarative) in the undergraduate curriculum. Some institutions believe that students should have early exposure to many paradigms, often as early as the first two courses.Many institutions emphasize object-oriented programming early in the curriculum. Some also include functional programming. Imperative topics are often covered in courses that emphasize object-oriented or functional issues. Where does declarative programming fit? Sometimes not until an upper-level language paradigms course or artificial intelligence course. Sometimes it never fits, at least not explicitly.

Solving problems is an integral part of learning in Computer Science. In order to provide students with a vast supply of problems with which to practice, we propose to use applets that automatically generate problems. In this paper, we first discuss the capabilities required of such applets, and then, present the design and features of an applet we have developed to automatically generate problems on static scope in Pascal.

How do on-line tests compare with written tests in Computer Science I? Do students who do well in written tests also do well in an online test? Is an online test better or worse than a written test at assessing the problem-solving skills of a student? This paper summarizes the answers to these questions that we found during our switch from written to on-line testing.

No abstract available.

This working group report proposes a set of criteria for effective design and use of educational multimedia.These criteria are organized around the mutually interdependent roles of teacher, learner, and technology in the educational process.The criteria constitute a first attempt, grounded in educational theory, at a response to important pedagogical and social issues that have been raised with respect to traditional instructional approaches.Some illustrations of potential uses of multimedia are discussed.Finally, recommendations are offered regarding public policy and institutional activities to promote development and dissemination of well-designed multimedia materials and equitable access to the technology necessary for their use.

We propose a fork diagram as a visual representation of the algorithm for binary selection. Among other things, fork diagrams can be used to teach students how to write correctly nested if-else statements, analyze nested selection code, appreciate the problem of dangling else, and understand short circuit evaluation of conditions with boolean operators (and and or). We have used C code to illustrate concepts, although fork diagrams can be used for any high level language.