In Software Engineering courses, students typically work in groups to collaborate on team projects. However, contributions to these projects are not always fairly distributed across team members and sometimes individuals are less consistent at making contributions over an academic term. This paper investigates discrepancies in the consistency of each member's contributions to software engineering teams. We studied 94 software engineering teams from undergraduate software engineering courses and gathered evidence of each member's (n=465) activities on GitHub, including commits, pull requests, issues, code reviews, and comments. We analyzed whether each member exhibited any of these activities for each week of the academic term to measure the consistency of their contributions. We calculated the (dis)parity of how consistently each team member made contributions using the Gini coefficient, which measures (in)equity within groups. Each team also self-reported their frequency of conflicts and cohesiveness toward a common goal. This exploratory study investigates the relationship between parity among teammates' consistency and the team's overall cohesiveness and lack of conflict. While parity of consistency varied between teams, there was no association between it and teams' cohesiveness or conflicts. However, median team consistency was a significant predictor of team cohesiveness and a lack of conflicts.

Course-based Undergraduate Research Experience (CURE) is a pedagogy for engaging an entire class in the scientific exploration of real research problems with unknown solutions that have impact beyond the classroom. Since CUREs originated in biological sciences, there are unique challenges to adapting the CURE model for computer science. We designed a CURE by aligning its principles with entrepreneurial mindset (EM) and software development practices by applying them in a programming project. This experience report introduces the design and adoption of a CURE project for an introductory programming class (CS1).

There is an increasing variety of tools available to support students outside of the classroom as they learn how to program. However, broadening participation in computer science will also require these tools to reinforce inclusivity, diversity, and equity. To address these needs, two Hispanic-Serving Institutions, California State University, Chico (Chico State) and University of California, Santa Barbara (UCSB), are collaborating to produce video tutorials featuring students from historically marginalized communities. In these videos, students demonstrate uses of programming concepts in contexts that relate to their lives and interests. This experience report describes the development of the videos and associated programming practice problems. In a mixed-method observational study (n=188) of CS1 courses at the two universities, we investigated students' engagement with the videos and practice problems. Preliminary findings revealed that when the videos and practice problems were provided as separate links, students tended to watch the videos but not engage in the practice platform. Consequently, this paper also describes the development of Codewit.us, which integrates video lessons with an adaptive drill-and-practice programming interface. We found that when presented with the integrated interface, students were more likely to engage with practice in tandem with watching videos (instead of just watching videos without practice, as previously observed). The paper also describes thematic analysis of students' qualitative feedback on the videos and practice problems.

To broaden participation in computing, learning materials should relate to a diverse spectrum of student perspectives. This paper introduces Codewit.us, an online platform for under-represented minority (URM) students in computing to share examples of how coding fundamentals can be demonstrated in contexts relevant to their own lives. Codewit.us features brief video demonstrations by Black, Indigenous, and People of Color (BIPOC) and female students, paired with interactive drill-and-practice coding problems on matching programming concepts.

With an increasing emphasis on unit testing in computer science curricula, we examined students' work on testing assignments to investigate their adoption of test smells---practices that indicate potential problems in unit tests. We discovered three common causes of test smells in students' unit tests: multiple member function calls, multiple assertions, and conditional logic. We also explored how each might be associated with test inaccuracies.

Computing has, for many years, been one of the least demographically diverse STEM fields, particularly in terms of women's participation [2] and those from minoritized racial and ethnic groups. In the case of higher education, one of the most powerful sites of intervention is the classroom. The last decade has seen a proliferation of research exploring new teaching techniques and course sequencing and their effect on the retention of students who have historically been excluded from computing. This research suggests interventions and practices that can affect the inclusiveness of the computer science classroom and potentially improve learning outcomes for all students. But research needs to be translated into practice, and practices need to be taken up in real classrooms. The goal of this working group (WG) is to conduct a systemic "state-of-the-art" review of recent empirical studies of teaching practices that have some explicit test of the impact on women (or other under-represented groups) in computing. The WG will produce an annotated bibliography and a report that distills the research into specific, actionable practices.

Version control systems are widely used in industry and form a staple of modern software development, and this panel addresses the benefits and difficulties of introducing version control in education. Version control systems such as Git, and software development platforms such as GitHub, Bitbucket and GitLab are industry standards for which there is an increasing appeal within education. In general, students and teachers are motivated to use these tools in the classroom to increase graduates' competitiveness in the job market. Although highly desirable, the adoption of version control is hindered by a steep learning curve for both teachers and students, a lack of resources tailored to specific instructional and learning needs, as well as inadequate integration and alignment with the learning outcomes in the program curricula. The panel will analyze these challenges, solicit feedback from the audience's experience with version control tools, and facilitate discussions around promising strategies that could benefit student learning and graduates' professional competencies expected in the workplace.

Software Engineering courses often incorporate large-scale projects with collaboration between students working in teams. However, it is difficult to objectively assess individual students when their projects are a product of collaborative efforts. This study explores measurements of individuals' contributions to their respective teams.

Unit testing is widely adopted in software development to automate verification of individual functions. Likewise, computer science curricula are increasing their emphasis on teaching unit testing skills. However, coverage and other measurements of test quality have limitations and may even encourage poor practices when used as an assessment and/or feedback mechanism for students. In this paper, we introduce test accuracy as a measurement that compares how well unit tests perform at distinguishing acceptable from unacceptable function implementations. To study measurements of test quality, we evaluated an assignment where students (n=103) developed a class along with accompanying tests for each of its functions. Our quantitative analysis compares test accuracy with coverage and bug identification measurements by examining each of their relationships with a lack of bugs in the students' implementations. Of the three measurements, accuracy had the strongest positive correlation (p=.648, p<.001) with a lack of bugs in the entire program. When examining the subset of tests for a particular function-under-test, both accuracy and coverage were significant predictors of a lack of bugs within the corresponding function's implementation.

This Innovative Practice Full Paper describes a pedagogical technique for introducing unit testing within software engineering courses. The Bug Hide-and-Seek educational game reinforces testing principles by requiring students to develop some correct solutions as well as some other solutions that intentionally contain bugs. While developing the correct and buggy solutions, students also write corresponding tests that should identify whether each solution contains bugs or exhibits acceptable behavior. Consequently, the first goal of the game is to hide a clever bug that will trick other students' tests into passing the implementation, despite the hidden bug. The second goal is to write thorough tests that can accurately differentiate correct from incorrect software behavior. We introduce the motivation, pedagogy, and preliminary analysis of two variations of the Bug Hide-and-Seek game, while comparing their tradeoffs. The between-subject variation considers each student's test suite in its entirety. The within-subject variation requires more sophisticated analysis, but considers each individual function along with its corresponding tests, which provides more granular insight and specific feedback to students. We conducted a pilot study of both variations over two semesters of students (n=87) playing the Bug Hide-and-Seek game. We found that students' test True Positive Rate and True Negative Rate at verifying implementations are both significant predictors of a lack of bugs in their own solution.

By working on open source software projects, software engineering students can benefit from working on more realistic products than traditional, educational programming assignments. However, careers in software engineering demand learning how to work within a professional environment and how to follow software development processes. We studied the impact of students' interactions with external collaborators on open source projects and found many similar outcomes between those who communicated remotely and those who communicated face-to-face. However, we also discovered that face-to-face interactions with local software professionals following the Localized Free and Open Source (LFOSS) model had particular advantages in teaching Agile methods, holding students accountable, and introducing professional networking opportunities.

The Tech Startup model is an approach to learning software engineering methods by partnering with students studying entrepreneurship to collaborate on real software products. Agile software development methods align with Lean Startup practices so that students in sister classes experience leading contemporary practices in their respective fields. This paper describes a pilot study of interdisciplinary Tech Startup projects with a heuristic evaluation of software engineering realism and formative assessment of students' surveyed experiences. The study found several similar student outcomes to other project models; however, it also identified limitations in the pilot with corresponding recommendations for future implementations.

A common refrain from Senior Exit Surveys and Alumni Surveys is the desire to work on "real-world," "practical" and "hands-on" projects using industry-ready tools and development environments. To assuage this, institutions have moved towards adopting Free and Open Source Software (FOSS) as an avenue to provide meaningful, applied learning interventions to students. Through these experiences, students benefit from engagement with various communities including: the community of contributors to the FOSS project; the community of local software developers; the community of citizens who reside in the local area; the community of students at their institution and others; and, the community of people impacted by the FOSS project. These engagements motivate students, enhance their communication and technical skills, allow them to grow and become more confident, help them form professional networks, and provide the "real-world" projects they seek. In this panel, we will discuss our experiences in engaging students with five different types of communities as part of incorporating FOSS into our courses, focusing on how other educators can provide the same benefits to their students as well. In order to satisfy the time constraints of the panel, the last two authors will present together.

Involving computer science students in open source software projects provides opportunities for them to contribute to real products with more authentic scope than typical computer science assignments. However, the environment of collaborating with external, distributed teams also poses unique challenges and may distance students from the potential for valuable, direct contact and mentorship from software professionals. In addition, while the technology industry continues to grow, smaller communities have a vested interest in growing a culture for collaboration between students and local software developers. We formed a local open source organization to collaborate on a product by combining efforts from students and professionals. This paper describes the localized free and open source software (LFOSS) organization and reports initial findings from software engineering students' involvement.

Writing meaningful software tests requires students to think critically about a problem and consider a variety of cases that might break the solution code. Consequently, to overcome bugs in their code, it would be beneficial for students to reflect over their work and write robust tests rather than relying on trial-and-error techniques. Automated grading systems provide students with prompt feedback on their programming assignments and may help them identify where their interpretation of requirements do not match the instructor's expectations.

As students learn to program they also learn basic software development methods and techniques, but educators do not often directly assess students' development processes or evaluate their adherence to specific techniques. However, automated grading systems provide opportunities to evaluate students' programming and provide feedback while the student is still in the process of developing. Consequently, automated adaptive feedback may help reinforce effective techniques and processes.

Online environments introduce unprecedented scale for formal and informal learning communities. In these environments, user-contributed content enables social constructivist approaches to education. In particular, students can help each other by providing hints and suggestions on how to approach problems, by rating each other's suggestions, and by engaging in discussions about the questions. In addition, students can also learn through composing their own questions. Furthermore, with grounding in Item Response Theory, data mining and statistical student models can assess questions and hints for their quality and effectiveness. As a result, internet-scale learning environments allow us to move from simple, canned quizzing systems to a new model where automated, data-driven analysis continuously assesses and refines the quality of teaching material. Our poster describes a framework and prototype of an online drill-and-practice system that leverages user-contributed content and large-scale data to organically improve itself.

While Computer Science curricula teach students strategic software development processes, assessment is often product-instead of process-oriented. Test-Driven Development (TDD) has gained popularity in computing education, but evaluating students' adherence to TDD requires analyzing their development processes instead of only their final product. Consequently, we designed an adaptive feedback system for reinforcing incremental testing behaviors. In this paper, we compare the results of the system with different reinforcement schedules and with- or without- visually salient testing goals. We analyzed snapshots of students' programming projects gathered during development and interviewed students at the end of the academic term. From our findings, we identify potential for influencing student development behaviors and suggest future direction for designing adaptive reinforcement.

Rudimentary programming skills are essential to developing fundamental proficiency in computer science. However, learning programming techniques can be challenging and frustrating for many students. CodeWorkout is an online learning environment that offers drill-and-practice exercises with novel social and adaptive scaffolding. Learners can track their progress on an assortment of computer science areas and skills while taking advantage of social features to discuss questions and help teach each other. Meanwhile, objective measurements of questions and teaching hints help promote the best, most effective content for learning. Our poster demonstrates how both computer science students and teachers benefit from joining the CodeWorkout community and taking advantage of its unique features.

This data-driven paper quantitatively evaluates software testing behaviors that students exhibited in introductory computer science courses. The evaluation includes data collected over five years (10 semesters) from 49,980 programming assignment submissions by 883 different students. To examine the effectiveness of software testing behaviors, we investigate the quality of their testing at different stages of their development. We partition testing behaviors into four groups according to when in their development they first achieve substantial (at least 85%) test coverage.

Studies have found that following Test-Driven Development (TDD) can improve code and testing quality. However, a preliminary investigation was consistent with concerns raised by other educators about programmers resisting TDD. In this paper, we describe an adaptive, pedagogical system for tracking and encouraging students' adherence to TDD. Along with an empirical evaluation of the system, we discuss challenges and opportunities for persuading student behavior through adaptive technology.

In computing education, students must learn techniques practiced in relevant professions. Test-Driven Development (TDD) is one such technique popular in the software industry. Preliminary reports suggest that TDD helps produce higher-quality code. However, motivating novice programmers to adopt TDD is also recognized as a distinct challenge. My studies and proposed work address this challenge with the following objectives: measuring adherence to TDD and its consequential outcomes; understanding students' reasons for non-adherence; and influencing students' attitudes and behavior via pedagogical interventions.

Test-Driven Development (TDD) is a software development process with a test-first approach that shows promise for improving code quality. Our research addresses concerns raised in both academia and industry about a lack of motivation or acceptance in adopting TDD. In a CS2 class, we used an automated testing tool and post-class surveys to observe patterns of behavior in testing as well as changes in attitudes. We found significant positive outcomes for students following TDD. We also identified obstacles deterring students from adhering to TDD and discuss reasons and possible remedies.