Interest in teaching-track faculty positions has been steadily increasing as enrollments in computer science degree programs continue to trend upward. While departments have welcomed these new teaching-track faculty members, senior faculty, department chairs, and university committees often struggle with how to best evaluate these faculty members during the promotion process. In our experience, some universities try to use a "watered-down" version of the tenure-track promotion standards with the intent of uniformity. Other universities have created whole new processes, which may be better at capturing the differences in teaching-track positions, but also can create a "second-class citizen" status for the teaching-track faculty members.

The computing education community endeavors to consistently move forward, improving the educational experience of our students. As new innovations in computing education practice are learned and shared, however, these papers may not exhibit the desired qualities that move simple experience reports to true Scholarship of Teaching and Learning (SoTL). We report on our six years of experience in running professional development for computing educators in empirical research methods for social and behavioral studies in the classroom. Our goal is to have a direct impact on instructors who are in the beginning stages of transitioning their educational innovations from anecdotal to empirical results that can be replicated by instructors at other institutions. To achieve this, we created a year-long mentoring experience, beginning with a multi-day workshop on empirical research methods during the summer, followed by regular mentoring sessions with participants, and culminating in a follow-up session at the following year's SIGCSE Technical Symposium. From survey results and as evidenced by eventual research results and publications from participants, we believe that our method of structuring empirical research professional development was successful and could be a model for similar programs in other areas.

We present an eight-year curriculum redesign effort impacting al-most every course in our computer science department. Having not made a major update to our curriculum in two decades, complications began to arise from significant increases in enrollment and instituting multiple degrees in computing in the same department. Starting from a desire to adjust a few courses, we systematically collected a broad set of requirements and blue-sky ideas from many stakeholders, resulting in an unsatisfiable set of content, ordering, and course boundary constraints. After multiple rounds of conversation with our stakeholders in and out of the department, we evolved and relaxed several of our constraints, allowing us to develop a compromise plan for seven new courses and a new prerequisite system. We then piloted five of the new courses and collected feedback on results, iterating on these courses each semester for two years. We worked with registrars, advisors, and administrators to develop a transition plan from old to new courses. This paper presents highlights of each step of this process, a summary of the resulting curriculum design, and reflections and recommendations for other departments that may want to undertake a similar update.

Context. Computing Education Research (CER) is critical to help the computing education community and policy makers support the increasing population of students who need to learn computing skills for future careers. For a community to systematically advance knowledge about a topic, the members must be able to understand published work thoroughly enough to perform replications, conduct meta-analyses, and build theories. There is a need to understand whether published research allows the CER community to systematically advance knowledge and build theories. Objectives. The goal of this study is to characterize the reporting of empiricism in Computing Education Research literature by identifying whether publications include content necessary for researchers to perform replications, meta-analyses, and theory building. We answer three research questions related to this goal: (RQ1) What percentage of papers in CER venues have some form of empirical evaluation? (RQ2) Of the papers that have empirical evaluation, what are the characteristics of the empirical evaluation? (RQ3) Of the papers that have empirical evaluation, do they follow norms (both for inclusion and for labeling of information needed for replication, meta-analysis, and, eventually, theory-building) for reporting empirical work? Methods. We conducted a systematic literature review of the 2014 and 2015 proceedings or issues of five CER venues: Technical Symposium on Computer Science Education (SIGCSE TS), International Symposium on Computing Education Research (ICER), Conference on Innovation and Technology in Computer Science Education (ITiCSE), ACM Transactions on Computing Education (TOCE), and Computer Science Education (CSE). We developed and applied the CER Empiricism Assessment Rubric to the 427 papers accepted and published at these venues over 2014 and 2015. Two people evaluated each paper using the Base Rubric for characterizing the paper. An individual person applied the other rubrics to characterize the norms of reporting, as appropriate for the paper type. Any discrepancies or questions were discussed between multiple reviewers to resolve. Results. We found that over 80% of papers accepted across all five venues had some form of empirical evaluation. Quantitative evaluation methods were the most frequently reported. Papers most frequently reported results on interventions around pedagogical techniques, curriculum, community, or tools. There was a split in papers that had some type of comparison between an intervention and some other dataset or baseline. Most papers reported related work, following the expectations for doing so in the SIGCSE and CER community. However, many papers were lacking properly reported research objectives, goals, research questions, or hypotheses; description of participants; study design; data collection; and threats to validity. These results align with prior surveys of the CER literature. Conclusions. CER authors are contributing empirical results to the literature; however, not all norms for reporting are met. We encourage authors to provide clear, labeled details about their work so readers can use the study methodologies and results for replications and meta-analyses. As our community grows, our reporting of CER should mature to help establish computing education theory to support the next generation of computing learners.

One of the most important, and difficult, aspects of starting an education research project is identifying an interesting, answerable, repeatable, measurable, and appropriately scoped research question. The lack of a valid research question reduces the potential impact of the work and could result in wasted effort. The goal of this workshop is to help educational researchers get off on the right foot by defining such a research question. This workshop is part of the larger Designing Empirical Education Research Studies (DEERS) project, which consists of an ongoing series of workshops in which researcher cohorts work with experienced empirical researchers to design, implement, evaluate, and publish empirical work in computer science education. In addition to instruction on the various aspects of good research questions, DEERS alumni will join us to mentor attendees in development of their own research questions in small group breakout sessions. At the end of the workshop, attendees will leave with a valid research question that can then be the start for designing a research study. Attendees will also receive information on how to apply to attend the full summer workshop, where they can fully flesh out the empirical study design, and join a DEERS research cohort. More information about DEERS can be found at http://empiricalcsed.org.

Many computer science departments have chosen to hire faculty to teach in a teaching-track position that parallels the standard tenure-track position, providing the possibility of promotion, longer-term contracts, and higher pay for excellence in teaching and service. This birds-of-a-feather is designed to gather educators who are currently in such a position to share their experiences as members of the faculty of their departments and schools, and to provide opportunities for schools considering such positions to gather information.

Capstone and large projects in computing education are used as a vehicle for giving students as close to a “real-world” experience in software development as possible within the constraints of a computing degree program. This special issue presents four articles that focus on empirical research on capstone or other large-scale projects. These articles discuss areas such as project selection, working with external stakeholders, choosing the appropriate development methodology, incorporating creative activities to support student engagement, and learning.

One of the most important, and difficult, aspects of starting an education research project is identifying an interesting, answerable, repeatable, measurable, and appropriately scoped research question. The lack of a valid research question reduces the potential impact of the work and could result in wasted effort. The goal of this workshop is to help educational researchers get off on the right foot by defining such a research question. This workshop is part of the larger Designing Empirical Education Research Studies (DEERS) project, which consists of an ongoing series of workshops in which researcher cohorts work with experienced empirical researchers to design, implement, evaluate, and publish empirical work in computer science education. In addition to instruction on the various aspects of good research questions, DEERS alumni will join us to mentor attendees in development of their own research questions in small group breakout sessions. At the end of the workshop, attendees will leave with a valid research question that can then be the start for designing a research study. Attendees will also receive information on how to apply to attend the full summer workshop, where they can fully flesh out the empirical study design, and join a DEERS research cohort. More information about DEERS can be found at http://empiricalcsed.org.

Many computer science departments have chosen to hire faculty to teach in a teaching-track position that parallels the standard tenure-track position. These teaching-track positions include the possibility for promotion, longer-term contracts, and greater pay as demonstrated by excellence in teaching, educational material development, and service. As this type of position has been growing rapidly in popularity, CS departments have been developing the expectations and responsibilities for such faculty almost ad hoc, creating policies that can be vastly different similarly-named positions at other universities. This birds-of-a- feather is designed to gather educators who are currently in such a position to share their experiences as members of the faculty of their departments and schools, and to provide opportunities for schools considering such positions to gather information. In past sessions, we have laid the groundwork for keeping this community together through online discussions and consolidating information about teaching-track positions at various universities together into an online database.

The computer science education (CSEd) research community consists of a large group of passionate CS educators who often contribute to other disciplines of CS research. There has been a trend in other disciplines toward more rigorous and empirical evaluation of various hypotheses. Prior investigations of the then-current state of CSEd research showed a distinct lack of rigor in the top research publication venues, with most papers falling in the general category of experience reports. In this paper, we present our examination of the two most recent proceedings of the SIGCSE Technical Symposium, providing a snapshot of the current state of empiricism at the largest CSEd venue. Our goal to categorize the current state of empiricism in the SIGCSE Technical Symposium and identify where the community might benefit from increased empiricism when conducting CSEd research. We found an increase in empirical validation of CSEd research to over 70%; however, our findings suggest that current CSEd research minimizes replication precluding meta-analysis and theory building.

Many computer science departments have chosen to hire faculty to teach in a teaching-track position that parallels the standard tenure-track position. These teaching-track positions include the possibility for promotion, longer-term contracts and greater pay as demonstrated by excellence in teaching, educational material development, and service. Since this type of position is relatively new in CS departments, the expectations and responsibilities for such faculty varies from one university to another, but they certainly differ from the traditional tenure-track positions. This birds-of-a-feather is designed to gather educators who are currently in such a position to share their experiences as members of the faculty of their departments and schools, and to provide opportunities for schools considering such positions to gather information. In past sessions, we have laid the groundwork for keeping this community together through online discussions and consolidating information about teaching-track positions at various universities together into an online database.

A great CS assignment is a delight to all, but the path to one can be most roundabout. Many CS students have had their characters built up on assignments that worked better as an idea than as an actual assignment. Assignments are hard to come up with, yet they are the key to student learning. The Nifty Assignments special session is all about promoting and sharing the ideas and ready-to-use materials of successful assignments.

Many computer science departments have chosen to hire faculty to teach in a teaching-track position that parallels the standard tenure-track position. These teaching-track positions include the possibility for promotion, longer-term contracts and greater pay as demonstrated by excellence in teaching, educational material development, and service. Since this type of position is relatively new in CS departments, the expectations and responsibilities for such faculty varies from one university to another, but they certainly differ from the traditional tenure-track positions. This birds-of-a-feather is designed to gather educators who are currently in such a position to share their experiences as members of the faculty of their departments and schools, and to provide opportunities for schools considering such positions to gather information. In past sessions, we have laid the groundwork for keeping this community together through online discussions and consolidating information about teaching-track positions at various universities together into an online database.

Over the past several years, there has been a marked increase in empirical validation in research papers presented at the SIGCSE Symposium. This birds-of-a-feather session intends to bring together faculty interested in empirical research in computer science education to share methodologies and techniques, while also continuing the conversation about how to move empirical computer science education research forward. The computer science education (CSEd) research community consists of a large group of passionate CS educators who often contribute to other disciplines of CS research. There has been a trend in other disciplines toward more rigorous and empirical evaluation of various hypotheses. Prior meta-analyses of the then-current state of CSEd research showed a distinct lack of rigor in the top CSEd publication venues, with most papers falling in the general category of experience reports. Our goal is to engage the SIGCSE community in a discussion as to how we can move empirical research in CSEd forward. We hope to help researchers establish connections that will allow them to share research methodologies and challenges.

We present the design and execution of a Service Learning Practicum (SLP) course sequence intended to be year-long capstone for computer science seniors. Students are teamed into groups of six, and develop software for local nonprofit organizations. In addition to the structure of the course, we describe the challenges faced (legal, organizational, etc.), student perceptions via survey results, and provide a number of suggestions for other institutions who are looking to create a similar course sequence. At the end of the cap- stone experience, the customers are provided with working software that meet their current needs.

Many computer science departments have chosen to hire faculty to teach in a teaching-track position that parallels the standard tenure-track position. These teaching-track positions include the possibility for promotion, longer-term contracts and greater pay as demonstrated by excellence in teaching, educational material development, and service. Since this type of position is relatively new in CS departments, the expectations and responsibilities for such faculty varies from one university to another, but they certainly differ from the traditional tenure-track positions. This birds-of-a-feather is designed to gather educators who are currently in such a position to share their experiences as members of the faculty of their departments and schools, and to provide opportunities for schools considering such positions to gather information. In past sessions, we have laid the groundwork for keeping this community together through online discussions and consolidating information about teaching-track positions at various universities together into an online database.

Many computer science departments have chosen to hire faculty to teach in a teaching-track position that parallels the standard tenure-track position. These teaching-track positions include the possibility for promotion, longer-term contracts and greater pay as demonstrated by excellence in teaching, educational material development, and service. Since this type of position is relatively new in CS departments, the expectations and responsibilities for such faculty varies from one university to another, but they certainly differ from the traditional tenure-track positions. This birds-of-a-feather is designed to gather educators who are currently in such a position to share their experiences as members of the faculty of their departments and schools, and to provide opportunities for schools considering such positions to gather information. In past sessions, we have laid the groundwork for keeping this community together through online discussions and consolidating information about teaching-track positions at various universities together into an online database.

During the summer of 2011, twenty-four high school students participated in an intense, three-week computer science course at the University of Virginia. The course met for twenty-one three-hour sessions, thus encompassing more contact time than a standard college-level course. The course was structured in an “Inform, Experience, Implement” active-learning format: students were exposed to the history of a particular problem in context and participated in an active learning lesson regarding the topic before learning how to address the examined problem through programming. This structure helped integrate into the course best practices from experiential learning, kinesthetic outreach activities, and active learning pedagogy. Utilizing this three-part rotation curriculum achieved some important goals, including holding the interest of students during the summer for six hours a day and successfully motivating students who had no programming background.

Many computer science departments have chosen to hire faculty to teach in a teaching-track position that parallels the standard tenure-track position, providing the possibility of promotion, longer-term contracts, and higher pay for excellence in teaching and service. This birds-of-a-feather is designed to gather educators who are currently in such a position to share their experiences as members of the faculty of their departments and schools, and to provide opportunities for schools considering such positions to gather information.

Over the past two years, second-year Java and software engineering courses have been taught at the University of Virginia and North Carolina State University utilizing the Android OS platform. Instructors taught a variety of traditional second-year topics, including abstraction, design, requirements, and testing, utilizing a variety of Android-based mobile devices. Anecdotal responses from student surveys and evaluations from five course sessions indicate that teaching lower-level courses with more advanced and current technology, even with a steeper learning curve, is beneficial. In this tutorial proposal, we outline our plan for presenting a session that would help educators incorporate the Android OS into their curriculum and how to use the system even if mobile devices are not available.

This paper describes the benefits of using LeJOS and the Lego Mindstorms NXT set for teaching advanced software development. While Lego Mindstorms has been used in introduction to computer science courses, it is not reported to be widely used in a simulated production environment requiring such things as threading, network communications, and the implementation of command protocols. Additionally, because the Mindstorms NXT system supports Bluetooth communications with multiple devices, it is possible to use this system as the basis for a complex, communicating system requiring multiple software artifacts on different machines.

This paper details the results of an investigation into the compatibility of partnered computer science students. The study involved approximately 290 students at the University of Virginia (UVA). This study builds on the work of researchers at North Carolina State University (NCSU). NCSU researchers have conducted a number of studies on the compatibility of pair programmers. We examined many of the factors that the NCSU researchers explored in their studies (including personality type, learning style, skill level, programming self esteem, work ethic, and time management choices) in order to determine whether the conclusions of the research at NCSU also hold true at UVA. Consistent with the NCSU studies, we found that skill level continues to be the most important factor in student compatibility.

Verification and validation techniques often generate various forms of software development artifacts. Change records created from verification and validation efforts show how files in the system tend to change together in response to fixes for identified faults and failures. We propose a methodology for determining the impact of a new system modification by analyzing software change records through singular value decomposition. This methodology generates clusters of files that historically tend to change together to address faults and failures found in the code base. We performed a post hoc case study using this technique on five open source software systems. We determined that our technique was effective in identifying impacted files in a system from an introduced change when the developers tended to make small, targeted updates to the source system regularly. We further compared our technique against two other impact analysis techniques (Pathlmpact and Coveragelmpact) and found that our technique provided comparable results, while also identifying non-source files that could be impacted by the change.

A critical problem facing todaypsilas Internet community is the increasing number of attacks exploiting flaws found in Web applications. This paper specifically targets input validation vulnerabilities found in SQL queries that may lead to SQL Injection Attacks (SQLIAs). We introduce a tool that automatically detects and suggests fixes to SQL queries that are found to contain SQL Injection Vulnerabilities (SQLIVs). Testing was performed against phpBB v2.0, an open source forum package, to determine the accuracy and efficacy of our software.

During development and testing, changes made to a system to repair a detected fault can often inject a new fault into the code base. These injected faults may not be in the same files that were just changed, since the effects of a change in the code base can have ramifications in other parts of the system. We propose a methodology for determining the effect of a change and then prioritizing regression test cases by gathering software change records and analyzing them through singular value decomposition. This methodology generates clusters of files that historically tend to change together. Combining these clusters with test case information yields a matrix that can be multiplied by a vector representing a new system modification to create a prioritized list of test cases. We performed a post hoc case study using this technique with three minor releases of a software product at IBM. We found that our methodology suggested additional regression tests in 50% of test runs and that the highest-priority suggested test found an additional fault 60% of the time.

In this paper, we propose a technique for leveraging historical field failure records in conjunction with automated static analysis alerts to determine which alerts or sets of alerts are predictive of a field failure. Our technique uses singular value decomposition to generate groupings of static analysis alert types, which we call alert signatures, that have been historically linked to field failure-prone files in previous releases of a software system. The signatures can be applied to sets of alerts from a current build of a software system. Files that have a matching alert signature are identified as having similar static analysis alert characteristics to files with known field failures in a previous release of the system. We performed a case study involving an industrial software system at IBM and found three distinct alert signatures that could be applied to the system. We found that 50% of the field failures reported since the last static analysis run could be discovered by examining the 10% of the files and static analysis alerts indicated by these three alert signatures. The remaining failures were either not detected by a signature which could be an indication of a new type of error in the field, or they were on areas of the code where no static analysis alerts were detected.

During the course of software development, developers will employ several different verification and validation (V&V) practices with their software. However, these efforts might not be recorded or maintained in an effective manner. We have built defect estimation with V&V Certificates on Programming (DevCOP), a software certificate management system. With DevCOP, developers can automatically track and maintain a persistent record of the V&V practices used during development via certificates. With this V&V information, developers and managers can better manage their V&V efforts within a system. Detailed information such as coverage of particular V&V techniques over the system or the amount of V&V performed on a single function can be provided. Developers can also use this V&V information post hoc to see which techniques were more effective at removing defects. In our future work, we are researching a parametric model which utilizes these certificates to estimate defect density as development proceeds

In industry, information on defect density of a product tends to become available too late in the software development process to affordably guide corrective actions. Our research objective is to build a parametric model which utilizes a persistent record of the validation and verification (V&V) practices used with a program to estimate the defect density of that program. The persistent record of the V&V practices are recorded as certificates which are automatically recorded and maintained with the code.PhD Advisor: Dr. Laurie Williams, [email protected]