This study examines sensemaking in student test case design, showing a reliance on conceptual knowledge learned during programming courses over exploratory testing strategies. Of the three identified approaches taken by students, the “developer approach” is used most often, suggesting a gap in software engineering education. We hypothesise that software testing should be taught in computer science programs using a design paradigm based on empiricism instead of rationalism. Based on these results, and our hypothesis, we will further analyse the sensemaking processes of both students and experts, and create an instructional design to improve software testing education in computer science programs.

Many educators face problems with integrating testing into programming education. For instance: existing courses are already fully packed; testing requires skills that students might not yet have; and testing is, although considered important, not always given priority by students. Educators, in general, do not have time to overhaul a programming course to fully integrate testing, resulting in a situation in which the improvement of testing education seems to have slowed down. In this paper, we propose Test Informed Learning with Examples (TILE), a new concept to create test-awareness in introductory programming courses. TILE aims to introduce testing as early as possible and in a subtle way. As a result, integration into existing curricula can be done seamlessly and requires less effort than completely overhauling existing programming courses. The contributions of this paper are: the presentation of TILE; experiences of having applied this method in the classroom; and an open repository with assignments using our approach. Applying TILE seems to be a promising approach to introduce testing in early programming. Moreover, some TILEs can be added to existing courses with almost no effort from day one. More research is needed to gain confidence in the benefits of using TILE over time and to collect evidence that we reached the final aim of TILE, i.e. students that test because that inherently belongs to programming, and not because it is explicitly asked from them.

Software testing is the most used technique for quality assurance in industry. However, in computer science education software testing is still treated as a second-class citizen and students are unable to test their software well enough. One reason for this is that teaching the subject of software testing is difficult as it is a complex intellectual activity for which students need to allocate multiple cognitive resources at the same time. A myriad of primary and secondary studies have tried to solve this problem in education, however still with very limited results. Before we can design interventions to improve our pedagogical approaches, we need to gain more in-depth understanding and recognition of sensemaking as it is happening when students design test cases. An initial exploratory study identified four different sensemaking approaches used by students while creating test models. In this paper we present a follow-up study with 50 students from a large university in Spain. The used methodology was based on the previous study with the improvements that originated from its evaluation. We asked the participants to create a test model based on a description of a test problem using a specialized web-based tool for modeling test cases. We measured how well these models fit the test problem, the sensemaking process that students went through when creating the models, and the students’ perception of the modeling task. The participants received no compensation for their efforts, and we scheduled the experiment during a regular class. Apart from the created models and their metadata, we also collected recordings of the students’ computer screens made during the experiment and used a questionnaire to study their perspectives on the assignment. All the collected textual, graphical, and video data was analyzed using an iterative inductive analysis process to allow new information about the different sensemaking approaches to emerge. We gained better insights into the sensemaking processes of students while modeling test cases for a problem. The results enabled us to refine our previous findings, and we identified new sensemaking approaches. Based on these results, we can further investigate ways to influence the sensemaking process in education, the possible misconceptions that have a negative influence on it, and the desired mental model we want our students to have to design test cases.

Software Testing is one of the most effective ways to increase software quality, it is a complex learning problem requiring different approaches then the learning of software programming. The focus of my research is to design an evidence based instructional design for software testing in computer science education.