While professional integrated programming environments support developers with advanced debugging functionality, block-based programming environments for young learners often provide no support for debugging at all, thus inhibiting debugging and preventing debugging education. In this paper we introduce NuzzleBug, an extension of the popular block-based programming environment Scratch that provides the missing debugging support. NuzzleBug allows controlling the executions of Scratch programs with classical debugging functionality such as stepping and breakpoints, and it is an omniscient debugger that also allows reverse stepping. To support learners in deriving hypotheses that guide debugging, NuzzleBug is an interrogative debugger that enables to ask questions about executions and provides answers explaining the behavior in question. In order to evaluate NuzzleBug, we survey the opinions of teachers, and study the effects on learners in terms of debugging effectiveness and efficiency. We find that teachers consider NuzzleBug to be useful, and children can use it to debug faulty programs effectively. However, systematic debugging requires dedicated training, and even when NuzzleBug can provide correct answers learners may require further help to comprehend faults and necessary fixes, thus calling for further research on improving debugging techniques and the information they provide.

Abstract The importance of programming education has led to dedicated educational programming environments, where users visually arrange block-based programming constructs that typically control graphical, interactive game-like programs. TheScratchprogramming environment is particularly popular, with more than 90 million registered users at the time of this writing. While the block-based nature ofScratchhelps learners by preventing syntactical mistakes, there nevertheless remains a need to provide feedback and support in order to implement desired functionality. To support individual learning and classroom settings, this feedback and support should ideally be provided in an automated fashion, which requires tests to enable dynamic program analysis. In prior work we introducedWhisker, a framework that enables automated testing ofScratchprograms. However, creating these automated tests forScratchprograms is challenging. In this paper, we therefore investigate how to automatically generateWhiskertests. Generating tests forScratchraises important challenges: First, game-like programs are typically randomised, leading to flaky tests. Second,Scratchprograms usually consist of animations and interactions with long delays, inhibiting the application of classical test generation approaches. Thus, the new application domain raises the question of which test generation technique is best suited to produce high coverage tests capable of detecting faulty behaviour. We investigate these questions using an extension of theWhiskertest framework for automated test generation. Evaluation on common programming exercises, a random sample of 1000Scratchuser programs, and the 1000 most popularScratchprograms demonstrates that our approach enablesWhiskerto reliably accelerate test executions, and even though manyScratchprograms are small and easy to cover, there are many unique challenges for which advanced search-based test generation using many-objective algorithms is needed in order to achieve high coverage.