How to measure students' Computational Problem-Solving (CPS) competencies is an ongoing research topic. Prevalent approaches vary by measurement tools (e.g., interactive programming, multiple-choice tests, or programming-independent tests) and task types (e.g., debugging problems or Parson problems). However, few studies have examined the measurement tools of CPS competencies themselves: affordances and limitations of the measurement tools and how they compare, or whether different task types might elicit CPS competencies differently. Research needs to address these questions in order to better understand how to design robust, generalizable, and effective measurement tools for CPS competencies. This paper presents an exploratory study that contributes to this research direction. It is part of a larger international project to develop an open-access formative assessment platform for CPS, which includes a novel authoring tool for a wide range of task types for interactive block-based programming. We used the tool to create an interactive programming experience with multiple task types and gave it to more than 300 secondary school students from different countries. We also administered a validated multiple-choice measurement of Computational Thinking with block-based programs. We focused on task complexity as a characteristic of task type, using a classification scheme based on task design features. Comparing students' performances on tasks of different complexity and using two distinct measurement tools, we found that the multiple-choice measurement only partially predicts performance in the interactive programming task. Additionally, its predictive capacity varies significantly between task types of differing complexity.

Abstract Research has shown that Educational Robotics (ER) enhances student performance, interest, engagement and collaboration. However, until now, the adoption of robotics in formal education has remained relatively scarce. Among other causes, this is due to the difficulty of determining the alignment of educational robotic learning activities with the learning outcomes envisioned by the curriculum, as well as their integration with traditional, non-robotics learning activities that are well established in teachers’ practices. This work investigates the integration of ER into formal mathematics education, through a quasi-experimental study employing the Thymio robot and Scratch programming to teach geometry to two classes of 15-year-old students, for a total of 26 participants. Three research questions were addressed: (1) Should an ER-based theoretical lecture precede, succeed or replace a traditional theoretical lecture? (2) What is the students’ perception of and engagement in the ER-based lecture and exercises? (3) Do the findings differ according to students’ prior appreciation of mathematics? The results suggest that ER activities are as valid as traditional ones in helping students grasp the relevant theoretical concepts. Robotics activities seem particularly beneficial during exercise sessions: students freely chose to do exercises that included the robot, rated them as significantly more interesting and useful than their traditional counterparts, and expressed their interest in introducing ER in other mathematics lectures. Finally, results were generally consistent between the students that like and did not like mathematics, suggesting the use of robotics as a means to broaden the number of students engaged in the discipline.