Enhanced Programming Error Messages are one type of feedback that may be useful during the learning process of novice programmers. One way to improve such messages is by using the context given by an auto-grader from students' errors when solving programming exercises in those environments. We investigate how Large Language Models (LLM) may be used to provide additional feedback beyond simple compiler or interpreter error messages by garnering input from a programming exercise description, the student's code to solve that exercise, and the error generated by a Python interpreter and generating an LLM prompt from that given information. In an experiment, we asked instructors to evaluate the prompt text generated by an LLM from a variety of exercises and the students' errors. Our results show a high rate of adequate feedback by means of explanations and code examples to fix the errors without providing a full code solution. However, some cons have also been noticed, such as vague explanations, hallucinations, and dissociation from explanations and examples. The results suggest that LLM feedback is a promising approach to provide feedback to novice programmers by means of enhanced programming error messages.

Computational Thinking (CT) is a prominent concept that is part of several computing curricula. However, there is a lack of valid and reliable assessment instruments aimed at assessing the acquisition of Concepts, Practices, and Perspectives of CT. This work aims to develop an assessment instrument for the acquisition of CT to the target audience of primary school graduates, and to carry out its theoretical evaluation using semantic analysis and content analysis. The proposed assessment considers the dimensions of Concepts, Practices and Perspectives of CT. Based on a previously designed operational definition of CT validated by an expert panel, we developed a test bank of questions with 117 items in total, of which 63 items were from the Concepts dimension, 45 items were from the Practices dimension and nine scenarios were from the Perspectives dimension. The items' theoretical analysis was performed in sequence. The semantic analysis sought to verify the items' intelligibility. Content analysis sought to verify whether the developed items refer to the latent trait that was intended to be measured. After the theoretical evaluation, the final instrument has 55 items, being 35 from Concepts, 13 from Practices and seven scenarios from Perspectives. The items that make up the instrument are intelligible to the target audience. Furthermore, according to the expert panel, each item refers to only one of the latent traits among the categories of Concepts, Practices and Perspectives.

This full Research-to-Practice paper evaluates a Machine Learning (ML) course as a strategy to introduce Artificial Intelligence (AI) in middle school. AI is a technology that is increasingly present in our daily lives, and it is important that K-12 schools offer their students some basic first steps in this universe. Nonetheless, most initiatives to introduce ML aim at higher education, in undergraduate computing programs, and school initiatives usually lack the use of hardware to learn ML. In this context, we designed and implemented an introductory workshop on AI and ML for middle school students on the fundamentals of AI using TinyML and Arduino, and we assessed their attitudes towards 21st Century skills. Results show some ways how middle school students are impacted with the presentation of ML concepts and practices by building small applications, in addition to providing practice grounding to future educational interventions using TinyML as a tool to familiarize K-12 students with ML. Survey results point to very few post-intervention changes regarding 21st Century skills. Learned lessons point to a need to increase the course workload for more significant changes in students' perceptions.

Computational Thinking (CT) is a set of thinking processes used by computer scientists to formulate problems and describe solutions. In recent years, CT has been largely explored by the Computing Education community. Due to its potential to contribute to problem solving and analytical thinking, CT could also be a relevant subject in K-12 education. For such, there is a need to qualify educators in CT. In this paper, we describe our experience in a large-scale course on Computational Thinking for pre-service teachers. Our goal is to qualify pre-service teachers in CT, both in the comprehension of its definition and how it can be incorporated in classroom. Our experience indicates that the participants had a positive awareness of CT concepts, as well as an adequate understanding of the pedagogical approaches to teach the subject. They also developed positive attitudes towards the field of Computer Science.

Schools around the world have included computing in K-12 education. However, a challenge remains, which is the scarcity of coherent computing textbooks for primary school. Despite the existing reference curricula and guidelines, there is a lack of instructional materials, especially evaluated textbooks. In any field of knowledge, textbooks are of paramount importance, serving as a guide for teachers in class. In Brazil, the Fundamental Computing book series was developed based on the CSTA curriculum standards. The books cover from the sixth to the ninth grades of Brazilian middle school, addressing knowledge of Computer Science, with lesson plans and activities. The goal of this work is to evaluate, from the perspective of teachers, the sixth-grade textbook of the Fundamental Computing series, based on the quality of instructional materials and the adequacy of the content to the target audience's age and development. Twelve researchers and teachers answered surveys based on the evaluation standards of Project 2061 and the TEC rubric. Evaluation criteria were Substance, Sophistication, Educational Design, Teacher Accessibility, Equity and Content. Results show that, in general, the book is a good option for introducing computing contents to sixth-grade students. As issues for improvement, there is the need to add assessment materials to the book, and to better address equity issues. As positive aspects, the spiral approach, the detailed lesson plans and encouraging student collaboration and participation can be highlighted. From the results, the authors may improve the textbook, and potential users may take advantage of the detailed assessment.

This paper explores a survey instrument on Computational Thinking (CT) and attitudes towards Computing for educational purposes. Given the well-known skills demanded for the 21st century, CT needs to be known in the teacher community. One crucial aspect in this regard is the perceptions that students hold on the subject as well as on Computing in general. Various studies have been carried out to measure students' attitudes towards Computing. However, assessing the understanding of CT and Computing attitudes in undergraduate education, and particularly in teacher education, has not been fully explored yet. Most current instruments are either not aimed at educators or not designed to cover elements related to both CT and Computing. In this work, our goal was to evaluate a survey instrument on perceptions of CT and attitudes towards Computing for undergraduate students. Our evaluation focused on verifying whether the instrument could grasp the aspects involved in five different intended constructs: 1) understanding of CT, 2) application of CT in classroom, 3) confidence to learn Computing, 4) interest for Computing and 5) perception of Computing usefulness. After applying the instrument with 2,290 undergraduate students, we computed its reliability and used confirmatory factor analysis results to measure its validity. Results showed evidence that the instrument is both reliable and valid to measure impressions of CT and Computing attitudes in an educational context.

Undergraduate computing programs usually have high failure and dropout rates in the CS1 courses due to the variety of concepts, models and strategies related to programming to be learned in a short time span. This paper presents an experience of a CS0 remote workshop, planned and led by peers, with the goal of teaching introductory programming in Python to first-year Computer Engineering undergraduates. The workshop happened during the Covid-19 pandemic and took advantage of calendar postponements to offer a 12-week remote experience. From the data collected, we perceived high student motivation, medium to high engagement and medium self-regulation both in general and in programming issues. Analysis of the experience suggests that, after the workshop, students have positive perceptions of their own non-cognitive issues, in addition to feeling prepared to learn programming in the CS1 course. Peer learning allowed novice students to reduce barriers to learning, and to feel closer to their field of knowledge. To the senior students who led the workshop, it allowed them to mediate the learning process, to collectively build knowledge, and to learn while teaching.

Difficulties in introductory programming courses have been reported by several studies. Some authors associate these difficulties with students' sentiments, such as motivation and engagement, others with the learning approach. Active learning approaches, such as PBL, have been used to stimulate motivation and engagement as well as to improve learning outcomes. However, there is little evidence on learning improvements with this approach as well as its association with students' sentiments in CS1 courses. From this perspective, our study carried out an analysis of a hybrid PBL approach adopted in an introductory programming course for majors seeking to understand how students' performances and sentiments are expressed in this case. We performed an exploratory analysis of students' sentiments and performances to understand the relation of this hybrid approach on these variables throughout the course. We also computed correlations and multiple regressions to understand how sentiments in a PBL approach relate to students' performances. Our results suggest that PBL has a positive effect on students' motivation and engagement, being larger in the former. They also confirm the significance of motivation over learning outcomes, with an emphasis on the confidence and satisfaction categories. Engagement is also significant to learning, with an emphasis on the the dedication category. Finally, students' performances were better in PBL tutorials than in lectures, but we found no significant correlation between performance in PBL tutorials and a language-independent CS1 concept inventory.

This innovative practice full paper presents a report that describes an active learning experience with undergraduate freshmen from a Computer Engineering program in an introductory programming course using a Problem-Based Learning (PBL) approach. Programming is complex for many students and solutions such as the use of active learning methodologies have been addressed over the years. Since its creation, our Computer Engineering program uses PBL as a fundamental building block of its curriculum, and the CS1 course fully adopts this approach. This paper reports the planning, execution and analysis of our experience of teaching programming in CS1 using PBL. The main lessons learned were: building step-by-step examples helps the learning process; student participation happens in different ways in lectures and in the PBL sessions; the PBL tutorial brings relevance to the lectures; there must be a careful design of the PBL problems; the PBL approach works on technical, personal and interpersonal skills; and the lectures should be one step behind the PBL sessions.

This a Research Full Paper. The requirements of Software Engineering need a learning environment that is not only practical but true to the reality of the market. As a student centered approach, Problem-Based Learning (PBL) enables students to be collaborative and attitude-oriented during problem-solving. Despite this, PBL has its cultural challenges. Thus, this work intends to better understand how the students learn and behave through student meaningful learning profiles and Myers Briggs Type Indicator (MBTI). Based on the principle that these students participate or participated in a discipline that used the PBL method and are students of a course in Computer Science area. In this context, this article proposes a descriptive study to make initial analyses and kick start the research. Based on this study and its early results, we can conclude that learning dimensions require greater stimulation and is perceived an initial relationship between personality profiles and meaningful learning profiles.

This Research-to-Practice Full Paper evaluates an online programming course for K-12 educators that aimed to teach Computational Thinking (CT). CT is widely defined as a set of skills, such as problem solving and systems design, that rely on Computer Science principles. As these skills are increasingly demanded in modern society, it is critical to include CT in K-12 education to boost not only computing courses, but various other disciplines. However, most educators do not yet know the basics of CT and Computer Science fundamentals, particularly in Brazil. In this context, we designed and implemented an online programming course for K-12 educators to provide qualification in CT and Programming basics, as well as to assess learners' attitudes towards Computing, how their motivation are expressed during the course, and the correlation between motivation and attitudes. The results indicate that participants' attitudes towards Computing has become more positive after the course, primarily their comprehension of CT and confidence to learn Computer Science. The outcomes also suggest a high level of learners' motivation over the course, and reveal a high correlation between the relevance expressed in the course with their interest and confidence towards Computing.

This Research to Practice Full Paper describes a case study of a course that uses problem-based learning (PBL) to integrate the subjects of Object-Oriented Programming, Data Structures and Software Design. We aimed to explore student learning when this teaching-learning approach was applied in the second term of a Computer Engineering undergraduate program. Data collection and analysis was qualitative, using semi-structured interviews and both open and axial coding. Results were organized in terms of qualitative themes describing course organization, problem conception, learned concepts and skills. This thorough exploration of such issues allows course developers to reflect on the impact of PBL and curriculum integration in student learning.

This Research to Practice Full Paper presents a proposal for monitoring student progress in Problem-Based Learning (PBL). The adoption of the PBL approach has been growing in computer education, where problem-solving and group work are essential. Despite the compatibility and benefits of PBL, some challenges remain, in particular, with respect to the assessment process. For an effective assessment process, it needs to be well defined and managed by both teachers and students themselves, considering that, in PBL, the students are at the center of the teaching and learning process, they are active, and self-regulating. In this context, this paper proposes an interface for student progress monitoring (a “student board”) based on an authentic assessment model called PBL-SEE. Constructed using the Design Science Research (DSR) method, this interface was initially prototyped and validated by PBL specialists. The results showed a good acceptance of the student board and important recommendations for improvements.

This Research to Practice Full Paper analyzes the impact of using a problem-based learning (PBL) approach on student motivation and engagement in an introductory programming course (CS1). Learning programming is a difficult task, and CS1 courses have experienced high dropout and failure rates. Some authors associate this to students' lack of motivation and recommend the use of active learning methods to stimulate their motivation and engagement. Educators have been interested in using the PBL approach, an active instructional method based on problem solving, for its motivational potential. In this context, we surveyed students with the ARCS IMMS and UWES-S questionnaires to respectively measure their levels of motivation and engagement in a CS1 course. Results show that, on the categories of motivation, the PBL approach relates to students' attention and relevance. It also relates to confidence and satisfaction, either positively or negatively, depending on course planning. The approach, on the other hand, does not have much relation with the categories of engagement. Finally, we also found that there must be careful problem design and planning in order to positively relate to student motivation and engagement.

This innovative practice full paper presents an experience report of an approach carried out with students of a vocational secondary program in Informatics, in an introductory course on Computer Programming. In Brazil, in addition to the undergraduate education, an important part of professional education is based on technical vocational secondary programs, geared towards faster training and practice. In information technology (IT) vocational programs, high levels of dropout and failure are frequent. Much of this failure happens in computer programming courses. Various factors contribute to the lack of student motivation and, thus, to the high professional deficit in the field of IT. This work presents an experience report of an approach used with students of a technical vocational IT program. The approach is based on activity contextualization, environments for novices and a spiral learning approach. The experience was split into three contextualized blocks: Figures, Games and Images. In each block, concepts are (re)introduced and deepened in a spiral way. Lessons learned suggest that: context change impacts student motivation, the approach is partially adequate to the audience, the Scratch tool contributes to learning programming concepts, and appropriate tools help transitioning between block-based and text-based programming languages.

This research full paper investigates the development of soft skills of undergraduate students in a Computer Engineering program that uses problem-based learning (PBL) as its main learning approach. As the role of engineers have changed over the years, their need for a different set of skills has increased. Communication, leadership, creativity and management, known as soft skills, are increasingly needed in their positions. For that reason, undergraduate programs have been pushing to add the development of soft skills in their curriculum. One way to accomplish that change is using active learning approaches. Such approaches usually lead to more student participation in the learning process and, consequently, the development of particular soft skills. This research investigates the use of the active learning approach of Problem-Based Learning (PBL) to enable the development of soft skills of Computer Engineering undergraduate students. Students' skills were assessed through a Likert Scale questionnaire, investigating eight skill categories: Self-Management, Management, Interpersonal skills, Communication, Leadership, Decision Making and Problem Solving, Organization, and Analysis and Creativity. Results show that the students' perceptions of their own skills drop during the course of their studies, but rise when they get closer to their graduation.

This research to practice full paper investigates software engineering students' perceptions of their contact with open source projects as a real-world experience. Working with open source projects (OSPs) has been shown as an interesting option in software engineering courses to bringing students closer to more realistic environments. However, when instructors use this approach, it is not clear whether students perceive the OSP as a real industrial software project, or whether they perceive the tasks they perform over OSPs as typical or close to industrial software project activities. The goal of this work was to identify students' perceptions of their interaction with an open source project as a real world experience. To do so, we performed three mixed-methods case studies with three different undergraduate classes. Each class had a different focus: i) software maintenance and evolution, ii) software testing, and iii) reverse engineering of software requirements. Results show that students perceived features that make OSPs close to industrial projects, realized that their OSP tasks are close to the ones in industrial projects, and also faced difficulties typical of working with real world software. Moreover, students forged a view of the skills needed for their future professional success. We conclude that students realized that performing tasks in OSPs was a real world experience they took part, contributing to their background both for the competencies they acquired and the difficulties they had to overcome.

This innovative practice full paper presents an experience report of an approach integrating the teaching and learning of Object-Oriented Programming, Data Structures and Software Design in the second term of a Computer Engineering undergraduate program. Learning object-oriented programming requires acquiring high-level skills, which is negatively affected by present curricula and pedagogies. Isolated courses with teacher-centered approaches do not allow for the appropriate practice of such skills. For 15 years, our Computing Engineering undergraduate program has been pursuing an effort of curriculum integration and active learning practices based on problems and projects. This paper presents an experience report of our approach. This experience led us to important lessons learned with our approach: the acquisition of personal, interpersonal and technical skills provided by the approach; the benefits of knowledge integration through more authentic experiences and a more disciplined practice of software production; the need for careful problem planning; the main difficulties faced by instructors to manage the course; and the challenges faced by students to develop their skills.

This innovative practice full paper reports our attempts to increase motivation of CS1 non-major students through an approach contextualized by games and media. We report our experience to conceive and deliver an introductory programming course for CS non-majors. We used a mixed approach which combines Scratch with game creation, and Python with both turtle graphics and image manipulation in order reduce initial barriers to learning and increase student motivation by using different contexts. The course was given to university-level, freshman, civil engineering students who take programming as a required first-term course. The main lessons learned were: a spiral approach over programming content reduces cognitive load and facilitates learning; careful design for a transition between tools/languages is crucial to minimize difficulties; games and media bring freshmen closer to the programming world; and playful contexts improve motivation, but only to a certain extent.

This research to practice full paper investigates motivation of engineering students with a mixed-contexts approach to introductory programming. Non-major CS1 students usually have more difficulties learning computer programming than CS majors, for reasons such as lack of interest in computing or courses formatted for CS majors. Student motivation is vital in the learning process, and fostering it is important to promote more effective learning. Various initiatives have been proposed to increase engagement and reduce the difficulties faced by non-majors in CS1 courses. We believe that motivation can be improved by using a mix of languages and tools already tested in CS1 teaching. In this paper, we evaluate a teaching approach for non-majors, which combines the use of Scratch in a context of game creation, and Python with both turtle graphics and image manipulation. We performed an exploratory case study with CS1 civil engineering students from our institution to analyze the impact of this approach on student motivation. Our results describe motivation arisen during the course in terms of attention, relevance, confidence and satisfaction (ARCS), and as practical factors that contribute to either increase or decrease motivation.

This research full paper characterizes the literature on academic initiatives to foster computational thinking and programming (CT&P) in Brazilian K-12 education. Context: Mapping and analyzing the diversity of experiences and studies that address CT&P in K-12 education can bring valuable data to researchers. This work delimits such study to the Brazilian scenario to allow a more in-depth view, given the Brazilian context. Previous surveys and systematic mapping studies present recent publications in major Brazilian computing journals and conferences. Although they offer important contributions, they do not comprehensively cover the Brazilian literature on CT&P in K-12 education, since they focus the search in the Brazilian Computer Society publications alone. Objective: This work proposes to characterize the literature on CT&P in K-12 education in Brazil. Results: through a systematic mapping study, we collected information on year, venue, type, K-12 education stage and modality, methodological contexts, and used tools and programming languages from 338 selected primary studies from 2001 until 2016. Conclusions: there is a significant increase in the number of Brazilian studies in the latest years, showing a growing interest on this research area as well as several trends and gaps to be addressed by both researchers and practitioners.

This research to practice full paper explores the role of programming workshops for girls of primary education in changing perceptions and interests of this audience regarding joining this field. A programming workshop with the playful environment Scratch and a teaching-learning approach based on challenges was held in a middle school of city in the northeast of Brazil, and was analyzed using a mixed-methods case study. Results point to increased interest in the field after the workshop and no confirmation of erroneous stereotypes about the area. We conclude that this type of workshop is relevant to better understand and potentially improve girls' perceptions about the field of computing.

Previous studies have presented approaches to teach programming based on contexts close to students, such as games, robotics, and media. Those contexts may turn learning easier and more motivating. Media manipulation is one of such contexts relevant to teenage students, for their thorough use of image applications and social networks. In this work, we design and evaluate a spiral approach to teach programming to ninth-grade students of a middle school in Brazil's countryside. The approach is contextualized by media, i.e., image creation and manipulation, and we use the Python language with turtle graphics and JES tools to support it. Results point out to the influence of context and tools on learning, significant changes of perception about computing, high motivation to learn how to code, as well as a positive correlation between learning and motivation.

Helping college freshmen to learn basic computer programming is a longstanding research topic. Various environments, tools and languages have been developed to ease the initial steps of novice programmers. However, to be used to their full extent, such artifacts should be better combined with an appropriate learning approach. This work describes pre-term workshops offered to Computer Engineering undergraduate freshmen that combines the use of the Scratch learning environment with a learning approach based on peer support, game development and a strategy of challenge-response. Results are derived from quantitative data analysis: surveys and artifact analysis. Students evaluated the workshop well: they found it stimulant, lightweight, with good teaching, useful, well-organized, and conducive to learning. Results suggest the effectiveness of the approach to reduce initial difficulties for freshmen learning programming concepts.

Problem-based learning (PBL) is a well-known active learning approach, and it is becoming increasingly popular in computing. The Computer Engineering undergraduate program at UEFS adopts PBL since 2003. Previous studies thoroughly describe PBL course design and practices, but there is a lack of reports about the teaching experience, i.e., how faculty live the process of teaching PBL courses. Thus, we took advantage of this PBL experience to uncover such issues. We developed an interpretative phenomenological analysis (IPA) to learn the essences of being a PBL teacher, using a qualitative research methodology. Data collection and analysis entailed semi-structured interviews with five UEFS computer engineering professors, interview transcription and memo writing, open coding, code memo writing, code grouping and abstraction, and description of essences. Results led to the following essences: 1) feedback is essential for student success; 2) PBL develops better students and professionals; 3) assessment is complex and multifaceted; 4) developing good problems is a difficult skill; 5) PBL requires strong teacher engagement and background; 6) it is essential to keep a motivating scenario; and 7) good coordination and group dynamics is required. In this paper, we describe the four first essences in detail.

Context: It is common practice in academia to have students work with “toy” projects in software engineering courses. One way to make such courses more realistic and reduce the gap between academic courses and industry needs is getting students involved in Open Source Projects with faculty supervision. Objective: This study aims to summarize existing information on how open source projects have been used to facilitate students' learning of software engineering. Method: A systematic mapping study was undertaken by identifying, filtering and classifying primary studies using a predefined strategy. Results: 53 papers were selected and classified. The main results were: a) most studies focus on comprehensive software engineering courses, although some papers deal with specific areas; b) the most prevalent approach was the traditional project method; c) surveys are the main learning assessment instrument, especially for student self-assessment; d) conferences are the typical publication venue; and e) more than half of the studies were published in the last five years. Conclusions: The resulting map gives an overview of the existing initiatives in this context and shows gaps where further research can be pursued.