Evaluation of educational innovation at the course level is dominated by frameworks that are linear, built on simple models, lack grounding in theory, and are generally not flexible enough to provide actionable recommendations for improvements. Moreover, the guidelines provided by existing literature are difficult to transfer to other contexts as most of these are aimed at specific innovations only. Therefore, there is a need for a new approach that is flexible enough to enable adaptation to different contexts, and that draws on the strengths of existing evaluation traditions. This paper forms a starting point for a larger research initiative for developing a comprehensive innovation evaluation framework. The context of the paper is a project based within the four universities of technology in The Netherlands (4TU) dedicated to strengthening collaboration in engineering education and research needed to take on global societal problems. The potential reach of our framework has implications for the wider field. This paper describes the results of an exploratory literature review to draw on the strengths of various evaluation theories to develop a new approach to evaluating educational innovation in courses. Value, Methods, and Use are the main components identified for this approach. Secondly, we developed a workshop to pilot this approach. Lastly, the workshop was presented to educational researchers, engineering educators, and educational advisors. We share lessons learned from the workshop and conclude with descriptions of future research to refine the framework to prepare its application in real innovation initiatives.

This research paper aimed to evaluate the effectiveness of AI bots in simulating human subjects for research purposes. In recent years, the use of artificial intelligence (AI) in education has gained significant attention. While most approaches explore possibilities for teaching and learning, the possibilities for human subject research have been under-explored. By utilizing AI bots as a tool for data collection and analysis, researchers can potentially reduce costs, increase efficiency, and improve overall accuracy in their studies. This project focused on replicating a traditional qualitative research study on the topic of “humility in engineering“ with real human subjects in a simulated study using several publicly available generative AI tools - ChatGPT, Gemini, DebateAI, MasterDebater - to evaluate the effectiveness of AI bots as simulators of human subject research. The project assessed the “personalities“ and response patterns of AI tools leading to ethical implications of using AI bots as research simulators. In addition to evaluating the effectiveness and ethical implications of AI bots as simulators of human subject research, the project also explored the potential benefits and limitations of this approach. Overall, this project represents an important step towards understanding the role of AI bots in research and their potential impact on the engineering education research community. By evaluating the use of AI bots as simulators of human subject research, researchers can gain valuable insights into the benefits, limitations, and ethical considerations of this approach. Ultimately, this project seeks to advance the field of research methodology and contribute to the ongoing dialogue surrounding the use of AI in scientific studies.

Daily stand-up meetings (DSMs) are the most popular technique in agile methodologies and are deemed crucial for communication among the individual agile software developers to perform. Only a handful of studies in the past have shown the effect of the characteristics of DSMs on the attitude of agile developers. Little is known about how agile developers experience DSMs in their roles as junior/senior developers, and what feelings evoke these experiences. The purpose of this study was to describe and interpret agile developers’ lived experiences with DSMs in their diverse roles. We conducted a hermeneutic phenomenological study with 19 professional agile developers. The lived experiences across the interviews revealed as an interaction between four categories: rationalising irrelevancy of DSMs, experiencing challenges with DSMs, conflicted opinion on the advantages of DSMs, and finding solutions. Developers experienced DSMs were too short to facilitate clear problem identification, or solve problems, or have a meaningful outcome. Senior developers experienced DSMs differently than junior developers in terms of sharing information, interest in other’s work, monitoring progress, and facilitating decision making. Based on these findings, we discuss the theoretical contributions of our study, and offer recommendations for practitioners.

The purpose of this work-in-progress study was to research teachers' emotional reactions to engineering employing a metaphoric approach. There are two research questions: What is the range of emotional and metaphoric responses of precollege teachers to engineering? What are prevalent and recurring themes and emotional undertones of precollege teachers' when describing engineering in metaphors? Using a phenomenological approach, we analyzed 298 statements from the data collection process by asking participants questions such as “Engineering is like ... Explain why”, for example “If engineering were a plant, it would be... Explain why”. These metaphorical statements were grouped under two core categories to answer our research questions: A) Categories of Metaphoric Responses, and B) Categories of Emotional. Responses Those categories reflect pre-service teachers' beliefs, attitudes and emotions in regards to engineering subjects and should be further investigated for insights about pre-service/in-service teacher education.

This work-in-progress reports on a project to establish differences between STEM education, and specifically Engineering Education, and other education fields/disciplines based on empirical observations. In this study we report on first steps towards a literature overview of such differences and on the development of an analytical framework to analyze the publications.

Technical and interpersonal competences are essential characteristics that engineers need to obtain nowadays. Prior studies suggest that empathy and care are important skills in engineering within an increasingly global and innovative work environment. However, there is still limited literature about engineers‘ perceptions of empathy and care particularly when developing curricular approaches to strengthen engineers‘ empathic competences. Thereby, the current study aimed first to validate the factor structure of the US-American Empathy and Care Questionnaire (ECQ), by Hess, Strobel, Pan, and Wachter Morris (2017), regarding German engineers‘ perceptions of empathy and care at their workplaces. Second, the study sought to investigate the differences of perceived empathy of German engineers concerning engineers‘ years of work experience. Results suggest the five-factor model fit was acceptable and that German engineers with more professional experience perceived empathy and care as a more important aspect of their work in comparison to those with less professional experience.

This Innovative Practice Work-In-Progress paper elucidates the approach of the NSF-funded CAHSI INCLUDES Alliance for creating change in students' competencies by an effort across eight institutions to support the delivery of one-and two-credit hour courses for three levels of problem solving in Computer Science: general problem solving, computational thinking in problem solving, and algorithmic thinking in problem solving. The courses were developed to address industry's need for improved problem-solving skills, incorporating consistent, deep collaboration with Google technical staff The first of its kind for CAHSI, the problem-solving courses are fewer credit hours than typical courses in order to fit within a traditional curriculum. The intent is to instill complementary problem-solving, computational thinking skills, and logical reasoning needed to succeed in computer science, and make this content available across different student populations at various stages in their academic pathways. Advanced problem solving prepares students for competitive interviews. The courses create opportunities to learn across academic levels, and create new student communities, mentorship opportunities, and social connections to support retention. The paper reports on the course design, student reflection, assessment and evaluation, and an ethnographic study of the courses.

This study uses social network analysis to analyze teacher social capital in a NSF-funded, DRK-12 cyber-enabled engineering education teacher professional development program. The social capital leveraged among the teachers can be visually represented by attributes such as schools, grades, cohorts, and subgroups. Each teacher's social capital is calculated to complement the analysis.

This paper explores what is included in the worldview of the modern engineer and how this compares to the concept of sustainability. Worldviews are important to humanity because they are interwoven throughout civilizations. Societies do not contain but one homogenous worldview, however, they do essentially contain a dominant worldview characterized by the collection of values, beliefs, habits, and norms. This dominant worldview forms the frame of reference for a collectivity of people, such as a nation or culture. In this paper, we attempt to articulate modern worldviews, the contemporary engineering worldview, and the sustainability worldview. We use the concept of worldviews to address the compatibility of sustainability and engineering. Our synthesis suggests that the two ideologies are misaligned and incompatible in many respects. We suggest that for sustainability to gain prominence within an engineering context, engineers and engineering educators must first become conscious of these inconsistencies. Through the philosophical synthesis presented in this paper, it is our goal to begin rethinking how we educate engineering students about engineering and sustainability.

A new model of metacognition scaffolding, Metacognition in the Engineering Design Process (MCinEDP), which parallels instruction in engineering design processes (EDP), has been designed for use in pre-college, particularly 2-6, engineering classrooms. MCinEDP utilizes engineering as the context for implementation of metacognitive activities that are designed to aid students in building mastery of metacognition and to improve specific abilities in engineering design. This paper outlines the design process in order to reveal the intersection of instructional design theory, instructional design models, cognitive theory and learning theory that supported this project.

In this paper, we describe the design of a goal-based problem solving oriented interactive game on biodiversity. The audience is children ranging from age 6 to 14. We focus in our paper especially on a) the instructional design of the game and b) the support structures. We differentiate between modeling, coaching, scaffolding, and general support structures and focus especially on the design of scaffolds that were integrated in order to provide learners with feedback, guidelines, and help structure.