Developing problem-solving competency is central to Science, Technology, Engineering, and Mathematics (STEM) education, yet translating this priority into effective approaches to problem-solving instruction and assessment has been a significant challenge. The recent proliferation of generative artificial intelligence (genAI) tools like ChatGPT in higher education introduces new considerations: how to define problem-solving competency in a genAI era, and how these tools can help or hinder students' development of STEM problem-solving competency. Our research takes steps in examining these considerations by studying how and why college students are currently using genAI tools in their STEM coursework, with a specific focus on how they employ these tools to support their problem-solving. We conducted an online survey of 40 STEM college students from diverse institutions across the US. In addition, we surveyed 28 STEM faculty to understand instructor views on effective and ineffective genAI tool use in STEM courses and their guidance for students. Our findings reveal high adoption rates and diverse applications of genAI tools among STEM students. The most common use cases of genAI tools in STEM coursework include finding explanations, exploring related topics, summarizing readings, and helping with problem-set questions. The primary motivation for using genAI tools in STEM coursework was to save time. Moreover, we found that over half of the student participants reported simply inputting a problem for AI to generate solutions, potentially bypassing their own problem-solving processes. These findings indicate that despite high adoption rates, students' current approaches to utilizing genAI tools often fall short in enhancing their own STEM problem-solving competencies. The study also explored students' and STEM instructors' perceptions of the benefits and risks associated with using genAI tools in STEM education. Our findings provide insights into how to guide students on appropriate genAI use in STEM courses and how to design genAI-based tools to foster students' problem-solving competency.

A growing number of social science researchers, including educational researchers, have turned to online crowdsourcing platforms such as Prolific and MTurk for their experiments. However, there is a lack of research investigating the quality of data generated by online subjects and how they compare with traditional subject pools of college students in studies that involve cognitively demanding tasks. Using an interactive problem-solving task embedded in an educational simulation, we compare the task engagement and performance based on the interaction log data of college students recruited from Prolific to those from an introductory physics course. Results show that Prolific participants performed on par with participants from the physics class in obtaining the correct solutions. Furthermore, the physics course students who submitted incorrect answers were more likely than Prolific participants to make rushed cursory attempts to solve the problem. These results suggest that with thoughtful study design and advanced learning analytics and data mining techniques, crowdsourcing platforms can be a viable tool for conducting research on teaching and learning in higher education.

Learning how to solve authentic problems is an important goal of education, yet how to assess and teach problem solving are research topics to be further explored. This study examines how interaction log data from a computerized task environment could be used to extract meaningful features in order to automate the assessment of reflective problem-solving practices. We collected survey responses and interaction log data of 40 college students working to solve the mass of a ”mystery object” in an interactive physics simulation. The log data was parsed to reveal both the test trials conducted to solve the problem and the pauses in-between test trials, where potential monitoring and reflection of the problem-solving process took place. The results show that reflective problem-solving practices, as indicated by meaningful pauses, can predict problem-solving performance above and beyond participants’ application of physics knowledge. Our approach to log data processing has implications for how we study problem solving using interactive simulations.

Interactive simulations provide an exciting opportunity to assess and teach students the practices used by scientists and engineers to solve real-world problems. This study examines how the logged interaction data from a simulation-based task could be used to automate the assessment of complex problem-solving practices. A total of 73 college students worked on an interactive circuit puzzle embedded in a science simulation in an interview setting. Their problem-solving processes were videotaped and logged in the backend of the simulation. We extracted different sets of features from the log data and evaluated their effectiveness as predictors of students' problem-solving success and evidence for specific problem-solving practices. Our results indicate that the application of data mining techniques guided by knowledge gained from qualitative observation was instrumental in the discovery of semantically meaningful features from the raw log data. These knowledge-grounded features were significant predictors of students' overall problem-solving success and provided evidence on how well they adopted specific problem-solving practices, including decomposition, data collection, and data recording. The results point to promising directions for how scaffolding/feedback could be provided in educational simulations to enhance student learning in problem-solving skills.

Work in progress: Engineering graduates rank problem-solving as one of the most important skills they use in their careers. Despite the widely recognized importance of problem-solving, there is little evidence to suggest that we are teaching our students how to solve the complex problems they will encounter in their future careers. Part of the reason we lack such evidence is that almost no tools exist that measure meaningful dimensions of authentic problem-solving. We have developed an assessment of authentic problem-solving based on studies of how expert scientists and engineers solve problems. In the chemical engineering version of this assessment, students are asked to troubleshoot a flawed chemical process design; this requires them to make many of the same decisions that an expert engineer does when they solve real-world problems. In previous studies, we have validated the use of this assessment to measure differences between novice and more advanced students and to identify best practices for administering the assessment. In this study, we use the assessment to measure the problem-solving outcomes of two different versions of a capstone design course in chemical engineering: one that focuses on product design, and one on traditional plant design. We discuss preliminary results from two highly selected private universities in the United States. This work is providing valuable information as to the problem solving learning differences from different capstone design course experiences.

Teaching students to become skillful problem solvers is a goal of university education, but it has been difficult to measure such skill or demonstrate the benefits of particular educational experiences. This paper presents a study of college students solving a problem unrelated to their academic majors. The analysis suggests that the educational experiences of Computer Science (CS) students may better train them in problem-solving than the experiences of other majors. In this study, students from a variety of undergraduate majors and grade levels were given a 15-minute problem-solving task embedded in an interactive science simulation. The complex task calls upon many problem-solving practices needed by scientists and engineers in their professions. Although this task has little resemblance to the problems encountered in a computer science course, CS students performed significantly better than students in any other major. In addition, only for CS students was there an indication of improvement in problem-solving from lower to upper grade levels. We propose that general problem-solving and computational thinking share some common practices, such as problem decomposition and comprehensive data collection. Furthermore, we present preliminary evidence that training in computational thinking is transferable to problem-solving tasks across domains and discuss how the unique features of CS programming assignments could be generalized to other science and engineering courses to foster students' general problem-solving skills.

Guided by experimental tests of theory and practice, science has advanced rapidly in the past 500 years. Guided primarily by tradition and dogma, science education meanwhile has remained largely medieval. Research on how people learn is now revealing how many teachers badly misinterpret what students are thinking and learning from traditional science classes and exams. However, research is also providing insights on how to do much better. The combination of this research with modern information technology is setting the stage for a new approach that can provide the relevant and effective science education for all students that is needed for the 21st century. I will discuss the failures of traditional educational practices, even as used by "very good" teachers, and the successes of some new practices and technology that characterize this more effective approach, and how these results are highly consistent with findings from cognitive science.