Developing problem-solving competency is central to Science, Technology, Engineering, and Mathematics (STEM) education, yet translating this priority into effective approaches to problem-solving instruction and assessment has been a significant challenge. The recent proliferation of generative artificial intelligence (genAI) tools like ChatGPT in higher education introduces new considerations: how to define problem-solving competency in a genAI era, and how these tools can help or hinder students' development of STEM problem-solving competency. Our research takes steps in examining these considerations by studying how and why college students are currently using genAI tools in their STEM coursework, with a specific focus on how they employ these tools to support their problem-solving. We conducted an online survey of 40 STEM college students from diverse institutions across the US. In addition, we surveyed 28 STEM faculty to understand instructor views on effective and ineffective genAI tool use in STEM courses and their guidance for students. Our findings reveal high adoption rates and diverse applications of genAI tools among STEM students. The most common use cases of genAI tools in STEM coursework include finding explanations, exploring related topics, summarizing readings, and helping with problem-set questions. The primary motivation for using genAI tools in STEM coursework was to save time. Moreover, we found that over half of the student participants reported simply inputting a problem for AI to generate solutions, potentially bypassing their own problem-solving processes. These findings indicate that despite high adoption rates, students' current approaches to utilizing genAI tools often fall short in enhancing their own STEM problem-solving competencies. The study also explored students' and STEM instructors' perceptions of the benefits and risks associated with using genAI tools in STEM education. Our findings provide insights into how to guide students on appropriate genAI use in STEM courses and how to design genAI-based tools to foster students' problem-solving competency.

A growing number of social science researchers, including educational researchers, have turned to online crowdsourcing platforms such as Prolific and MTurk for their experiments. However, there is a lack of research investigating the quality of data generated by online subjects and how they compare with traditional subject pools of college students in studies that involve cognitively demanding tasks. Using an interactive problem-solving task embedded in an educational simulation, we compare the task engagement and performance based on the interaction log data of college students recruited from Prolific to those from an introductory physics course. Results show that Prolific participants performed on par with participants from the physics class in obtaining the correct solutions. Furthermore, the physics course students who submitted incorrect answers were more likely than Prolific participants to make rushed cursory attempts to solve the problem. These results suggest that with thoughtful study design and advanced learning analytics and data mining techniques, crowdsourcing platforms can be a viable tool for conducting research on teaching and learning in higher education.

Digital games offer promising platforms for assessing student higher-order competencies such as problem-solving. However, processing and analyzing the large volume of interaction log data generated in these platforms to uncover meaningful behavioral patterns remain a complex research challenge. In this study, we employ sequence mining and clustering techniques to examine students’ log data in an interactive puzzle game that requires player to change rules to win the game. Our goal is to identify behavioral characteristics associated with the problem-solving practices adopted by individual students. The findings indicate that the most effective problem solvers made fewer rule changes and took longer time to make those changes across both an introductory and a more advanced level of the game. Conversely, rapid rule change actions were linked to ineffective problem-solving. This research underscores the potential of sequence mining and cluster analysis as generalizable methods for understanding student higher-order competencies through log data in digital gaming and learning environments. It also suggests future directions on how to provide just-in-time, in-game feedback to enhance student problem-solving competences.

Learning how to solve authentic problems is an important goal of education, yet how to assess and teach problem solving are research topics to be further explored. This study examines how interaction log data from a computerized task environment could be used to extract meaningful features in order to automate the assessment of reflective problem-solving practices. We collected survey responses and interaction log data of 40 college students working to solve the mass of a ”mystery object” in an interactive physics simulation. The log data was parsed to reveal both the test trials conducted to solve the problem and the pauses in-between test trials, where potential monitoring and reflection of the problem-solving process took place. The results show that reflective problem-solving practices, as indicated by meaningful pauses, can predict problem-solving performance above and beyond participants’ application of physics knowledge. Our approach to log data processing has implications for how we study problem solving using interactive simulations.

Interactive simulations provide an exciting opportunity to assess and teach students the practices used by scientists and engineers to solve real-world problems. This study examines how the logged interaction data from a simulation-based task could be used to automate the assessment of complex problem-solving practices. A total of 73 college students worked on an interactive circuit puzzle embedded in a science simulation in an interview setting. Their problem-solving processes were videotaped and logged in the backend of the simulation. We extracted different sets of features from the log data and evaluated their effectiveness as predictors of students' problem-solving success and evidence for specific problem-solving practices. Our results indicate that the application of data mining techniques guided by knowledge gained from qualitative observation was instrumental in the discovery of semantically meaningful features from the raw log data. These knowledge-grounded features were significant predictors of students' overall problem-solving success and provided evidence on how well they adopted specific problem-solving practices, including decomposition, data collection, and data recording. The results point to promising directions for how scaffolding/feedback could be provided in educational simulations to enhance student learning in problem-solving skills.

Teaching students to become skillful problem solvers is a goal of university education, but it has been difficult to measure such skill or demonstrate the benefits of particular educational experiences. This paper presents a study of college students solving a problem unrelated to their academic majors. The analysis suggests that the educational experiences of Computer Science (CS) students may better train them in problem-solving than the experiences of other majors. In this study, students from a variety of undergraduate majors and grade levels were given a 15-minute problem-solving task embedded in an interactive science simulation. The complex task calls upon many problem-solving practices needed by scientists and engineers in their professions. Although this task has little resemblance to the problems encountered in a computer science course, CS students performed significantly better than students in any other major. In addition, only for CS students was there an indication of improvement in problem-solving from lower to upper grade levels. We propose that general problem-solving and computational thinking share some common practices, such as problem decomposition and comprehensive data collection. Furthermore, we present preliminary evidence that training in computational thinking is transferable to problem-solving tasks across domains and discuss how the unique features of CS programming assignments could be generalized to other science and engineering courses to foster students' general problem-solving skills.