This work-in-progress research paper explores the effects of generative AI design tools on engineering students' design artifacts during computer-aided design (CAD) tasks with the objective to better understand the impact of emergent design tools on design artifacts to better inform the development of curriculum surrounding generative design. Our research explored the question: to what extent does engaging in generative design produce a different quality artifact as compared to traditional design? This study utilized a mixed methods approach to compare students'$(\mathrm{n}=20)$CAD artifact quality between two separate tasks; one completed with traditional parametric modeling and the other with generative design tools. Preliminary findings indicate that students produced a significantly higher quality design artifact using generative design as compared to artifacts created using traditional parametric methods.

Generative design uses artificial intelligence-driven algorithms to create and optimize concept variants that meet or exceed performance requirements beyond what is currently possible using the traditional design process. However, current generative design tools lack the integration of human factors, which diminishes the efforts to understand and inject a broad set of human capabilities, limitations, and potential emotional responses for future human-centered product and service innovation. This paper demonstrates collaborative research in formulating a human-centered generative design framework that injects human factors early in the design for quick-and-dirty concept creation and evaluation. Three case studies overviewing our ongoing multidisciplinary research efforts in synthesizing human and mechanical attributes are presented. The results show that the framework has the potential to enhance human factors representation within generative design workflow. Strategies from a computational design perspective, such as data-driven generative design, digital human modeling, and mixed-reality validation, are discussed as alternative approaches that could be implemented to augment designers.

This work-in-progress, set in the context of an introductory design and graphics course, explores student reasoning when tasked with choosing a “best” option among solutions developed from a CAD-based generative design solution space. Students tend to cite rationalistic reasoning in design decision-making, rather than relying on intuition or on empathy for users. Future work will explore the relationship between traditional undergraduate engineering design task decision-making and generative design decision-making.

(Work-in-Progress) One of the key features of engineering that differentiates it from science is trade-off decisions which affect the overall quality of a design artifact. Making trade-offs is a complex cognitive process that involves weighing possible outcomes against their respective benefits and costs in areas such as aesthetics, cost, degree of safety, and various performance indicators. Making trade-off decisions is an effective design practice, and is a key dimension of successful performance. Designers and design researchers have methods to assess design artifact quality. While useful, these methods are not necessarily easy to use nor do they indicate design competency. Moreover, they are not grounded in a definition of engineering design. The objective of this study was to complete a comprehensive review of design literature to synthesize common tools used by designers and design students to assess design artifact quality as well as approaches used to assess design quality, where trade-offs are inherently integrated into this decision-making to compare design process to design outcome measures. Results and visuals compare and contrast how these tools and approaches assess design artifacts in terms of: (1) encompassing multiple complementary and competing dimensions, (2) consistent and systematic application, and (3) design competency level communication. This work shows the common pitfalls in common design artifacts assessment tools and methods with respect to understanding a key design behavior, making trade-offs, and establishes the need for a comprehensive way to assess student designer trade-off decisions in design that are easy to use and conceptually grounded in a definition of engineering design.

Engineering design is a complex experience for students to undertake and for instructors to assess. Conducting tests is a critical design practice, yet the research on K-12 students' conceptions on conducting experiments is limited. Such research is essential as we attempt to understand how students become informed designers and ways in which we can support their transformation. By analyzing students' prioritization and re-prioritization of design strategies after participating in a design activity, this study examined how students' conceptions of design activities change over time. The study took place in three middle schools with 746 students. The main data source was the “conceptions of design test,” which students completed a pre- and post-test, before and after completing a design project. We performed McNemar tests to quantitatively analyze students' changing conceptions of design. Results suggest that after a design activity, “conducting tests” became a statistically more important concept to students. Future work will investigate student rationale for this increased importance placed on “conducting tests” in design.

Design is recognized as a key engineering activity, and engineering is a fundamental part of science education at the K-12 level. However, it is difficult to assess student designs when the range of “correct” answers is wide. Feedback in the form of assessment helps students learn from a design activity and can direct students along the pathway of improvement. The purpose of this paper is to develop an assessment protocol to measure solution quality taking into account both objective and subjective design criteria (e.g. measurements of cost/energy used along with aesthetics). Three protocols are developed by analyzing and comparing 109 high school students' design solutions to a zero-energy home design task. Lessons learned from the first two approaches informed the third approach that highlights the importance of balancing trade-offs. Results suggest the Trade-off Value approach provides an intuitive and accurate way to understand how well a designer has balanced both complementary and competing design criteria. These results can be used as feedback to support a systems design process and to evaluate the relationship between design quality and design behavior.

Engineering design is a complex activity for students to undertake and for instructors to assess. This research uses large learner data sets collected through automatic, unobtrusive logging of student actions in a CAD platform to address this difficulty in observing design behavior. We used a computer-aided design software that captured student design activities to investigate patterns of student design behaviors that are associated with idea fluency. We show how micro-level process data can be used to validate observations made from viewing the student design process through design replays. Students who engaged in high idea fluency showed evidence of fluency in both process data and design replays. Similar patterns were observed for low idea fluency students. There is great potential to investigate student design learning through system-collected data. Yet, how to justify the inferences made about students based on their process data is largely unexplored. Our results demonstrate how traditional forms of assessment data can be used to validate inferences made by process data. Implications of this work would be highly relevant to engineering educators as well as researchers who are interested in understanding the relationship between learner analytics and student learning.