This research paper describes the concept evolution of a student design team's early design process as they participated in NASA's 2021 BIG Idea Challenge. The team and final design concept began as a course project that was accepted by the BIG Idea judges and led to the filing of a patent application. The BIG Idea Challenge addressed the high-risk issue of lunar dust mitigation, requiring the team of student engineers to design for an environment in which they had no previous knowledge or experience. An analysis of the students' use of design tools and methods that led to this successful design result in the context of a difficult and unknown problem, as well as the pedagogical framework within which the methods were taught, can be instructive for design course pedagogy and for guiding design methods in the industry. An analysis of design team documentation required by the course and digitized for team-sharing during the COVID pandemic produced a detailed understanding of the concept generation, evolution, and selection process. The structured application of multiple concept generation and refinement tools, including 6-3-5, TRIZ, and bioinspired design, allowed for the student to explore a wide range of solutions, and contribute meaningful improvements to their leading concepts. Additionally, extensive background research conducted before and during the concept generation phase enabled the design team to incorporate existing technologies in novel form factors. The final concept selection phase also included multiple consultations with experts and a critical analysis of the lunar environment to ensure that the team might be best positioned to have their product considered for implementation.

Academic makerspaces have continued to rise in popularity as research shows the diverse benefits they provide to students. More and more engineering curriculums are incorporating makerspaces and as such there is a need to better understand how their student users can best be served. Surveys administered to makerspace users at a public research university in the Southwest United States during Fall 2020, Spring 2021, Spring 2022, and Fall 2022 tracked student tool usage trends with academic career stages. The survey asked questions about prior experience, motivation, tool usage, and demographics. Analyzed results for mechanical engineering student users provide insight into how curriculum and class year affect the specific tools used and the percentage of student who used a particular tool. The survey results also create a bipartite network model of students and tools, mimicking plant-pollinator type mutualistic networks in ecology. The bipartite network models the student interactions with the tools and visualizes how students interact with the tools. This network modeling enables ecological network analysis techniques to identify key makerspace actors quantitatively. Ecological modularity, for example, identifies divisions in the student-tool makerspace network that highlight how students from different majors (here we investigate mechanical) utilize the makerspace's tools. Modularity is also able to identify “hub” tools in the space, defined as tools central to a student's interaction within the space, based on student-tool connectivity data. The analysis finds that tools commonly used for class by mechanical engineering students, such as the 3D printer or laser cutter, act as gateway tools that bring users into the space and help spark interest in the space's other tools. Using the combined insights from the survey results and the network analysis, ecological network metrics are shown here to be a promising route for informing makerspace policy, tool purchases, and curriculum development. The results can help ensure that the space is set up to give students the best learning opportunities.

This Work In Progress research investigates sketching and visualization experts' perspectives on the definitions and alignment of object assembly sketching exercises with experience from their professional practice. Learning to sketch is a key skill for developing strong visualization and spatial reasoning skills, as well as communication, representation, idea generation, and idea fluency during engineering design. However, manual sketching has largely been replaced by computer graphics tools in undergraduate engineering classrooms. The expert feedback of architecture, civil engineering, and mechanical engineering instructors are reported on relative importance of eight sketching skills, as well as grading practices and discipline-specific practices. Experts generally valued shape quality metrics over line quality, and suggested new interpretations of rubric levels and criteria. We discuss recommended changes to the rubric and exercises.

This Research Work In Progress Paper examines empirical evidence on the impacts of feedback from an intelligent tutoring software on sketching skill development. Sketching is a vital skill for engineering design, but sketching is only taught limitedly in engineering education. Teaching sketching usually involves one-on-one feedback which limits its application in large classrooms. To meet the demands of feedback for sketching instruction, SketchTivity was developed as an intelligent tutoring software. SketchTivity provides immediate personalized feedback on sketching freehand practice. The current study examines the effectiveness of the feedback of SketchTivity by comparing students practicing with the feedback and without. Students were evaluated on their motivation for practicing sketching, the development of their skills, and their perceptions of the software. This work in progress paper examines preliminary analysis in all three of these areas.

This Research Work-In-Progress reports the implementation of an Object Assembly Test for sketching skills in an undergraduate mechanical engineering graphics course. Sketching is essential for generating and refining ideas, and for communication among team members. Design thinking is supported through sketching as a means of translating between internal and external representations, and creating shared representations of collaborative thinking. While many spatial tests exist in engineering education, these tests have not directly used sketching or tested sketching skill. The Object Assembly Test is used to evaluate sketching skills on 3-dimensional mental imagery and mental rotation tasks in 1- and 2-point perspective. We describe revisions to the Object Assembly Test skills and grading rubric since its pilot test, and implement the test in an undergraduate mechanical engineering course for further validation. We summarize inter-rater reliability for each sketching exercise and for each grading metric for a sample of sketches, with discussion of score use and interpretation.

This research Work In Progress systematically reviews the current literature on sketching assessment in engineering education. Sketching is an integral part of the engineering curriculum for conceptual understanding, communication, and design. Sketching enables designers to offload, view, share, and test their ideas. In addition, sketching serves as a tool to increase students' spatial reasoning skills, which is critical to retention and success in engineering. Due to its impact, sketching has been studied in a variety of ways and settings and there are a wide array of methods for assessing sketching. Researchers often assess sketching skill through expert judgment, and when actual sketches are assessed, there are many different metrics that are used. This study is a systematic literature review of sketching assessment exploring applications, cognitive dimensions, and metrics. Databases namely Engineering Village, APA PsycInfo, and Education Source were searched for finding relevant literature related to sketching assessment. Data collection criteria included papers at the high school and college level in engineering, design, architecture, and art. In this paper, our search strings and summary of the final literature sample at the abstract level in terms of publication sources, year, and reviewer decisions are presented. Future directions include continuation of content analysis at the full paper level and assigning quality rankings. The end goal of the project is to provide the design and education communities with a succinct recommendation on sketching assessment to unify efforts in sketching research across the literature.

Engineering students need practical, open-ended problems to help them build their problem-solving skills and design abilities. However, large class sizes create a grading challenge for instructors as there is simply not enough time nor support to provide adequate feedback on many design problems. In this work, we describe an intelligent user interface to provide automated real-time feedback on hand-drawn free body diagrams that is capable of analyzing the internal forces of a sketched truss to evaluate open-ended design problems. The system is driven by sketch recognition algorithms developed for recognizing trusses and a robust linear algebra approach for analyzing trusses. Students in an introductory statics course were assigned a truss design problem as a homework assignment using either paper or our software. We used conventional content analysis on four focus groups totaling 16 students to identify key aspects of their experiences with the design problem and our software. We found that the software correctly analyzed all student submissions, students enjoyed the problem compared to typical homework assignments, and students found the problem to be good practice. Additionally, students using our software reported less difficulty understanding the problem, and the majority of all students said they would prefer the software approach over pencil and paper. We also evaluated the recognition performance on a set of 3000 sketches resulting in an f-score of 0.997. We manually reviewed the submitted student work which showed the handful of student complaints about recognition were largely due to user error.

Sketching is a practical and useful skill that can benefit communication and problem solving. However, it remains a difficult skill to learn because of low confidence and motivation among students and limited availability for instruction and personalized feedback among teachers. There is an need to improve the educational experience for both groups, and we hypothesized that integrating technology could provide a variety of benefits. We designed and developed an intelligent tutoring system for sketching fundamentals called Sketchtivity, and deployed it in to six existing courses at the high school and university level during the 2017-2018 school year. 268 students used the tool and produced more than 116,000 sketches of basic primitives. We conducted semi-structured interviews with the six teachers who implemented the software, as well as nine students from a course where the tool was used extensively. Using grounded theory, we found ten categories which unveiled the benefits and limitations of integrating an intelligent tutoring system for sketching fundamentals in to existing pedagogy.

Providing opportunities for students to exercise their creative skills in large, entry engineering classes challenges most faculty. This paper presents a study of a large statics class provided with a homework problem that asks them to design a truss structure. Automatic grading was done by Mechanix, an AI tutor-based software package that can automatically recognize a free-body diagram or a planar, 2d, statically determinate truss structure. The paper presents a study done in two different semesters, comparing the students using Mechanix to a control (problem on paper). To ease grading, the control group's trusses were analyzed by Mechanix after submission. No mean homework grade differences were observed, but students in the Mechanix group produced trusses that could withstand higher loads. This is despite the fact the only guidance or feedback Mechanix provides was if the students' calculated max load was correct, and if it was not, which member failed. This study occurred in Fall 2019 and Spring 2020. Students also submitted more attempts in Mechanix than the control. It may be students in the control group only submitted correct answers despite being asked to submit all attempts. Future work will provide more incentive for students to submit all attempts on paper. Mechanix automatically records all attempts. During high stress (Covid-19), more students in the Mechanix group submitted the assignment indicating that students may find this system less mentally taxing to use, less stressful, or something else led to this difference. It will be explored with focus groups in the future. AI tools have the potential to provide automatic grading for open-ended, creativity required, design problems, and to engage students more, allowing universities to develop more innovative engineers while also deepening their knowledge.

This Full Research paper presents a study to determine the correlations between student involvement in an academic makerspace and academic achievement as measured by GPA. University makerspaces are open environments designed to encourage creative collaborations and innovative exploration by providing students access to a variety of machines and tools typically focused on rapid prototyping. In an effort to understand the benefits these spaces in academia, there have been several studies on their founding and facilitation; however, there is a lack of data-driven studies of student involvement and the impact of makerspace use on student development. This paper presents preliminary results from an investigation to so show the impact of involvement in an academic makerspace on GPA. By measuring cohorts of students at different stages in the mechanical engineering curriculum, correlations are made between the level of involvement in makerspaces and GPA. The results indicate that students who use the space to build and create, either by choice or for a course requirement, have been shown to have a higher average in-major GPA than students who do not use the space. These findings and more encourage the further exploration of the impact of makerspaces through both quantitative and qualitative methods.

Design sketching is an important skill for designers, engineers, and creative professionals, as it allows them to express their ideas and concepts in a visual medium. Being a critical and versatile skill for many different disciplines, courses on design sketching are often taught in universities. Courses today predominately rely on pen and paper; however, this traditional pedagogy is limited by the availability of human instructors, who can provide personalized feedback. Using a stylus-based intelligent tutoring system called SketchTivity , we aim to eventually mimic the feedback given by an instructor and assess student-drawn sketches to give students insight into areas for improvement. To provide effective feedback to users, it is important to identify what aspects of their sketches they should work on to improve their sketching ability. After consulting with several domain experts in sketching, we came up with several classes of features that could potentially differentiate expert and novice sketches. Because improvement on one metric, such as speed, may result in a decrease in another metric, such as accuracy, the creation of a single score may not mean much to the user. We attempted to create a single internal score that represents overall drawing skill so that the system can track improvement over time and found that this score correlates highly with expert rankings. We gathered over 2,000 sketches from 20 novices and four experts for analysis. We identified key metrics for quality assessment that were shown to significantly correlate with the quality of expert sketches and provide insight into providing intelligent user feedback in the future.

In academic makerspaces, students explore innovative practices. Whether they see these spaces as a means to pursue innovation, however, remains to be understood. This paper examines how makerspaces and innovation are connected by the meanings that students and faculty attribute to makerspaces. Ethnographic techniques were used as the methodology to uncover these meanings. The findings presented in this paper are from the Fall 2016 semester where one graduate researcher explored and observed the engineering makerspaces, an elite student group, and an interdisciplinary collaborative research team. Through the field notes, meanings of interaction, functionality, environment, and innovation provide insight into how students and faculty perceive the value and impact of engineering makerspaces.

Engineering students need to learn highly effective processes for pursuing difficult design problems that require highly innovative solutions. Few studies exist of highly successful expert design teams. The current paper presents the results from a multi-million dollar department of energy research project which reduced the racking hardware and mounting installation costs for commercial applications by more than 50%. This was an extremely challenging goal which was met. This study focuses on the prototyping processes of the team in order to determine effective approaches. Structured interviews with documentations of the prototypes were conducted. Results show the team while the team started with prototyping the complete system they often iterated at the component then integrating it into the complete system prototypes. The early tests of the prototypes tended to be less formal and the number of test increased with each prototype. The professional team also reverted to earlier versions and restarting their processes when a given design path was not successful.

Student designers and professionals alike have difficulty accessing appropriate analogies for design problems. Recognizing the advantages of Design-by-Analogy (DbA), the Design-Analogy Performance Parameter System (D-APPS) tool was developed to include a library of analogy entries and a matching algorithm. These components are combined into the Design Repository & Analogy Computation via Unit-Language Analysis (DRACULA) software package that maps functions across domains in order to present analogies to designers as initiated through engineering performance metrics and critical functions. Most tools developed for DbA emphasize the searching by function feature. Since analogies are based on more than function, DRACULA incorporates both performance and function for the user to identify relevant analogous solutions. Prior to exposing engineering students to this tool, we investigated their ability to use analogies when crossing domains. During this process, we identified three function concepts to be vital for students to effectively use analogies across domains: reoccurring functions, critical functions, and mapping functions. The results establish a better understanding of the information that students utilize in order to formulate appropriate and creative analogous design solutions.

Analogy and bioinspired design have demonstrated applicability as effective tools for innovation, but they can be very difficult to implement. One challenge faced by students and other novice designers is their lack of knowledge to base analogies on. This challenge is also faced by engineers making distant-domain analogies to biology. Other open research questions surround effective strategies for students attempting to implement analogies. When students seek analogies should potential analogues be presented one at a time or should multiple analogues be presented simultaneously (presentation format)? This study implements a 2×2 between-subject factorial design to further explore the impact of presentation format and students' perceived valuableness of passages. Impact is measured in terms of the quality, novelty, and variety of ideas generated. Two additional control conditions were also included, one with no passages and another with random passages. The results from this study have shown that quality is significantly affected by valuableness of passages, novelty is not affected by any factor, and variety is significantly affected by both factors.

Physical prototypes play a crucial role in any design project; but all the critical effects that physical prototypes have need to be further studied. Often, design courses (including capstone courses) include some type of prototyping. More needs to be understood about what students need to learn from prototyping and the benefits of hands-on learning. Physical prototypes help designers by providing critical feedback on their designs. Existing literature shows the importance of prototyping in design projects, while some researchers are concerned with the design fixation caused by prototyping. In order to maximize the benefits of prototyping, it is essential to study the design thinking involved in it. This knowledge may help designers, especially novice designers and students in making decisions about which prototyping method to choose. In this study, data are collected from a realistic design project carried out by a team of professional designers. Through interviews with the designers and direct observations on the prototyping and testing cycle, the following hypotheses are investigated: (1) Building and testing prototypes helps to supplement designers' incomplete mental models leading them to better ideas and (2) Prototyping leads designers to design fixation. The results strongly support the hypotheses. Data from the current study is also compared to prior research on semester-long graduate student team projects. This prior research indicates that many unarticulated tests occur when prototypes are built and this leads to a significant number of improvements to the product. Surprisingly, this also occurred with the practicing designers. This demonstrates that the engineering design curriculum may need to adapt to better teach students to take advantage of the unexpected and students must have building skills in order to leverage this advantage. The data also show that building simple physical prototypes frequently in a design project helps to eliminate the shortcomings in initial ideas and lead designers to better ideas. In order to reduce the fixation associated with prototyping, it is essential to minimize the cost (in terms of money, time and effort) associated with prototyping. Due to the need for low sunk cost and opportunities to learn in unexpected ways from prototypes, engineering students need to have proficient building and testing skills so prototypes do not induce fixation due to sunk cost. The professional design team also did things the student design teams did not. The professional team built representational prototypes and those with selected functionalities at times. They also used strategies that likely reduce design fixation including prototyping only parts of the system and systematically used low cost materials like wood and plastic when testing physical interfaces for fit and assembly, not structural capacity.

Design-by-analogy is considered to be a powerful tool for engineering design. The difficulty of finding suitable analogies for solving a given design problem gives rise to the current efforts on computational tools for analogy-based design. In searching for analogies, the critical functions in a design problem are potential search criteria. The study described in this paper investigates whether novice designers identify an expert-derived critical function in a design problem under three scenarios: when they are asked to report the important functions in the problem, when they are directly asked to report the critical function and when they are asked to use design-by-analogy. It is observed that student designers do not identify the expert-derived critical functions when directly asked or when asked to list important functions. However, they inherently use the expert-derived critical functions in their analogical mapping process. This suggests that, during analogical reasoning, designers tend to identify and use the same critical functions regardless of experience, and also that critical functions are valid search criteria for deriving analogies from a computational database. This insight is highly valuable for current efforts to develop computational tools for analogical reasoning.

Physical models are widely used as idea generation tools by industrial designers, engineers, engineering educators and government agencies. Many schools promote the use of physical models in their engineering curricula. Despite the apparent popularity of physical models, little is known about their cognitive impacts and when they should be implemented in the design process. A few studies have explored physical models and their use as idea generation tools; however the guidelines from them are conflicting. Based upon these conflicting guidelines, a series of controlled and qualitative studies are conducted by the authors to understand the cognitive impacts of physical models in engineering idea generation. In addition to the insights from these studies, data are collected from a project-based graduate design course. The reports from design teams prototyping their ideas as a part of a class project are studied. These reports provide insights about the conceptual errors that student designers make as they build and test physical models of their designs. To reduce the two most critical errors, a design method (Model Error Reduction Method) is formulated. This design method forces the designers to think about two potential conceptual errors in their designs and provides guidance to rectify the issues. The two conceptual errors that the method addresses are: failure to account for critical loads and failure to design connections. This paper presents a controlled experiment evaluating the effectiveness of the method. The preliminary results show that novice designers find the design method extremely useful; moreover, the method seems to help eliminate, to a large extent, said conceptual errors. These findings suggest that the method might augment existing engineering design curricula.

Introductory engineering courses within large universities often have annual enrollments which can reach up to a thousand students. It is very challenging to achieve differentiated instruction in classrooms with class sizes and student diversity of such great magnitude. Professors can only assess whether students have mastered a concept by using multiple choice questions, while detailed homework assignments, such as planar truss diagrams, are rarely assigned because professors and teaching assistants would be too overburdened with grading to return assignments with valuable feedback in a timely manner. In this paper, we introduce Mechanix, a sketch-based deployed tutoring system for engineering students enrolled in statics courses. Our system not only allows students to enter planar truss and free body diagrams into the system just as they would with pencil and paper, but our system checks the student’s work against a hand-drawn answer entered by the instructor, and then returns immediate and detailed feedback to the student. Students are allowed to correct any errors in their work and resubmit until the entire content is correct and thus all of the objectives are learned. Since Mechanix facilitates the grading and feedback processes, instructors are now able to assign free response questions, increasing teacher’s knowledge of student comprehension. Furthermore, the iterative correction process allows students to learn during a test, rather than simply displaying memorized information.

Mechanix is a sketch recognition tool that provides an efficient means for engineering students to learn how to draw truss free-body diagrams (FBDs) and solve truss problems. The system allows students to sketch these FBDs into a tablet computer or by using a mouse just as they would by hand, a mouse can also be used to draw the sketch using a regular computer and monitor. Mechanix is then able to provide immediate feedback to the students and tell them if they are missing any components of the FBD, without providing answers. The program is also able to tell them whether their solved reaction or member forces are correct or not. This paper presents a study to evaluate the effectiveness and advantages of using Mechanix in the classroom, as a supplement to traditional teaching and learning methods. Current results demonstrate that students believe Mechanix enhances their learning and are highly engaged when using it. Future work on the refinement of the program is also discussed.

Many capstone courses include design methodology but the characteristics of design problems that provide the best learning opportunities need to be defined and the effect on student perceptions measured. In an industrial setting, it is generally up to the engineer to decide which design methods should be applied in order to obtain the desired outcome. Thus if students do not believe the design methods to be effective, they will not choose to use them in the future. Functional modeling is a technique for systematically breaking down a design problem into its sub-functions. Groups were given either a simple or complex problem to solve for their senior design project. At the end of the semester, a quiz over the functional modeling concepts was administered, and a questionnaire was answered by the students to measure their perceptions of the design methods. An ANOVA shows an interaction between student opinions and the complexity of the design problem on the students' functional modeling ability. Results indicate the complexity of the problem and perceptions of the design methods are significant factors when determining the students' functional modeling abilities. Additionally, the complexity of the design problem likely affects other design skills.

Designers use various kinds of physical models throughout their design process to enhance creativity. The existing literature provides conflicting guidelines about their implementation. The effects of physical models on design cognition remains largely unknown. Prior laboratory studies show that physical models supplement designers' erroneous mental models and thereby lead to higher quality ideas. These prior studies fail to demonstrate any design fixation associated with the use of physical models. In contrast, a few prior observational studies on practicing designers show that the use of physical models causes design fixation. Based on these conflicting results, this study investigates the role of physical models in industry-sponsored projects and in the development of award-winning products through a qualitative research approach. This study explores two hypotheses: The Mental Models Hypothesis - physical models supplement designers' mental models and the Fixation Hypothesis - physical models cause design fixation during the idea generation process. The data are coded qualitatively and then tested quantitatively. The results are triangulated with the results from the prior controlled study. The results provide support to the hypotheses. The differences observed between current and prior studies point to the potential role of the Sunk Cost Effect in engineering idea generation with physical models.

In an introductory Engineering course with an annual enrollment of over 1000 students, a professor has little option but to rely on multiple choice exams for midterms and finals. Furthermore, the teaching assistants are too overloaded to give detailed feedback on submitted homework assignments. We introduce Mechanix, a computer-assisted tutoring system for engineering students. Mechanix uses recognition of freehand sketches to provide instant, detailed, and formative feedback as the student progresses through each homework assignment, quiz, or exam. Free sketch recognition techniques allow students to solve free-body diagram and static truss problems as if they were using a pen and paper. The same recognition algorithms enable professors to add new unique problems simply by sketching out the correct answer. Mechanix is able to ease the burden of grading so that instructors can assign more free response questions, which provide a better measure of student progress than multiple choice questions do.