This is a WIP progress paper belongs to innovative practice section. According to Bandura, self-efficacy refers to “beliefs in one's capabilities to organize and execute the courses of action required to produce given attainments.” Self-efficacy plays an essential role in skill development and engineering education. Sketching is a critical skill for engineering students, and it has various benefits, especially in developing spatial visualization skills. Sketching enables students to effectively communicate graphically, represent ideas, and brainstorm ideas, especially in the early design stages. No formal research has been conducted concerning the role of self-efficacy in learning to sketch. An instrument that measures sketching self-efficacy is beneficial for gaining insight into the levels of sketching self-efficacy of engineering students. A sketching self-efficacy instrument was developed for high school students, however it needs revision and validation for use in undergraduate engineering classrooms. This work-in-progress study aims to revise the sketching self-efficacy survey instrument items based on expert feedback to ensure the validity of the instrument's content. We have gathered feedback from nine engineering instructors from various disciplines who are experts in sketching on the items of the current sketching self-efficacy instrument. Our research question is: In what areas do engineering students need to develop self-efficacy in sketching according to engineering instructors, and how do these areas vary across mechanical and industrial engineering disciplines? We conducted five interviews virtually and semi-structured through the Zoom platform. The interview transcripts were anonymized, cleaned, and loaded into qualitative data analysis software, MAXQDA. Qualitative contnet analysis was utilized to analyze the interview data. Based on the interview analysis results, we will discuss the recommended changes to the sketching self-efficacy instrument. Our final study will include psychometric analysis, such as exploratory Factor analysis and Confirmatory factor analysis, to test the instrument's validity for college students.

This Work In Progress research investigates sketching and visualization experts' perspectives on the definitions and alignment of object assembly sketching exercises with experience from their professional practice. Learning to sketch is a key skill for developing strong visualization and spatial reasoning skills, as well as communication, representation, idea generation, and idea fluency during engineering design. However, manual sketching has largely been replaced by computer graphics tools in undergraduate engineering classrooms. The expert feedback of architecture, civil engineering, and mechanical engineering instructors are reported on relative importance of eight sketching skills, as well as grading practices and discipline-specific practices. Experts generally valued shape quality metrics over line quality, and suggested new interpretations of rubric levels and criteria. We discuss recommended changes to the rubric and exercises.

This Research Work In Progress Paper examines empirical evidence on the impacts of feedback from an intelligent tutoring software on sketching skill development. Sketching is a vital skill for engineering design, but sketching is only taught limitedly in engineering education. Teaching sketching usually involves one-on-one feedback which limits its application in large classrooms. To meet the demands of feedback for sketching instruction, SketchTivity was developed as an intelligent tutoring software. SketchTivity provides immediate personalized feedback on sketching freehand practice. The current study examines the effectiveness of the feedback of SketchTivity by comparing students practicing with the feedback and without. Students were evaluated on their motivation for practicing sketching, the development of their skills, and their perceptions of the software. This work in progress paper examines preliminary analysis in all three of these areas.

This Research Work-In-Progress reports the implementation of an Object Assembly Test for sketching skills in an undergraduate mechanical engineering graphics course. Sketching is essential for generating and refining ideas, and for communication among team members. Design thinking is supported through sketching as a means of translating between internal and external representations, and creating shared representations of collaborative thinking. While many spatial tests exist in engineering education, these tests have not directly used sketching or tested sketching skill. The Object Assembly Test is used to evaluate sketching skills on 3-dimensional mental imagery and mental rotation tasks in 1- and 2-point perspective. We describe revisions to the Object Assembly Test skills and grading rubric since its pilot test, and implement the test in an undergraduate mechanical engineering course for further validation. We summarize inter-rater reliability for each sketching exercise and for each grading metric for a sample of sketches, with discussion of score use and interpretation.

Sketchtivity is a stylus-based intelligent tutoring system that can help instructors automatically provide feedback to their students, saving them the time and effort of providing personalized feedback themselves. The system uses a generic evaluation of perspective, direction, and accuracy to give students feedback on the quality of their sketches. If instructors want to personalize the metrics, the system would require them to provide multiple sets of samples. Therefore, instructors may use instructional team members such as teaching and graduate teaching assistants to provide feedback on the required samples. Compared to that of assistants, the feedback they produce might vary due to expertise and create noise in the training data. To address this problem, we implement a deep neural network that leverages learning to reweight algorithms. The data collected by the instructor from undergraduate and graduate-level rectilinear perspectives sketching is considered the validated sample. In this study, we analyzed the training size requirement for a Multi-Layer Perceptron (MLP) to accurately predict whether or not a stroke was a perspective stroke. We observed that the training data required to predict stroke accuracy is small. In addition, the performance of the algorithm in terms of accuracy was good even under extreme conditions such as having highly unbalanced data and having a small valid set of data. The results from the study support the use of these types of algorithms for future system personalizing to support scalable feedback systems in education.

This research Work In Progress systematically reviews the current literature on sketching assessment in engineering education. Sketching is an integral part of the engineering curriculum for conceptual understanding, communication, and design. Sketching enables designers to offload, view, share, and test their ideas. In addition, sketching serves as a tool to increase students' spatial reasoning skills, which is critical to retention and success in engineering. Due to its impact, sketching has been studied in a variety of ways and settings and there are a wide array of methods for assessing sketching. Researchers often assess sketching skill through expert judgment, and when actual sketches are assessed, there are many different metrics that are used. This study is a systematic literature review of sketching assessment exploring applications, cognitive dimensions, and metrics. Databases namely Engineering Village, APA PsycInfo, and Education Source were searched for finding relevant literature related to sketching assessment. Data collection criteria included papers at the high school and college level in engineering, design, architecture, and art. In this paper, our search strings and summary of the final literature sample at the abstract level in terms of publication sources, year, and reviewer decisions are presented. Future directions include continuation of content analysis at the full paper level and assigning quality rankings. The end goal of the project is to provide the design and education communities with a succinct recommendation on sketching assessment to unify efforts in sketching research across the literature.

This research Work-In-Progress presents a survey of advanced learners' motivation in highly-technical advanced engineering MOOCs. Advanced engineering courses as MOOCs are increasingly prevalent in graduate and professional learning and are gaining importance for credentialing and accredited degrees. However, these courses are open to the public as well as formal learners, making it difficult to generalize experiences for all. To understand what prevents advanced learners from reaching their goals and to better support them in meeting goals, there is a need for more sensitive tools to measure motivation. We have revised the Expectancy-Value-Cost scale and examined its functioning on pilot data to begin looking at validity evidence for using the revised scale with professional engineers. We analyze motivations, intentions, and course ratings of learners from two online MOOCs who are enrolled in formal degree programs at a four-year institution, learners completing a MOOC master's degree, and independent learners. We also perform bivariate correlation of motivation items to test the performance of the instrument. Preliminary results show high motivation for all learner groups, but greater differences between independent and formal learners, with formal learners reporting a wider range of costs.