This work-in-progress research paper focuses on the first-year engineering teams' peer evaluation processes. Teamwork has been identified as a necessary skill for professional engineers and thus an ABET learning outcome. Reliable systematic processes to assess teamwork effectiveness are crucial for improving team outcomes and identifying dysfunctional teams in a classroom setting. Evaluating team effectiveness and identifying dysfunctional teams has been traditionally done through peer evaluations. However, there is a lack of evidence on what/how students' perceptions of effective teammates and functional teams impact peer evaluations. Identifying common indicators and behaviors that students consider when evaluating their peers is a first step to exploring the reliability of peers as evaluators, understanding potential biases, and using peer feedback to investigate dysfunctional teams. This paper aims to take this first step and will explore the following question: “What evidence do students provide to describe poor team behaviors?” The study was conducted at a large, public, urban, Midwestern R1 institution. For the first two semesters of the engineering curriculum, students are required to participate actively in team-based projects as part of engineering design thinking courses. The following procedure was used to answer our research question. Students were first asked to respond to a peer evaluation instrument based on a four-factor (Interdependency, Goal setting, Potency, and Trust) team effectiveness model. Then, students were asked to grade their teammates on a cumulative score of 100 for the team and support their distribution choices with open-ended comments on team behaviors. We started with an analysis of students' comments regarding negative behaviors to support their low rating for each teammate. We selected comments that had a high misalignment between the model-based peer evaluation and corresponding distributed ratings, resulting in a sample of 93 students. Next, we conducted inductive coding on the comments to identify frequently mentioned negative behaviors and see if these behaviors were addressed in the model-based peer evaluation. Eight major themes emerged from students' comments.

The research-to-practice paper presents applications of genetic algorithms using multi-criterion optimization to designate student-teams in an academic setting. Teamwork skills are becoming a more desirable trait in industry today and hence more instructors are using teamwork in their engineering courses. ABET also requires engineering degree programs to have student outcomes that provide them with "an ability to function effectively on a team whose members together provide leadership, create a collaborative and inclusive environment, establish goals, plan tasks, and meet objectives." This, however, comes with its challenges because the literature suggests that random or student-selected teams tend to lead to dysfunctional behaviors. Instructor-designated academic teams based on skills, learning personalities, and demographics are known to be more effective in promoting learning and positive interactions. Creating optimal homogeneous or heterogeneous teams based on multiple criteria (e.g., prior knowledge, skills, abilities, psychosocial, demographics) can be a complex and time-consuming task when done manually, especially when large numbers of students are involved.This study presents an expansion of previous work that used single-criterion genetic algorithm optimization of teams in a first-year classroom. The research question answered in this study is, how do teams formed using an algorithm that optimizes multiple criteria with genetic algorithms represent heterogeneity when compared to teams formed manually? Using a Vector Evaluated Genetic Algorithm (VEGA), optimization of multiple skills is conducted to attain uniform heterogeneity in the classroom. The algorithm uses discrete integer-type representations as a basic unit of team configuration. Self-reported student competency data on three different computational skills were used. Teams were formed for approximately 1300 students enrolled in a first-year engineering design thinking course at a large Midwestern University. Four-member teams were maximized with a minimum number of teams with 3 students in each section. Average skills of the teams are calculated and the standard deviation in class is minimized for each of three skills parallelly. The algorithm also aims to maintain diversity of teams based on gender and ethnicity.Results presented include visualization of team-configurations and comparison with teams formed manually using the same criteria. The aim of the algorithm is to optimize students into teams that have skills and demographics uniformly distributed across the classroom. Future implications of this study include the potential to use flexible algorithm-based team formations that are built with standard genetic operators so that optimization criteria can be modified by the instructor. Such algorithms can reduce the time needed to manually form teams by a considerable amount and optimize the distribution of skills more uniformly. Steps to further this study will involve investigating whether teams formed using these criteria are more effective.

Plagiarism in student submissions is a continual threat to the integrity of engineering education. Online learning environments enhance the accessibility of many forms of plagiarism, and a corresponding increase in the prevalence of academic misconduct has been observed. When unmitigated, plagiarism has severe detrimental effects on student learning outcomes, and therefore it is the responsibility of instructors to detect plagiarism by all practical means. Plagiarism identification techniques typically seek to discover abnormal similarities in student submissions, and although imperfect, research has shown that this approach is highly effective. This work in progress paper seeks to augment traditional similarity-focused plagiarism detection strategies through the use of watermarks, or student-specific unique identifiers embedded in files. This paper shows how existing watermarking techniques can be applied in the context of plagiarism detection by integrating watermarks into assignment template files. The applicability of watermarking to multiple submission types, including source code files and Microsoft Office documents, is demonstrated. Additionally, this paper presents methods for automated generation of watermarked templates, distribution of marked files, and evaluation of submissions for plagiarism. Finally, preliminary results are presented, and future work necessary to complete this in-progress project is discussed.

Academic misconduct in programming assignments, such as excessive collaboration with peers or use of online resources, is of growing concern to the integrity of engineering education. Software development skills have become increasingly relevant to many fields, and consequentially the number of students submitting software assignments has also increased dramatically. The online learning environment caused by current global conditions broadens opportunities for students to engage in unethical academic behavior. Current techniques for the identification of academic misconduct in software submissions suffer from multiple issues such as the production of an excessive amount of difficult to interpret data, and the use of algorithms with limited effectiveness on short program submissions, such as those containing fewer than fifty lines. This paper presents how several existing techniques for identifying software similarity are made more effective when combined and fine-tuned in a way that increases sensitivity and decreases noise for short source code submissions. This paper shows how similarity results can be applied in a robust framework for determining which submissions are similar enough to warrant investigation. Finally, this paper introduces a new technique for grouping similar submissions, which helps identify collections of student submissions that all contain matching features. This increases efficiency by reducing the number of submission pairs that require human analysis. The application of these techniques enables comprehensive analysis of similarity in short software submissions, reduction of noise through the use of robust methods for determining which submissions were likely involved in academic misconduct, and improvements in human review efficiency by grouping sets of similar submissions and reducing the total amount of data for review.

Diversity statements are becoming mandatory in increasing numbers of job searches by increasing numbers of institutions. However, how to use diversity statements is seldom discussed with search committee members or persons interviewing the potential candidate. Some institutions, even if required, do not provide guidance or expectation to the use of the statement. Some institutions and/or search committees use diversity statements in a very basic way as tiebreakers. Institutions seeking improved inclusivity and convergent thinking have found that by reading the diversity statement first, higher numbers of traditionally underrepresented candidates are remaining in the pool at each stage of the hiring process [1], [2]. This workshop provides participants with a framework to create and use rubrics to evaluate diversity statements for improved inclusivity.

This work in progress study aims at developing a system to designate teams to relatively large groups of students in a classroom setting. It is motivated by recent changes in the ABET Criterion 3 accreditation guideline that requires students to demonstrate "an ability to function effectively on a team whose members together provide leadership, create a collaborative and inclusive environment, establish goals, plan tasks, and meet objectives." Outside of accreditation guidelines, team-based learning is becoming the prescribed pedagogical tool to enhance student collaboration and prepare them for professional work environments upon graduation. One important factor in effective team-based learning is to ensure teams are balanced in their abilities and characteristics across team members. Demographical memberships of students can also play an important role in team dynamics and impact the overall success in content learning. Instructors usually assign teams to students by manually looking into their abilities from previous grades and other performance factors. This can then become tedious when aiming to maintain uniform heterogeneity in classrooms of students coming from diverse backgrounds of knowledge, skills, ethnicity, and gender. While some studies have presented the use of computer-aided tools, visualization categorization, and other Artificial Intelligence techniques, this work proposes the use of the Genetic Algorithm (GA) to form teams optimized for heterogeneity. The Genetic Algorithm presented uses a discrete integer-based chromosome representation and groups alleles to represent each team. Standard GA operators enable the algorithm to be adapted to any optimizing criteria deemed fit by the instructor. The algorithm presented in this work uses self-reported competency data on 3 different computational skills for 1300 students enrolled in a firstyear engineering design thinking course divided into over 20course sections of strength 40-60 each. Net team scores for each computational tool are calculated and the variation across each skill minimized by the fitness function for individual sections. Constraints based on gender and ethnicity are applied to minimize demographical imbalance between teams. The algorithm is tested for all sections in the Fall 2019 cohort to give consistent team configuration. Discussions on the stability and validity of configurations generated are discussed. Future work include methods that use a fuzzy logic decision-making tool for multiple criteria optimization.

At FIE 2002, 13 engineering educators assembled to address a variety of topics and predict the “Future of Engineering Education.” Larry Shuman organized and moderated the session [1]. Topics included the changing demographics and economics of the country, technological advances, the engineering pipeline, the state of the University and forces driving change, engineering as a liberal art, the accreditation process and faculty reward system, the role of technology in delivering engineering education, educating for higher levels of performance, research in engineering education, research applications, and outcomes assessment. This panel includes five of the original authors and some new contributors who are active in FIE. We will examine the predictions made in 2002, and ask where we were right, where we were wrong, what has come to pass, what is still in progress, and what concerns have faded from view. Many of the issues previously discussed are still hot topics 15 years later.

Student diversity in higher education tends to focus on gender, ethnicity/race, and socio-economic status. However, these factors do not address cognitive diversity. Cognitive diversity, within the context of this study, refers to the varying ability of brain functions such as reasoning and memory, excluding persons with a developmental disability. Students with learning disabilities (LD), specifically dyslexia, contribute to this cognitive diversity. This study aims to initiate scholarly research on academic success factors for First-Year Engineering (FYE) students with dyslexia. FYE student performances on the Purdue Spatial Visualization Test-Rotations (PSVT-R) and Student Attitudinal Success Instrument (SASI) have been found to be predictors of academic success in engineering. A preliminary analysis of entering FYE student performance on the PSVT-R and SASI is conducted for three populations: students with dyslexia, students with a LD, and students without a LD. The anticipated findings will support the inclusion of cognitive ability, with an emphasis on LD and dyslexia, in FYE engineering diversity programs.

We want our students to understand and apply the concepts in each course. Therefore, we work hard to help our students master often-difficult concepts; however, our evaluation of their conceptual understanding often occurs simultaneously with evaluation of other learning goals through use of traditional problem-solving tests. Seldom do we measure pre-to-post learning gains. Often, instruments that would facilitate pre-to-post learning evaluation are not available. Creation, development, and use of such instruments would likely promote constructive conversations between engineering students and faculty members. Assessment instruments that have been designed to evaluate only conceptual understanding are often referred to as concept inventories, following a convention established by the Force Concept Inventory. Concept inventories have a range of possible uses, e.g., a pre-course diagnostic to understand conceptual understanding of students at the beginning of a course, early course formative assessment to guide instructional planning, summative assessment to evaluate conceptual understanding at the end of the course, and pre-post assessment to aid evaluation of instructional strategies. Concept inventories have been used at both course and program levels. What distinguishes concept inventories from typical engineering course assessment methods is focus on a small set of key constructs, focus on a specific domain of academic content, and focus on conceptual understanding or qualitative reasoning, as opposed to computational problem solving. Considerable scholarship informs selection of the situations, formulation of the question, and development of plausible distracters. During the workshop, participants will (i) be provided an overview of research on conceptual understanding, (ii) be provided an overview of the historical development of concept inventories, (iii) engage in activities to describe effective uses and some misuses of concept inventories in their courses, (iv) learn how to access existing concept inventories via the developing ciHUB.org platform, (v) discuss psychometric properties of existing instruments, (vi) learn how psychometric analysis can aid development of concept inventories, and (vii) have opportunities to become active members in a growing community of users.

This work in progress presents a developmental model representing the ability of students to negotiate shared meanings with cultural others in order to build sustainable and mutually beneficial partnerships. The goal of this research is to locate students within this continuum and provide a student-centered starting point in the ways students construct meaning around cultural differences. This paper uses a qualitative inquiry and analysis methodology with a focus on first-year engineering students at a large Midwestern public university and a similar large public university in Australia. The data collected were interviews and focus group discussions probing their experiences with cultural differences. Initial findings demonstrate that in order for students to be able to acknowledge and express their understanding of differences, they need and want models, tools and techniques to be able to communicate their thoughts about cultural differences and to negotiate bridges of mutual understanding. Student interviews in the US reflected more polarizing messages while focus groups in Australia generated more minimizing messages. Engineering educators encourage students to approach and explore both their own cultures (self-knowledge), internal dialogues and other cultures (perceived through the student's own cultural lenses), and the language they use to describe others.

The appropriate placement for mathematics courses for first-year engineering students is an important factor in ensuring the success of engineering students in terms of retention and graduation rates. However, often the advisors only are able to look at the cutoffs of a few factors when making their recommendations. We propose that modeling offers advisors a more effective method to integrate multiple factors in order to make the most appropriate recommendation for first year math courses to entering students and help ensure student success. Results to be reported in this study are effectiveness of predicting engineering students' success in Calculus I courses using regression models, significant predictors to Calculus I grades, the actual recommendations that students received from advisors for appropriate calculus placement, whether students took the advisors recommendations in choosing a calculus placement, and the relationship between the model and what advisors are actually advising. The results of this study will provide information that helps to find the most effective means of advising to help ensure student success.

Female enrollment in engineering in the United States has remained at or below 20% for decades. Enrollment of students from traditionally underrepresented groups has also remained below desired level for years. A systematic understanding of important factors leading to persistence and success in undergraduate engineering programs for female and underrepresented minority students would be very valuable for recruiting, retaining and educating young engineers with diverse perspectives. This paper discusses the significant predictors for retention and academic performance of female engineering students, and reports the difference in comparison with male engineering students. Similar results on the important predictors for retention and performance of underrepresented minority engineering students will also be reported and compared with the ethnic majority students. The findings from this study suggest it is potentially advantageous to develop student success models specific for female or minority engineering student populations, rather than using the same model developed for the whole population. New knowledge obtained through this study will lead to the development of necessary strategies, interventions or programs to help improve retention and academic success of our engineering students.

Developing students teaming skills has become common place in engineering education as a pedagogical tool to facilitate learning of technical content as well as to prepare students for professional practice. Engineering faculty typically determine the degree to which students have had an effective team experience by indirect methods such as homework or project grades along with self report team member peer-evaluations. Such methods tend to place a greater emphasis on the outcome (or product) of teaming rather than on the process of teaming itself. The use of standalone peer-evaluations to indirectly determine team effectiveness has also been shown to be problematic, since students are not typically taught how to properly evaluate their peers. This lack of training generally results in a significant amount evaluation bias. This research presents a theoretical framework to indirectly measure team effectiveness using a calibrated peer evaluation system The system provides students feedback on their rating ability as well as quantifies (as a 1st order approximation) their evaluation bias. The system can be used by faculty for early identification of dysfunctional teams as well as to determine the degree to which students are engaged in effective team behaviors.