Software design is a complex process and requires that software designers follow best practices while solving business problems.A software design errors taxonomy (SDET) [1] was compiled as an outcome of a systematic literature review.This paper furnishes the insights from a survey of industry experts on the relevance, usability and completeness of SDET.The paper further identifies additional software design errors that are prevalent across a diverse set of industries.The authors conducted a survey of software professionals to investigate the common set of software design problems that occur in the industry and later updated the taxonomy with appropriate findings.Findings reveal a convergence between academic literature and industry experiences, highlighting the critical need for improved design practices.The findings from the industry survey were synthesized to improve the SDET and publish the results Keywords-software design errors, error prevention, software design errors taxonomy, software quality improvement I.

This innovative practice full paper explores the use of AI technologies in user story generation. With the emergence of agile software development, generating comprehensive user stories that capture all necessary functionalities and perspectives has become crucial for software development. Every computing program in the United States requires a semester-or year-long senior capstone project, which requires student teams to gather and document technical requirements. Effective user story generation is crucial for successfully implementing software projects. However, user stories written in natural language can be prone to inherent defects such as incompleteness and incorrectness, which may creep in during the downstream development activities like software designs, construction, and testing. One of the challenges faced by software engineering educators is to teach students how to elicit and document requirements, which serve as a blueprint for software development. Advanced AI technologies have increased the popularity of large language models (LLMs) trained on large multimodal datasets. Therefore, utilizing LLM-based techniques can assist educators in helping students discover aspects of user stories that may have been overlooked or missed during the manual analysis of requirements from various stakeholders. The main goal of this research study is to investigate the potential application of OpenAI techniques in software development courses at two academic institutions to enhance software design and development processes, aiming to improve innovation and efficiency in team project-based educational settings. The data used for the study constitute student teams generating user stories by traditional methods (control) vs. student teams using OpenAI agents (treatment) such as gpt-4-turbo for generating user stories. The overarching research questions include: RQ-l) What aspects of user stories generated using OpenAI prompt engineering differ significantly from those generated using the traditional method? RQ-2) Can the prompt engineering data provide insights into the efficacy of the questions/prompts that affect the quality and comprehensiveness of user stories created by software development teams? Industry experts evaluated the user stories created and analyzed how prompt engineering affects the overall effectiveness and innovation of user story creation, which provided guidelines for incorporating AI-driven approaches into software development practices. Overall, this research seeks to contribute to the growing body of knowledge on the application of AI in software engineering education, specifically in user story generation. Investigating the use of AI technologies in user story generation could further enhance the usability of prompt engineering in agile software development environments. We plan to expand the study to investigate the long-term effects of prompt engineering on all phases of software development.

This full research paper describes the analysis of common software vulnerabilities that are introduced by students enrolled in four-year computing and cybersecurity majors from two different higher education institutions in Georgia. As the demand for secure coding education continues to grow, pedagogical improvements need to be made in identifying key software vulnerabilities students commit during code development (from the first programming course to the exit senior design capstone) which in turn can be analyzed to inform the pedagogical interventions focused at preparing students with skill sets for writing secure code and entering the professional workforce. While code security is emphasized throughout the computing curriculum, this research is focused on training individuals to be aware of common vulnerabilities and tailoring programming concept knowledge that has been shown to have a positive effect on code security. Existing research has mainly focused on developing vulnerability analysis tools rather than collecting data (and subsequently analyzing) regarding the types of vulnerabilities produced by students at their institutions. In this paper, we analyzed student code across different courses and reported the types of vulnerabilities produced by students in their assignment submission code from two different higher education institutions across different levels of the four-year curriculum. The reported vulnerabilities are grouped by CWE-ID, which is a standard and common way to categorize and identify software vulnerabilities. The resulting CWE-IDs are then grouped per student submission and per semester (across curriculum levels) to discover the common types of software vulnerabilities committed across cross sections of students. Our results from the analysis of vulnerabilities (ranging from CS1 courses to capstone courses) are organized around the following research questions: 1) What are the most common software vulnerabilities produced by computing majors at different levels through the computing curriculum?; and 2) Do these vulnerabilities persist throughout their curriculum as they advance into higher-level courses? We report that students commonly make mistakes related to variable usage, null pointer checks, hard-coding sensitive information, and improperly validating input. Vulnerabilities such as CWE-489 (“ Active Debug Code”) and CWE-215 (“Insertion of Sensitive Information Into Debugging Code”) tend to persist across multiple course levels and may need to be focused in the computing curriculum. The number of vulnerabilities introduced in assignment code increases as course complexity increases. We also find that vulnerabilities produced by students have little overlap with what software vulnerability researchers commonly study, potentially leading to a mismatch in priority for secure coding topics. Our findings have implications for computer science and cybersecurity curriculum design and delivery.

This lightning talk focuses on the efforts of software engineering (SE) academics to keep curriculum relevant. SE evolves rapidly, with changing technology and industry expectations. The curriculum review bodies (e.g. ACM and IEEE-CS working groups) provide a foundation for curriculum, but can have refresh cycles measured in years. We aim to identify, and potentially increase, the confidence level of SE educators with respect to the relevance of their curriculum. It is also of interest to identify the sources academia uses to keep their courses current. and the role of gray literature (GL). Other fields have found GL useful in bridging academic research and industry needs. GL can be extended to SE to aid faculty preparing students for industry.

This Research to Practice Full Paper presents the use of data collected by our Non-Intrusive Classroom Attention Tracking System (NiCATS) to evaluate student comprehension. Quantifying students' cognitive processes in classrooms in a non-intrusive way is challenging. By analyzing various aspects of the eye metrics against defined regions of interest (ROI), instructors can better understand students’ cognitive processes as they acquire new knowledge. Eye-tracking studies primarily define ROIs based on commonly used metrics (source code complexity, significant fixation durations, etc.). While helpful, these metrics, when used independently, do not accurately represent their comprehension patterns. This paper contributes an alternative, multilayered approach for calculating gaze metrics against automatically defined ROIs. The work utilizes the AI-based Non-Intrusive Classroom Attention Tracking System (NiCATS - developed by the researchers), collecting raw-gaze data in real-time as information is presented on a computer screen. This paper reports the results of a study in which undergraduate students in a CS programming course were asked to identify defects seeded in Java programs. Each JAVA program included its own unique sets of ROIS defined using two different granularities: lexer-based and line-based. The ROI sets were then used to calculate relevant eye metrics in the context of each ROI layout. The results of the eye metric analysis at specific ROIs w.r.t their code review task provide insights into the cognitive processes students undergo when trying to comprehend new material. Subdividing this region into lexer-based regions, we determined “content topics” students struggled with (e.g., using complex data types) in a specific area. This feedback is valuable to the instructor as it enables the ability to identify hard-to-comprehend content topics post-hoc and gives the ability to validate student learning in the classroom. While this experiment focused on students in introductory programming courses, we intend to conduct experiments in other learning settings where students are expected to read material on a computer screen or solve actual problems. To summarize, the analysis of these eye metrics using more fine-grained ROIs (lexer-based, line-based) as an extension of complexity-based ROIs provides instructors with deeper insights into the cognitive processes used by students when compared to the current state-of-the-art techniques.

The Covid-19 pandemic, as Henry Kissinger mentions, will not only "forever alter the world order," but also potentially transform the ever-changing higher education world. The recent increase in technological innovations in information, communications, and computer technology has profoundly transformed traditional teaching-learning processes and peer-to-peer interactions for knowledge transfer. One such radical change in technology that researchers are continually working on is motivating collaborative learning and student interactions to improve their learning experiences. Collaborative Learning (CL), where students work in groups to achieve a specific learning objective, can facilitate a deep learning activity that promotes student participation. However, the potential of discussion forums is limited due to their unstructured nature in LMSs like Canvas.We propose and develop a structured discussion forum that can offer a platform to communicate and discuss problems and receive feedback, discuss solutions, and suggestions online. Students who participate in these discussion forums can benefit in multiple ways, including increased class preparedness and more active learning. The twofold objectives and outcomes include 1) analyzing discussion board data to reveal students’ interaction and their degree of participation in the course, and 2) developing a toolset to draw useful inferences from such collaboration networks. Specifically, our schema-based model can help students visualize the discussion board networks creating an engaged learning environment. Furthermore, the model can help draw valuable inferences of the patterns of student interactions and assess student participation and belonging in the course with greater precision.This paper demonstrates a schema-based discussion board model that can allow researchers to collect better-formatted discussion data and more reliable information about the posts, such as the type of posts and the relationships of each post with others. The reimagined discussion boards include the ability to classify discussion posts using various parameters, visualize the posts’ patterns of interactions, identify their relationships with other discussion posts, and precisely evaluate student participation in discussions to monitor the major topics of discussion. We believe that the result of increased participation in discussions with other students will have the effect of increasing students’ sense of belonging to the community of scholars.

This Research to Practice Full Paper presents our Non-intrusive Classroom Attention Tracking System (NiCATS) and discusses the data collected through it. Academic instructors and institutions desire the ability to accurately and autonomously measure students' attentiveness in the classroom. Generally, college departments use unreliable direct communication from students, observational sit-ins, and end-of-semester surveys to collect feedback regarding their courses. Each of these methods of collecting feedback is useful but does not provide automatic feedback regarding the pace and direction of lectures. It has been widely reported that attention levels during passive classroom lectures generally drop after about ten to thirty minutes and can be restored to normal levels with regular breaks, novel activities, mini-lectures, case studies, or videos. Tracking these “drops” in attention can be crucial for the accurate timing of these change-ups in activities. This allows for maximal attention and a greater amount of deeply learned material. Autonomously collected data can also be used either in real-time or post-hoc to alter the design and presentation of lectures. Keeping track of student attention is vital to having confidence in delivering material. Even if lectures do not break up presentation slides with attention-raising activities, they can still show more important information during periods of high attention and less important information during periods of low attention. This area of research has applications both in in-person classrooms and online learning environments. The long-term goals of this research can prove invaluable for large in-person classrooms or classrooms where students’ faces are obscured, such as behind computer monitors.

This paper describes objective technical results and analysis.Any subjective views or opinions that might be expressed

Testing Tutor is a web-based platform that helps instructors support software testing pedagogy by automatically diagnosing the fundamental testing concepts (e.g., boundary value) not covered in students' test suite and subsequently helping students initiate their own learning process about those concepts and systematically improve their test suites. The platform's differentiating features include 1) customizable feedback engine which allows instructors to scaffold the level of feedback (varying from conceptual to detailed), 2) a built-in repository of problems instructors can use, 3) access to digital learning content and 4) modes (learning and development) so instructors can scaffold the level of problems. This demo provides a brief overview of the platform and a walkthrough of two example use cases that illustrate the power of Testing Tutor from the perspectives of an instructor and a student. The two use cases will (1) demonstrate using Testing Tutor at different course levels by walking through the steps for an instructor to configure an assignment and the feedback engine and (2) demonstrate the student's experience submitting an assignment and receiving feedback. Information about Testing Tutor can be found at https://testingtutor.org. This work is supported by NSF IUSE grants 2013296 and 2013342.

This Innovative Practice Full-Paper presents a system for real-time accurate detection of classroom attentiveness using monitor-mounted webcams and eye trackers. Academic institutions and instructors cannot accurately assess the moment-to-moment attentiveness of students in classrooms where students' faces are obscured by computer monitors. This can cause the lectures of Computer Science, Information Technology, or other lab-based courses to be incorrectly paced, which leads to students having overall poorer grasps of the subject material. We present a system for accurate detection of classroom attentiveness using monitor-mounted webcams and eye trackers. To determine correlations for the attentiveness judging system, we compare an initial attentiveness score produced by trained labelers using an image of the student's face with a series of calculated eye metrics to determine a final attentiveness score. Because the student webcam images and eye coordinates are synchronously collected with the lecture, this final attentiveness score is used to provide post-hoc feedback to instructors on the status of their students via time-series graphs displayed on the instructor's computer monitor. The proposed system is invaluable for institutions seeking to improve student education, instructors striving to improve the flow of lectures, and students seeking a more accommodating learning environment. The primary source of innovation from this system comes from the correlation of extracted eye metrics with the face images labeled for attentiveness. Research exists about determining attentiveness using a convolutional neural network trained on face images and even determining attentiveness by correlating face-image-trained outputs, each of which we plan to incorporate to make our system real-time in the future. This novel research could prove helpful for the field of education.

The feedback provided by current testing education tools about the deficiencies in a student's test suite either mimics industry code coverage tools or lists specific instructor test cases that are missing from the student's test suite. While useful in some sense, these types of feedback are akin to revealing the solution to the problem, which can inadvertently encourage students to pursue a trial-and-error approach to testing, rather than using a more systematic approach that encourages learning. In addition to not teaching students why their test suite is inadequate, this type of feedback may motivate students to become dependent on the feedback rather than thinking for themselves. To address this deficiency, there is an opportunity to investigate alternative feedback mechanisms that include a positive reinforcement of testing concepts. We argue that using an inquiry-based learning approach is better than simply providing the answers. To facilitate this type of learning, we present Testing Tutor, a web-based assignment submission platform that supports different levels of testing pedagogy via a customizable feedback engine. We evaluated the impact of the different types of feedback through an empirical study in two sophomore-level courses. We use Testing Tutor to provide students with different types of feedback, either traditional detailed code coverage feedback or inquiry-based learning conceptual feedback, and compare the effects. The results show that students that receive conceptual feedback had higher code coverage (by different measures), fewer redundant test cases, and higher programming grades than the students who receive traditional code coverage feedback.

Background: Open source software repositories like GitHub are mined to gain useful empirical software engineering insights and answer critical research questions. However, the present state of the art mining approaches suffers from high error rate in the labeling of data that is used for such analysis. This is particularly true when labels are automatically generated from the commit message, and seriously undermines the results of these studies. Aim: Our goal is to label commit comments with high accuracy automatically. In this work, we focus on classifying a commit as a “Bug-Fix commit” or not. Method: Traditionally, researchers have utilized keyword-based approaches to identify bug fix commits that leads to a significant increase in the error rate. We present an alternative methodology leveraging a deep neural network model called Bidirectional Encoder Representations from Transformers (BERT) that can understand the context of the commit message. We provide the rules for semantic interpretation of commit comments. We construct a hand-labeled dataset from real GitHub commits according to these rules and fine-tune BERT for classification. Results: Our initial evaluation shows that our approach significantly reduces the error rate, with up to 10% relative improvement in classification over keyword-based approaches. Future Direction: We plan on extending our dataset to cover more corner cases and reduce programming language specific biases. We also plan on refining the semantic rules. In this work, we have only considered a simple binary classification problem (Bug-Fix or not), which we plan to extend to other classes and extend the approach to consider multiclass problems. Conclusion: The rules, data, and the model proposed in this paper have the potential to be used by people analyzing open source repositories to improve the labeling of data used in their analysis.

Background: This Research Category Full Paper presents the results of authors' efforts to develop and evaluate learning materials for introducing Software Engineering (SE) students to the Cognitive Psychology concept of human errors (specifically to those human errors that occur during software development). During the last few years, the authors have developed, through a rigorous literature review and empirical investigation, human error intervention instrumentation and supporting training/teaching material. The intervention instrument consists of a corpus of human errors and a tool to support human error based software requirements inspections. The primary aim of developing this instrumentation and training material is to impart SE/CS students with the knowledge about the most frequently committed human errors during the software development process. Goal and Method: First, a study was conducted with Industry Practitioners with the goal of examining if the practitioners believed that human errors and human error training are useful and relevant to the software development process. Next, based on feedback from the practitioners, a study was conducted in an undergraduate Software Engineering course where students were trained using the human error instrument and were asked to perform error based requirements inspections. The high-level goal of this paper is to evaluate whether requirements inspections supported by human errors can be used to deliver knowledge about software engineering human errors as well as knowledge about requirements inspections (a key industry skill) to students. Results: Results showed that industry practitioners found the human error instrumentation and training useful. Based on their feedback, when the training was administered to students, it helped students understand those human errors that are the frequently committed during the software development process.

This full research category paper reports the result of an experiment that designed and implemented peer code review (PCR) to help introductory computer science (CS1) students understand programming concepts and improve their programming skills. Instructors at North Dakota State University have observed that students enrolled in CS1 programming course experience difficulty understanding programming concepts. We performed an empirical study that evaluated if PCR can help address the problem. We analyzed data collected from the PCR session, code development activity and end-of-study questionnaire. Our result provides insights into the most and least reported error types. Students were able to systematically improve the review output. More importantly, PCR had a significant positive impact on students' performance when developing their own code. PCR can be an effective teaching tool that when used in CS1 programming course can improve students' understanding of programming concepts and improve their programming skills.

This work in progress research paper discusses importance of appropriate order of content delivery in an introductory programming course (CS1). A majority of students face problems when programming concepts are introduced to them in disorderly fashion, reducing their retention. It has been observed that the instructors use their intrinsic judgement to order course contents without considering effective student learning outcome. This traditional approach of teaching a course in an unstructured way leads to high failure and dropout rate in CS1 course. In this study, an association rule mining (ARM) based approach is being used to understand the order of information that students should be exposed to in CS1 courses. Our proposed approach follows an empirical research methodology and generates strong ARM rules. These rules can assist instructors to structure CS1 topics and reflect on required prerequisites for problematic topics to improve students' learning. Our initial results and observations yielded promising and logical inferences and uncovered few anomalies with traditional pedagogy during course offering.

Programming is a skill, often acquired through repeated practice and feedback. During traditional lectures, students not actively engaged in their own learning. It is imperative to pique students motivation and direct their focus on gaining the requisite knowledge. As the class size grows, instructors feedback is delayed that impacts student engagement and learning. Educational researchers have supported using web-based tools to help evaluate student work, provide timely feedback and increase the amount of time they spend improving their skills. Motivated by the previous work, our team has developed the SEP-CyLE (Software Engineering and Programming Cyber Learning Environment) - a cyber learning environment that contains digital learning content of software programming and testing concepts. SEP-CyLE incorporates collaborative learning, social networking and gamification-based learning engagement strategies (LESs) that has led to an improved motivation and understanding of programming concepts. This paper aims to assess the impact of different combinations of these LESs on student learning in the context of CS1 classrooms. We coordinated studies at two universities wherein different combination of LESs were utilized using SEP-CyLE in CS1 classrooms. We analyzed the impact of LESs on students' acquisition of programming concepts, their engagement and usage of SEP-CyLE. The pre and post test results indicated that the assorted LEs have shown a positive impact on student learning across all the institutions. The correlation results demonstrated that there is meaningful relationship between the LEs and the student performance.

[Background] Software inspections are extensively used for requirements verification. Our research uses the perspective of human cognitive failures (i.e., human errors) to improve the fault detection effectiveness of traditional fault-checklist based inspections. Our previous evaluations of a formal human error based inspection technique called Error Abstraction and Inspection (EAI) have shown encouraging results, but have also highlighted a real need for improvement. [Aims and Method] The goal of conducting the controlled study presented in this paper was to identify the specific tasks of EAI that inspectors find most difficult to perform and the strategies that successful inspectors use when performing the tasks. [Results] The results highlighted specific pain points of EAI that can be addressed by improving the training and instrumentation.

In spite of the human-centric aspect of software engineering (SE) discipline, human error knowledge has been ignored by SE educators as it is often thought of as something that belongs in the realm of Psychology. SE curriculum is also severely devoid of educational content on human errors, while other human-centric disciplines (aviation, medicine, process control) have developed human error training and other interventions. To evaluate the feasibility of using such interventions to teach students about human errors in SE, this paper describes an exploratory study to evaluate whether requirements inspections driven by human errors can be used to deliver both requirements validation knowledge (a key industry skill) and human error knowledge to students. The results suggest that human error based inspections can enhance the fault detection abilities of students, a primary learning outcome of inspection exercises conducted in software engineering courses. Additionally, results showed that students found human error information useful for understanding the underlying causes of requirement faults.

Background -- Inspecting requirements and design artifacts to find faults saves rework effort significantly. While inspections are effective, their overall team performance rely on inspectors' ability to detect and report faults. Our previous research showed that individual inspectors have varying LSs (i.e., they vary in their ability to process information recorded in requirements document). To extend the results of our previous LS research, this paper utilizes the concept of eye tracking (to record eye movements of inspectors) along with their LSs to detect reading patterns of inspectors during requirements inspections. Aim -- The objective of this research is to analyze the reading trends of effective and efficient inspectors using eye movement and LS data of individual inspectors and virtual inspection teams. Method -- The current research uses data (LS, eye tracking, and inspection) from thirteen inspectors to find its impact on inspection effectiveness and efficiency. Results -- Results from this study show that, inspectors who detect more faults during inspection, focus significantly more at the fault region to find and report faults as opposed to comprehending requirements information. Results also showed Inspection teams with diverse inspectors outperform similar teams and spend more time in comprehending information at the fault region. Additionally, results showed that inspectors with SEQ LS significantly tends to focus more at fault locations and are preferred for inspection. Conclusion -- These results can aid the selection of inspectors during the inspection process thus improving software quality

Background: Developing correct software requirements is important for overall software quality. Most existing quality improvement approaches focus on detection and removal of faults (i.e. problems recorded in a document) as opposed identifying the underlying errors that produced those faults. Accordingly, developers are likely to make the same errors in the future and fail to recognize other existing faults with the same origins. Therefore, we have created a Human Error Taxonomy (HET) to help software engineers improve their software requirement specification (SRS) documents. Aims: The goal of this paper is to analyze whether the HET is useful for classifying errors and for guiding developers to find additional faults. Methods: We conducted a empirical study in a classroom setting to evaluate the usefulness and feasibility of the HET. Results: First, software developers were able to employ error categories in the HET to identify and classify the underlying sources of faults identified during the inspection of SRS documents. Second, developers were able to use that information to detect additional faults that had gone unnoticed during the initial inspection. Finally, the participants had a positive impression about the usefulness of the HET. Conclusions: The HET is effective for identifying and classifying requirements errors and faults, thereby helping to improve the overall quality of the SRS and the software.

Software inspections are an effective method for early detection of faults present in software development artifacts (e.g., requirements and design documents). However, many faults are left undetected due to the lack of focus on the underlying sources of faults (i.e., what caused the injection of the fault?). To address this problem, research work done by Psychologists on analyzing the failures of human cognition (i.e., human errors) is being used in this research to help inspectors detect errors and corresponding faults (manifestations of errors) in requirements documents. We hypothesize that the fault detection performance will demonstrate significant gains when using a formal taxonomy of human errors (the underlying source of faults). This paper describes a newly developed Human Error Taxonomy (HET) and a formal Error-Abstraction and Inspection (EAI) process to improve fault detection performance of inspectors during the requirements inspection. A controlled empirical study evaluated the usefulness of HET and EAI compared to fault based inspection. The results verify our hypothesis and provide useful insights into commonly occurring human errors that contributed to requirement faults along with areas to further refine both the HET and the EAI process.

Software inspections are an effective method for achieving high quality software.We hypothesize that inspections focused on identifying errors (i.e., root cause of faults) are better at finding requirements faults when compared to inspection methods that rely on checklists created using lessons-learned from historical fault-data.Our previous work verified that, error based inspections guided by an initial requirements errors taxonomy (RET) performed significantly better than standard fault-based inspections.However, RET lacked an underlying human information processing model grounded in Cognitive Psychology research.The current research reports results from a systematic literature review (SLR) of Software Engineering and Cognitive Science literature -Human Error Taxonomy (HET) that contains requirements phase human errors.The major contribution of this paper is a report of control group study that compared the fault detection effectiveness and usefulness of HET with the previously validated RET.Results of this study show that subjects using HET were not only more effective at detecting faults, but they found faults faster.Post-hoc analysis of HET also revealed meaningful insights into the most commonly occurring human errors at different points during requirements development.The results provide motivation and feedback for further refining HET and creating formal inspection tools based on HET.

Advances in the psychological understanding of the origins and manifestations of human error have led to tremendous reductions in errors in fields such as medicine, aviation, and nuclear power plants. This workshop is intended to foster a better understanding of software engineering errors and how a psychological perspective can reduce them, improving software quality and reducing maintenance costs. The workshop goal is to develop a body of knowledge that can advance our understanding of the psychological processes (of human reasoning, planning, and problem solving) and how they fail during the software development. Applying human error research to software quality improvement will provide insights to the cognitive aspects of software development. The workshop will include interactive session to discuss common themes of errors in different fields, and structure software error information to detect and prevent software errors during the development.

Inspections of software artifacts during early software development aids managers to detect early faults that may be hard to find and fix later.While inspections are effective, evidence suggests that inspection abilities of individuals vary widely which affect overall inspection effectiveness.Cognitive psychologists have used Learning Styles (LS) to measure an individual's characteristic strength and ability to acquire and process information.This concept of LS is being utilized in software engineering domain as a means to improve inspection performance.This paper presents the results from an industrial empirical study, wherein the LS's of individual inspectors were manipulated to measure its impact on the fault detection effectiveness of inspection teams.Using inspection data from nineteen professional developers, we developed virtual teams with varying LS's of individual inspectors and analyzed the team performance.The results from the current study show that, teams of inspectors with diverse LS's are significantly more effective at detecting faults as compared to teams of inspectors with similar LS's.Therefore, LS's can aid software managers to create high performance inspection team(s) and manage software quality.

Managers recognize that software development project teams need to be developed and guided.Although technical skills are necessary, non-technical (NT) skills are equally, if not more, necessary for project success.Currently, there are no proven tools to measure the NT skills of software developers or software development teams.Behavioral markers (observable behaviors that have positive or negative impacts on individual or team performance) are beginning to be successfully used by airline and medical industries to measure NT skill performance.The purpose of this research is to develop and validate the behavior marker system tool that can be used by different managers or coaches to measure the NT skills of software development individuals and teams.This paper presents an empirical study conducted at the Software Factory where users of the behavior marker tool rated video clips of software development teams.The initial results show that the behavior marker tool can be reliably used with minimal training.

Inspections of software artifacts during early software development aids managers to detect early faults that may be hard to find and fix later. Results showed inspection ability does not depend on educational background and technical knowledge. This paper presents the results from an industrial empirical study, wherein the Learning Styles (i.e. ability to perceive and process information) of individual inspectors were manipulated to measure its impact on the fault detection effectiveness of inspection teams. Using inspection data from professional developers, we developed virtual teams with varying LS’s of individual inspectors and analyzed the team performance. The results from the current study show that teams of inspectors with diverse LS’s are significantly more effective at detecting faults as compared to teams of inspectors with similar LS’s. Therefore, LS’s can aid software managers to create high performance inspection team(s) and manage software quality.

Managers recognize that software development teams need to be developed. Although technical skills are necessary, non-technical (NT) skills are equally, if not more, necessary for project success. Currently, there are no proven tools to measure the NT skills of software developers or software development teams. Behavioral markers (observable behaviors that have positive or negative impacts on individual or team performance) are successfully used by airline and medical industries to measure NT skill performance. This research developed and validated a behavior marker system through an empirical study conducted at the Software Factory where users of the behavior marker tool rated video clips of software development teams. The initial results show that the behavior marker tool can be reliably used with minimal training.

No abstract available.

Large paper sheets are still the most preferred medium used by engineers to inspect remote sites. However, these paper documents are hard to modify and retrieve. This paper presents a novel, spatially aware, mobile system (called PhoneLens) which combines the merits of paper documents and mobile devices. It augments paper documents with digital information. Different from previous approaches, PhoneLens is inexpensive. It includes two infrared LEDs (i.e., light-emitting diodes), one Wiimote, and one Android device. Based on the hardware setting, we developed an efficient spatial-tracking algorithm to record the movement of a mobile device within a large workspace. Our approach is robust and applicable to various scenarios. PhoneLens provides different functions to browse a multivalent document, such as browsing different layers, searching annotations, and zooming. We conducted a controlled study that compared the participants' performance with PhoneLens against a traditional paper-based method with a multivalent paper document at a significance level of p <; 0.05. The following results were obtained from the study: 1) PhoneLens was significantly more efficient than the paper-based method on search and measurement tasks; 2) PhoneLens was rated higher on subjects' overall experience than the paper-based method; and 3) the usefulness of the training on PhoneLens was positively correlated with subjects' browsing efficiency.

Inspections aid software managers by early detection and removal of faults committed during the creation of requirements and design documents. This helps reduce the rework during the later stages of software development. While inspections are effective in practice, the evidence suggests that the effectiveness of inspectors varies widely. Cognitive psychologists have used Learning Style (LS) to show the improvement in student's score by considering their characteristic strength and preferences to acquire and process information. This concept of LS can cross over to software engineering as a means of increasing the inspection effectiveness. This paper investigates the effect of the LS of inspectors on fault detection abilities of inspection teams and individual inspectors. Using the inspection data with varying number of participants, we analyzed the effect of the LS of inspectors across various inspection team sizes on the inspection performance. We also analyzed the effect of LS categories on the individual inspection performance. The initial results show that the teams composed of inspectors with different LS preferences are more effective and efficient than the teams of inspectors who had similar LS's. The results also provide insights into the LS categories that favor requirements inspection.

Although computer science, information systems, and information technology educators often do an exemplary job of preparing their students for jobs in industry or for further education, there are still many areas where these students do not possess the necessary skills or knowledge based on the expectations of employers or academia. These gaps between the abilities of graduating students and those expected to have can prevent them from succeeding in their careers. This paper presents the results of a systematic literature review conducted to determine which areas graduating students most frequently fall short of the expectations of industry or macademia. The results of this review indicate that graduating students are lacking in many different areas, including technical abilities (design, testing, configuration management tools, etc.) personal skills (communication, teamwork, etc.) and professional qualities (e.g. ethics). By raising awareness of these areas, it is possible for educators to become aware of areas where students most frequently fail to meet expectations and to make curriculum changes or adjustments to address these problems

Inspections and testing are two widely recommended techniques for improving software quality. While testing cannot be conducted until software is implemented, inspections can help find and fix the faults right after their injection in the requirements and design documents. It is estimated that majority of testing cost is spent on fault rework and can be saved by inspections of early software products. However there is a lack of evidence regarding the testing costs saved by performing inspections. This research analyzes the costs and benefits of inspections and testing to decide on whether to schedule an inspection. We also analyzed the effect of the team size on the decision of how to organize the inspections. Another aspect of our research evaluates the use of Capture Recapture (CR) estimation method when the actual fault count of software product is unknown. Using data from 73 inspectors, we applied the Kusumoto metric to evaluate the cost-effectiveness of the inspections with varying team size. Our results provide a detailed analysis of the number of inspectors required for varying levels of cost-effectiveness during inspections; and the number of inspectors required by the CR estimators to provide estimates within 5% to 20% of the actual.

Teamwork is essential in industry and a university is an excellent place to assess which skills are important and for students to practice those skills. A positive teamwork experience can also improve student learning outcomes. Prior research has established that teams with high levels of social sensitivity tend to perform well when completing a variety of specific, short-team, collaborative tasks. Social sensitivity is the personal ability to perceive and understand the feelings and viewpoints of others, and it is reliably measurable. Our hypothesis is that, social sensitivity can be a key component in positively mediating teamwork task activities and member satisfaction. Our goal is to bring attention to the fact that social sensitivity is an asset to teamwork. We report the results from an empirical study that investigates whether social sensitivity is correlated with the effectiveness of processes involved in teamwork and team member satisfaction in an educational setting. The results support our hypothesis that the social sensitivity is highly correlated with team effectiveness. It suggests, therefore, that educators in computer-related disciplines, as well as computer professionals in the workforce, should take the concept of social sensitivity seriously as an aid or obstacle to team performance and the teamwork experience.

This paper presents ongoing research into the use of mental model consistency (MMC) to produce more effective student programming pairs. Previous studies have found that pair programming is highly useful in improving students' enjoyment of programming as well as improving the retention rates of students enrolled in computer science programs. However, existing research provides little support that pair programming actually benefits student learning in terms of improved test or exam scores. This research focuses on evaluating the use of MMC-based student pairs to increase student performance in introductory programming courses. Empirical studies were conducted over two semesters to determine if pairings based on different levels of MMC produced more effective pairs. The results from this study indicate that MMC is a good predictor of success in a course when using pair programming and that students who migrate towards greater consistency tend to do better than those who do not migrate. However, the current results do not support that pairs based on any combination of mental models are more effective than others. Still, the authors of this paper feel that MMC is a valuable method and that if combined with other techniques to produce more compatible pairs, may yet produce substantial results. Other potential uses for MCC are also discussed.

Team work is the norm in major development projects and industry is continually striving to improve team effectiveness. Researchers have established that teams with high levels of social sensitivity tend to perform well when completing a variety of specific collaborative tasks. Social sensitivity is the personal ability to perceive, understand, and respect the feelings and viewpoints of others, and it is reliably measurable. However, the tasks in recent research have been primarily short term, requiring only hours to finish, whereas major project teams work together for longer durations and on complex tasks. Our claim is that, social sensitivity can be a key component in predicting the performance of teams that carry out major projects. Our goal is to determine if previous research, which was not focused on students or professionals in scientific or technical fields, is germane for people in computing disciplines. This paper reports the results from an empirical study that investigates whether social sensitivity is correlated with the performance of student teams on large semester-long projects. The overall result supports our claim that the team social sensitivity is highly correlated with successful team performance. It suggests, therefore, that educators in computer-related disciplines, as well as computer professionals in the workforce, should take the concept of social sensitivity seriously as an aid or obstacle to productivity.

Pair Programming has been shown to be beneficial to student learning. Much research has been conducted to effectively create student pairs when using pair programming in introductory computer science courses. This paper reports results of research investigating the effectiveness of pairing students based on their mental model consistency. Prior research has found a strong correlation between mental model consistency and performance in introductory computer programming courses. Evaluating students' mental models helps to provide insights into how students approach problem solving and may indicate how to effectively pair students to improve their programming ability and learning. The results from an empirical study conducted to investigate these effects indicate that mental model consistency is a predictor of student success in an introductory programming course. Future goals of this research are to fully evaluate all possible pairing arrangements and to produce tests that can be used to evaluate mental model consistency for other computer science concepts.

At North Dakota State University, there are multiple sections of the CS1 and CS2 introductory computer science courses. A large number of students are enrolled in each section, making it difficult to hold laboratory sessions as there is not enough space for all of the students in one room. This results in diminished student attendance and a decrease in student understanding of the course material. Pair programming has been shown to have multiple benefits in educational use. Previous research has shown that it benefits student learning in addition to increasing the student retention in computer science programming courses. Using pair programming would also allow students to share laboratory resources and make it possible to accommodate more students in laboratory session. To study the effects of pair programming on student-instructor interactions in laboratory sessions of introductory computer science courses, an empirical study was conducted at North Dakota State University. Data about student-instructor interactions was collected by monitoring the laboratory sessions during the study run, as well as through a post-study survey given to students and interviews with the instructors. The results from this study indicate that having students work in pairs as opposed to individually reduces the number of questions from students and decreases the amount of time that a student must wait for instructor assistance.

Pair programming is a programming technique where two programmers work together on the same programming task. Previous research has shown that it is effective for improving the learning effectiveness, efficiency, and enjoyment of students in introductory programming courses. Much research has also been dedicated to determining effective strategies for forming pairs. This paper discuss two different empirical studies conducted at North Dakota State University to a) test the feasibility of using pair programming in introductory computer science courses and b) determine whether or not major-based pairing produces effective pairs. The results of these studies provide support for implementing pair programming in introductory computer science courses and show that pairing of computer science and non-computer science students may produce pairs which are less compatible than other pairing methods.

Defect prevention techniques can be used during the creation of software artifacts to help developers create high-quality artifacts. These artifacts should have fewer faults that must be removed during inspection and testing. The Requirement Error Taxonomy that we have developed helps focus developers' attention on common errors that can occur during requirements engineering. Our claim is that, by focusing on those errors, the developers will be less likely to commit them. This paper investigates the usefulness of the Requirement Error Taxonomy as a defect prevention technique. The goal was to determine if making requirements engineers' familiar with the Requirement Error Taxonomy would reduce the likelihood that they commit errors while developing a requirements document. We conducted an empirical study in which the participants were given the opportunity to learn how to use the Requirement Error Taxonomy by employing it during the inspection of a requirements document. Then, in teams of four, they developed their own requirements document. This requirements document was then evaluated by other students to identify any errors made. The hypothesis was that participants who find more errors during the inspection of a requirements document would make fewer errors when creating their own requirements document. The overall result supports this hypothesis.

Project managers can use the capture-recapture models to estimate the number of faults in a software artifact. The capture-recapture estimates are calculated using the number of unique faults and the number of times each fault is found. The accuracy of the estimates is affected by the number of inspectors and the number of faults. Our earlier research investigated the effect that the number of inspectors had on the accuracy of the estimates. In this paper, we investigate the effect of the number of faults on the performance of the estimates using real requirement artifacts. These artifacts have an unknown amount of naturally occurring faults. The results show that while the estimators generally underestimate, they improve as the number of faults increases. The results also show that the capture-recapture estimators can be used to make correct re-inspection decisions.

Project managers can use capture-recapture models to manage the inspection process by estimating the number of defects present in an artifact and determining whether a reinspection is necessary. Researchers have previously evaluated capture-recapture models on artifacts with a known number of defects. Before applying capture-recapture models in real development, an evaluation of those models on naturally-occurring defects is imperative. The data in this study is drawn from two inspections of real requirements documents (that later guided implementation) created as part of a capstone course (i.e. with naturally occurring defects). The major results show that: a) estimators improve from being negatively biased after one inspection to being positively biased after two inspections, b) the results contradict the earlier result that a model that includes two sources of variation is a significant improvement over models with one source of variation, and c) estimates are useful in determining the need for artifact reinspection.

Inspections can be made more cost-effective by using capture-recapture methods to estimate post-inspection defects. Previous capture-recapture studies of inspections used relatively small data sets compared with those used in biology and wildlife research (the origin of the models). A common belief is that capture-recapture models underestimate the number of defects but their performance can be improved with data from more inspectors. This increase has not been evaluated in detail. This paper evaluates new estimators from biology not been previously applied to inspections. Using a data from seventy-three inspectors, we analyze the effect of the number of inspectors on the quality of estimates. Contrary to previous findings indicating that Jackknife is the best estimator, our results show that the SC estimators are better suited to software inspections. Our results also provide a detailed analysis of the number of inspectors necessary to obtain estimates within 5% to 20% of the actual.

Project managers use inspection data as input to capture-recapture (CR) models to estimate the total number of faults present in a software artifact. The CR models use the number of faults found during an inspection and the overlap of faults among inspectors to calculate the estimate. A common belief is that CR models underestimate the number of faults but their performance can be improved with more input data. This paper investigates the minimum number of faults that has to be present in an artifact before the CR method can be used. The result shows that the minimum number of faults varies from ten faults to twenty-three faults for different CR estimators.

This paper is the second in a series of empirical studies about requirement error abstraction and classification as a quality improvement approach. The Requirement error abstraction and classification method supports the developers' effort in efficiently identifying the root cause of requirements faults. By uncovering the source of faults, the developers can locate and remove additional related faults that may have been overlooked, thereby improving the quality and reliability of the resulting system. This study is a replication of an earlier study that adds a control group to address a major validity threat. The approach studied includes a process for abstracting errors from faults and provides a requirement error taxonomy for organizing those errors. A unique aspect of this work is the use of research from human cognition to improve the process. The results of the replication are presented and compared with the results from the original study. Overall, the results from this study indicate that the error abstraction and classification approach improves the effectiveness and efficiency of inspectors. The requirement error taxonomy is viewed favorably and provides useful insights into the source of faults. In addition, human cognition research is shown to be an important factor that affects the performance of the inspectors. This study also provides additional evidence to motivate further research.