Technology is advancing at an unprecedented pace. With the advent of cutting-edge technologies, keeping up with rapid changes are becoming increasingly challenging. In addition to that, increasing dependencies on the cloud technologies have imposed enormous pressure on modern web browsers leading to adapting new technologies faster and making them more susceptible to defects/bugs. Although, many studies have explored browser bugs, a comparative study among the modern browsers generalizing the bug categories and their nature was still lacking. To fill this gap, we undertook an empirical investigation aimed at gaining insights into the prevalent bugs in Google Chromium and Mozilla Firefox as the representatives of modern web browsers. We used GPT-4.o to identify the defect (bugs) categories and analyze the clusters of the most commonly appeared bugs in the two prominent web browsers. Additionally, we compared our LLM based bug categorization with the traditional NLP based approach using TF-IDF and K-Means clustering. We found that although Google Chromium and Firefox have evolved together since almost around the same time (2006-2008), Firefox suffers from high number of bugs having extremely high defect-prone components compared to Chromium. This exploratory study offers valuable insights on the browser bugs and defect-prone components to the developers, enabling them to craft web browsers and web-applications with enhanced resilience and reduced errors.

The advent of Large Language Models (LLMs) has revolutionized various domains of artificial intelligence, including the realm of software engineering. In this research, we evaluate the efficacy of pre-trained LLMs in replicating the tasks traditionally performed by developers in response to code review comments. We provide code contexts to five popular LLMs and obtain the suggested code-changes (patch sets) derived from real-world code-review comments. The performance of each model is meticulously assessed by comparing their generated patch sets against the historical data of human-generated patch-sets from the same repositories. This comparative analysis aims to determine the accuracy, relevance, and depth of the LLMs' feedback, thereby evaluating their readiness to support developers in responding to code-review comments. Novelty: This particular research area is still immature requiring a substantial amount of studies yet to be done. No prior research has compared the performance of existing Large Language Models (LLMs) in code-review comments. This in-progress study assesses current LLMs in code review and paves the way for future advancements in automated code quality assurance, reducing context-switching overhead due to interruptions from code change requests.

Feature toggles control the state of features and allow exposing unfinished features to a reduced cohort of users without affecting the general software operation. It is basically a variable used in if conditions to control the flow of program execution. Since there is no universal standard of using feature toggles established yet, developers write code around feature toggles and use them spontaneously. Certain usage patterns of feature toggles may even lead to code smells. In this short paper I introduce six different toggle usage patterns from Google Chromium and discuss the possible reasons, consequences, and detection methods. I further conduct a mixed-method approach to analyze them. Since this study is still in progress, I report the early results only for the three most commonly appeared usage patterns. I validate the quantitative findings with the qualitative results obtained by interviewing 15 Google developers. I found that there are 3.1K toggles present in 38 components of Chromium. In the median case, nested toggles are shared by 5 different files, spread toggles span 2 different components, and dead toggles cover an average of 4 lines of code (loc).Novel aspects:- Although usage patterns of C pre-processors (#ifdefs) are studied in the past, I did not find any study particularly focusing on the run-time feature toggles usage patterns. Hence, I have been inspired to do an exploratory study to investigate different usage patterns of feature toggles. I chose Google Chromium as a case study at this point since Google developers use feature toggles extensively.

Project-based learning (PjBL) is a desirable form of active learning that facilitate student engagement, team work and problem solving. Current literature in PjBL have studied the merits, demerits, implementation strategies and the impact of PjBL on student performance. However, PjBL has not been properly studied from the lens of Industry Standard Tools and Practices (IST&Ps). Currently, the specific learning effectiveness of PjBL vis-a-vis IST&Ps are largely unknown. To provide insight into the effectiveness of PjBL in relation to IST&Ps, we implemented PjBL in our class using 5 popular IST (SQL, Atlassian Jira, GitHub, Jenkins, and planning poker) and 3 most common Agile Development practices as ISP. We collected data from 120 students juniors and seniors using RIMMS as our instrument. Data was analyzed both qualitatively and quantitatively. Our preliminary results shows that IST&P has significantly positive impact over learning effectiveness, and students' engagement.

Predicting fault-prone components at an early stage is useful for any organization to ensure quality software delivery. Prioritizing tests becomes easier with the prediction of faultproneness of the components since developers can allocate more time and resources to the “High” fault-prone components. Furthermore, test prioritization helps reduce the cost of regression and allows developers to take careful decisions regarding the sensitive components. This paper performs an empirical study to predict fault-prone components and their evolution on two popular open-source projects, Chromium web browser and ProFTPD. First, we construct and compare two prediction models: Random Forest (RF) and Support Vector Machine (SVM) for classifying components as “High” or “Low” fault prone. Second, we analyze the evolution of the fault proneness of 22 components of Chromium in 42 releases and 12 components of ProFTPD in 15 releases. Chromium has a median of 3. 9k commits per release with a standard deviation of lk while ProFTPD has 578 commits per release with a standard deviation of 654. We consider the total churns, bug-fix commits, bug-fix churns, rush period changes, number of developers, and number of files modified as the measures to construct our prediction models. Our models are able to successfully predict the fault-proneness of the components having the Random Forest outperforming SVM with an accuracy of 96% and 95% for Chromium and ProFTPD respectively. We found that the majority of the Chromium components are high fault-prone. For ProFTPD, components are found to be high and low fault-prone interchangeably. The fault proneness of the components has evolved over the releases where “Chromecast” in Chromium seems to be gradually turning into a low fault-prone component.

This research to practice full paper presents our investigation into the use of gamification in teaching cybersecurity awareness to students. Although there are evidences of increasing research activities in cybersecurity, cyberattacks continue to be pervasive. Academic institutions are largely responsible for educating and producing skilled professionals with cybersecurity competences. However, cybersecurity education can be challenging, especially to smaller institutions, usually characterized by meagre resources. In literature, one of the beneficial pedagogical methods for teaching cybersecurity awareness is gamification, which is based on constructivist theoretical framework. While this method has proved very useful, one major challenge is that gamification platforms can be costly to develop and difficult to maintain. This may discourage smaller institutions from using gamification to teach cybersecurity awareness. Freemium gamified platforms e.g., Kahoot! can offer an alternative that is affordable, easy to use and requires very little to no overhead cost. The research questions under investigation are: what is the impact of gamification (as an instructional method) on students’ learning of cybersecurity awareness?, which game elements and what aspect of gamification best motivate students?. Using questionnaire, we asked the students, from a small university in Northwestern Pennsylvania, USA, to rate their knowledge and awareness of cyberattacks. Afterwards, we taught a cyber awareness module with 5 learning objectives to these students using gamification in Kahoot! platform. At the end of the class, we administered another questionnaire to the students and asked them to rate their knowledge and awareness of those same cyberattacks. Our analysis and statistical results show that gamification is an effective technique for knowledge acquisition in cybersecurity awareness. Furthermore, students are mostly motivated by game elements that give them a sense of achievement. Finally we found that students are more interested in the knowledge acquisition aspect of gamification rather than the entertainment and winning aspects. The results of our study may be beneficial for instructional design of introductory cybersecurity awareness courses.

For more than 2 decades, online or e-learning has been the major approach to distant education. However, a new variant of online learning AKA emergency remote teaching or ERT has emerged and increasingly becoming popular. ERT refers to the temporary transition of educational activities (instruction, assessment, advising) from the traditional to online to avert the crisis. This differs from a typical online or e-learning wherein educational activities are intended to be delivered online and are thus carefully designed, planned and implemented to fulfill this intention. With regards to existing literature, few publications have identified the differences between online learning and ERT, and expressed concerns over the quality of educational activities in ERT. Currently, studies that validate student learning experience in ERT are lacking in literature. ERT is an emerging pedagogical approach that was widely adopted in Spring 2020 due to COVID-19, hence it is imperative to validate its impact in students’ learning experience as well as instructors’ teaching experience. In this research, we focus on the following research question: what factors affect students’ learning experience and instructors’ teaching experience in an emergency remote teaching? To answer this question, we collected data from 240 students and 98 instructors during the implementation of ERT in our institution in Spring and Fall of 2020. Using a combination of ANOVA and Turkey’s Honestly Significantly Difference (HSD), we analyze the data to determine the factors that can be used to predict student learning experience and teaching experience in ERT. Our results of this study will inspire more studies in ERT and inform effective delivery of instructional activities in time of crisis.

This Full Paper (Research) presents an analysis on the competences in software testing for the fresh graduates in computer science. Software Testing education (ST) is receiving increasing attention in literature, recent studies have evaluated instructional methods used in ST education. However, analysis of competences (skills, knowledge, and ability) required in ST education are lacking in literature. Competences play critical roles in curriculum development e.g., they inform the design of student learning outcomes, learning objectives and program outcomes. This full paper in the research category aims to analyze competences in ST education and then examine the gap between these competences and the current ST curriculum. Using natural language processing (NLP) techniques, we collect 2033 job descriptions from three popular job portals (indeed, monster, and career builder) in the USA and Canada. Also, we collected course syllabi from 20 universities offering ST courses and use these to assess the current curriculum in ST. We analyzed the data using thematic analysis and found that the current software testing curricula do not always teach or equip students with some of the soft skills they require to be successful in software testing career. For instance, our result shows that soft skills such as teamwork, communication, leadership, which are often required by software testing employers are not always taught in ST courses.

Testing is an integral part of release engineering and continuous integration. In theory, a failed test on a build indicates a problem that should be fixed and the build should not be released. In practice, tests decay and developers often release builds, ignoring failing tests. In this paper, we studying the link between builds with failing tests and the number of crash reports on the Firefox webbrowser. Builds with all tests passing have a median of only two crash reports. In contrast, builds with one or more failing tests are associated with a median of 508 and 291 crash reports for Beta and Production builds, respectively. We further investigate the impact of ``flaky'' tests, which can both pass and fail on the same build, and find that they have a median of 514 and 234 crash reports for Beta and Production builds. Finally, building on previous research that has shown that tests that have failed frequently in the past will fail frequently in the future, we find that Builds with HighFailureTests have a median of 585 and 780 crash reports for Beta and Production builds. Unlike other types of test failures, HighFailureTests have a larger impact on Production releases than on Beta builds, and they have a median of 2.7 times more crashes than builds with normal test failures. We conclude that ignoring test failures is related to a dramatic increase in the number of crashes reported by users.

Continuous delivery and rapid releases have led to innovative techniques for integrating new features and bug fixes into a new release faster. To reduce the probability of integration conflicts, major software companies, including Google, Facebook and Netflix, use feature toggles to incrementally integrate and test new features instead of integrating the feature only when it's ready. Even after release, feature toggles allow operations managers to quickly disable a new feature that is behaving erratically or to enable certain features only for certain groups of customers. Since literature on feature toggles is surprisingly slim, this paper tries to understand the prevalence and impact of feature toggles. First, we conducted a quantitative analysis of feature toggle usage across 39 releases of Google Chrome (spanning five years of release history). Then, we studied the technical debt involved with feature toggles by mining a spreadsheet used by Google developers for feature toggle maintenance. Finally, we performed thematic analysis of videos and blog posts of release engineers at major software companies in order to further understand the strengths and drawbacks of feature toggles in practice. We also validated our findings with four Google developers. We find that toggles can reconcile rapid releases with long-term feature development and allow flexible control over which features to deploy. However they also introduce technical debt and additional maintenance for developers.

In my PhD research I will focus on modern release engineering practices. First, I have quantified the time and effort that is involved in stabilizing a release. I found that despite using rapid release, the Chrome and Linux projects still have a period where they rush changes into a release. Second, developers typically isolate unrelated changes on branches. However, developers at major companies, such as Google and Facebook, commit all changes to a single branch. They isolate unrelated changes using feature-flags, which allows them to disable works in progress. My goal is to empirically determine the best practices when using flags and identify dead code. Finally, I will develop tool support to manage feature flags.