Amiangshu Bosu

dblp:18/11159 · DBLP profile ↗
← Back
30ranked-venue papers
10as first author
12since 2021 · last 2026
0000-0002-3178-6232ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Software engineering, systems software and programming languages · 29 · 9 first-author · 12 since 2021Databases, data management, data science and information retrieval · 2 · 2 first-authorSecurity and privacy · 1 · 1 first-authorApplied, interdisciplinary, general and emerging computing · 1
YearPublicationVenuePosition
2026 Cold-Start Anti-Patterns and Refactorings in Serverless Systems: An Empirical Study
abstract
Serverless computing simplifies deployment and scaling, yet cold-start latency remains a major performance bottleneck. Unlike prior work that treats mitigation as a black-box optimization, we study cold starts as a developer-visible design problem. From 81 adjudicated issue reports across open-source serverless systems, we derive taxonomies of initialization anti-patterns, remediation strategies, and diagnostic challenges spanning design, packaging, and runtime layers. Building on these insights, we introduce SCABENCH, a reproducible benchmark, and INITSCOPE, a lightweight analysis framework linking what code is loaded with what is executed. On SCABENCH, INITSCOPE improved localization accuracy by up to 40% and reduced diagnostic effort by 64% compared with prior tools, while a developer study showed higher task accuracy and faster diagnosis. Together, these results advance evidence-driven, performance-aware practices for cold-start mitigation in serverless design. Availability: The research artifact is publicly accessible for future studies and improvements.
Syed Salauddin Mohammad Tariq, Foyzul Hassan, Amiangshu Bosu, Probir Roy
SANER3
2026 Automated Identification of Sexual Orientation and Gender Identity Discriminatory Texts from Issue Comments
abstract
In an industry dominated by straight men, many developers representing other gender identities and sexual orientations often encounter hateful or discriminatory messages. Such communications pose barriers to participation for women and LGBTQ+ persons. Due to sheer volume, manual inspection of all communications for discriminatory communication is infeasible for a large-scale Free Open Source Software (FLOSS) community. To address this challenge, this study proposes an automated mechanism to identify Sexual Orientation and Gender Identity Discriminatory (SGID) texts in software developers’ communications. On this goal, we trained and evaluated SGID4SE (Sexual orientation and Gender Identity Discriminatory text identification for (4) Software Engineering texts), a supervised learning-based tool. SGID4SE incorporates six preprocessing steps and 10 state-of-the-art algorithms. SGID4SE employs six distinct strategies to enhance the performance of the minority class. We empirically evaluated each strategy and identified an optimum configuration for each algorithm. In our ten-fold cross-validation-based evaluations, a BERT-based model achieves the best performance with 85.9% precision, 80.0% recall, and 82.9% F1-score for the SGID class. This model achieves 95.7% accuracy and a Matthews Correlation Coefficient of 80.4%. Our dataset and tool establish a foundation for further research in this direction.
Sayma Sultana, Jaydeb Sarker, Farzana Israt, Rajshakhar Paul, Amiangshu Bosu
ACM Trans. Softw. Eng. Methodol.5
2025 Beyond Binary Moderation: Identifying Fine-Grained Sexist and Misogynistic Behavior on GitHub with Large Language Models
abstract
Background: Sexist and misogynistic behavior significantly hinders inclusion in technical communities like GitHub, causing developers, especially minorities, to leave due to subtle biases and microaggressions. Current moderation tools primarily rely on keyword filtering or binary classifiers, limiting their ability to detect nuanced harm effectively. Aims: This study introduces a fine-grained, multi-class classification framework that leverages instruction-tuned Large Language Models (LLMs) to identify twelve distinct categories of sexist and misogynistic comments on GitHub. Method: We utilized an instruction-tuned LLM-based framework with systematic prompt refinement across 20 iterations, evaluated on 1,440 labeled GitHub comments across twelve sexism/misogyny categories. Model performances were rigorously compared using precision, recall, F1-score, and the Matthews Correlation Coefficient (MCC). Results: Our optimized approach (GPT-4o with Prompt 19) achieved an MCC of 0.501, significantly outperforming baseline approaches. While this model had low false positives, it struggled to interpret nuanced, contextdependent sexism and misogyny reliably. Conclusion: Welldesigned prompts with clear definitions and structured outputs significantly improve the accuracy and interpretability of sexism detection, enabling precise and practical moderation on developer platforms like GitHub.
Tanni Dev, Sayma Sultana, Amiangshu Bosu
ESEM3
2025 From First Patch to Long-Term Contributor: Evaluating Onboarding Recommendations for OSS Newcomers
abstract
Attracting and retaining a steady stream of new contributors is crucial to ensuring the long-term survival of open-source software (OSS) projects. However, there are two key research gaps regarding recommendations for onboarding new contributors to OSS projects. First, most of the existing recommendations are based on a limited number of projects, which raises concerns about their generalizability. If a recommendation yields conflicting results in a different context, it could hinder a newcomer's onboarding process rather than help them. Second, it's unclear whether these recommendations also apply to experienced contributors. If certain recommendations are specific to newcomers, continuing to follow them after their initial contributions are accepted could hinder their chances of becoming long-term contributors. To address these gaps, we conducted a two-stage mixed-method study. In the first stage, we conducted a Systematic Literature Review (SLR) and identified 15 task-related actionable recommendations that newcomers to OSS projects can follow to improve their odds of successful onboarding. In the second stage, we conduct a large-scale empirical study of five Gerrit-based projects and 1,155 OSS projects from GitHub to assess whether those recommendations assist newcomers’ successful onboarding. Our results suggest that four recommendations positively correlate with newcomers’ first patch acceptance in most contexts. Four recommendations are context-dependent, and four indicate significant negative associations for most projects. Our results also found three newcomer-specific recommendations, which OSS joiners should abandon at non-newcomer status to increase their odds of becoming long-term contributors.
Asif Kamal Turzo, Sayma Sultana, Amiangshu Bosu
IEEE Trans. Software Eng.3
2024 What makes a code review useful to OpenDev developers? An empirical investigation
Asif Kamal Turzo, Amiangshu Bosu
Empir. Softw. Eng.2
2023 ToxiSpanSE: An Explainable Toxicity Detection in Code Review Comments
abstract
Background: The existence of toxic conversations in open-source platforms can degrade relationships among software developers and may negatively impact software product quality. To help mitigate this, some initial work has been done to detect toxic comments in the Software Engineering (SE) domain. Aims: Since automatically classifying an entire text as toxic or non-toxic does not help human moderators to understand the specific reason(s) for toxicity, we worked to develop an explainable toxicity detector for the SE domain. Method: Our explainable toxicity detector can detect specific spans of toxic content from SE texts, which can help human moderators by automatically highlighting those spans. This toxic span detection model, ToxiSpanSE, is trained with the 19,651 code review (CR) comments with labeled toxic spans. Our annotators labeled the toxic spans within 3,757 toxic CR samples. We explored several types of models, including one lexicon-based approach and five different transformer-based encoders. Results: After an extensive evaluation of all models, we found that our fine-tuned RoBERTa model achieved the best score with 0.88$F1$, 0.87 precision, and 0.93 recall for toxic class tokens, providing an explainable toxicity classifier for the SE domain. Conclusion: Since ToxiSpanSE is the first tool to detect toxic spans in the SE domain, this tool will pave a path to combat toxicity in the SE community.
Jaydeb Sarker, Sayma Sultana, Steven R. Wilson 0001, Amiangshu Bosu
ESEM4
2023 Towards Automated Classification of Code Review Feedback to Support Analytics
abstract
Background: As improving code review (CR) effectiveness is a priority for many software development organizations, projects have deployed CR analytics platforms to identify potential improvement areas. The number of issues identified, which is a crucial metric to measure CR effectiveness, can be misleading if all issues are placed in the same bin. Therefore, a finer-grained classification of issues identified during CRs can provide actionable insights to improve CR effectiveness. Although a recent work by Fregnan et al. proposed automated models to classify CR-induced changes, we have noticed two potential improvement areas – i) classifying comments that do not induce changes and ii) using deep neural networks (DNN) in conjunction with code context to improve performances. Aims: This study aims to develop an automated CR comment classifier that leverages DNN models to achieve a more reliable performance than Fregnan et al. Method: Using a manually labeled dataset of 1,828 CR comments, we trained and evaluated supervised learning-based DNN models leveraging code context, comment text, and a set of code metrics to classify CR comments into one of the five high-level categories proposed by Turzo and Bosu. Results: Based on our 10-fold cross-validation-based evaluations of multiple combinations of tokenization approaches, we found a model using CodeBERT achieving the best accuracy of 59.3%. Our approach outperforms Fregnan et al.'s approach by achieving 18.7% higher accuracy. Conclusion: In addition to facilitating improved CR analytics, our proposed model can be useful for developers in prioritizing code review feedback and selecting reviewers.
Asif Kamal Turzo, Fahim Faysal, Ovi Poddar, Jaydeb Sarker, Anindya Iqbal, Amiangshu Bosu
ESEM6
2023 Code reviews in open source projects : how do gender biases affect participation and outcomes?
Sayma Sultana, Asif Kamal Turzo, Amiangshu Bosu
Empir. Softw. Eng.3
2023 Automated Identification of Toxic Code Reviews Using ToxiCR
abstract
Toxic conversations during software development interactions may have serious repercussions on a Free and Open Source Software (FOSS) development project. For example, victims of toxic conversations may become afraid to express themselves, therefore get demotivated, and may eventually leave the project. Automated filtering of toxic conversations may help a FOSS community maintain healthy interactions among its members. However, off-the-shelf toxicity detectors perform poorly on a software engineering dataset, such as one curated from code review comments. To counter this challenge, we presentToxiCR, a supervised learning based toxicity identification tool for code review interactions. ToxiCR includes a choice to select one of the 10 supervised learning algorithms, an option to select text vectorization techniques, eight preprocessing steps, and a large-scale labeled dataset of 19,651 code review comments. Two out of those eight preprocessing steps are software engineering domain specific. With our rigorous evaluation of the models with various combinations of preprocessing steps and vectorization techniques, we have identified the best combination for our dataset that boosts 95.8% accuracy and an 88.9% F1-score in identifying toxic texts. ToxiCR significantly outperforms existing toxicity detectors on our dataset. We have released our dataset, pre-trained models, evaluation results, and source code publicly, which is available at https://github.com/WSU-SEAL/ToxiCR .
Jaydeb Sarker, Asif Kamal Turzo, Ming Dong 0001, Amiangshu Bosu
ACM Trans. Softw. Eng. Methodol.4
2021 A Rubric to Identify Misogynistic and Sexist Texts from Software Developer Communications
abstract
Background: As contemporary software development organizations are dominated by males, occurrences of misogynistic and sexist remarks are abundant in many communities. Such remarks are barriers to promoting diversity and inclusion in the software engineering (SE) domain.
Sayma Sultana, Jaydeb Sarker, Amiangshu Bosu
ESEM3
2021 Why Security Defects Go Unnoticed during Code Reviews? A Case-Control Study of the Chromium OS Project
abstract
Peer code review has been found to be effective in identifying security vulnerabilities. However, despite practicing mandatory code reviews, many Open Source Software (OSS) projects still encounter a large number of post-release security vulnerabilities, as some security defects escape those. Therefore, a project manager may wonder if there was any weakness or inconsistency during a code review that missed a security vulnerability. Answers to this question may help a manager pinpointing areas of concern and taking measures to improve the effectiveness of his/her project's code reviews in identifying security defects. Therefore, this study aims to identify the factors that differentiate code reviews that successfully identified security defects from those that missed such defects. With this goal, we conduct a case-control study of Chromium OS project. Using multi-stage semi-automated approaches, we build a dataset of 516 code reviews that successfully identified security defects and 374 code reviews where security defects escaped. The results of our empirical study suggest that the are significant differences between the categories of security defects that are identified and that are missed during code reviews. A logistic regression model fitted on our dataset achieved an AUC score of 0.91 and has identified nine code review attributes that influence identifications of security defects. While time to complete a review, the number of mutual reviews between two developers, and if the review is for a bug fix have positive impacts on vulnerability identification, opposite effects are observed from the number of directories under review, the number of total reviews by a developer, and the total number of prior commits for the file under review.
Rajshakhar Paul, Asif Kamal Turzo, Amiangshu Bosu
ICSE3
2021 Using a balanced scorecard to identify opportunities to improve code review effectiveness: an industrial experience report
Masum Hasan, Anindya Iqbal, Mohammad Rafid Ul Islam, A. J. M. Imtiajur Rahman, Amiangshu Bosu
Empir. Softw. Eng.5
2020 A Benchmark Study of the Contemporary Toxicity Detectors on Software Engineering Interactions
abstract
Automated filtering of toxic conversations may help an Open-source software (OSS) community to maintain healthy interactions among the project participants. Although, several general purpose tools exist to identify toxic contents, those may incorrectly flag some words commonly used in the Software Engineering (SE) context as toxic (e.g., `junk', `kill', and `dump') and vice versa. To encounter this challenge, an SE specific tool has been proposed by the CMU Strudel Lab (referred as the `STRUDEL' hereinafter) by combining the output of the Perspective API with the output from a customized version of the Stanford's Politeness detector tool. However, since STRUDEL's evaluation was very limited with only 654 SE text, its practical applicability is unclear. Therefore, this study aims to empirically evaluate the Strudel tool as well as four state-of-the-art general purpose toxicity detectors on a large scale SE dataset. On this goal, we empirically developed a rubric to manually label toxic SE interactions. Using this rubric, we manually labeled a dataset of 6,533 code review comments and 4,140 Gitter messages. The results of our analyses suggest significant degradation of all tools' performances on our datasets. Those degradations were significantly higher on our dataset of formal SE communication such as code review than on our dataset of informal communication such as Gitter messages. Two of the models from our study showed significant performance improvements during 10-fold cross validations after we retrained those on our SE datasets. Based on our manual investigations of the incorrectly classified text, we have identified several recommendations for developing an SE specific toxicity detector.
Jaydeb Sarker, Asif Kamal Turzo, Amiangshu Bosu
APSEC3
2019 Identifying the Challenges of the Blockchain Community from StackExchange Topics and Trends
abstract
Software developers around the globe have shown tremendous interests in blockchain with more than seven thousand active blockchain software (BCS) projects on Github. Yet, little research has focused on understanding the challenges encountered by the developers of those projects as well as its' users. Therefore, the objective of this study is to better understand the primary areas of challenges encountered by the BCS community. Using a Latent Dirichlet Allocation based topic modeling, we identify discussion topics from the two Blockchain related StackExchange sites. We manually investigated the posts belonging to each topic to understand challenges encountered by the developers. The results of our study revealed that while the ratios of posts on BCS development are increasing, the ratios of posts on mining cryptocurrencies are decreasing. Due to the scarcity of expert blockchain developers, posts on BCS development are more likely to go either answered or encounter more delays than posts on other topics. Based on our findings, we recommend project maintainers to spend efforts to improve documentations on BCS development as the community lacks supporting materials on that area the most.
Irfan Alahi, Mubassher Islam, Anindya Iqbal, Amiangshu Bosu
COMPSAC (1)4
2019 Diversity and Inclusion in Open Source Software (OSS) Projects: Where Do We Stand?
abstract
Background: As the area of computing has thrived over the years, the participation of women in computing declined. Currently women represent less than 24% of the computing workforce and that number is declining. On the other hand, the ratios of women in Open Source Software (OSS) projects are even lower.Aims: The primary objective of this study is to determine the level of gender diversity among popular OSS projects and identify the presence of gender biases that may discourage females' participation.Method: On this goal, we mined the code review repositories of ten popular OSS projects. We used a semi-automated approach followed by a manual validation to identify the genders of the active contributors.Results: Our results suggest that lack of gender diversity remains an ongoing issue among all the ten projects as each of the projects had less than 10% female developers. However, many of the projects also suffer from lack of inclusion of females to leadership positions. Although none of the projects suggest significant differences between male and female developers in terms of productivity based on three different measures, data from three out of the ten projects indicate technical biases against female developers with lower code acceptance rates as well as delayed feedback during code reviews. However, biases against females are not universal as majority of the projects do not discriminate against females. The two projects with the least ratios of female contributors as core developers showed the most biases against females.Conclusion: Based on our findings, we conclude that promoting and mentoring females to leadership positions may be an effective solution to foster gender diversity.
Amiangshu Bosu, Kazi Zakia Sultana
ESEM1
2019 Expressions of Sentiments during Code Reviews: Male vs. Female
abstract
As most of the software development organizations are male-dominated, many female developers encounter negative workplace experiences and report feeling like they “do not belong”. Exposures to discriminatory expletives or negative critiques from their male colleagues may further exacerbate those feelings. The primary goal of this study is to identify the differences in expressions of sentiments between male and female developers during various software engineering tasks. On this goal, we mined the code review repositories of six popular open source projects. We used a semi-automated approach leveraging the name as well as multiple social networks to identify the gender of a developer. Using SentiSE, a customized and state-of-the-art sentiment analysis tool for the software engineering domain, we classify each communication as negative, positive, or neutral. We also compute the frequencies of sentiment words, emoticons, and expletives used by each developer. Our results suggest that the likelihood of using sentiment words, emoticons, and expletives during code reviews varies based on the gender of a developer, as females are significantly less likely to express sentiments than males. Although female developers were more neutral to their male colleagues than to another female, male developers from three out of the six projects were not only writing more frequent negative comments but also withholding positive encouragements from their female counterparts. Our results provide empirical evidence of another factor behind the negative work place experiences encountered by the female developers that may be contributing to the diminishing number of females in the SE industry.
Rajshakhar Paul, Amiangshu Bosu, Kazi Zakia Sultana
SANER2
2019 Understanding the motivations, challenges and needs of Blockchain software developers: a survey
Amiangshu Bosu, Anindya Iqbal, Rifat Shahriyar, Partha Chakraborty
Empir. Softw. Eng.1
2018 SOQDE: A Supervised Learning Based Question Difficulty Estimation Model for Stack Overflow
abstract
StackOverflow (SO), the most popular community Q&A site rewards answerers with reputation scores to encourage answers from volunteer participants. However, irrespective of the difficulty of a question, the contributor of an accepted answer is awarded with the same 'reputation' score, which may demotivate an user's additional efforts to answer a difficult question. To facilitate a question difficulty aware rewarding system, this study proposes SOQDE (Stack Overflow Question Difficulty Estimation), a supervised learning based Question difficulty estimation model for the StackOverflow. To design SOQDE, we randomly selected 936 questions from a SO datadump exported during September 2017. Two of the authors independently labeled those questions into three categories (basic, intermediate, or advanced), where conflicting labels were resolved through tie-breaking votes from a third author. We performed an empirical study to determine how the difficulty of a question impacts its outcomes, such as number of votes, resolution time, and number of votes. Our results suggest that the answers of a basic question receive more votes and therefore would generate more reputation points for an answerer. Due to less incentives relative to efforts spent by an answerer, intermediate and advanced questions encounter significantly more delays than the basic questions, which further validates the need of a model like SOQDE. To build our model, we have identified textual and contextual features of a question and divided them into two categories-pre-hoc and post-hoc features. We observed a model based on Random Forest achieving the highest mean accuracy (67.6%), using only answer-independent pre-hoc features. Accommodating answer-dependent post-hoc features, we were able to improve the mean accuracy of our model to 75.2%.
Sk Adnan Hassan, Dipto Das, Anindya Iqbal, Amiangshu Bosu, Rifat Shahriyar, Toufique Ahmed
APSEC4
2018 A Comparison of Nano-Patterns vs. Software Metrics in Vulnerability Prediction
abstract
Context: Software security is an imperative aspect of software quality. Early detection of vulnerable code during development can better ensure the security of the codebase and minimize testing efforts. Although traditional software metrics are used for early detection of vulnerabilities, they do not clearly address the granularity level of the issue to precisely pinpoint vulnerabilities. The goal of this study is to employ method-level traceable patterns (nano-patterns) in vulnerability prediction and empirically compare their performance with traditional software metrics. The concept of nano-patterns is similar to design patterns, but these constructs can be automatically recognized and extracted from source code. If nano-patterns can better predict vulnerable methods compared to software metrics, they can be used in developing vulnerability prediction models with better accuracy. Aims: This study explores the performance of method-level patterns in vulnerability prediction. We also compare them with method-level software metrics. Method: We studied vulnerabilities reported for two major releases of Apache Tomcat (6 and 7), Apache CXF, and two stand-alone Java web applications. We used three machine learning techniques to predict vulnerabilities using nano-patterns as features. We applied the same techniques using method-level software metrics as features and compared their performance with nano-patterns. Results: We found that nano-patterns show lower false negative rates for classifying vulnerable methods (for Tomcat 6, 21% vs 34.7%) and therefore, have higher recall in predicting vulnerable code than the software metrics used. On the other hand, software metrics show higher precision than nano-patterns (79.4% vs 76.6%). Conclusion: In summary, we suggest developers use nano-patterns as features for vulnerability prediction to augment existing approaches as these code constructs outperform standard metrics in terms of prediction recall.
Kazi Zakia Sultana, Byron J. Williams, Amiangshu Bosu
APSEC3
2018 Understanding the software development practices of blockchain projects: a survey
abstract
Background: The application of the blockchain technology has shown promises in various areas, such as smart-contracts, Internet of Things, land registry management, identity management, etc. Although Github currently hosts more than three thousand active blockchain software (BCS) projects, a few software engineering research has been conducted on their software engineering practices. Aims: To bridge this gap, we aim to carry out the first formal survey to explore the software engineering practices including requirement analysis, task assignment, testing, and verification of blockchain software projects. Method: We sent an online survey to 1,604 active BCS developers identified via mining the Github repositories of 145 popular BCS projects. The survey received 156 responses that met our criteria for analysis. Results: We found that code review and unit testing are the two most effective software development practices among BCS developers. The results suggest that the requirements of BCS projects are mostly identified and selected by community discussion and project owners which is different from requirement collection of general OSS projects. The results also reveal that the development tasks in BCS projects are primarily assigned on voluntary basis, which is the usual task assignment practice for OSS projects. Conclusions: Our findings indicate that standard software engineering methods including testing and security best practices need to be adapted with more seriousness to address unique characteristics of blockchain and mitigate potential threats.
Partha Chakraborty, Rifat Shahriyar, Anindya Iqbal, Amiangshu Bosu
ESEM4
2017 Collusive Data Leak and More: Large-scale Threat Analysis of Inter-app Communications
abstract
Inter-Component Communication (ICC) provides a message passing mechanism for data exchange between Android applications. It has been long believed that inter-app ICCs can be abused by malware writers to launch collusion attacks using two or more apps. However, because of the complexity of performing pairwise program analysis on apps, the scale of existing analyses is too small (e.g., up to several hundred) to produce concrete security evidence. In this paper, we report our findings in the first large-scale detection of collusive and vulnerable apps, based on inter-app ICC data flows among 110,150 real-world apps. Our system design aims to balance the accuracy of static ICC resolution/data-flow analysis and run-time scalability. This large-scale analysis provides real-world evidence and deep insights on various types of inter-app ICC abuse. Besides the empirical findings, we make several technical contributions, including a new open-source ICC resolution tool with improved accuracy over the state-of-the-art, and a large database of inter-app ICCs and their attributes.
Amiangshu Bosu, Danfeng Yao, Gang Wang 0011
AsiaCCS1
2017 Understanding the impressions, motivations, and barriers of one time code contributors to FLOSS projects: a survey
abstract
Successful Free/Libre Open Source Software (FLOSS) projects must attract and retain high-quality talent. Researchers have invested considerable effort in the study of core and peripheral FLOSS developers. To this point, one critical subset of developers that have not been studied are One-Time code Contributors (OTC) - those that have had exactly one patch accepted. To understand why OTCs have not contributed another patch and provide guidance to FLOSS projects on retaining OTCs, this study seeks to understand the impressions, motivations, and barriers experienced by OTCs. We conducted an online survey of OTCs from 23 popular FLOSS projects. Based on the 184 responses received, we observed that OTCs generally have positive impressions of their FLOSS project and are driven by a variety of motivations. Most OTCs primarily made contributions to fix bugs that impeded their work and did not plan on becoming long term contributors. Furthermore, OTCs encounter a number of barriers that prevent them from continuing to contribute to the project. Based on our findings, there are some concrete actions FLOSS projects can take to increase the chances of converting OTCs into long-term contributors.
Amanda Lee, Jeffrey C. Carver, Amiangshu Bosu
ICSE3
2017 SentiCR: a customized sentiment analysis tool for code review interactions
abstract
Sentiment Analysis tools, developed for analyzing social media text or product reviews, work poorly on a Software Engineering (SE) dataset. Since prior studies have found developers expressing sentiments during various SE activities, there is a need for a customized sentiment analysis tool for the SE domain. On this goal, we manually labeled 2000 review comments to build a training dataset and used our dataset to evaluate seven popular sentiment analysis tools. The poor performances of the existing sentiment analysis tools motivated us to build SentiCR, a sentiment analysis tool especially designed for code review comments. We evaluated SentiCR using one hundred 10-fold cross-validations of eight supervised learning algorithms. We found a model, trained using the Gradient Boosting Tree (GBT) algorithm, providing the highest mean accuracy (83%), the highest mean precision (67.8%), and the highest mean recall (58.4%) in identifying negative review comments.
Toufique Ahmed, Amiangshu Bosu, Anindya Iqbal, Nick Rahimi
ASE2
2017 Process Aspects and Social Dynamics of Contemporary Code Review: Insights from Open Source Development and Industrial Practice at Microsoft
abstract
Many open source and commercial developers practice contemporary code review, a lightweight, informal, tool-based code review process. To better understand this process and its benefits, we gathered information about code review practices via surveys of open source software developers and developers from Microsoft. The results of our analysis suggest that developers spend approximately 10-15 percent of their time in code reviews, with the amount of effort increasing with experience. Developers consider code review important, stating that in addition to finding defects, code reviews offer other benefits, including knowledge sharing, community building, and maintaining code quality. The quality of the code submitted for review helps reviewers form impressions about their teammates, which can influence future collaborations. We found a large amount of similarity between the Microsoft and OSS respondents. One interesting difference is that while OSS respondents view code review as an important method of impression formation, Microsoft respondents found knowledge dissemination to be more important. Finally, we found little difference between distributed and co-located Microsoft teams. Our findings identify the following key areas that warrant focused research: 1) exploring the non-technical benefits of code reviews, 2) helping developers in articulating review comments, and 3) assisting reviewers' program comprehension during code reviews.
Amiangshu Bosu, Jeffrey C. Carver, Christian Bird, Jonathan D. Orbeck, Christopher Chockley
IEEE Trans. Software Eng.1
2015 Characteristics of Useful Code Reviews: An Empirical Study at Microsoft
abstract
Over the past decade, both open source and commercial software projects have adopted contemporary peer code review practices as a quality control mechanism. Prior research has shown that developers spend a large amount of time and effort performing code reviews. Therefore, identifying factors that lead to useful code reviews can benefit projects by increasing code review effectiveness and quality. In a three-stage mixed research study, we qualitatively investigated what aspects of code reviews make them useful to developers, used our findings to build and verify a classification model that can distinguish between useful and not useful code review feedback, and finally we used this classifier to classify review comments enabling us to empirically investigate factors that lead to more effective code review feedback. In total, we analyzed 1.5 millions review comments from five Microsoft projects and uncovered many factors that affect the usefulness of review feedback. For example, we found that the proportion of useful comments made by a reviewer increases dramatically in the first year that he or she is at Microsoft but tends to plateau afterwards. In contrast, we found that the more files that are in a change, the lower the proportion of comments in the code review that will be of value to the author of the change. Based on our findings, we provide recommendations for practitioners to improve effectiveness of code reviews.
Amiangshu Bosu, Michaela Greiler, Christian Bird
MSR1
2014 Impact of developer reputation on code review outcomes in OSS projects: an empirical investigation
abstract
Context: Gaining an identity and building a good reputation are important motivations for Open Source Software (OSS) developers. It is unclear whether these motivations have any actual impact on OSS project success. Goal: To identify how an OSS developer's reputation affects the outcome of his/her code review requests. Method: We conducted a social network analysis (SNA) of the code review data from eight popular OSS projects. Working on the assumption that core developers have better reputation than peripheral developers, we developed an approach, Core Identification using K-means (CIK) to divide the OSS developers into core and periphery groups based on six SNA centrality measures. We then compared the outcome of the code review process for members of the two groups. Results: The results suggest that the core developers receive quicker first feedback on their review request, complete the review process in shorter time, and are more likely to have their code changes accepted into the project codebase. Peripheral developers may have to wait 2 - 19 times (or 12 - 96 hours) longer than core developers for the review process of their code to complete. Conclusion: We recommend that projects allocate resources or create tool support to triage the code review requests to motivate prospective developers through quick feedback.
Amiangshu Bosu, Jeffrey C. Carver
ESEM1
2014 Identifying the characteristics of vulnerable code changes: an empirical study
abstract
To focus the efforts of security experts, the goals of this empirical study are to analyze which security vulnerabilities can be discovered by code review, identify characteristics of vulnerable code changes, and identify characteristics of developers likely to introduce vulnerabilities. Using a three-stage manual and automated process, we analyzed 267,046 code review requests from 10 open source projects and identified 413 Vulnerable Code Changes (VCC). Some key results include: (1) code review can identify common types of vulnerabilities; (2) while more experienced contributors authored the majority of the VCCs, the less experienced contributors' changes were 1.8 to 24 times more likely to be vulnerable; (3) the likelihood of a vulnerability increases with the number of lines changed, and (4) modified files are more likely to contain vulnerabilities than new files. Knowing which code changes are more prone to contain vulnerabilities may allow a security expert to concentrate on a smaller subset of submitted code changes. Moreover, we recommend that projects should: (a) create or adapt secure coding guidelines, (b) create a dedicated security review team, (c) ensure detailed comments during review to help knowledge dissemination, and (d) encourage developers to make small, incremental changes rather than large changes.
Amiangshu Bosu, Jeffrey C. Carver, Munawar Hafiz, Patrick Hilley, Derek Janni
SIGSOFT FSE1
2014 Peer impressions in open source organizations: A survey
Amiangshu Bosu, Jeffrey C. Carver, Rosanna E. Guadagno, Blake Bassett, Debra McCallum, Lorin Hochstein
J. Syst. Softw.1
2013 Impact of Peer Code Review on Peer Impression Formation: A Survey
abstract
Peer code review has been adopted as an effective quality improvement practice by many Open Source Software (OSS) communities. In addition to increasing software quality, there is anecdotal evidence that peer code review has other benefits, including: sharing knowledge, sharing expertise, sharing development techniques, and most importantly building accurate peer impressions between the code review participants. To further investigate the presence of these benefits, we surveyed members of popular OSS communities who were involved with peer code review. We used established scales from Psychology, Information science, and Organizational Behavior to create survey questions. We also enforced multiple reliability and validity measures to ensure higher confidence in the survey results. In this paper, we present a subset of the surveys results focused on better understanding four aspects of peer impression formation: trust, reliability, perception of expertise, and friendship. The results indicate that there is indeed a high level of trust, reliability, perception of expertise, and friendship between OSS peers who have participated in code review for a period of time. Because code review involves examining someone else's code, unsurprisingly, peer code review helped most in building a perception of expertise between code review partners.
Amiangshu Bosu, Jeffrey C. Carver
ESEM1
2013 Building reputation in StackOverflow: an empirical investigation
abstract
StackOverflow (SO) contributors are recognized by reputation scores. Earning a high reputation score requires technical expertise and sustained effort. We analyzed the SO data from four perspectives to understand the dynamics of reputation building on SO. The results of our analysis provide guidance to new SO contributors who want to earn high reputation scores quickly. In particular, the results indicate that the following activities can help to build reputation quickly: answering questions related to tags with lower expertise density, answering questions promptly, being the first one to answer a question, being active during off peak hours, and contributing to diverse areas.
Amiangshu Bosu, Christopher S. Corley, Dustin Heaton, Debarshi Chatterji, Jeffrey C. Carver, Nicholas A. Kraft
MSR1