The rapid adoption of large language models has led to the emergence of AI coding agents that autonomously create pull requests on GitHub. However, how these agents differ in their pull request description characteristics, and how human reviewers respond to them, remains underexplored. In this study, we conduct an empirical analysis of pull requests created by five AI coding agents using the AIDev dataset. We analyze agent differences in pull request description characteristics, including structural features, and examine human reviewer response in terms of review activity, response timing, sentiment, and merge outcomes. We find that AI coding agents exhibit distinct PR description styles, which are associated with differences in reviewer engagement, response time, and merge outcomes. We observe notable variation across agents in both reviewer interaction metrics and merge rates. These findings highlight the role of pull request presentation and reviewer interaction dynamics in human–AI collaborative software development.

As software engineering moves toward SE 3.0, AI agents are increasingly used to carry out development tasks and contribute changes to software projects. It is therefore important to understand the extent of these contributions and how human developers review and intervene, since these factors shape the risks of delegating work to AI agents. While recent studies have examined how AI agents support software development tasks (e.g., code generation, issue resolution, and PR automation), their role in documentation tasks remains underexplored–even though documentation is widely consumed and shapes how developers understand and use software.

Recently, research on predicting match outcomes in esports has been actively conducted, but much of it is based on match log data and statistical information. This research targets the FPS game VALORANT, which requires complex strategies, and aims to build a round outcome prediction model by analyzing minimap information in match footage. Specifically, based on the video recognition model TimeSformer, we attempt to improve prediction accuracy by incorporating detailed tactical features extracted from minimap information, such as character position information and other in-game events. This paper reports preliminary results showing that a model trained on a dataset augmented with such tactical event labels achieved approximately$\mathbf{8 1 \%}$prediction accuracy, especially from the middle phases of a round onward, significantly outperforming a model trained on a dataset with the minimap information itself. This suggests that leveraging tactical features from match footage is highly effective for predicting round outcomes in VALORANT.

This paper presents eye2vec, an infrastructure for analyzing software developers' eye movements while reading source code. In common eye-tracking studies in program comprehension, researchers must preselect analysis targets such as control flow or syntactic elements, and then develop analysis methods to extract appropriate metrics from the fixation for source code. Here, researchers can define various levels of AOIs like words, lines, or code blocks, and the difference leads to different results. Moreover, the interpretation of fixation for word/line can vary across the purposes of the analyses. Hence, the eye-tracking analysis is a difficult task that depends on the time-consuming manual work of the researchers. eye2vec represents continuous two fixations as transitions between syntactic elements using distributed representations. The distributed representation facilitates the adoption of diverse data analysis methods with rich semantic interpretations.

Social media platforms have become more influential than traditional news sources, shaping public discourse and accelerating the spread of information. With the rapid advancement of artificial intelligence (AI), open-source software (OSS) projects can leverage these platforms to gain visibility and attract contributors. In this study, we investigate the relationship between Hacker News, a social news site focused on computer science and entrepreneurship, and the extent to which it influences developer activity on the promoted GitHub AI projects. We analyzed 2,195 Hacker News (HN) stories and their corresponding comments over a two-year period. Our findings reveal that at least 19 % of AI developers promoted their GitHub projects on Hacker News, often receiving positive engagement from the community. By tracking activity on the associated 1,814 GitHub repositories after they were shared on Hacker News, we observed a significant increase in forks, stars, and contributors. These results suggest that Hacker News serves as a viable platform for AI-powered OSS projects, with the potential to gain attention, foster community engagement, and accelerate software development.

In an era increasingly shaped by generative AI, code understanding remains a critical foundation for ensuring software quality, reliability, and maintainability. While AI systems can accelerate code generation, developers still face substantial challenges in comprehending, debugging, and effectively integrating the resulting artifacts. Code proficiency, defined not only by comprehension but also by the ability to write efficient, idiomatic code, plays a central role in addressing these challenges. Existing tools, such as those assigning CEFR-based levels, which outline six levels of proficiency from A1 to C2, to code constructs that offer initial frameworks for assessing code difficulty. However, their manually derived classifications lack empirical validation and often diverge from pedagogical progressions found in computer science textbooks. This research seeks to address the absence of a standardized, data-driven metric for determining the proficiency levels required to understand specific programming elements. We propose an automated framework grounded in textbook analysis and clustering techniques to establish a scalable proficiency metric applicable to both human-written and AI-generated code. Preliminary findings reveal strong alignment with educational sequencing and suggest promising applications in AI-assisted software development, particularly in enhancing code review workflows and tailoring AI-generated code to developers' proficiency levels.

Assessing developer proficiency in open-source software (OSS) projects is essential for understanding project dynamics, especially for expertise. This paper presents "PyGress", a web-based tool designed to automatically evaluate and visualize Python code proficiency using pycefr, a Python code proficiency analyzer. By submitting a GitHub repository link, the system extracts commit histories, analyzes source code proficiency across CEFR-aligned levels (A1-C2), and generates visual summaries of individual and project-wide proficiency. The PyGress tool visualizes per-contributor proficiency distribution and tracks project code proficiency progression over time. PyGress offers an interactive way to explore contributor coding levels in Python OSS repositories. The video demonstration of the PyGress tool can be found at https://youtu.be/hxoeK-ggcWk, and the source code of the tool is publicly available at https://github.com/MUICT-SERU/PyGress.

The Log4j-Core vulnerability, known as Log4Shell, exposed significant challenges to dependency management in software ecosystems. When a critical vulnerability is disclosed, it is imperative that dependent packages quickly adopt patched versions to mitigate risks. However, delays in applying these updates can leave client systems exposed to exploitation. Previous research has primarily focused on NPM, but there is a need for similar analysis in other ecosystems, such as Maven. Leveraging the 2025 mining challenge dataset of Java dependencies, we identify factors influencing update lags and categorize them based on version classification (major, minor, patch release cycles). Results show that lags exist, but projects with higher release cycle rates tend to address severe security issues more swiftly. In addition, over half of vulnerability fixes are implemented through patch updates, highlighting the critical role of incremental changes in maintaining software security. Our findings confirm that these lags also appear in the Maven ecosystem, even when migrating away from severe threats.

Log4j has become a widely adopted logging library for Java programs due to its long history and high reliability. Its widespread use is notable not only because of its maturity but also due to the complexity and depth of its features, which have made it an essential tool for many developers. However, Log4j 1.x, which reached its end of support (deprecated), poses significant security risks and has numerous deprecated features that can be exploited by attackers. Despite this, some clients may still rely on this library. We aim to understand whether clients are still using Log4j 1.x despite its official support ending. We utilized the Mining Software Repositories 2025 challenge dataset, which provides a large and representative sample of open-source software projects. We analyzed over 10,000 log entries from the Mining Software Repositories 2025 challenge dataset using the Goblin framework to identify trends in usage rates for both Log4j 1.x and Log4j-core 2.x. Specifically, our study addressed two key issues: (1) We examined the usage rates and trends for these two libraries, highlighting any notable differences or patterns in their adoption. (2) We demonstrate that projects initiated after a deprecated library has reached the end of its support lifecycle can still maintain significant popularity. These findings highlight how deprecated are still popular, with the next step being to understand the reasoning behind these adoptions.

GitHub Sponsors was launched in 2019, enabling donations to open-source software developers to provide financial support, as per GitHub's slogan: "Invest in the projects you depend on". However, a 2022 study on GitHub Sponsors found that only two-fifths of developers who were seeking sponsorship received a donation. The study found that, other than internal actions (such as offering perks to sponsors), developers had advertised their GitHub Sponsors profiles on social media, such as Twitter (also known as X). Therefore, in this work, we investigate the impact of tweets that contain links to GitHub Sponsors profiles on sponsorship, as well as their reception on Twitter/X. We further characterize these tweets to understand their context and find that (1) such tweets have the impact of increasing the number of sponsors acquired, (2) compared to other donation platforms such as Open Collective and Patreon, GitHub Sponsors has significantly fewer interactions but is more visible on Twitter/X, and (3) developers tend to contribute more to open-source software during the week of posting such tweets. Our findings are the first step toward investigating the impact of social media on obtaining funding to sustain open-source software.

Ensemble learning methods have been used to enhance the reliability of defect prediction models. However, there is an inconclusive stability of a single method attaining the highest accuracy among various software projects. This work aims to improve the performance of ensemble-learning defect prediction among such projects by helping select the highest accuracy ensemble methods. We employ bandit algorithms (BA), an online optimization method, to select the highest-accuracy ensemble method. Each software module is tested sequentially, and bandit algorithms utilize the test outcomes of the modules to evaluate the performance of the ensemble learning methods. The test strategy followed might impact the testing effort and prediction accuracy when applying online optimization. Hence, we analyzed the test order's influence on BA's performance. In our experiment, we used six popular defect prediction datasets, four ensemble learning methods such as bagging, and three test strategies such as testing positive-prediction modules first (PF). Our results show that when BA is applied with PF, the prediction accuracy improved on average, and the number of found defects increased by 7% on a minimum of five out of six datasets (although with a slight increase in the testing effort by about 4% from ordinal ensemble learning). Hence, BA with PF strategy is the most effective to attain the highest prediction accuracy using ensemble methods on various projects.

Large Language Models (LLMs) have taken the world by storm, demonstrating their ability not only to automate tedious tasks, but also to show some degree of proficiency in completing software engineering tasks. A key concern with LLMs is their “black-box” nature, which obscures their internal workings and could lead to societal biases in their outputs. In the software engineering context, in this early results paper, we empirically explore how well LLMs can automate recruitment tasks for a geographically diverse software team. We use OpenAI's ChatGPT to conduct an initial set of experiments using GitHub User Profiles from four regions to recruit a six-person software development team, analyzing a total of 3,657 profiles over a five-year period (2019–2023). Results indicate that ChatGPT shows preference for some regions over others, even when swapping the location strings of two profiles (counterfactuals). Furthermore, ChatGPT was more likely to assign certain developer roles to users from a specific country, revealing an implicit bias. Overall, this study reveals insights into the inner workings of LLMs and has implications for mitigating such societal biases in these models.

Python, one of the most prevalent programming languages today, is widely utilized in various domains, including web development, data science, machine learning, and DevOps. Recent scholarly efforts have proposed a methodology to assess Python competence levels, similar to how proficiency in natural languages is evaluated. This method involves assigning levels of competence to Python constructs—for instance, placing simple ‘print’ statements at the most basic level and abstract base classes at the most advanced. The aim is to gauge the level of proficiency a developer must have to understand a piece of source code. This is particularly crucial for software maintenance and evolution tasks, such as debugging or adding new features. For example, in a code review process, this method could determine the competence level required for reviewers. However, categorizing Python constructs by proficiency levels poses significant challenges. Prior attempts, which relied heavily on expert opinions and developer surveys, have led to considerable discrepancies. In response, this paper presents a new approach to identifying Python competency levels through the systematic analysis of introductory Python programming textbooks. By comparing the sequence in which Python constructs are introduced in these textbooks with the current state of the art, we have uncovered notable discrepancies in the order of introduction of Python constructs. Our study underscores a misalignment in the sequences, demonstrating that pinpointing proficiency levels is not trivial. Insights from the study serve as pivotal steps toward reinforcing the idea that textbooks serve as a valuable source for evaluating developers' proficiency, and particularly in terms of their ability to undertake maintenance and evolution tasks.

This paper introduces DevGPT, a dataset curated to explore how software developers interact with ChatGPT, a prominent large language model (LLM). The dataset encompasses 29,778 prompts and responses from ChatGPT, including 19,106 code snippets, and is linked to corresponding software development artifacts such as source code, commits, issues, pull requests, discussions, and Hacker News threads. This comprehensive dataset is derived from shared ChatGPT conversations collected from GitHub and Hacker News, providing a rich resource for understanding the dynamics of developer interactions with ChatGPT, the nature of their inquiries, and the impact of these interactions on their work. DevGPT enables the study of developer queries, the effectiveness of ChatGPT in code generation and problem solving, and the broader implications of AI-assisted programming. By providing this dataset, the paper paves the way for novel research avenues in software engineering, particularly in understanding and improving the use of LLMs like ChatGPT by developers.

The use of libraries is crucial in software development. Library users should update their libraries to address bugs and vulnerabilities that are fixed in newer versions. However, updating libraries can lead to software malfunction due to backward incompatibilities. Therefore, it is necessary to carefully examine the changes in the library, identify incompatible behavior, and modify the software accordingly when applying updates. Identifying the cause of incompatibility is challenging as updates often include changes to APIs other than the one used by the user. We propose a method to detect candidate library methods that cause backward incompatibilities in client-side library updates using Merkle tree. Our approach involves conducting unit tests on the client software, which includes library API calls, before and after the library updates. The execution traces of these tests are collected at the Java bytecode instruction level. By constructing Merkle trees for each execution trace before and after the update, we efficiently compare the control structures and return values to identify the differences indicating backward incompatibilities. To validate the effectiveness of our method, we conducted a case study on three instances of incompatibility in open-source software.

In an assignment-based introductory programming course, a teacher makes a daily class plan based on a textbook and assigns tasks to the students. Assignments are prepared by the teacher so that students can have a better understanding of programming language constructs explained in the course. On the other hand, it is unclear how those language constructs are useful for practical programming tasks. To analyze the practicality of a programming course, this study proposes to compare the vocabularies of code used in textbooks, assignments, and regular programming tasks. If the vocabularies of the textbooks and assignments are closer to that of source code in actual projects, the programming course is considered more practical. As a case study, we have applied the method to evaluate a programming course focusing on data science for graduate students. The result revealed inconsistency between the programming language constructs taught in the course and frequently used in data analysis programs on the Kaggle platform.

Using libraries in applications has helped developers reduce the costs of reinventing already existing code. However, an increase in diverse technology stacks and third-party library usage has led developers to inevitably switch technologies and search for similar libraries implemented in the new technology. To assist with searching for these replacement libraries, maintainers have started to release their libraries to multiple ecosystems. Our goal is to explore the extent to which these libraries are intertwined between ecosystems. We perform a large-scale empirical study of 1.1 million libraries from five different software ecosystems, i.e., PyPI, CRAN, Maven, RubyGems, and NPM, to identify 4,146 GitHub repositories. As a starting point, insights from the study raise implications for library maintainers, users, contributors, and researchers into understanding how these different ecosystems are becoming more intertwined with each other.

Biosignals reflect the mental states of software developers and could improve support technologies for software development activities.Although several technologies for software development support using biosignals (BioSDS) have been proposed, BioSDS has not yet been deployed in actual software development workplaces.As a prerequisite for industrial deployment, BioSDS must be well understood and accepted by software developers.However, the current level of their acceptance has not been comprehensively assessed.In this study, we conducted a survey to clarify the current level of acceptance of BioSDS and potential attributes that influence the level of acceptance.We defined eleven use-cases based on six previous primary studies related to BioSDS, and then asked developers at Hitachi, a Japanese IT company in the FORTUNE 500, about the level of acceptance of each use-case.Our analysis of eighty-six responses revealed that four out of eleven use-cases had some level of acceptance by software developers.In addition, we found four attributes that affect the level of acceptance: subject to be measured, objectives, interventions, and timing.These findings help to identify barriers to the adoption of BioSDS in the workplace.

A risk in adopting third-party dependencies into an application is their potential to serve as a doorway for malicious code to be injected (most often unknowingly). While many initiatives from both industry and research communities focus on the most critical dependencies (i.e., those most depended upon within the ecosystem), little is known about whether the rest of the ecosystem suffers the same fate. Our vision is to promote and establish safer practises throughout the ecosystem. To motivate our vision, in this paper, we present preliminary data based on three representative samples from a population of 88,416 pull requests (PRs) and identify unsafe dependency updates (i.e., any pull request that risks being unsafe during runtime), which clearly shows that unsafe dependency updates are not limited to highly impactful libraries. To draw attention to the long tail, we propose a research agenda comprising six key research questions that further explore how to safeguard against these unsafe activities. This includes developing best practises to address unsafe dependency updates not only in top-tier libraries but throughout the entire ecosystem.

The widespread adoption of third-party libraries for contemporary software development has led to the creation of large inter-dependency networks, where sustainability issues of a single library can have widespread network effects. Maintainers of these libraries are often overworked, relying on the contributions of volunteers to sustain these libraries. To understand these contributions, in this work, we leverage socio-technical techniques to introduce and formalise dependency-contribution congruence (DC congruence) at both ecosystem and library level, i.e., to understand the degree and origins of contributions congruent to dependency changes, analyze whether they contribute to library dormancy (i.e., a lack of activity), and investigate similarities between these congruent contributions compared to typical contributions. We conduct a large-scale empirical study to measure the DC congruence for the npm ecosystem using 1.7 million issues, 970 thousand pull requests (PRs), and over 5.3 million commits belonging to 107,242 npm libraries. We find that the most congruent contributions originate from contributors who can only submit (not commit) to both a client and a library. At the project level, we find that DC congruence shares an inverse relationship with the likelihood that a library becomes dormant. Specifically, a library is less likely to become dormant if the contributions are congruent with upgrading dependencies. Finally, by comparing the source code of contributions, we find statistical differences in the file path and added lines in the source code of congruent contributions when compared to typical contributions. Our work has implications to encourage dependency contributions, especially to support library maintainers in sustaining their projects.

Python is known to be used by beginners to professional programmers. Python provides functionality to its community of users through PyPI libraries, which allows developers to reuse functionalities to an application. However, it is unknown the extent to which these PyPI libraries require proficient code in their implementation. We conjecture that PyPI contributors may decide to implement more advanced Pythonic code, or stick with more basic Python code. Are complex codes only committed by few contributors, or only to specific files? The new idea in this paper is to confirm who and where complex code is implemented. Hence, we present a visualization to show the relationship between proficient code, contributors, and files. Analyzing four PyPI projects, we are able to explore which files contain more elegant code, and which contributors committed to these files. Our results show that most files contain more basic competency files, and that not every contributor contributes competent code. We show how our visualization is able to summarize such information, and opens up different possibilities for understanding how to make elegant contributions.

To enhance the reliability of software, it is important is to review all software artifacts (e.g., design documents) to remove defects as earlier as possible. There are various review methods available, and project managers face the challenge of choosing a suitable method for their current projects. One of approaches to support the selection of review methods is to evaluate review methods beforehand, to identify the most effective method on average. However, past studies have not evaluated review methods thoroughly as the process can be time-consuming. We propose a bandit-algorithm (BA) based method to evaluate and then dynamically select a suitable review method (from a list of candidates). In our experiments, we assume that the proposed method is applied to design document review on basic design phase. We performed experiments based on a simulation, instead of using an actual dataset. On our simulation, when a review method is selected by our BA method, productivity (i.e., total development time) was improved by about 1.25 times, and it was the second highest among candidates of review methods.

The reuse of already existing code is widely considered a popular software development practice, that provides both benefits and drawbacks for all stakeholders involved. Prior work reports on how code reuse is a common practice in software development projects and data science projects such as machine learning pipelines. Recently, there has been much code reuse work in the context of competitive programming. Although there is work such as detecting plagiarism, there is no work that studies how a competitor will reuse their own code. In this paper, we present a preliminary study on the code reuse behavior of three grandmasters’ Jupyter notebooks in the Kaggle Competitions, an online competition platform for data scientists, and report the types of code they often reuse. Grandmasters are the highest level reached in competitions (novice, expert, master, and grandmaster). We find that Grandmasters are less likely to reuse specialized code, but instead, tend to reuse common functions like importing packages (importing the pandas library). They are most likely to reuse common abstractions like importing packages, configurations, file IO operations, show data, plotting graphs, defining functions, and exploring files. The work opens up new research potential into recommending how developers can reuse their own code.

Data visualization becomes a crucial component in data analytics, especially data exploration, understanding, and analysis. Effective data visualization impacts decision-making and aids in discovering and understanding relationships. It leads to benefits in data-intensive software development tasks e.g., feature engineering in machine learning-based software projects. However, it is unknown how visualizations are used in competitive programming. The idea of this paper is to report early results on what visualizations are prevalent in competitive programming. Grandmasters are the highest level reached in competitions (novice, expert, master, and grandmaster). Analyzing the visualizations of 7 high-rank competitors (i.e., Grandmaster) in Kaggle, we identify and present a catalog of visualizations used to both tell a story from the data, as well as explain the process and pipelines involved to explain their coding solutions. Our taxonomy includes nine types from over 821 visualizations in 68 instances of Jupyter notebooks. Furthermore, most visualizations are for data analysis for distribution (DA Distribution), and frequency (DA Frequency) are most used. We envision that this catalog can be useful to better understand different situations in which to employ these visualizations.

Third-party library dependencies are commonplace in today’s software development. With the growing threat of security vulnerabilities, applying security fixes in a timely manner is important to protect software systems. As such, the community developed a list of software and hardware weakness known as Common Weakness Enumeration (CWE) to assess vulnerabilities. Prior work has revealed that maintenance activities such as refactoring code potentially correlate with security-related aspects in the source code. In this work, we explore the relationship between refactoring and security by analyzing refactoring actions performed jointly with vulnerability fixes in practice. We conducted a case study to analyze 143 maven libraries in which 351 known vulnerabilities had been detected and fixed. Surprisingly, our exploratory results show that developers incorporate refactoring operations in their fixes, with 31.9% (112 out of 351) of the vulnerabilities paired with refactoring actions. We envision this short paper to open up potential new directions to motivate automated tool support, allowing developers to deliver fixes faster, while maintaining their code.

GitHub Sponsors, launched in 2019, enables donations to individual open source software (OSS) developers. Financial support for OSS maintainers and developers is a major issue in terms of sustaining OSS projects, and the ability to donate to individuals is expected to support the sustainability of developers, projects, and community. In this work, we conducted a mixed-methods study of GitHub Sponsors, including quantitative and qualitative analyses, to understand the characteristics of developers who are likely to receive donations and what developers think about donations to individuals. We found that: (1) sponsored developers are more active than non-sponsored developers, (2) the possibility to receive donations is related to whether there is someone in their community who is donating, and (3) developers are sponsoring as a new way to contribute to OSS. Our findings are the first step towards data-informed guidance for using GitHub Sponsors, opening up avenues for future work on this new way of financially sustaining the OSS community.

Python is known to be a versatile language, well suited both for beginners and advanced users. Some elements of the language are easier to understand than others: some are found in any kind of code, while some others are used only by experienced programmers. The use of these elements lead to different ways to code, depending on the experience with the language and the knowledge of its elements, the general programming competence and programming skills, etc. In this paper, we present pycefr, a tool that detects the use of the different elements of the Python language, effectively measuring the level of Python proficiency required to comprehend and deal with a fragment of Python code. Following the well-known Common European Framework of Reference for Languages (CEFR), widely used for natural languages, pycefr categorizes Python code in six levels, depending on the proficiency required to create and understand it. We also discuss different use cases for pycefr: identifying code snippets that can be understood by developers with a certain proficiency, labeling code examples in online resources such as Stackoverflow and GitHub to suit them to a certain level of competency, helping in the onboarding process of new developers in Open Source Software projects, etc. A video shows availability and usage of the tool: https://tinyurl.com/ypdt3fwe.

A key threat to the usage of third-party dependencies has been the threat of security vulnerabilities, which risks unwanted access to a user application. As part of an ecosystem of dependencies, users of a library are prone to both the direct and transitive dependencies adopted into their applications. Recent work involves tool supports for vulnerable dependency updates, rarely showing the complexity of the transitive updates. In this paper, we introduce our solution to support vulnerability updating in npm. V-Achilles is a prototype that shows a visualization (i.e., using dependency graphs) affected by vulnerability attacks. In addition to the tool overview, we highlight three use cases to demonstrate the usefulness and application of our prototype with real-world npm packages. The prototype is available at https://github.com/MUICT-SERU/V-Achilles, with an accompanying video demonstration at https://www.youtube.com/watch?v=tspiZfhMNcs.

Background: Selecting a suitable feature reduction technique. when building a defect prediction model, can be challenging. Different techniques can result in the selection of different independent variables which have an impact on the overall performance of the prediction model. To help in the selection, previous studies have assessed the impact of each feature reduction technique using different datasets. However, there are many reduction techniques, and therefore some of the well-known techniques have not been assessed by those studies. Aim: The goal of the study is to select a high-accuracy reduction technique from several candidates without preliminary assessments. Method: We utilized bandit algorithm (BA) to help with the selection of best features reduction technique for a list of candidates. To select the best feature reduction technique, BA evaluates the prediction accuracy of the candidates, comparing testing results of different modules with their prediction results. By substituting the reduction technique for the prediction method, BA can then be used to select the best reduction technique. In the experiment, we evaluated the performance of BA to select suitable reduction technique. We performed cross version defect prediction using 14 datasets. As feature reduction techniques, we used two assessed and two non-assessed techniques. Results: Using BA, the prediction accuracy was higher or equivalent than existing approaches on average, compared with techniques selected based on an assessment. Conclusions: BA can have larger impact on improving prediction models by helping not only on selecting suitable models, but also in selecting suitable feature reduction techniques.

Forking is a common practice for developers when building upon on already existing projects. These forks create variants, which have a common code base but then evolve the code in different directions, which is specific to that forked project requirements. An interesting side-effect of having multiple forks is the ability to select between different evolution directions of the code which is based on developers fixing bugs in the code base. However, the key issue that this decentralized form of information is difficult to analyze. In this study, we propose a visualization to analyze active changes in fork repositories that have not been merged back to the original project. Our visualization shows code commit activities in multiple forks with highlight on bug fix commits in the history of forks. While the commit activity of each repository is visualized similarly to the code frequency view of GitHub, our view shows only commits unique to fork repositories. To illustrate the effectiveness of our visualization, we have applied our view to two use cases: identifying forks from a repository no longer maintained, and identifying a bug fix among forks. In the first case, we identify a fork of a suspended project named Obfuscator-LLVM. Our view shows the original repository and its most active fork that continue the development on the top. In the second case, we identify a bug fix in a fork of Clipy project. Our view shows that the most active fork has its own bug fixes; we could easily identify a patch for the bug highlighted in the view. As a new ideas paper, we then present our outline of three research questions to spark real world use-cases and goals for our visualization has the potential to uncover. A prototype of our visualization is available at https://naist-se.github.io/vissoft2022/

Abstract The ability of an Open Source Software (OSS) project to attract, onboard, and retain any newcomer is vital to its livelihood. Although, evidence suggests an upsurge in novice developers joining social coding platforms (such as GitHub), the extent to which their activities result in a OSS contribution is unknown. Henceforth, we execute the protocols of a registered report to study activities of a “Newcomer OSS-Candidate”, who is a novice developer that is new to that social coding platform, and has the intention to later onboard an OSS project. Using GitHub as a case platform, we analyze 171 identified Newcomer OSS-Candidates to characterize their contribution activities. Results show that Newcomer OSS-Candidates are likely to target software based repositories (i.e., 66%), and their first contributions are mainly associated with development (commits) and maintenance (PRs). Newcomer OSS-Candidates are less likely to practice social coding, but eventually end up onboarding (i.e., 30% quantitative, 70% follow-up survey) an OSS project. Furthermore, they cite finding a way to start as the most challenging barrier to contribute. Our work reveals insights on how newcomers to social coding platforms are potential sources of OSS contributions.

Traceability between published scientific breakthroughs and their implementation is essential, especially in the case of open-source scientific software which implements bleeding-edge science in its code. However, aligning the link between GitHub repositories and academic papers can prove difficult, and the current practice of establishing and maintaining such links remains unknown. This paper investigates the role of academic paper references contained in these repositories. We conduct a large-scale study of 20 thousand GitHub repositories that make references to academic papers. We use a mixed-methods approach to identify public access, traceability and evolutionary aspects of the links. Although referencing a paper is not typical, we find that a vast majority of referenced academic papers are public access. These repositories tend to be affiliated with academic communities. More than half of the papers do not link back to any repository. We find that academic papers from top-tier SE venues are not likely to reference a repository, but when they do, they usually link to a GitHub software repository. In a network of arXiv papers and referenced repositories, we find that the most referenced papers are (i) highly-cited in academia and (ii) are referenced by repositories written in different programming languages.

Code review is a broadly adopted software quality practice where developers critique each others’ patches. In addition to providing constructive feedback, reviewers may provide a score to indicate whether the patch should be integrated. Since reviewer opinions may differ, patches can receive both positive and negative scores. If reviews with divergent scores are not carefully resolved, they may contribute to a tense reviewing culture and may slow down integration. In this article, we study patches with divergent review scores in theOpenStackandQtcommunities. Quantitative analysis indicates that patches with divergent review scores: (1) account for 15–37 percent of patches that receive multiple review scores; (2) are integrated more often than they are abandoned; and (3) receive negative scores after positive ones in 70 percent of cases. Furthermore, a qualitative analysis indicates that patches with strongly divergent scores that: (4) are abandoned more often suffer from external issues (e.g., integration planning, content duplication) than patches with weakly divergent scores and patches without divergent scores; and (5) are integrated often address reviewer concerns indirectly (i.e., without changing patches). Our results suggest that review tooling should integrate with release schedules and detect concurrent development of similar patches to optimize review discussions with divergent scores. Moreover, patch authors should note that even the most divisive patches are often integrated through discussion, integration timing, and careful revision.

Defect prediction models are proposed to help a team prioritize the areas of source code files that need Software Quality Assurance (SQA) based on the likelihood of having defects. However, developers may waste their unnecessary effort on the whole file while only a small fraction of its source code lines are defective. Indeed, we find that as little as 1-3 percent of lines of a file are defective. Hence, in this work, we propose a novel framework (calledLine-DP) to identify defective lines using a model-agnostic technique, i.e., an Explainable AI technique that provides information why the model makes such a prediction. Broadly speaking, ourLine-DPfirst builds a file-level defect model using code token features. Then, ourLine-DPuses a state-of-the-art model-agnostic technique (i.e., LIME) to identify risky tokens, i.e., code tokens that lead the file-level defect model to predict that the file will be defective. Then, the lines that contain risky tokens are predicted as defective lines. Through a case study of 32 releases of nine Java open source systems, our evaluation results show that ourLine-DPachieves an average recall of 0.61, a false alarm rate of 0.47, a top 20%LOC recall of 0.27, and an initial false alarm of 16, which are statistically better than six baseline approaches. Our evaluation shows that ourLine-DPrequires an average computation time of 10 seconds including model construction and defective line identification time. In addition, we find that 63 percent of defective lines that can be identified by ourLine-DPare related to common defects (e.g., argument change, condition change). These results suggest that ourLine-DPcan effectively identify defective lines that contain common defects while requiring a smaller amount of inspection effort and a manageable computation cost. The contribution of this paper builds an important step towards line-level defect prediction by leveraging a model-agnostic technique.

Technical Debt is a metaphor used to describe the situation in which long-term software artifact quality is traded for short-term goals in software projects. In recent years, the concept of self-admitted technical debt (SATD) was proposed, which focuses on debt that is intentionally introduced and described by developers. Although prior work has made important observations about admitted technical debt in source code, little is known about SATD in build systems. In this paper, we set out to better understand the characteristics of SATD in build systems. To do so, through a qualitative analysis of 500 SATD comments in the Maven build system of 291 projects, we characterize SATD by location and rationale (reason and purpose). Our results show that limitations in tools and libraries, and complexities of dependency management are the most frequent causes, accounting for 50% and 24% of the comments. We also find that developers often document SATD as issues to be fixed later. As a first step towards the automatic detection of SATD rationale, we train classifiers to detect the two most frequently occurring reasons and the four most frequently occurring purposes of SATD in the content of comments in Maven build systems. The classifier performance is promising, achieving an F1-score of 0.71–0.79. Finally, within 16 identified ‘ready-to-be-addressed’ SATD instances, the three SATD submitted by pull requests and the five SATD submitted by issue reports were resolved after developers were made aware. Our work presents the first step towards understanding technical debt in build systems and opens up avenues for future work, such as tool support to track and manage SATD backlogs.

Background: defect prediction model is built using historical data from previous versions/releases of the same project. However, such historical data may not exist in case of newly developed projects. Alternatively, one can train a model using data obtained from external projects. This approach is known as cross-project defect prediction (CPDP). In CPDP, it is still difficult to utilize external projects' data or decide which particular project to use to train a model. Aim: to address this issue, we apply bandit algorithm (BA) to CPDP in order to select the most suitable training project from a set of projects. Method: BA-based prediction iteratively reselects the project after each module is tested, considering the accuracy of the predictions. As baselines, we used simple CPDP methods such as training a model with randomly selected project. All models were built using logistic regression. Results: We experimented our approach on two datasets (NASA and DAMB, with a total of 12 projects). The BA-based defect prediction models resulted in, on average, a higher accuracy (AUC and F1 score) than the baselines. Conclusion: in this preliminarily study, we demonstrate the feasibility of using BA in the context of CPDP. Our initial assessment shows that the use BA for predicting defects in CPDP is promising and may outperform existing approaches.

The management of third-party package dependencies is crucial to most technology stacks, with package managers acting as brokers to ensure that a verified package is correctly installed, configured, or removed from an application. Diversity in technology stacks has led to dozens of package ecosystems with their own management features. While recent studies have shown that developers struggle to migrate their dependencies, the common assumption is that package ecosystems are used without any issue. In this study, we explore 13 package ecosystems to understand whether their features correlate with the experience of their users. By studying experience through the questions that developers ask on the question-and-answer site Stack Overflow, we find that developer questions are grouped into three themes (i.e., Package management, Input-Output, and Package Usage). Our preliminary analysis indicates that specific features are correlated with the user experience. Our work lays out future directions to investigate the trade-offs involved in designing the ideal package ecosystem.

Self-Admitted Technical Debt (SATD) is a special form of technical debt in which developers intentionally record their hacks in the code by adding comments for attention. Here, we focus on issue-related "On-hold SATD", where developers suspend proper implementation due to issues reported inside or outside the project. When the referenced issues are resolved, the On-hold SATD also need to be addressed, but since monitoring these issue reports takes a lot of time and effort, developers may not be aware of the resolved issues and leave the On-hold SATD in the code. In this paper, we propose FixMe, a GitHub bot that helps developers detecting and monitoring On-hold SATD in their repositories and notify them whenever the On-hold SATDs are ready to be fixed (i.e. the referenced issues are resolved). The bot can automatically detect On-hold SATD comments from source code using machine learning techniques and discover referenced issues. When the referenced issues are resolved, developers will be notified by FixMe bot. The evaluation conducted with 11 participants shows that our FixMe bot can support them in dealing with On-hold SATD. FixMe is available at https://www.fixmebot.app/ and FixMe's VDO is at https://youtu.be/YSz9kFxN_YQ.

Recommendation systems have played a large role in the Software Engineering research landscape. Applications have ranged from source code elements, APIs and reviewer recommendations, with techniques borrowed from the Information Retrieval, and Machine Learning domains. In recent times, there has been work into a new method of interaction, which is ChatBots, especially for Software Engineering. Early work has been aimed at using bots for mining software repositories, providing task-oriented feedback for the software developer. In this work, we would like to take the ChatBots one step forward, but using them inconjunction with recommendation systems to provide an interactive experience for recommendations. As a case study, we focus on the existing reviewer recommendation systems, and propose how using a ChatBot may enhance the solution, to provide a more accurate and realistic recommendation for the practitioner. In the end, we highlight the potential and next steps to utilize ChatBots into existing Software Engineering recommendation systems.

Nowadays, software development increasingly depends on third-party library packages to reuse functionality and save the costs of building themselves. Since dependency is constantly evolving, developers struggle to update dependencies. In this work, we take the first exploration into the responsibility of updating a dependency. Analyzing 89,393 npm packages, we mine the repositories to understand who is the person responsible (i.e., dependency author) for the library update and whether or not the spread of responsibility of updating has an impact on what libraries will get updated. Our results show that 64.24% packages have only one dependency author who is responsible for the dependency. Furthermore, the number of dependency authors correlates with dependency changes, hinting that updating dependencies correlates with having more responsible developers. Lastly, we find that npm packages with just a single dependency author update different libraries compared to those with more dependency authors.

This paper proposes a conceptual framework for developing new technologies that will solve today’s technical issues in software development and operations (DevOps) and support the future software ecosystems. The proposed framework perceives resources essential for software DevOps from three perspectives: products, people, and technical information, and actively utilizes and link the latest digital technologies such as AI, natural language processing, microservices, and blockchain. The goal is not to aim fully automate software DevOps, but also to achieve high economic efficiency and sustainability by eliminating waste in software DevOps, assuming a human-centered society. The principal approaches of new technology development in the framework are "product up-cycling", "placement of the right people and AI in the right places", and "quality control linked to external technical information." New technologies to be developed with these approaches will expand conventional concepts in software DevOps with three dimensions of "reuse," "human resources," and "quality control."

Modern code review (MCR) is now broadly adopted as an established and effective software quality assurance practice, with an increasing number of open-source as well as commercial software projects identifying code review as a crucial practice. During the MCR process, developers review, provide constructive feedback, and/or critique each others’ patches before a code change is merged into the codebase. Nevertheless, code review is basically a human task that involves technical, personal and social aspects. Existing literature hint the existence of poor reviewing practices i.e., anti-patterns, that may contribute to a tense reviewing culture, degradation of software quality, slow down integration, and may affect the overall sustainability of the project. To better understand these practices, we present in this paper the concept of Modern Code Review Anti-patterns (MCRA) and take a first step to define a catalog that enumerates common poor code review practices. In detail we explore and characterize MCRA symptoms, causes, and impacts. We also conduct a series of preliminary experiments to investigate the prevalence and co-occurrences of such anti-patterns on a random sample of 100 code reviews from various OpenStack projects.

Automatic Static Analysis Tools (ASATs) detect coding rule violations, including mistakes and bad practices that frequently occur during programming. While ASATs are widely used in both OSS and industry, the developers do not resolve more than 80% of the detected violations. As one of the reasons, most ASATs users do not customize their ASATs to their projects after installation; the ASATs with the default configuration report many rule violations that confuse developers. To reduce the ratio of such uninteresting warning messages, we propose a method to customize ASATs according to the product source code automatically. Our fundamental hypothesis is: A software project has interesting ASAT rules that are consistent over time. Our method takes source code as input and generates an ASAT configuration. In particular, the method enables optional (i.e., disabled by default) rules that detected no violations on the version because developers are likely to follow the rules in future development. Our method also disables violated rules because developers were unlikely to follow them. To evaluate the method, we applied our method to 643 versions of four JavaScript projects. The generated configurations for all four projects increased the ASAT precision. They also increased recall for two projects. The result shows that our method helps developers to focus on their attractive rule violations. Our implementation of the proposed method is available at https://github.com/devreplay/linter-maintainer

Abstract Security vulnerability in third-party dependencies is a growing concern not only for developers of the affected software, but for the risks it poses to an entire software ecosystem, e.g., Heartbleed vulnerability. Recent studies show that developers are slow to respond to the threat of vulnerability, sometimes taking four to eleven months to act. To ensure quick adoption and propagation of a release that contains the fix (fixing release), we conduct an empirical investigation to identify lags that may occur between the vulnerable release and its fixing release (package-side fixing release). Through a preliminary study of 231 package-side fixing release of npm projects on GitHub, we observe that a fixing release is rarely released on its own, with up to 85.72% of the bundled commits being unrelated to a fix. We then compare the package-side fixing release with changes on a client-side (client-side fixing release). Through an empirical study of the adoption and propagation tendencies of 1,290 package-side fixing releases that impact throughout a network of 1,553,325 releases of npm packages, we find that stale clients require additional migration effort, even if the package-side fixing release was quick (i.e., package-side fixing releasetypeSpatch). Furthermore, we show the influence of factors such as the branch that the package-side fixing release lands on and the severity of vulnerability on its propagation. In addition to these lags we identify and characterize, this paper lays the groundwork for future research on how to mitigate propagation lags in an ecosystem.

A Correction to this paper has been published: 10.1007/s10664-021-09939-7

Abstract Although many software development projects have moved their developer discussion forums to generic platforms such as Stack Overflow, Eclipse has been steadfast in hosting their self-supported community forums. While recent studies show forums share similarities to generic communication channels, it is unknown how project-specific forums are utilized. In this paper, we analyze 832,058 forum threads and their linkages to four systems with 2,170 connected contributors to understand the participation, content and sentiment. Results show that Seniors are the most active participants to respond bug and non-bug-related threads in the forums (i.e., 66.1% and 45.5%), and sentiment among developers are inconsistent while knowledge sharing within Eclipse. We recommend the users to identify appropriate topics and ask in a positive procedural way when joining forums. For developers, preparing project-specific forums could be an option to bridge the communication between members. Irrespective of the popularity of Stack Overflow, we argue the benefits of using project-specific forum initiatives, such as GitHub Discussions, are needed to cultivate a community and its ecosystem.

Abstract Code reviews serve as a quality assurance activity for software teams. Especially for Modern Code Review, sharing a link during a review discussion serves as an effective awareness mechanism where “Code reviews are good FYIs [for your information].”. Although prior work has explored link sharing and the information needs of a code review, the extent to which links are used to properly conduct a review is unknown. In this study, we performed a mixed-method approach to investigate the practice of link sharing and their intentions. First, through a quantitative study of the OpenStack and Qt projects, we identify 19,268 reviews that have 39,686 links to explore the extent to which the links are shared, and analyze a correlation between link sharing and review time. Then in a qualitative study, we manually analyze 1,378 links to understand the role and usefulness of link sharing. Results indicate that internal links are more widely referred to (93% and 80% for the two projects). Importantly, although the majority of the internal links are referencing to reviews, bug reports and source code are also shared in review discussions. The statistical models show that the number of internal links as an explanatory factor does have an increasing relationship with the review time. Finally, we present seven intentions of link sharing, with providing context being the most common intention for sharing links. Based on the findings and a developer survey, we encourage the patch author to provide clear context and explore both internal and external resources, while the review team should continue link sharing activities. Future research directions include the investigation of causality between sharing links and the review process, as well as the potential for tool support.

To attract, onboard, and retain any newcomer in Open Source Software (OSS) projects is vital to their livelihood. Recent studies conclude that OSS projects risk failure due to abandonment and poor participation of newcomers. Evidence suggests more new users are joining GitHub, however, the extent to which they contribute to OSS projects is unknown. In this study, we coin the term `newcomer candidate' to describe new users to the GitHub platform. Our objective is to track and characterize their initial contributions. As a preliminary survey, we collected 208 newcomer candidate contributions in GitHub. Using this dataset, we then plan to track their contributions to reveal insights. We will use a mixed-methods approach, i.e., quantitative and qualitative, to identify whether or not newcomer candidates practice social coding, the kinds of their contributions, projects they target, and the proportion that they eventually onboard to an OSS project.

With one of the largest available collection of reusable packages, the JavaScript runtime environment Node.js is one of the most popular programming application. With recent work showing evidence that known vulnerabilities are prevalent in both open source and industrial software, we propose and implement a viable code-based vulnerability detection tool for Node.js applications. Our case study lists the challenges encountered while implementing our Node.js vulnerable code detector.

Modern software is developed under considerable time pressure, which implies that developers more often than not have to resort to compromises when it comes to code that is well written and code that just does the job. This has led over the past decades to the concept of “technical debt”, a short-term hack that potentially generates long-term maintenance problems. Self-admitted technical debt (SATD) is a particular form of technical debt: developers consciously perform the hack but also document it in the code by adding comments as a reminder (or as an admission of guilt). We focus on a specific type of SATD, namely “On-hold” SATD, in which developers document in their comments the need to halt an implementation task due to conditions outside of their scope of work (e.g., an open issue must be closed before a function can be implemented).We present an approach, based on regular expressions and machine learning, which is able to detect issues referenced in code comments, and to automatically classify the detected instances as either “On-hold” (the issue is referenced to indicate the need to wait for its resolution before completing a task), or as “cross-reference”, (the issue is referenced to document the code, for example to explain the rationale behind an implementation choice). Our approach also mines the issue tracker of the projects to check if the On-hold SATD instances are “superfluous” and can be removed (i.e., the referenced issue has been closed, but the SATD is still in the code). Our evaluation confirms that our approach can indeed identify relevant instances of On-hold SATD. We illustrate its usefulness by identifying superfluous On-hold SATD instances in open source projects as confirmed by the original developers.

Abstract Self-admitted technical debt refers to situations where a software developer knows that their current implementation is not optimal and indicates this using a source code comment. In this work, we hypothesize that it is possible to develop automated techniques to understand a subset of these comments in more detail, and to propose tool support that can help developers manage self-admitted technical debt more effectively. Based on a qualitative study of 333 comments indicating self-admitted technical debt, we first identify one particular class of debt amenable to automated management: on-hold self-admitted technical debt (on-hold SATD), i.e., debt which contains a condition to indicate that a developer is waiting for a certain event or an updated functionality having been implemented elsewhere. We then design and evaluate an automated classifier which can identify these on-hold instances with an area under the receiver operating characteristic curve (AUC) of 0.98 as well as detect the specific conditions that developers are waiting for. Our work presents a first step towards automated tool support that is able to indicate when certain instances of self-admitted technical debt are ready to be addressed.

Abstract Automatic identification of the differences between two versions of a file is a common and basic task in several applications of mining code repositories. Git, a version control system, has a diff utility and users can select algorithms of diff from the default algorithm Myers to the advanced Histogram algorithm. From our systematic mapping, we identified three popular applications of diff in recent studies. On the impact on code churn metrics in 14 Java projects, we obtained different values in 1.7% to 8.2% commits based on the different diff algorithms. Regarding bug-introducing change identification, we found 6.0% and 13.3% in the identified bug-fix commits had different results of bug-introducing changes from 10 Java projects. For patch application, we found that the Histogram is more suitable than Myers for providing the changes of code, from our manual analysis. Thus, we strongly recommend using the Histogram algorithm when mining Git repositories to consider differences in source code.

Defect models that are trained on class imbalanced datasets (i.e., the proportion of defective and clean modules is not equally represented) are highly susceptible to produce inaccurate prediction models. Prior research compares the impact of class rebalancing techniques on the performance of defect models but arrives at contradictory conclusions due to the use of different choice of datasets, classification techniques, and performance measures. Such contradictory conclusions make it hard to derive practical guidelines for whether class rebalancing techniques should be applied in the context of defect models. In this paper, we investigate the impact of class rebalancing techniques on the performance measures and interpretation of defect models. We also investigate the experimental settings in which class rebalancing techniques are beneficial for defect models. Through a case study of 101 datasets that span across proprietary and open-source systems, we conclude that the impact of class rebalancing techniques on the performance of defect prediction models depends on the used performance measure and the used classification techniques. We observe that the optimized SMOTE technique and the under-sampling technique are beneficial when quality assurance teams wish to increase AUC and Recall, respectively, but they should be avoided when deriving knowledge and understandings from defect models.

Technical Debt (TD) introduces a quality problem and increases maintenance cost since it may require improvements in the future. Several studies show that it is possible to automatically detect TD from source code comments that developers intentionally created, so-called self-admitted technical debt (SATD). Those studies proposed to use binary classification technique to predict whether a comment shows SATD. However, SATD has different types (e.g. design SATD and requirement SATD). In this paper, we therefore propose an approach using N-gram Inverse Document Frequency (IDF) and employ a multi-class classification technique to build a model that can identify different types of SATD. From the empirical evaluation on 10 open-source projects, our approach outperforms alternative methods (e.g. using BOW and TF-IDF). Our approach also improves the prediction performance over the baseline benchmark by 33%.

This Research to Practice Full Paper presents the development of the algorithmic expressions for the assessment tools for assessing algorithmic thinking ability of elementary school children. In Japan, elementary school children will be required to learn computer programming as an interdisciplinary element appearing throughout the curriculum in 2020. The purpose of this programming education is to nurture Computational Thinking (CT) for elementary school children in Japan. However, almost no discussion has been conducted in Japan on how to measure the level of CT an elementary school child has acquired. Since the definition of CT is not very firm, it is not easy to measure the levels of CT. Therefore, several organizations have issued operational definitions of CT. Among the concepts of CT in those operational definitions, Algorithmic Thinking was chosen as a representative of CT, and the assessment tools for evaluating Algorithmic Thinking ability have been developed in this research. The assessment tool was conducted in the experimental Computer Science Unplugged classes and in the control classes in two elementary schools in Japan. There were in total 152 children in the classes, and all of them were 5th grade children. By answering the questions in the assessment tool, each child got a score between 0 and 15. The scores were statistically analyzed.

Modern Code Review (MCR) is a pillar of contemporary quality assurance approaches, where developers discuss and improve code changes prior to integration. Since review interactions (e.g., comments, revisions) are archived, analytics approaches like reviewer recommendation and review outcome prediction have been proposed to support the MCR process. These approaches assume that reviews evolve and are adjudicated independently; yet in practice, reviews can be interdependent.

We consider Free/libre and open source software (FLOSS) as a common pool resource (CPR). In economics, CPRs are frequently associated with markets, and it is reported that without appropriate agreement, monitoring and sanction, the resource will be overused. Toward building sustainable communities in FLOSS development, we first study our car-sharing experiment at NAIST, as a common pool resource management. We report the details of the car uses in our experiment, and describe the design of our new system to make better CPR management.

Social coding platforms (SCPs) have realized spontaneous software evolution, where new source code and ideas are spontaneously proposed by altruistic developers. Although there are many projects operated by active communities performing spontaneous evolution, it is yet unclear that how such successful projects and communities have been formed and governed. In this paper, we propose a method that can investigate the history of every project in the SCP. Introducing the concept of project as a city, we consider every project in the SCP as a city, where a government and citizens develop a city through collaborative activities. We then identify essential attributes that characterize a state of a city. For each attribute, we develop metrics that quantity the state S(p, t) of a project p at time t. An experimental evaluation investigating GitHub projects of famous code editors shows that the proposed metrics well visualize the history of the projects from essential perspectives of a city.

Defect prediction models-classifiers that identify defect-prone software modules-have configurable parameters that control their characteristics (e.g., the number of trees in a random forest). Recent studies show that these classifiers underperform when default settings are used. In this paper, we study the impact of automated parameter optimization on defect prediction models. Through a case study of 18 datasets, we find that automated parameter optimization: (1) improves AUC performance by up to 40 percentage points; (2) yields classifiers that are at least as stable as those trained using default settings; (3) substantially shifts the importance ranking of variables, with as few as 28 percent of the top-ranked variables in optimized classifiers also being top-ranked in non-optimized classifiers; (4) yields optimized settings for 17 of the 20 most sensitive parameters that transfer among datasets without a statistically significant drop in performance; and (5) adds less than 30 minutes of additional computation to 12 of the 26 studied classification techniques. While widely-used classification techniques like random forest and support vector machines are not optimization-sensitive, traditionally overlooked techniques like C5.0 and neural networks can actually outperform widely-used techniques after optimization is applied. This highlights the importance of exploring the parameter space when using parameter-sensitive classification techniques.

Due to the distributed nature of Modern Code Review (MCR) tools, developers risk submitting similar patches (i.e., patches that attempt to achieve similar objectives), which potentially causes extra efforts both for the contributors and reviewers. Although researches on other duplicate software artifact exist, there is no prior work that explores the impact of such similar patches in MCR. In this paper, we conduct an empirical study to understand the impact of similar patches on reviewing efforts in MCR. We extracted over 3,400 similar patches from the OpenStack project. Results of the exploratory study confirm that similar patches take just as much time and patch revisions as merged patches.

Background: Code review is a well-established software quality practice where developers critique each others' changes. A shift towards automated detection of low-level issues (e.g., integration with linters) has, in theory, freed reviewers up to focus on higher level issues, such as software design. Yet in practice, little is known about the extent to which design is discussed during code review.

It has become common practice for software projects to adopt third-party libraries, allowing developers full access to functions that otherwise will take time and effort to create them-selves. Regardless of migration effort involved, developers are encouraged to maintain their library dependencies by updating any outdated dependency, so as to remain safe from potential threats such as vulnerabilities. Through a manual inspection of a total of 60 client projects from three cases of high severity vulnerabilities, we investigate whether or not clients are really safe from these threats. Surprisingly, our early results show evidence that up to 73.3% of outdated clients were actually safe from the threat. This is the first work to confirm that analysis at the library level is indeed an overestimation. This result to pave the path for future studies to empirically investigate and validate this phenomena, and is towards aiding a smoother library migration for client developers.

Software ecosystems have had a tremendous impact on computing and society, capturing the attention of businesses, researchers, and policy makers alike. Massive ecosystems like the JavaScript node package manager (npm) is evidence of how packages are readily available for use by software projects. Due to its high-dimension and complex properties, software ecosystem analysis has been limited. In this paper, we leverage topological methods in visualize the high-dimensional datasets from a software ecosystem. Topological Data Analysis (TDA) is an emerging technique to analyze high-dimensional datasets, which enables us to study the shape of data. We generate the npm software ecosystem topology to uncover insights and extract patterns of existing libraries by studying its localities. Our real world example reveals many interesting insights and patterns that describes the shape of a software ecosystem.

In this research, a tool for evaluating algorithmic thinking ability of the primary schoolchildren was developed. This tool is based on the three categories of operations used to construct algorithms, namely, sequential operations, conditional branching operations, and iterative operations. Each question in the tool checks to see if the examinee understands the concept of one of the three categories. The tool was developed to evaluate the educational effect of programming education for middle to upper grade (third to sixth grade) primary schoolchildren. Since both Visual Programming Language (VPL) and Textual Programming Language (TPL) could be used, it was required that the tool could be used by both the group of children who use VPLs and the group of children who use TPLs. To make it possible, no programming language appeared in the questions in the tool. The teaching materials for the programming education were also developed in such a way that the three basic concepts of algorithm, namely, sequential processing, conditional branching, and repetitive processing, were clearly taught. The target VPL in this research was Scratch. The evaluation tool was conducted in a weekend class of programming education for primary schoolchildren, and the algorithmic thinking ability of the schoolchildren was analyzed.

Previous studies have found that a significant number of bug reports are misclassified between bugs and nonbugs, and that manually classifying bug reports is a time-consuming task. To address this problem, we propose a bug reports classification model with N-gram IDF, a theoretical extension of Inverse Document Frequency (IDF) for handling words and phrases of any length. N-gram IDF enables us to extract key terms of any length from texts, these key terms can be used as the features to classify bug reports. We build classification models with logistic regression and random forest using features from N-gram IDF and topic modeling, which is widely used in various software engineering tasks.With a publicly available dataset, our results show that our N-gram IDF-based models have a superior performance than the topic-based models on all of the evaluated cases. Our models show promising results and have a potential to be extended to other software engineering tasks.

For developers concerned with a performance drop or improvement in their software, a profiler allows a developer to quickly search and identify bottlenecks and leaks that consume much execution time. Non real-time profilers analyze the history of already executed stack traces, while a real-time profiler outputs the results concurrently with the execution of software, so users can know the results instantaneously. However, a real-time profiler risks providing overly large and complex outputs, which is difficult for developers to quickly analyze. In this paper, we visualize the performance data from a real-time profiler. We visualize program execution as a three-dimensional (3D) city, representing the structure of the program as artifacts in a city (i.e., classes and packages expressed as buildings and districts) and their program executions expressed as the fluctuating height of artifacts. Through two case studies and using a prototype of our proposed visualization, we demonstrate how our visualization can easily identify performance issues such as a memory leak and compare performance changes between versions of a program. A demonstration of the interactive features of our prototype is available at https://youtu.be/eleVo19Hp4k.

A key goal of this research is to understand the relationship between adoption of software library versions and its release cycle. In detail, we conducted an empirical study of the release cycle of 23 libraries and how they were adopted by 415 Apache Software Foundation (ASF) client projects. Our preliminary findings show that software projects are quicker to update earlier rapid-release libraries compared to library projects with a longer release cycle.

When developing and maintaining a software project, many issues about bug fixing or feature addition are reported on the Bug Tracking System (BTS) and the Issue Tracking System (ITS). Bountysource is a web founding platform that awards developers who have solved issues on the BTS/ITS. Users can post a bounty for the issues, and a developer who solves the issue can get that bounty. This research analyzes Bountysource to clarify how bounties act in open source software projects and discusses further research topics in open-source bounties.

Open Source Software (OSS) cannot exist without contributions from the community. In particular, long-term contributors (LTCs) (e.g., committer), defined as contributors who spend at least one year on OSS projects, play crucial role in a project success because they would have permission to add (commit) code changes to a project's version control system, and to become a mentor for a beginner in OSS projects. However, contributors often leave a project before becoming a LTC because most contributors are volunteers. If contributors are motivated in their work in OSS projects, they might not leave the projects. In this study, we examine the phenomena involved in becoming a LTC in terms of motivation to continue in OSS projects. In particular, our target motivation is to understand what is involved in long-term contribution with other expert contributors. We study classifier to identify a LTC who will contribute patch submissions for more than one year based on collaboration in terms of the code review process. In detail, we analyze what review feedbacks encourage a contributor to continue with OSS project. Using a Qt project dataset, we understand review feedback which affected contribution period of the developer.

Defect prediction models help software quality assurance teams to allocate their limited resources to the most defect-prone modules. Model validation techniques, such as$k$-fold cross-validation, use historical data to estimate how well a model will perform in the future. However, little is known about how accurate the estimates of model validation techniques tend to be. In this paper, we investigate the bias and variance of model validation techniques in the domain of defect prediction. Analysis of 101 public defect datasets suggests that 77 percent of them are highly susceptible to producing unstable results– - selecting an appropriate model validation technique is a critical experimental design choice. Based on an analysis of 256 studies in the defect prediction literature, we select the 12 most commonly adopted model validation techniques for evaluation. Through a case study of 18 systems, we find that single-repetition holdout validation tends to produce estimates with 46-229 percent more bias and 53-863 percent more variance than the top-ranked model validation techniques. On the other hand, out-of-sample bootstrap validation yields the best balance between the bias and variance of estimates in the context of our study. Therefore, we recommend that future defect prediction studies avoid single-repetition holdout validation, and instead, use out-of-sample bootstrap validation.

In a software development project, project management is indispensable, and effort estimation is one of the important factors on the management. To improve estimation accuracy, outliers are often removed from dataset used for estimation. However, the influence of the outliers to the estimation accuracy is not clear. In this study, we added outliers to dataset experimentally, to analyze the influence. In the analysis, we changed the percentage of outliers, the extent of outliers, variable including outliers, and location of outliers on the dataset. After that, effort was estimated using the dataset. In the experiment, the influence of outliers was not very large, when they were included in the software size metric, the percentage of outliers was 10%, and the extent of outliers was 100%.

Recently, information system operation becomes more important because of increasing size of information system and outsourcing the system operation. However, it is not easy for customers to judge the validity of the system operation cost. To provide information which helps the judgment, we analyzed factors which affect system operation cost. Working time of system operation service provider has the strong relationship to the cost. So, if customers know the working time, they estimate the cost properly. However, it is difficult for customers to know the working time generally. So, we assumed that customers estimate unit cost and working time, to speculate total operation cost roughly. To help the estimation, we analyzed factors affected working time and unit cost. The analysis results show that working time is settled based on the software size and the number of users, and the unit cost of the engineers increases when network range of the system is wide.

In the research of Mining Software Repositories, source code repositories are one of the core sources since it contains the product and the process of software development. A source code repository stores the versions of files and makes it possible to browse the histories of files, such as modification dates, authors, messages, so on. Although such rich information of file histories is easily available, extracting the histories of methods/functions, which are elements of source code files, is not easy from general code repositories. To tackle this difficulty, we have developed Historage, a fine-grained version control system. Historage repository is a Git repository, which is built upon an original Git repository. Therefore, similar mining techniques for general Git repositories are applicable to Historage repositories. We also have developed Kataribe, a hosting service of Historage repositories, which contains hundreds of Historage repositories constructed from repositories in GitHub, which are written in C#, Java, Python and Ruby. The list of all Historage and original repositories are available at http://kataribe.naist.jp/public. With this dataset, we will promote in-depth and fine-grained software evolution research with diversity of programming languages.

Effort-inconsistency is a situation where historical software project data used for software effort estimation (SEE) are contaminated by many project cases with similar characteristics but are completed with significantly different amount of effort. Using these data for SEE generally produces inaccurate results; however, an effective technique for its handling is yet made to be available. This study approaches the problem differently from common solutions, where available techniques typically attempt to remove every project case they have detected as outliers. Instead, we hypothesize that data inconsistency is caused by only a few deviant project cases and any attempt to remove those other cases will result in reduced accuracy, largely due to loss of useful information and data diversity. Filter-INC (short for Filtering technique for handling effort-INConsistency in SEE datasets) implements the hypothesis to decide whether a project case being detected by any existing technique should be subject to removal. The evaluation is carried out by comparing the performance of 2 filtering techniques between before and after having Filter-INC applied. The results produced from 8 real-world datasets together with 3 machine-learning models, and evaluated by 4 performance measures show a significant accuracy improvement at the confident interval of 95%. Based on the results, we recommend our proposed hypothesis as an important instrument to design a data preprocessing technique for handling effort-inconsistency in SEE datasets, definitely an important step forward in preprocessing data for a more accurate SEE model.

The purpose of this research is to compare textual programming languages and visual programming languages from the aspect of motivation. As a textual programming language, Processing programming language was used, and as visual programming languages, Scratch, a derivation of Scratch, Teaching materials offered by code.org, and LEGO Mindstorms EV3 were used. Teaching materials using the textual programming language, and those using the visual programming languages were developed separately. A trial experiment of programming education with the textual programming language was conducted to a cohort of seven primary schoolchildren. Trial experiments with the visual programming languages were conducted twice. In each of them, a cohort of eight primary schoolchildren participated. The motivation of the children was assessed using the questionnaire based on the ARCS (Attention, Relevance, Confidence, and Satisfaction) motivation model. The results with the visual programming languages suggested that the motivation scores of the children increased as the class progressed when visual programming languages were used. On the other hand, the results with Processing suggested that the variance of Satisfaction factor increased as the class progressed when textual programming languages were used, which further suggested that the Satisfaction scores of the children spread as the class progressed when textual programming languages were used.

Defect prediction models are classifiers that are trained to identify defect-prone software modules. Such classifiers have configurable parameters that control their characteristics (e.g., the number of trees in a random forest classifier). Recent studies show that these classifiers may underperform due to the use of suboptimal default parameter settings. However, it is impractical to assess all of the possible settings in the parameter spaces. In this paper, we investigate the performance of defect prediction models where Caret --- an automated parameter optimization technique --- has been applied. Through a case study of 18 datasets from systems that span both proprietary and open source domains, we find that (1) Caret improves the AUC performance of defect prediction models by as much as 40 percentage points; (2) Caret-optimized classifiers are at least as stable as (with 35% of them being more stable than) classifiers that are trained using the default settings; and (3) Caret increases the likelihood of producing a top-performing classifier by as much as 83%. Hence, we conclude that parameter settings can indeed have a large impact on the performance of defect prediction models, suggesting that researchers should experiment with the parameters of the classification techniques. Since automated parameter optimization techniques like Caret yield substantially benefits in terms of performance improvement and stability, while incurring a manageable additional computational cost, they should be included in future defect prediction studies.

It is not unreasonable to say that examples are one of the most commonly used knowledge sources when learning the usage and best practices of a new API. That being said, in many cases the examples provided on the APIs' Web site are lacking in quantity or quality, so developers have to resort to other information sources, namely blogs and coding forums. Moreover, there is no good way for API developers to measure anything concerning the examples they are creating. In order to resolve the problem of lacking examples information and feedback in their creation, our goal is to develop metrics for empirical measurement of examples and to offer support during the APIs example creation steps, and this paper represents the starting point towards that aim. We have analyzed the source code examples provided on the API's Web site or GitHub directory for seven popular API libraries written in Java and measured certain metrics, such as example coverage, example code to source code ratio, class coverage percentage, and problems that occur with the compilation and execution of existing examples. The purpose of this paper is to investigate the current situation of examples and provide a starting knowledge base for building an automatic tool for source code example metrics analysis.

Bug reports are one of the most crucial information sources for software engineering offering answers to many questions. Yet, getting these answers is not always easy; the information in bug reports is often implicit and some processes are required to extract the meaning of these reports. Most research in this area employ a supervised learning approach to classify bug reports so that required types of reports could be identified. However, this approach often requires an immense amount of time and effort, the resources that already too scarce in many projects. We aim to develop an automated framework that can categorize bug reports, according to their grammatical structure without the need for labeled data. Our framework categorizes bug reports according to their text similarity using topic modeling and a clustering algorithm. Each group of bug reports are labeled with our new clustering labeling algorithm specifically made for clusters in the topic space. Our framework is highly customizable with a modular approach and options to incorporate available background knowledge to improve its performance, while our cluster labeling approach make use of natural language process (NLP) chunking to create the representative labels. Our experiment results demonstrate that the performance of our unsupervised framework is comparable to a supervised learning one. We also show that our labeling process is capable of labeling each cluster with phrases that are representative for that cluster's characteristics. Our framework can be used to automatically categorize the incoming bug reports without any prior knowledge, as an automated labeling suggestion system or as a tool for obtaining knowledge about the structure of the bug report repository.

Shepperd et al. find that the reported performance of a defect prediction model shares a strong relationship with the group of researchers who construct the models. In this paper, we perform an alternative investigation of Shepperd et al.'s data. We observe that (a) research group shares a strong association with other explanatory variables (i.e., the dataset and metric families that are used to build a model); (b) the strong association among these explanatory variables makes it difficult to discern the impact of the research group on model performance; and (c) after mitigating the impact of this strong association, we find that the research group has a smaller impact than the metric family. These observations lead us to conclude that the relationship between the research group and the performance of a defect prediction model are more likely due to the tendency of researchers to reuse experimental components (e.g., datasets and metrics). We recommend that researchers experiment with a broader selection of datasets and metrics to combat any potential bias in their results.

Data quality is an essential aspect in any empirical study, because the validity of models and/or analysis results derived from an empirical data is inherently influenced by its quality. In this empirical study, we focus on data consistency as a critical factor influencing the accuracy of prediction models in software engineering. We propose a software metric called Cases Inconsistency Level (CIL) for analyzing conflicts within software engineering data sets by leveraging probability statistics on project cases and counting the number of conflicting pairs. The result demonstrated that CIL is able to be used as a metric to identify either consistent data sets or inconsistent data sets, which are valuable for building robust prediction models. In addition to measuring the level of consistency, CIL is proved to be applicable to predict whether or not an effort model built from data set can achieve higher accuracy, an important indicator for empirical experiments in software engineering.

In this research, a Textual Programming Language (TPL) is used in programming education for primary schoolchildren because of the following reasons: (1) it is more practical to use the programming languages similar to the ones used for developing real applications, (2) typing statements could be easier for primary schoolchildren than generally thought, (3) there exist programming environments such as Processing that are easy to use and produce very attractive graphical outcomes. Teaching material for programming education with Processing was developed. In this teaching material, cartoons were used to explain difficult concepts. The learners who use this teaching material were supposed to draw some computational figures with chosen colors. Trial experiments of programming education using this teaching material was conducted to a cohort of seven primary schoolchildren (six 4th grade and one 5th grade children) in two consecutive weekend classes (one hour each). Since the authors' aim of this programming education was to create a sense of fun and excitement in the children and inculcate a desire to engage with computing, the motivation of the children was assessed using the questionnaire based on the ARCS (Attention, Relevance, Confidence, and Satisfaction) motivation model. The results were encouraging and suggested that TPLs could be used in programming education for primary schoolchildren.

The reliability of a prediction model depends on the quality of the data from which it was trained. Therefore, defect prediction models may be unreliable if they are trained using noisy data. Recent research suggests that randomly-injected noise that changes the classification (label) of software modules from defective to clean (and vice versa) can impact the performance of defect models. Yet, in reality, incorrectly labelled (i.e., mislabelled) issue reports are likely non-random. In this paper, we study whether mislabelling is random, and the impact that realistic mislabelling has on the performance and interpretation of defect models. Through a case study of 3,931 manually-curated issue reports from the Apache Jackrabbit and Lucene systems, we find that: (1) issue report mislabelling is not random; (2) precision is rarely impacted by mislabelled issue reports, suggesting that practitioners can rely on the accuracy of modules labelled as defective by models that are trained using noisy data; (3) however, models trained on noisy data typically achieve 56%-68% of the recall of models trained on clean data; and (4) only the metrics in top influence rank of our defect models are robust to the noise introduced by mislabelling, suggesting that the less influential metrics of models that are trained on noisy data should not be interpreted or used to make decisions.

The importance of supporting test and maintenance activities in software development has been increasing, since recent software systems have become large and complex. Although in the field of Mining Software Repositories (MSR) there are many promising approaches to predicting, localizing, and triaging bugs, most of them do not consider impacts of each bug on users and developers but rather treat all bugs with equal weighting, excepting a few studies on high impact bugs including security, performance, blocking, and so forth. To make MSR techniques more actionable and effective in practice, we need deeper understandings of high impact bugs. In this paper we introduced our dataset of high impact bugs which was created by manually reviewing four thousand issue reports in four open source projects (Ambari, Camel, Derby and Wicket).

Software code review is an inspection of a code change by an independent third-party developer in order to identify and fix defects before an integration. Effectively performing code review can improve the overall software quality. In recent years, Modern Code Review (MCR), a lightweight and tool-based code inspection, has been widely adopted in both proprietary and open-source software systems. Finding appropriate code-reviewers in MCR is a necessary step of reviewing a code change. However, little research is known the difficulty of finding code-reviewers in a distributed software development and its impact on reviewing time. In this paper, we investigate the impact of reviews with code-reviewer assignment problem has on reviewing time. We find that reviews with code-reviewer assignment problem take 12 days longer to approve a code change. To help developers find appropriate code-reviewers, we propose RevFinder, a file location-based code-reviewer recommendation approach. We leverage a similarity of previously reviewed file path to recommend an appropriate code-reviewer. The intuition is that files that are located in similar file paths would be managed and reviewed by similar experienced code-reviewers. Through an empirical evaluation on a case study of 42,045 reviews of Android Open Source Project (AOSP), OpenStack, Qt and LibreOffice projects, we find that RevFinder accurately recommended 79% of reviews with a top 10 recommendation. RevFinder also correctly recommended the code-reviewers with a median rank of 4. The overall ranking of RevFinder is 3 times better than that of a baseline approach. We believe that RevFinder could be applied to MCR in order to help developers find appropriate code-reviewers and speed up the overall code review process.

Context: Since human power is an essential resource, the number of contributors in a software development community is one of the health indicators of an open source software (OSS) project. For maintaining and increasing the populations in software development communities, both attracting new contributors and retaining existing contributors are important. Goal: Our goal is understanding the current status of projects' population, especially the different experienced contributors' composition of the projects. Method: We propose software population pyramids, a graphical illustration of the distribution of various experience groups in a software development community. Results: From the study with OSS projects in GitHub, we found that the shapes of software population pyramids varies depending on the current status of OSS development communities. Conclusions: This paper present a software population pyramid of the distribution of various experience groups in a software community population. Our results can be considered as predictors of the near future of a project.

In the past, the authors had been analyzing motivation of the learners in programming education using the ARCS assessment metric. This metric had been used in the application experiment in 13 programming courses, and about 1,700 sets of data was collected. From these data, the learners' model, characteristics of the change of motivation, and ways of improving teaching materials had been clarified. However, these study results were obtained after the terms, when the programming courses were over, and thus did not contribute much to the ongoing programming education. For this reason, in this research, the methods for predicting the change of learners' motivation were studied so that the learners who may need support could be identified. The idea came from the experiment the authors conducted, in which the motivation of learners was analyzed by plotting the motivation scores of each factor in the ARCS model as a 3D graph. As a result, a decreasing tendency of motivation was observed when the distribution of the plot widened. After studying the tendency in detail, it was thought to be due to the influence of the variance of sub-level category scores. In the proposed method, the motivation of each learner is assessed in each lesson using the ARCS assessment metric. If variance of the motivation scores of a learner in a lesson is above a certain threshold value AND if mean of the scores has not decreased from the previous lesson, then the learner is identified as a candidate of learner who needs support at that lesson. In the application experiment, a programming course with 9 lessons was offered and 9 learners attended all the 9 lessons. In the experiment, 7 cases had been identified as the candidates of learners who need support, and out of those 7 cases, a decrease of motivation to less than average was observed in 5 cases.

In the research of Mining Software Repositories, code repository is one of the core source since it contains the product of software development. Code repository stores the versions of files, and makes it possible to browse the histories of files, such as modification dates, authors, messages, etc. Although such rich information of file histories is easily available, extracting the histories of methods, which are elements of source code files, is not easy from general code repositories. To tackle this difficulty, we have developed Historage, a fine-grained version control system. Historage repository is a Git repository which is built upon original Git repository. Therefore, similar mining techniques for general Git repositories are applicable to Historage repositories. Kataribe is a hosting service of Historage repositories, which enables researchers and developers to browse method histories on the web and clone Historage repositories to local. The Kataribe project aims to maintain and expand the datasets and features.

Bug reports play essential roles in many software engineering tasks. Since validity and performance of these tasks definitely rely on the quality of bug reports, accurate information from bug reports is very important. However, as found in previous study, significant numbers of reports classified as bug are not really a bug. Recent studies proposed techniques to automatically classify bug reports into binary classes, yet there is still more to desire. These bug reports can be classified into multiple classes, which could help to identify what these reports are actually about. Moreover, previous study only looks into one possibility of topic modeling, that is, Latent Dirichlet Allocation (LDA). While LDA has its advantage, parameter tuning is required. In this paper, we propose a nonparametric approach to automatically classify bug reports with, another topic modeling method, Hierarchical Dirichlet Process (HDP). The result indicates that our nonparametric approach performance is comparable to the parametric one. We also examine various aspects of LDA to provide more thoroughly understanding of this process.

This paper proposes a fault-prone prediction approach that combines a fault-prone prediction model and manual inspection. Manual inspection is conducted by a predefined checklist that consists of questions and scoring procedures. The questions capture the fault signs or indications that are difficult to be captured by source code metrics used as input by prediction models. Our approach consists of two steps. In the first, the modules are prioritized by a fault-prone prediction model. In the second step, an inspector inspects and scores α percent of the prioritized modules. We conducted a case study of source code modules in commercial software that had been maintained and evolved over ten years and compared AUC (Area Under the Curve) values of Alberg Diagram among three prediction models: (A) support vector machines, (B) lines of code, and (C) random predictor with four prioritization orders. Our results indicated that the maximum AUC values under appropriate α and the coefficient of the inspection score were larger than the AUC values of the prediction models without manual inspection in each of the four combinations and the three models in our context. In two combinations, our approach increased the AUC values to 0.860 from 0.774 and 0.724. Our results also indicated that one of the combinations monotonically increased the AUC values with the numbers of manually inspected modules. This might lead to flexible inspection; the number of manually inspected modules has not been preliminary determined, and the inspectors can inspect as many modules as possible, depending on the available effort.

In an Open Source Software (OSS) project, many developers contribute by submitting source code patches. To maintain the quality of the code, certain experienced developers review each patch before it can be applied or committed. Ideally, within a short amount of time after its submission, a patch is assigned to a reviewer and reviewed. In the real world, however, many large and active OSS projects evolve at a rapid pace and the core developers can get swamped with a large number of patches to review. Furthermore, since these core members may not always be available or may choose to leave the project, it can be challenging, at times, to find a good reviewer for a patch. In this paper, we propose a graph-based method to automatically recommend the most suitable reviewers for a patch. To evaluate our method, we conducted experiments to predict the developers who will apply new changes to the source code in the Eclipse project. Our method achieved an average recall of 0.84 for top-5 predictions and a recall of 0.94 for top-10 predictions.

What types of developers do active software projects have? This paper presents a study of the characteristics of developers' activities in open source software development. GitHub, a widely-used hosting service for software development projects, provides APIs for collecting various kinds of GitHub data. To clarify the characteristics of developers' activities, we used these APIs to investigate GitHub events generated by each developer. Using this information, we categorized developers based on measures such as whether they prefer communication by coding or comments, or whether they are specialists or generalists. Our study indicates that active software projects have various kinds of developers characterized by different types of development activities.

The performance of software cost estimation based on analogy reasoning depends upon the measures that specifying the similarity between software projects. This paper empirically investigates the use of probabilistic-based distance functions to improve the similarity measurement. The probabilistic-based distance functions are considerably more robust, because they collect the implicit correlation between the occurrences of project feature attributes. This information gain enables the constructed estimation model to be more concise and comprehensible. The study compares 6 probabilistic-based distance functions against the commonly-used Euclidian distance. We empirically evaluate the implemented cost estimation model using 5 real-world datasets collected from the PROMISE repository. The result shows a significant improvement in terms of error reduction, that implies an estimation based on probabilistic-based distance functions achieve higher accuracy on average, and the peak performance significantly outperforms the Euclidian distance based on Wilcox on signed-rank test.

Bug reports are widely used in several research areas such as bug prediction, bug triaging, and etc. The performance of these studies relies on the information from bug reports. Previous study showed that a significant number of bug reports are actually misclassified between bugs and non-bugs. However, classifying bug reports is a time-consuming task. In the previous study, researchers spent 90 days to classify manually more than 7,000 bug reports. To tackle this problem, we propose automatic bug report classification techniques. We apply topic modeling to the corpora of pre-processed bug reports of three open-source software projects with decision tree, naive Bayes classifier, and logistic regression. The performance in classification, measured in F-measure score, varies between 0.66-0.76, 0.65-0.77, and 0.71-0.82 for HTTPClient, Jackrabbit, and Lucene project respectively.

In this paper, a method for improving the teaching materials of programming education is introduced, and the evaluation of the effects of using the strategy is presented. By using this method, the teachers of programming education will be able to assess and improve their teaching materials irrespective of their knowledge and experience of their teaching materials already used. In this method, the teaching materials were improved based on the statistical analysis of the motivation of students. Specifically, the motivation of students was measured for each lower category of ARCS motivation model with the authors' original questionnaire. The lower category in a particular lesson that showed a statistically significant decrease from the previous lesson was identified, and the improvement strategies for the lower category were selected from the list of motivation strategies in the ARCS model. The teaching materials of programming education were then improved based on the strategy. In this research, five lower categories of particular lessons in a programming course were identified, and the teaching materials were improved. The improved teaching materials were used in the following programming course, and the effects of the improvements were seen in three lower categories out of the identified five lower categories.

A large open source software (OSS) project receives many bug reports on a daily basis. Bug localization techniques automatically pinpoint source code fragments that are relevant to a bug report, thus enabling faster correction. Even though many bug localization methods have been introduced, their performance is still not efficient. In this research, we improved on existing bug localization methods by taking into account co-change histories. We conducted experiments on two OSS datasets, the Eclipse SWT 3.1 project and the Android ZXing project. We validated our approach by evaluating effectiveness compared to the state-of-the-art approach Bug Locator. In the Eclipse SWT 3.1 project, our approach reliably identified source code that should be fixed for a bug in 72.46% of the total bugs, while Bug Locator identified only 51.02%. In the Android ZXing project, our approach identified 85.71%, while Bug Locator identified 60%.

In past research, we proposed an authentication method that combines actions with spatiotemporal information such as location, time, and distance. With the method, a user succeeds in authentication when he/she performs preset actions such as pushing button n times on preset intervals defined by spatiotemporal information. In this paper, we improve the authentication method using a partial matching method. We propose two kinds of partial matching methods for pushing button and interval. A type I method assumes the number of pushing button is sometimes less than preset count, but the number never exceeds it, and a user never pushes the button out of preset areas. A type II method assumes the number of pushing button is less or more than preset count occasionally, and a user pushes the button out of preset areas. We showed how to calculate FAR when the type I or II is applied. In the experiment, we compared the type I and II methods with a conventional method to evaluate their security. As a result, the type I method improved false acceptance rate (FAR) from 0.097% to 0.053%. The type II method improved FAR from 0.097% to 0.035%.

Until now, various techniques for predicting fault-prone modules have been proposed and evaluated in terms of their prediction performance; however, their actual contribution to business objectives such as quality improvement and cost reduction has rarely been assessed. This paper proposes using a simulation model of software testing to assess the cost effectiveness of test effort allocation strategies based on fault prediction results. The simulation model estimates the number of discoverable faults with respect to the given test resources, the resource allocation strategy, a set of modules to be tested, and the fault prediction results. In a case study applying fault prediction of a small system to acceptance testing in the telecommunication industry, results from our simulation model showed that the best strategy was to let the test effort be proportional to "the number of expected faults in a module × log(module size)." By using this strategy with our best fault prediction model, the test effort could be reduced by 25 percent while still detecting as many faults as were normally discovered in testing, although the company required about 6 percent of the test effort for metrics collection, data cleansing, and modeling. The simulation results also indicate that the lower bound of acceptable prediction accuracy is around 0.78 in terms of an effort-aware measure, Norm(Popt). The results indicate that reduction of the test effort can be achieved by fault prediction only if the appropriate test strategy is employed with high enough fault prediction accuracy. Based on these preliminary results, we expect further research to assess their general validity with larger systems.

Background: Association rules are more comprehensive and understandable than fault-prone module predictors (such as logistic regression model, random forest and support vector machine). One of the challenges is that there are usually too many similar rules to be extracted by the rule mining. Aim: This paper proposes a rule reduction technique that can eliminate complex (long) and/or similar rules without sacrificing the prediction performance as much as possible. Method: The notion of the method is to removing long and similar rules unless their confidence level as a heuristic is high enough than shorter rules. For example, it starts with selecting rules with shortest length (length=1), and then it continues through the 2nd shortest rules selection (length=2) based on the current confidence level, this process is repeated on the selection for longer rules until no rules are worth included. Result: An empirical experiment has been conducted with the Mylyn and Eclipse PDE datasets. The result of the Mylyn dataset showed the proposed method was able to reduce the number of rules from 1347 down to 13, while the delta of the prediction performance was only. 015 (from. 757 down to. 742) in terms of the F1 prediction criteria. In the experiment with Eclipsed PDE dataset, the proposed method reduced the number of rules from 398 to 12, while the prediction performance even improved (from. 426 to. 441.) Conclusion: The novel technique introduced resolves the rule explosion problem in association rule mining for software proneness prediction, which is significant and provides better understanding of the causes of faulty modules.

One of the common problems in building an effort estimation model is that not all the effort factors are suitable as predictor variables. As a supplement of missing information in estimation models, this paper explores the project manager's knowledge about the target project. We assume that the experts can judge the target project's productivity level based on his/her own expert knowledge about the project. We also assume that this judgment can be further improved, because using the expert's judgment solely could incur subjective perception. This paper proposes a regression model building/selection method to address this challenge. In the proposed method, a fit dataset for model building is divided into two or three subsets by project productivity, and an estimation model is built on each data subset. The expert judges the productivity level of the target project and selects one of the models to be used. In the experiment, we used three datasets to evaluate the produced effort estimation models. In the experiment, we adjusted the error rate of the judgment and analyzed the relationship between the error rate and the estimation accuracy. As a result, the judgment-incorporating models produced significantly higher estimation accuracy than the conventional linear regression model, where the expert's error rate is less than 37%.

An efficient bug management process is critical for the success of software projects. Prior work has focused on improving this process, for example, by automating bug triaging, detecting duplicate bugs, and understanding the rationale for re-opening bugs. This paper continues this line of work by exploring the people who are involved in the bug management process. In particular we develop four patterns that distill the different relations between the people involved in the process: the reporter, triager, and fixer of a bug. Through a case study on the Eclipse Platform and JDT projects, we demonstrate that these patterns have an impact on the efficiency of the bug management process. For example, we find that using our patterns project personnel can improve their efficiency through better communication about bugs before assigning them.

There are two "barriers" in the research of software engineering. One is "the barrier of individuality" and other one is "the barrier of secret". These have obstructed prosperity of the software engineering. For these nine years, the authors considered overthrowing these barriers. The authors realized the powerful in process measurement platform for software development project and challenged to get empirical evidence of its usefulness and its propagation. Up to now, the authors realized two generations of measurement platform and distributed them to the industry, and also carried several empirical evaluation projects. Then the authors realized the new generation of in-process measurement platform, as for it, all the former days experiences were reflected. It was utilizing the structure of remarkable progressed software development environment in recent years. And the authors stated its distribution to the industrial society. This paper looks down former day's empirical evaluation activities and clarifies requirements for in-process measurement platform which were obtained from those activities. It also describes the new generation measurement platform which realized those former researches. Add to them, this paper proposes the broad participation in the empirical collaborative research using this platform. Finally it is introduced related research, and positioning of this research is shown.

In the beginning of a contracted based software development project, the RFP is provided by a software user company and used as an initial system requirements specification to ask software developer companies to propose their technical plans to fulfill the requirements. In this process, it is very important to evaluate the quality of the RFP to make sure that basic user requirements are written enough. Especially, non-functional requirements (NFRs) are important since the system architecture greatly depends on the NFRs such as response time and security issues. This paper proposes a simple evaluation model of NFRs included in the RFP, mainly focusing on the user maintenance and operation issues. This model consists of NFR categories, NFR metrics, description level grading and weight to each NFR. As a case study, RFPs of 29 projects were evaluated by the proposed model. As a result, we confirmed that the model could identify poorly-written NFR aspects in the RFP, which need refinement before asking the developer company for a proposal.

Time zone, different work schedule, limited real-time information sharing, steep learning curve and different personal specialties, these are common limitations in the collaborative studies, especially when a researcher has just been introduced to the research field or working in a different environment (i.e. Internship programs). This paper introduces you with the experiences, challenges, difficulties, lessons learned, common fallacies and pitfalls in the collaborative software engineering research through the experience of 2-months collaborative research program between Kasetsart University in Thailand and Nara Institute of Science and Technology in Japan. Good mentoring and flat-style communication between professors and students are good indicators of the high quality result in the internship program. These information can be useful to professors, young researchers and internship students who will be conducting researches in such manner.

In order to plan efficient and effective in-house training for programming, the characteristics of the motivation of learners in an in-house training of programming course, in an information and communication technology (ICT) industry, have been analyzed. In the in-house training course, 42 participants underwent a 16 day training program of Java language. The analysis of the motivation was carried out using a questionnaire based on the ARCS motivation model. As a result, it has been found that the motivation of those who have a technology background is not necessarily higher than that of those who have a non-technology background.

The goal of this research was to assess the consistency of source code comprehension strategies and comprehension effort estimation metrics, such as LOC, across different types of modification tasks in software maintenance and evolution. We conducted an empirical study with software development practitioners using source code from a small paint application written in Java, along with four semantics-preserving modification tasks (refactoring, defect correction) and four semantics-modifying modification tasks (enhancive and modification). Each task has a change specification and corresponding source code patch. The subjects were asked to comprehend the original source code and then judge whether each patch meets the corresponding change specification in the modification task. The subjects recorded the time to comprehend and described the comprehension strategies used and their reason for the patch judgments. The 24 subjects used similar comprehension strategies. The results show that the comprehension strategies and effort estimation metrics are not consistent across different types of modification tasks. The recorded descriptions indicate the subjects scanned through the original source code and the patches when trying to comprehend patches in the semantics-modifying tasks while the subjects only read the source code of the patches in semantics-preserving tasks. An important metric for estimating comprehension efforts of the semantics-modifying tasks is the Code Clone Subtracted from LOC(CCSLOC), while that of semantics-preserving tasks is the number of referred variables.

There are many roles to play in the bug fixing process in open source software development. A developer called ``Committer'', who has a permission to submit a patch into software repository, plays a major role in this process and hold a key to the successfulness of the project. In this work, we have observed each committer activities from the Eclipse-Platform bug tracking system and version archives. Despite the importance of committer's activities, we suspected that sometimes committers can make mistakes, which have negative consequences to the bug fixing process. Our research focus on studying the consequences of each committer's activities to this process. We collected each committer's historical data and evaluated each of them by comparing the more cautiousness to less cautiousness committers. Then we looked deeper into each committer's characteristics to see the reasons why some committers tend to make mistakes more than the others. From our results, we would like to make a humbly suggestion to the OSS's committers to be aware of their importance to the projects and be cautious before doing their jobs.

In this paper, we present a replicated study to predict fault-prone modules with code clone metrics to follow Baba's experiment. We empirically evaluated the performance of fault prediction models with clone metrics using 3 datasets from the Eclipse project and compared it to fault prediction without clone metrics. Contrary to the original Baba's experiment, we could not significantly support the effect of clone metrics, i.e., the result showed that F1-measure of fault prediction was not improved by adding clone metrics to the prediction model. To explain this result, this paper analyzed the relationship between clone metrics and fault density. The result suggested that clone metrics were effective in fault prediction for large modules but not for small modules.

Patch submission has been known as one of the most important activities to sustain the open source software (OSS). The patch archive can be analyzed to procure many benefit cognizance for supporting the OSS project works. The recent models and methods that analyze the patches acceptance are quite rack of comprehensive; hence, complex activities such as a committer portioning the Passed QA patch out and accept are still excluded from the analysis. Therefore, the results derived from those methods would be inadequate to conclude the actual patch acceptance. In this research, we introduce an algorithm for analyzing patch acceptance including the partial and gradually accepted conditions. Validating our algorithm, we present our methods for indicating the partial and gradual application of the Passed QA patch between either mailing list and SVN or Bugzilla and CVS which are the commonly deployed patch-activities related system. We studied on two well known OSS projects; Apache HTTP and Eclipse Platform. We obtained a fascinating conclusion that larger patches have more confident to be accepted than the smaller contradicted to other analysis that came from the recent methods.

This paper describes an exploratory study on theuse of screenshots for recording software inspection activities such as defect reproduction and correction. Although detected defects are usually recorded in writing, using screenshots to record detected defects should ecrease the percentage of irreproducible defects and the time needed to reproduce defects during the defect orrection phase. An experiment was conducted to clarify the efficiency of using screenshots to record detected defects. One practitioner group and two student groups participated in the experiment. The recorder in each group used a prototype support tool for capturing screenshots during the experiment. Each group conducted two trials: one with a general spreadsheet application to support recording, the other with the prototype tool that supportsrecording inspection activities. After the inspection meeting, the recorder was asked to reproduce the recorded defects. The percentage of reproduce defects and time to reproduce defects was measured. The results of the experiment show that use of screenshots increases the percentage of reproduced defects and decreases the time needed to reproduce the defects. The results also indicate that use of the recording tool affected the types of defects.

Recently, software functional size becomes larger, and consequently, not only a software developer but also a software purchaser suffers considerable losses by software project failure. So avoiding project failure is also important for purchasers. Project supervision (monitoring and control) is expected for the purchaser to suppress risk of project failure. It is performed by sharing software metrics during the project for the purchaser to grasp the status of the project, and corrective actions are done based on analysis results of the metrics. Although there are some software measurement models, the models are not enough to describe how to confirm effects of project supervision. To acquire the effects certainly, the purchaser and the developer should quantitatively confirm whether the effects are acquired or not by project supervision. In addition, the models cannot represent corrective actions when symptoms of project failure are found. We propose the model for project supervision. The model explains planning, collecting data, transforming data, analyzing data, reaction toward found issues, and confirming effect of project supervision. With our model, project supervision can be described more rigorously.

To clarify the characteristics of cost-overrun software projects, this paper focuses on the cost to sales ratio of software development, computed from financial information of a midsize software company in the embedded systems domain, and analyzes the correlation with outsourcing ratio as well as code reuse ratio and relative effort ratio per development phase. As a result, we found that the lower cost to sales ratio projects had the higher relative effort ratio in external design phase, which indicates that spending less effort in external design can cause decrease of profit. We also found that high outsourcing ratio projects had higher cost to sales ratio, and that projects having moderate code reuse ratio had lower and disperse cost to sales ratio, which suggests troubles in code reuse can damage the profit of a project.

This study describes a new method of development visualization along with empirical evidence of its usefulness. Typically, development activities such as program design, programming, and unit testing are not disclosed to the procurement organization (project owner). However, during integration and testing, various issues require collaboration between the procurement organization and developers. When this occurs, it is important to make the development process visible. Recent reports indicate the usefulness for project management of various in-process project measurements which allow visualization of the formerly invisible software project progress [1–6]. Based on this background, the authors investigated a case study where in-process measurement during the integration and test phase helped to make development issues visible. In this study, data obtained from the integration and testing phase were compared to a development process model. This model was based on the author's experience, and provided a vivid picture of the development activity. By applying in-process measurements in collaboration during the integration test phase, the development activity was clearly visualized, and the procurement organization understood problems.

Bug prediction models are often used to help allocate software quality assurance efforts (e.g. testing and code reviews). Mende and Koschke have recently proposed bug prediction models that are effort-aware. These models factor in the effort needed to review or test code when evaluating the effectiveness of prediction models, leading to more realistic performance evaluations. In this paper, we revisit two common findings in the bug prediction literature: 1) Process metrics (e.g., change history) outperform product metrics (e.g., LOC), 2) Package-level predictions outperform file-level predictions. Through a case study on three projects from the Eclipse Foundation, we find that the first finding holds when effort is considered, while the second finding does not hold. These findings validate the practical significance of prior findings in the bug prediction literature and encourage their adoption in practice.

To support legacy migration to service-oriented architecture (SOA), this paper presents a method that derives candidates of SOA services from procedural programs. In SOA, every service is supposed to be a process (procedure) with (1) open interface, (2) self-containedness, (3) coarse granularity for business. We identify such services from the source code and its data flow diagram (DFD), by analyzing data and control dependency among processes. Specifically, we first obtain the DFD with reverse-engineering techniques. For each layer of the DFD we classify every data flow into three categories. Using the data category and control among procedures, we characterize four types of dependency. We finally apply six rules that aggregate mutually dependent procedures and extract them as a service. A case study with a liquor shop inventory control system extracts service candidates with various granularity.

This paper proposes a novel method to generate synthetic projectcases and add them to a fit dataset for the purpose of improving the performance of analogy-based software effort estimation. The proposed method extends conventional over-sampling method, which is a preprocessing procedure for n-group classification problems, which makes it suitable for any imbalanced dataset to be used in analogy-based system. We experimentally evaluated the effect of the over-sampling method to improve the performance of the analogy-based software effort estimation by using the Desharnais dataset. Results show significant improvement to the estimation accuracy by using our approach.

This paper proposes a fault-prone module prediction method that combines association rule mining with logistic regression analysis. In the proposed method, we focus on three key measures of interestingness of an association rule (support, confidence and lift) to select useful rules for the prediction. If a module satisfies the premise (i.e. the condition in the antecedent part) of one of the selected rules, the module is classified by the rule as either fault-prone or not. Otherwise, the module is classified by the logistic model. We experimentally evaluated the prediction performance of the proposed method with different thresholds of each rule interestingness measure (support, confidence and lift) using a module set in the Eclipse project, and compared it with three well-known fault-proneness models (logistic regression model, linear discriminant model and classification tree). The result showed that the improvement of the F1-value of the proposed method was 0.163 at maximum compared to conventional models.

To construct a better multivariate regression model for software effort estimation, this paper proposes a method to select projects as a fit data from a given project data set based on estimation target's features. While regression models were often constructed from all available project data, this paper showed the necessity of fit data selection, and showed that the proposed method is one of the effective and systematic means to do the selection.

ESEM '08 : the Second ACM-IEEE international symposium on Empirical software engineering and measurement, October 09-10, 2008, Kaiserslautern, Germany

In this research, we have analyzed the transition of the motivation of the students in the art design faculty to learn programming using the questionnaire based on the ARCS motivation model. As a result, we have found that the change of the motivation of the students had been more prevalent in the first half of the course than the second half of the course.

To build high-reliability software in software development, software review is essential. Typically, software review requires documents from multiple phases such as requirements specification, design document and source code to reveal the inconsistencies among them and to ensure the traceability of deliverables. However, most previous studies on software review (reading) techniques focus on finding defects in a single document in their experiments. In this paper, we propose a multi-document review evaluation system, DRESREM2. This system records reviewers' eye movements and mouse/keyboard operations for analysis. We conducted eye gaze analysis of reviewers in design document review with multiple documents (including requirements specification, design document, etc.) to confirm the usefulness of the system. For the performance analysis, we recorded defect detection ratio, detection time per defect, and fixation ratio of eye movements on each document. As a result, reviewers who concentrated their eye movements on requirements specification found more defects in the design document. We believe this result is good evidence to encourage developers to read high-level documents when reviewing lowlevel documents.

This paper describes an empirical study to reveal factors influencing defect correction effort in software development. In the study we collected various attributes (metrics) of defects found in a typical medium-scale, multi-vendor information system development project in Japan over a six-month period. We then statistically analyzed the relationship between the defects' attributes and the correction effort. The analysis confirmed the well-known principle “defects are the more expensive the later they are detected” by revealing that defects detected in the “system test” were 4.88 times more expensive than those detected in the “coding/unit test”. Another principle “defects are more expensive the longer they survive in software” was also confirmed by revealing that defects, which survived two or more development phases, were 4.44 times more expensive than those detected immediately. We also identified other factors, such as defect reproducibility, severity, and the cause of detection delay, that had a significant influence on the correction effort.

The purpose of this paper is to analyze the relation between the teaching materials and motivation to learn programming when the results of programming are works of art, and the students are from the art and digital design faculty. The programming environment used in this study was Processing which can produce fine artwork with relatively simple codes. The programming courses with Processing were offered at two universities, and the SIEM assessment standard was used to evaluate the students' motivation levels to learn programming. It has been verified from the analysis of the SIEM assessment standard that the desire of the students to create aesthetically satisfying artwork is important for the motivation to learn programming. Also, we have outlined the items that affected the change of the motivation index using the statistical analysis method.

The goal of this paper is to improve the prediction performance of fault-prone module prediction models (fault-proneness models) by employing over/under sampling methods, which are preprocessing procedures for a fit dataset. The sampling methods are expected to improve prediction performance when the fit dataset is unbalanced, i.e. there exists a large difference between the number of fault-prone modules and not-fault-prone modules. So far, there has been no research reporting the effects of applying sampling methods to fault-proneness models. In this paper, we experimentally evaluated the effects of four sampling methods (random over sampling, synthetic minority over sampling, random under sampling and one-sided selection) applied to four fault-proneness models (linear discriminant analysis, logistic regression analysis, neural network and classification tree) by using two module sets of industry legacy software. All four sampling methods improved the prediction performance of the linear and logistic models, while neural network and classification tree models did not benefit from the sampling methods. The improvements of Fl-values in linear and logistic models were 0.078 at minimum, 0.224 at maximum and 0.121 at the mean.

In this paper, we experimentally evaluated the effect of outlier detection methods to improve the prediction performance of fault-proneness models. Detected outliers were removed from a fit dataset before building a model. In the experiment, we compared three outlier detection methods (Mahalanobis outlier analysis (MOA), local outlier factor method (LOFM) and rule based modeling (RBM)) each applied to three well-known fault-proneness models (linear discriminant analysis (LDA), logistic regression analysis (LRA) and classification tree (CT)). As a result, MOA and RBM improved F1-values of all models (0.04 at minimum, 0.17 at maximum and 0.10 at mean) while improvements by LOFM were relatively small (-0.01 at minimum, 0.04 at maximum and 0.01 at mean).

This paper focuses on in-process measurements during requirements definition where measurements of processes and products are relatively difficult. However, development processes in Japan based on the enterprise architecture method provide standardized formats for such upstream processes and products, allowing in-process measurements. Based on previous work and on this examination of in-process measurements of requirements definition with the enterprise architecture method and previous results of empirical studies of in-process measurements and empirically validates of later development processes, this paper proposes a new measurement model, the "full in-process process and product (I-PAP) measurement model," which includes the complete software development process from requirements to maintenance. Standardization of the requirements definition phase using the enterprise architecture method in Japan allows in-process measurement across the complete development lifecycle. Combining this with collaborative filtering and a project benchmark database will support project evaluation, estimation, and prediction.

Analogy-based cost estimation provides a useful and intuitive means to support decision making in software project management. It derives a cost estimate required for completing a project from information about similar past projects, namely the analogues. While on average this method provides a relatively accurate cost estimate there remains a possibility of large estimation errors. In this paper, we empirically tested the hypothesis that "using more homogeneous analogues produces a more reliable cost estimate" using a software engineering data repository established by the software engineering center (SEC), Information-technology Promotion Agency, Japan. This testing showed that low and high homogeneity projects had a large variation in estimation reliability. For instance, the difference was 22.9% (p = 0.021) in terms of percentage to get accurate estimates (better than Median of Magnitude of Relative Error).

The technology of a home network system (HNS) allows integration of several kinds of home appliances to provide a user with value-added integrated services. Development of the integrated HNS services requires implementation of the appliance components (with APIs) and the services, according to each home-network environment. There are various implementation standards such as DLNA, ECHONET, OSGi and Jinifor the HNS applications. Therefore, even if a developer can choose the optimal one, it's very difficult to develop the integrated services by composing a new HNS implementation. In this paper, we propose a model-driven development of integrated HNS service applications. In our former manuscript, platform-independent design language for verifying HNS service scenarios was proposed. Our model-driven development method uses this design language as a meta-model of integrated HNS services. By model transformation to concrete implementation together with verification by SMV (symbolic model checking), productivity and quality of this kind of HNS applications are improved.

In this paper, we introduce SCACS, a Social Context-Aware Communication System that facilitates face-to-face communications between old-timers and newcomers in a research community. SCACS provides users with information on coauthor relationships collocutors have in order to help users understand collocutors' research background and relations to own. While the system works so as to help newcomers get better understandings on the research community by meeting old-timers---central to the community, it also works to recruit newcomers who might bring new ideas and research topics, in order to make the community sustainable. One of the contributions of the paper is to show an example of a fusion of social networking and ubiquitous computing technologies, which have attracted a considerable amount of attentions in the last few years. In contrast to exploiting social interactions in real world to enhance experiences of social networking services in virtual world, SCACS collects information on social networks (e.g., coauthor relationships networks) from virtual spaces (that is, databases), and then visualizes them to facilitate face-to-face communications among people in physical environments through using wearable interfaces. Instead of providing users with complex social network graphs, SCACS transforms network graphs into tree maps so that users are able to better understand the community.

This paper proposes to use eye movements to characterize the performance of individuals in reviewing source code of computer programs. We first present an integrated environment to measure and record the eye movements of the code reviewers. Based on the fixation data, the environment computes the line number of the source code that the reviewer is currently looking at. The environment can also record and play back how the eyes moved during the review process. We conducted an experiment to analyze 30 review processes (6 programs, 5 subjects) using the environment. As a result, we have identified a particular pattern, called scan, in the subjects' eye movements. Quantitative analysis showed that reviewers who did not spend enough time for the scan tend to take more time for finding defects.

The authors describe a new research framework for applying empirical software engineering methods in industrial practice and accomplishments in using it. The selected target for applying the framework is a governmentally funded software development project involving multiple vendors. This project involved in-process project data measurement in real time, data sharing with industry and academia, data analysis, and feedback to the project members. Today the project is in the system integration process. This paper shows the value of this research framework and describes issues of empirical data sharing between industry and academia which have emerged while using it. This experiment raised two major issues. One is the necessity of a new research framework for project measurement called the "Macro Measurement Tool". The other is effects of the software industry structure on this framework.

The purpose of this research is to detect low usability web pages from the behavior of users, such as browsing time, mouse movement and eye movement. We experimented to investigate the relation between the quantitative data viewing behavior of users and web usability evaluation by subjects. We analyzed the data to detect low usability web pages using discriminant analysis. Low usability web pages, 94.4% (17pages / 18pages = detectable pages / low usability pages) were detectable from the moving speed of gazing points and the amount of wheel rolling of a mouse. Moreover, this detection reduced the number of web pages which should be evaluated by half (46% = 89 pages / 192 pages = detected pages / all pages).

This paper presents a framework that adapts the conventional home electric appliances with the infrared remote controls (legacy appliances) to the emerging home network system (HNS). The proposed method extensively uses the concept of service-oriented architecture to improve programmable interoperability among multi-vendor appliances. We first prepare APIs that assist a PC to send infrared signals to the appliances. We then aggregate the APIs within self-contained service components, so that each of the component achieves a logical feature independent of device(or vendor)-specific operations. The service components are finally exported to the HNS as Web services. Thus, the legacy appliances can be used as distributed components with open interfaces. To demonstrate the effectiveness, we also implement an actual HNS and integrated services with multi-vendor legacy appliances

Software engineers have to select some appropriate development technologies to use in the work; however, engineers sometimes cannot find the appropriate technologies because there are vast amount of options today. To solve this problem, we propose a software technology recommendation method based on collaborative filtering (CF). In the proposed method, at first, questionnaires are collected from concerned engineers about their technical interest. Next, similarities between an active engineer who gets recommendation and the other engineers are calculated according to the technical interests. Then, some similar engineers are selected for the active engineer. At last, some technologies are recommended which attract the similar engineers. An experimental evaluation showed that the proposed method can make accurate recommendations than that of a naive (non-CF) method.

This paper presents a framework to specify and verify integrated services of a home network system (HNS). We first develop a modeling language to describe the HNS and the integrated services. Complementing our previous work, the language captures each appliance as an object consisting of properties and methods, encapsulating the underlying protocols and platforms. We then present a method that verifies the integrated services with symbolic model checking, by translating the proposed language into the SMV (symbolic model verifier) language. Thus, it is possible to validate if the integrated service is specified as intended, automatically and exhaustively. Using the proposed framework, service developers can effectively detect design flaws in a single integrated service, as well as feature interactions among multiple services, in early stages of service development.

The scale-free network shown in the small world phenomenon indicates that our human society consists of a small number of people who play the role of hubs linked with many nodes (persons) and a large number of people as nodes linked with few nodes. From our analysis of a large-scale online community - SourceForge.net - which has a large number of developers and projects, we have found that SourceForge also exists as a scale-free network. That is, only a minority of developers joins many projects and has rich links with other developers, while the majority joins few projects and has very limited social relations with others. The goal of our study is to build a system that supports knowledge collaboration in a large-scale online community of software development projects. In this paper, we discuss the challenges of supporting knowledge collaboration in such a large online community that is a scale-free network and then introduce the prototype system called D-SNS (dynamic social networking system).

This paper presents a method to implement integrated services of networked home electric appliances, which provide more convenient and comfortable living for home users. The conventional methods generally employ a home server to achieve the integrated services. The server controls all the networked appliances in a centralized manner. However, as the number of sophisticated appliances increases, the centralized server suffers from the concentration of load, as well as a decline in the reliability and interoperability. To cope with this problem, we adopt the service-oriented architecture (SOA) for the implementation of the integrated services. In the proposed framework, the appliances export own features as services, and autonomously execute the exported services one another. Thus, the appliances are loosely coupled via the exported services without any centralized home server, which enables more flexible, balanced and reliable integrated services. We first present a framework to design and implement the integrated services based on SOA, and then illustrate a prototype system developed with Web services. We also define three kinds of metrics (i.e., reliability, workload, and coupling), and conduct a comparative evaluation between the proposed and the previous systems.

The programmable service for Internet telephony (VoIP) allows end-users or third parties to define their own customized services. However, it imposes a serious drawback that service description created by end-users is likely to contain problems that are semantically ambiguous or inconsistent. To cope with this problem, we have so far proposed semantic warnings, which are the guidelines to guarantee the semantic correctness for the CPL (call processing language) programmable service environment. We evaluate the proposed semantic warnings with practical VoIP system, VOCAL (Vovida open communication application library). In the experiment, the proposed warnings revealed a semantic redundancy in a ready-made feature of VOCAL. It is also shown that customized features containing the semantic warnings often led VOCAL to problematic situations. Thus, the proposed warnings can help feature provisioning system to detect semantic flaws in programmable service environment.

In this paper, we present a new method to protect software against illegal acts of hacking. The key idea is to add a mechanism of self-modifying codes to the original program, so that the original program becomes hard to be analyzed. In the binary program obtained by the proposed method, the original code fragments we want to protect are camouflaged by dummy instructions. Then, the binary program autonomously restores the original code fragments within a certain period of execution, by replacing the dummy instructions with the original ones. Since the dummy instructions are completely different from the original ones, code hacking fails if the dummy instructions are read as they are. Moreover, the dummy instructions are scattered over the program, therefore, they are hard to be identified. As a result, the proposed method helps to construct highly invulnerable software without special hardware.

Software development project managers have already known that adding manpower to a late software project makes it later. This is a famous sentence in "Mythical man-month" written by Brooks (1975). The managers also recognize that even if two man-month workloads are assigned to two developers, the development period is not one month. However, the managers want to see when the two developers finish the two man-months workload. Of course, the managers don't think that the development period reduces from two months to one month. If the development period in the two developers is shorter than two months, the managers will decide to add the new developer to the work. Therefore, we propose a new workload estimation model based on communications among shared works. The model includes the Concept Model of UML of a software development project. The classes and relationships in the Concept Model present the kinds of works and relationships among the works, respectively. Sharing works causes increments of communication workload. The communication workload is derived from the relationships among the classes in the Concept Model. In case studies using the model, we were able to reconfirm quantitatively the famous sentence of "Mythical man-month".

Since some application software provides users with too many functions, it is often difficult to find those that are useful. This paper proposes a recommendation system based on a collaborative filtering approach to let users discover useful functions at low cost for the purpose of improving productivity when using application software. The proposed system automatically collects histories of software function execution (usage histories) from many users through the Internet. Based on the collaborative filtering approach, collected histories are used for recommending a set of candidate functions that may be useful to the individual user. This paper illustrates conventional filtering algorithms and proposes a new algorithm suitable for recommendation of software functions. The result of an experiment with a prototype recommendation system showed that the average ndpm of our algorithm was smaller than that of conventional algorithms, and it also showed that the standard deviation of ndpm of our algorithm was smaller than that of conventional algorithms. Furthermore, while every conventional algorithm had a case whose recommendation was worse than the random algorithm, our algorithm did not.

This paper proposes an efficient technique for eye gaze interface suitable for the general GUI environments such as Microsoft Windows. Our technique uses an eye and a hand together: the eye for moving cursors onto the GUI button (move operation), and the hand for pushing the GUI button (push operation). We also propose the following two techniques to assist the move operation: (1) Automatic adjustment and (2) Manual adjustment. In the automatic adjustment, the cursor automatically moves to the closest GUI button when we push a mouse button. In the manual adjustment, we can move the cursor roughly by an eye, then move it a little more by the mouse onto the GUI button. In the experiment to evaluate our method, GUI button selection by manual adjustment showed better performance than the selection by a mouse even in the situation that has many small GUI buttons placed very closely each other on the GUI.

Java programs distributed through the Internet are now suffering from program theft. This is because Java programs can be easily decomposed into reusable class files and even decompiled into source code by program users. We propose a practical method that discourages program theft by embedding Java programs with a digital watermark. Embedding a program developer's copyright notation as a watermark in Java class files will ensure the legal ownership of class files. Our embedding method is indiscernible by program users, yet enables us to identify an illegal program that contains stolen class files. The result of the experiment to evaluate our method showed most of the watermarks (20 out of 23) embedded in class files survived two kinds of attacks that attempt to erase watermarks: an obfuscactor attack, and a decompile-recompile attack.

Presents an experiment in understanding how learners of the Java programming language comprehend its concepts, such as classes, inheritance, interfaces, etc., in lectures and exercises. The authors used an empirical technique to test conjectures about how we learn the programming language. Usually, observations about how we learn a programming language are treated anecdotally. In this experiment, learners received lectures and did an exercise. The comprehension states of the learners were measured by tests in three learning phases. The first phase was before the lecture. In this phase, the learners had no knowledge of the programming language. The second phase was after the lecture and before the exercise. Here, the learners acquired some basic knowledge. The third phase was after the exercise. In this phase, the learners put the acquired knowledge into practice. Factor analysis was used to obtain factors affecting the test result of each learning phase. Changes in comprehension states are explained as a result of tracing the factors between the learning phases.

One of the major problems in object-oriented software projects is the lack of management's ability to comprehend and control the development progress of a project. This is because traditional phases of software development are not appropriate for object-oriented development. The project manager's "road map" is likely to be different with different phases, different milestones, and different checkpoints. This paper proposes a new framework which gives us a guideline for generating software process with relevant milestones for object-oriented development methods. The framework provides algorithms for identifying development phases and baseline products based on relationships among activities and products of the development method. In addition, the framework defines a software process model to manage a development progress in which milestones are established at the end of each phase in order to check the baseline product and establish goals for the following phase. Results of the application of the proposed framework show that the framework can generate a software process customized for well-known object-oriented development methods in a systematic way.

Empirical software engineering can be viewed as a series of actions to obtain knowledge and a better understanding about some aspects of software development, given a set of problem statements in the form of issues, questions or hypotheses. Experience has made us aware of the criticality of integrating the various types of data that are collected and analyzed as well as the criticality of integrating the various types of activities that take place, such as experiment design and the experiment itself. This has led us to develop a Computer-Aided Empirical Software Engineering (CAESE) framework to support the empirical software engineering lifecycle. The paper first presents the CAESE framework that consists of three elements: (1) a process model for the "lifecycle" of empirical software engineering studies, including needs analysis, experiment design, actual experimentation, and analyzing and packaging results; (2) a model that helps empirical software engineers decide how to look at the "world" to be studied in a coherent manner; (3) an architecture, based on which CAESE environments can be built, consisting of tool sets for each phase of the process model, a process management mechanism, and the two types of integration mechanism that are vital for handling multiple types of data: data integration and control integration. Next, the paper describes the Ginger2 environment as an instantiation of our framework. It concludes with reports on case studies using Ginger2, which dealt with a variety of empirical data types including mouse and keystrokes, eye traces, 3D movement, skin resistance level, and videotaped data.

This paper summarizes a previously developed space-time avoidance method, which applies to autonomous vehicles with temporal uncertainty, and which eliminates a collision threat by shifting vehicle's trajectory laterally as well as temporally. The shift parameters are obtained from simple geometric operations in metric 3D space-time. Though safe and simple, this method lacks efficiency for systems with little temporal uncertainty: the achieved planar collision distance is sometimes much larger than required. Therefore, this paper further proposes a specific, efficient solution for the case of 2 vehicles meeting without temporal uncertainty. Experimental results are presented in which the vehicles collide, or avoid each other mainly laterally or temporally, and by a mixture of both, depending on the choice of the time scaling factor.

Many of the non conventional software development methodologies (such as object-oriented analysis methodology) and tools (such as visual programming environment) have been applied in real life projects. These projects have been started without sufficient previous training given to the developers. An increment in the productivity has been seen as the projects progress. This paper proposes a simulation model for software development which can deal with variances of developers' productivity during software development. As the proposed model takes into account the developer's learning curve, it can be used to compute a developer's productivity and the quantity of gain to the developer's knowledge in executing an activity. The proposed model has been applied to four typical scenarios in our case study. The results show that it is highly practicable. An outline of a project planning prototype which is based on the proposed model is presented. The prototype can be used to make project plans which take the developer's learning curve into consideration.

In this paper we propose a new tactile sensing element, acoustic resonant tensor cell. The structure has only a spherical cavity in an elastic tactile sensor body with two fine ultrasound paths extended respectively to an ultrasound transmitter and receiver which are placed at the bottom of the elastomer. The acoustic resonant frequency of the air in the cavity has an explicit relation with the principal stresses around it, which is easily detected by the ultrasound transducers. The sensor is simple and elastic. The single structure presents useful tactile features of multidimensions. The principle and experimental results are described.

We describe an experiment investigating the distribution of failure intensity in software reliability growth models.We found that the assumption of conventional models that the failure intensity follows a gamma distribution is not always true.Our new software reliability model does not make this assumption; rather, the failure intensity is calculated from failure data.We show that our new model predicts more accurately the number of detected faults for our study project than the conventional models.

This paper describes concept and realization of IRoN (Inter Robot Network), a wireless communication network to support cooperation among mobile robots. From the point of view of the user (application programmer), IRoN has the following features: a list of surrounding robots with current position and speed is always available (by so-called implicit communication). Additionally, communication with any robot can be initiated from the user program explicitly (by so-called explicit communication). The paper also describes a simple application example: how simultaneous motion of two mobile robots can be achieved. Three different user program examples using different features of IRoN are described. The achieved motion delay ranges from about 1000 ms in the first user program, down to 50 ms in the third user program. IRoN will be used for further cooperation experiments involving three robots; a first experiment is reported in the conclusion.

While small computer systems have provided us with a wide array of development and operational environments, implementation methods, and domains of the software to be developed, it has also made the software development more risky. This paper proposes a new framework for improving the software development process on small computer systems to reduce software development risks. The key idea of the proposed framework is that the solutions to the software development problems can be extracted by the project manager, with relatively low overload, from the development processes that were actually executed in order to reduce the development risks, and then be formalized and stored in a database. The formalized solutions can be used not only for process improvement, but also for devising new solutions if new problems should occur in the future. We believe that the proposed framework is capable of making up for the difficulties in implementing CMM on small computer systems, and it encourages us to apply CMM to such risky software development.

One of the major problems in object-oriented software projects is the lack of management ability to comprehend and control the development progress of a project. This is because traditional phases of software development are not appropriate for object-oriented development. The project manager's "road map" is likely to be different with different phases, different milestones, and different checkpoints. The paper proposes a new framework which gives one a rigorous guideline for generating software process with relevant milestones for various kinds of software development methods, especially for object-oriented development methods. The framework provides algorithms for identifying development phases and baseline products based on relationships among activities and products of the development method. In addition, the framework defines a software process model to manage a development progress in which milestones are established at the end of each phase in order to check the baseline product and establish goals for the following phase. Results of the application of the proposed framework show that the framework can generate a software process customized for well-known object-oriented development methods in a systematic way.

In many cases of small scale software development, downstream activities are actually carried out before upstream activities have completely finished in order to reduce the development time and meet the deadline. However, overlapping activities also increase the total work effort because it demands more communications between activities. It is very important to be able to estimate the tradeoff between development time and total work effort since constraints may be placed on them. The paper proposes a simulation model to describe the overlapping development processes and the effects between overlapped activities. Based on this model, a software process simulator was developed. This simulator is applicable for supporting project planning (scheduling and staffing) with constraints on cycle time and total work effort.

This paper describes an actual experience of software process improvement at OMRON corporation.posed procedure are proven to be effective for reclucing the development effort at OM RON corporation.

The paper proposes a new method for increasing the reliability of multiversion software systems. The software using software breeding is more reliable than one using N version programming. But software breeding is not suitable for real time application because program versions are executed several times for detecting faulty modules. In the proposed method, the detection of faulty modules is performed in the background when program versions fail and the software continues the execution in the foreground. When the detection of faulty modules is finished, the combination of module versions in program versions are changed. Ten simulations, each of which executed program versions 10/sup 6/ times, were performed to analyse the effectiveness of the new method. This resulted in the reduction of the number of failures to range from 33% to 76% with an average of 56%.

This paper compares Software Breeding, a new approach for improving the reliability of multiversion software, with Community Error Recovery by mathematical models of the reliability of the multiversion software by these. The following results were obtained: (1) the reliability of the Software Breeding increases when the number of modules of the multiversion software increases, (2) the multiversion software by Software Breeding is more reliable than by the Community Error Recovery unless the reliability of a program version is low and (3) the Software Breeding can improve the reliability of multiversion software more easily than the Community Error Recovery since the Software Breeding does not need any additional routine which must have very high reliability except for the driver module.

A new metric for evaluating the cost effectiveness of technical reviews is described. The proposed metric is based on the degree to which testing costs are reduced by technical reviews. The metric can be interpreted as combining two conventional metrics. Using an experimental evaluation of the conventional metrics and the proposed metric for data collected in an industrial environment, the authors show the validity and usefulness of the proposed metric. In particular, they present a method to estimate a value of the proposed metric by using only the values obtained at review phase.>