Accurately assessing software vulnerabilities is essential for effective prioritization and remediation. While various scoring systems exist to support this task, their differing goals, methodologies and outputs often lead to inconsistent prioritization decisions. This work provides the first large-scale, outcome-linked empirical comparison of four publicly available vulnerability scoring systems: the Common Vulnerability Scoring System (CVSS), the Stakeholder-Specific Vulnerability Categorization (SSVC), the Exploit Prediction Scoring System (EPSS), and the Exploitability Index. We use a dataset of 600 real-world vulnerabilities derived from four months of Microsoft's Patch Tuesday disclosures to investigate the relationships between these scores, evaluate how they support vulnerability management task, how these scores categorize vulnerabilities across triage tiers, and assess their ability to capture the real-world exploitation risk. Our findings reveal significant disparities in how scoring systems rank the same vulnerabilities, with implications for organizations relying on these metrics to make data-driven, risk-based decisions. We provide insights into the alignment and divergence of these systems, highlighting the need for more transparent and consistent exploitability, risk, and severity assessments.

Background: Sensitive Information Exposure (SIEx) vulnerabilities (CWE-200) remain a persistent and underaddressed threat across software systems, often leading to serious security breaches. Existing detection tools rarely target the diverse subcategories of CWE-200 or provide context-aware analysis of code-level data flows. Aims: This paper aims to present SIExVulTS, a novel vulnerability detection system that integrates transformer-based models with static analysis to identify and verify sensitive information exposure in Java applications. Method: SIExVulTS employs a three-stage architecture: (1) an Attack Surface Detection Engine that uses sentence embeddings to identify sensitive variables, strings, comments, and sinks; (2) an Exposure Analysis Engine that instantiates CodeQL queries aligned with the CWE-200 hierarchy; and (3) a Flow Verification Engine that leverages GraphCodeBERT to semantically validate source-to-sink flows. We evaluate SIExVulTS using three curated datasets, including real-world CVEs, a benchmark set of synthetic CWE-200 examples, and labeled flows from 31 open-source projects. Results: The Attack Surface Detection Engine achieved an average$\mathbf{F 1}$score greater than$\mathbf{9 3 \%}$, the Exposure Analysis Engine achieved an F 1 score of 85.71 %, and the Flow Verification Engine increased precision from 22.61 % to 87.23 %. Moreover, SIExVulTS successfully uncovered six previously unknown CVEs in major Apache projects. Conclusions: The results demonstrate that SIExVulTS is effective and practical for improving software security against sensitive data exposure, addressing limitations of existing tools in detecting and verifying CWE-200 vulnerabilities.

Detecting conflicting requirements early in the software development lifecycle is crucial to mitigating risks of system failures and enhancing overall reliability. While Large Language Models (LLMs) have demonstrated proficiency in natural language understanding tasks, they often struggle with the nuanced reasoning required for identifying complex requirement conflicts. This paper introduces a novel framework, SAT-LLM, which integrates Satisfiability Modulo Theories (SMT) solvers with LLMs to enhance the detection of conflicting software requirements. SMT solvers provide rigorous formal reasoning capabilities, complementing LLMs' proficiency in natural language understanding. By synergizing these strengths, SAT-LLM aims to overcome the limitations of standalone LLMs in handling intricate requirement interactions. The early experiments provide empirical evidence supporting the effectiveness of our SAT-LLM over pure LLM-based methods like ChatGPT in identifying and resolving conflicting requirements. These findings lay a foundation for further exploration and refinement of hybrid approaches that integrate NLP techniques with formal reasoning methodologies to address complex challenges in software development.

We investigate the use of Natural Language Inference (NLI) in automating requirements engineering tasks. In particular, we focus on three tasks: requirements classification, identification of requirements specification defects, and detection of conflicts in stakeholders' requirements. While previous research has demonstrated significant benefit in using NLI as a universal method for a broad spectrum of natural language processing tasks, these advantages have not been investigated within the context of software requirements engineering. Therefore, we design experiments to evaluate the use of NLI in requirements analysis. We compare the performance of NLI with a spectrum of approaches, including prompt-based models, conventional transfer learning, Large Language Models (LLMs)-powered chatbot models, and probabilistic models. Through experiments conducted under various learning settings including conventional learning and zero-shot, we demonstrate conclusively that our NLI method surpasses classical NLP methods as well as other LLMs-based and chatbot models in the analysis of requirements specifications. Additionally, we share lessons learned characterizing the learning settings that make NLI a suitable approach for automating requirements engineering tasks.

Object serialization and deserialization are widely used for storing and preserving objects in files, memory, or database as well as for transporting them across machines, enabling remote interaction among processes and many more. This mechanism relies on reflection, a dynamic language that introduces serious challenges for static analyses. Current state-of-the-art call graph construction algorithms do not fully support object serialization/deserialization, i.e., they are unable to uncover the callback methods that are invoked when objects are serialized and deserialized. Since call graphs are a core data structure for multiple types of analysis (e.g., vulnerability detection), an appropriate analysis cannot be performed since the call graph does not capture hidden (vulnerable) paths that occur via callback methods. In this paper, we present Seneca, an approach for handling serialization with improved soundness in the context of call graph construction. Our approach relies on taint analysis and API modeling to construct sound call graphs. We evaluated our approach with respect to soundness, precision, performance, and usefulness in detecting untrusted object deserialization vulnerabilities. Our results show that Seneca can create sound call graphs with respect to serialization features. The resulting call graphs do not incur significant runtime overhead and were shown to be useful for performing identification of vulnerable paths caused by untrusted object deserialization.

Security requirements engineering is a manual and error-prone activity that is often neglected due to the knowledge gap between cybersecurity professionals and software requirements engineers. In this paper, we aim to automate the process of recommending and synthesizing security requirements specifications and therefore supporting requirements engineers in soliciting and specifying security requirements. We investigate the use of Relational Generative Adversarial Networks (GANs) in automatically synthesizing security requirements specifications. We evaluate our approach using a real case study of the Court Case Management System (CCMS) developed for the Indiana Supreme Court's Division of State Court Administration. We present an approach based on RelGAN to generate security requirements specifications for the CCMS. We show that RelGAN is practical for synthesizing security requirements specifications as indicated by subject matter experts. Based on this study, we demonstrate promising results for the use of GANs in the software requirements synthesis domain. We also provide a baseline for synthesizing requirements, highlight limitations and weaknesses of RelGAN and define opportunities for further investigations.

Prior research has shown that public vulnerability systems such as US National Vulnerability Database (NVD) rely on a manual, time-consuming, and error-prone process which has led to inconsistencies and delays in releasing final vulnerability results. This work provides an approach to curate vulnerability reports in real-time and map textual vulnerability reports to machine readable structured vulnerability attribute data. Designed to support the time consuming human analysis done by vulnerability databases, the system leverages the Common Vulnerabilities and Exposures (CVE) list of vulnerabilities and the vulnerability attributes described by the National Institute of Standards and Technology (NIST) Vulnerability Description Ontology (VDO) framework. Our work uses Natural Language Processing (NLP), Machine Learning (ML) and novel Information Theoretical (IT) methods to provide automated techniques for near real-time publishing, and characterization of vulnerabilities using 28 attributes in 5 domains. Experiment results indicate that vulnerabilities can be evaluated up to 95 hours earlier than using manual methods, they can be characterized with F-Measure values over 0.9, and the proposed automated approach could save up to 47% of the time spent for CVE characterization.

The notion of Attack Surface refers to the critical points on the boundary of a software system which are accessible from outside or contain valuable content for attackers. The ability to identify attack surface components of software system has a significant role in effectiveness of vulnerability analysis approaches. Most prior works focus on vulnerability techniques that use an approximation of attack surfaces and there have not been many attempts to create a comprehensive list of attack surface components. Although limited number of studies have focused on attack surface analysis, they defined attack surface components based on project specific hypotheses to evaluate security risk of specific types of software applications. In this study, we leverage a qualitative analysis approach to empirically identify an extensive list of attack surface components. To this end, we conduct a Grounded Theory (GT) analysis on 1444 previously published vulnerability reports and weaknesses with a team of three software developers and security experts. We extract vulnerability information from two publicly available repositories: 1) Common Vulnerabilities and Exposures (CVE) and 2) Common Weakness Enumeration (CWE). We ask three key questions: where the attacks come from, what they target, and how they emerge, and to help answer these questions we define three core categories for attack surface components: Entry points, Targets, and Mechanisms. We extract attack surface concepts related to each category from collected vulnerability information using the GT analysis and provide a comprehensive categorization that represents attack surface components of software systems from various perspectives. The paper introduces 254 new attack surface components that did not exist in the literature. The comparison of the proposed attack surface model with prior works indicates that only 6.7% of the identified Code level attack surface components are studied before.

Background: Automated unit and integration tests allow software development teams to continuously evaluate their application's behavior and ensure requirements are satisfied. Interest in explicitly testing security at the unit and integration levels has risen as more teams begin to shift security left in their workflows, but there is little insight into any potential pain points developers may experience as they learn to adapt their existing skills to write these tests. Aims: Identify security unit and integration testing pain points that could negatively impact efforts to shift security (testing) left to this level. Method: An mixed-method empirical study was conducted on 525 Stack Overflow and Security Stack Exchange posts related to security unit and integration testing. Latent Dirichlet Allocation (LDA) was applied to identify commonly discussed topics, pain points were learned through qualitative analysis, and links were analyzed to study commonly-shared resources. Results: Nine topics representing security controls, components, and scenarios were identified; Authentication was the most commonly tested control. Developers experienced seven pain points unique to security unit and integration testing, which were all influenced by the complexity of security control designs and implementations. Most linked resources were other Q&A posts, but repositories and documentation for security tools and libraries were also common. Conclusions: Developers may experience several unique pain points when writing tests at this level involving security controls. Additional resources are needed to guide developers through these challenges, which should also influence the creation of strategies and tools to help shift security testing to this level. To accelerate this, actionable recommendations for practitioners and future research directions based on these findings are highlighted.

Software designers and developers are increasingly relying on application frameworks as first-class design concepts. They instantiate the services that frameworks provide to implement various architectural tactics and patterns. One of the challenges in employing frameworks for such tasks is the difficulty of learning and correctly using the APIs of the frameworks. This paper introduces a learning-based approach called ARCODE to help novice programmers correctly use frameworks’ APIs to implement architectural tactics and patterns. ARCODE has several novel components: a graph-based approach for learning specification of a framework from a limited number of training software, a program analysis algorithm to eliminate erroneous training data, and a recommender module to help programmers use APIs correctly and identify API misuses in their program. We evaluated our technique across two popular frameworks: JAAS security framework used for authentication and authorization tactic and Java RMI framework used to enable remote method invocation between client and server and other object oriented patterns. Our results demonstrate (i) the feasibility of using ARCODE to learn the specification of a framework; (ii) ARCODE generates accurate recommendations for finding the next API call to implement an architectural tactic/pattern based on the context of the programmer’s code; (iii) it accurately detects API misuses in the code that implements a tactic/pattern and provides fix recommendations. We also demonstrate that ArCode outperforms two famous techniques, MAPO and GrouMiner, on API recommendation and misuse detection tasks.

Over the past three decades software engineering researchers have produced a wide range of techniques and tools for understanding the architectures of large, complex systems. However, these have tended to be one-off research projects, and their idiosyncratic natures have hampered research collaboration, extension and combination of the tools, and technology transfer. The area of software architecture is rich with disjoint research and development infrastructures, and datasets that are either proprietary or captured in proprietary formats. This paper describes a concerted effort to reverse these trends. We have designed and implemented a flexible and extensible infrastructure (SAIN) with the goal of sharing, replicating, and advancing software architecture research. We have demonstrated that SAIN is capable of incorporating the constituent tools extracted from three independently developed, large, long-lived software architecture research environments. We discuss SAIN's ambitious goals, the challenges we have faced in achieving those goals, the key decisions made in SAIN's design and implementation, the lessons learned from our experience to date, and our ongoing and future work.

Integrated development environments (IDE) play an important role in supporting developers during program comprehension and completion. Many of these supportive features focus on low-level programming and debugging activities. Unfortunately, there is less support in understanding and implementing architectural concerns in the form of patterns, tactics and/or other concerns. In this paper we present ArCode, a tool designed as a plugin for a popular IDE, IntelliJ IDEA. ArCode is able to learn correct ways of using frameworks' API to implement architectural concerns such as Authentication and Authorization from two sources, sample programs that practice the framework, and the byte code of that framework. Analyzing a program under development, this tool is able to find deviations from correct implementation and provide fix recommendations alongside with graphical demonstrations to better communicate the recommendations with the developers. We showcase how programmers can benefit from ArCode by providing an example of API misuse detection and API recommendation for a popular Java framework, Java Authentication and Authorization (JAAS) security framework.

Static analysis has the advantage of reasoning over multiple possible paths. Thus, it has been widely used for verification of program properties. Property verification often requires inter-procedural analysis, in which control and data flow are tracked across methods. At the core of inter-procedural analyses is the call graph, which establishes relationships between caller and callee methods. However, it is challenging to perform static analysis and compute the call graph of programs with dynamic features. Dynamic features are widely used in software systems; not supporting them makes it difficult to reason over properties related to these features. Although state-of-the-art research had explored certain types of dynamic features, such as reflection and RMI-based programs, serialization-related features are still not very well supported, as demonstrated in a recent empirical study. Therefore, in this paper, we introduce Salsa (Static AnaLyzer for SeriAlization features), which aims to enhance existing points-to analysis with respect to serialization-related features. The goal is to enhance the resulting call graph's soundness, while not greatly affecting its precision. In this paper, we report our early effort in developing Salsa and its early evaluation using the Java Call Graph Test Suite (JCG).

Automated test case generation tools have been successfully proposed to reduce the amount of human and infrastructure resources required to write and run test cases. However, recent studies demonstrate that the readability of generated tests is very limited due to (i) uninformative identifiers and (ii) lack of proper documentation. Prior studies proposed techniques to improve test readability by either generating natural language summaries or meaningful methods names. While these approaches are shown to improve test readability, they are also affected by two limitations: (1) generated summaries are often perceived as too verbose and redundant by developers, and (2) readable tests require both proper method names but also meaningful identifiers (within-method readability).

Authentication is a critical security feature for confirming the identity of a system's users, typically implemented with help from frameworks like Spring Security. It is a complex feature which should be robustly tested at all stages of development. Unit testing is an effective technique for fine-grained verification of feature behaviors that is not widely-used to test authentication. Part of the problem is that resources to help developers unit test security features are limited. Most security testing guides recommend test cases in a "black box" or penetration testing perspective. These resources are not easily applicable to developers writing new unit tests, or who want a security-focused perspective on coverage.

Software defect prediction models play a key role to increase the quality and reliability of software systems. Because, they are used to identify defect prone source code components and assist testing activities during the development life cycle. Prior research used supervised and unsupervised Machine Learning models for software defect prediction. Supervised defect prediction models require labeled data, however it might be time consuming and expensive to obtain labeled data that has the desired quality and volume. The unsupervised defect prediction models usually use clustering techniques to relax the labeled data requirement, however labeling detected clusters as defective is a challenging task. The Pareto principle states that a small number of modules contain most of the defects. Getting inspired from the Pareto principle, this work proposes a novel, unsupervised learning approach that is based on outlier detection. We hypothesize that defect prone software components have different characteristics when compared to others and can be considered as outliers, therefore outlier detection techniques can be used to identify them. The experiment results on 16 software projects from two publicly available datasets (PROMISE and GitHub) indicate that the k-Nearest Neighbor (KNN) outlier detection method can be used to identify the majority of software defects. It could detect 94% of expected defects at best case and more than 63% of the defects in 75% of the projects. We compare our approach with the state-of-the-art supervised and unsupervised defect prediction approaches. The results of rigorous empirical evaluations indicate that the proposed approach outperforms existing unsupervised models and achieves comparable results with the leading supervised techniques that rely on complex training and tuning algorithms.

Industrial control systems (ICS) are systems used in critical infrastructures for supervisory control, data acquisition, and industrial automation. ICS systems have complex, component-based architectures with many different hardware, software, and human factors interacting in real time. Despite the importance of security concerns in industrial control systems, there has not been a comprehensive study that examined common security architectural weaknesses in this domain. Therefore, this paper presents the first in-depth analysis of 988 vulnerability advisory reports for Industrial Control Systems developed by 277 vendors. We performed a detailed analysis of the vulnerability reports to measure which components of ICS have been affected the most by known vulnerabilities, which security tactics were affected most often in ICS and what are the common architectural security weaknesses in these systems. Our key findings were: (1) Human-Machine Interfaces, SCADA configurations, and PLCs were the most affected components, (2) 62.86% of vulnerability disclosures in ICS had an architectural root cause, (3) the most common architectural weaknesses were “Improper Input Validation”, followed by “Im-proper Neutralization of Input During Web Page Generation” and “Improper Authentication”, and (4) most tactic-related vulnerabilities were related to the tactics “Validate Inputs”, “Authenticate Actors” and “Authorize Actors”.

Preventing vulnerability exploits is a critical software maintenance task, and software engineers often rely on Common Vulnerability and Exposure (CVEs) reports for information about vulnerable systems and libraries. These reports include descriptions, disclosure sources, and manually-populated vulnerability characteristics such as root cause from the NIST Vulnerability Description Ontology (VDO). This information needs to be complete and accurate so stakeholders of affected products can prevent and react to exploits of the reported vulnerabilities. In this study, we demonstrate that VDO characteristics can be automatically detected from the textual descriptions included in CVE reports. We evaluated the performance of 6 classification algorithms with a dataset of 365 vulnerability descriptions, each mapped to 1 of 19 characteristics from the VDO. This work demonstrates that it is feasible to train classification techniques to accurately characterize vulnerabilities from their descriptions. All 6 classifiers evaluated produced accurate results, and the Support Vector Machine classifier was the best-performing individual classifier. Automating the vulnerability characterization process is a step towards ensuring stakeholders have the necessary data to effectively maintain their systems.

Through a set of well-defined interfaces, plug-and-play architectures enable additional functionalities to be added or removed from a system at its runtime. However, plug-ins can also increase the application’s attack surface or introduce untrusted behavior into the system. In this paper, we (1) use a grounded theory-based approach to conduct an empirical study of common vulnerabilities in plug-and-play architectures; (2) conduct a systematic literature survey and evaluate the extent that the results of the empirical study are novel or supported by the literature; (3) evaluate the practicality of the findings by interviewing practitioners with several years of experience in plug-and-play systems. By analyzing Chromium, Thunderbird, Firefox, Pidgin, WordPress, Apache OfBiz, and OpenMRS, we found a total of 303 vulnerabilities rooted in extensibility design decisions and observed that these plugin-related vulnerabilities were caused by 16 different types of vulnerabilities. Out of these 16 vulnerability types we identified 19 mitigation procedures for fixing them. The literature review supported 12 vulnerability types and 8 mitigation techniques discovered in our empirical study, and indicated that 5 mitigation techniques were not covered in our empirical study. Furthermore, it indicated that 4 vulnerability types and 11 mitigation techniques discovered in our empirical study were not covered in the literature. The interviews with practitioners confirmed the relevance of the findings and highlighted ways that the results of this empirical study can have an impact in practice.

Whether software architects should also code is an enduring question. In order to satisfy performance, security, reliability and other quality concerns, architects need to compare and carefully choose a combination of architectural patterns, styles or tactics. Then later in the development cycle, these architectural choices must be implemented completely and correctly so there will not be any drift from envisioned design. In this paper, we use data analytics-based techniques to study five large-scale software systems, examining the impact and the role of software architects who write code on software quality. Our quantitative study is augmented with a follow up interview of architects. This paper provides empirical evidence for supporting the pragmatic opinions that architects should write code. Our analysis shows that implementing architectural tactics is more complex than delivering functionality, tactics are more error prone than software functionalities, and the architects tend to introduce fewer bugs into the implementation of architectural tactics compared to the developers.

With the increasing popularity of open-source software development, there is a tremendous growth of software artifacts that provide insight into how people build software. Researchers are always looking for large-scale and representative software artifacts to produce systematic and unbiased validation of novel and existing techniques. For example, in the domain of software requirements traceability, researchers often use software applications with multiple types of artifacts, such as requirements, system elements, verifications, or tasks to develop and evaluate their traceability analysis techniques. However, the manual identification of rich software artifacts is very labor-intensive. In this work, we first conduct a large-scale study to identify which types of software artifacts are produced by a wide variety of open-source projects at different levels of granularity. Then we propose an automated approach based on Machine Learning techniques to identify various types of software artifacts. Through a set of experiments, we report and compare the performance of these algorithms when applied to software artifacts.

To satisfy security requirements, software architects often adopt security tactics. These architectural tactics provide mechanisms for resisting, detecting, reacting to and recovering from attacks. Consequently, flaws in the implementation of security tactics or their deterioration during software evolution and maintenance can introduce severe vulnerabilities that could be exploited by attackers. However, we currently lack an in-depth understanding of the types and impact of vulnerabilities related to security tactics. Therefore, in this paper, we conduct a first-of-its-kind in-depth case study involving three large-scale open-source systems. We investigate the most common types of vulnerabilities associated with security tactics, how frequently they may occur over time, and how fixing them differs from fixing vulnerabilities that are not related to security tactics. Key findings are (i) most tactic-related vulnerabilities were related to the tactics "Validate Inputs" and "Authorize Actors", (ii) vulnerabilities related to tactics have a similar distribution over time and software releases as vulnerabilities that are not related to tactics, (iii) fixing tactic-related vulnerabilities is not necessarily more complex than fixing vulnerabilities that are not related to security tactics. This study highlights the importance of ensuring an appropriate implementation of security-related design decisions in code to avoid vulnerabilities rooted in the architecture.

Architectural tactics such as heartbeat, resource pooling, and scheduling provide solutions to satisfy reliability, security, performance, and other critical characteristics of a software system. Current design practices advocate rigorous up-front analysis of the system's quality concerns to identify tactics and where in the code they should be used. In this paper, we explore a bottom-up approach to recommend architectural tactics based on latent topics discovered in the source code of projects. We present a recommender system developed by building predictor models which capture relationships between topical concepts in source code and the use of specific architectural tactics in that code. Based on an extensive analysis of over 116,000 open source systems, we identify significant correlations between latent topics in source code and the usage of architectural tactics. We use this information to construct a predictor for generating tactic recommendations. Our approach is validated through a series of experiments which demonstrate the ability to generate package-level tactic recommendations. We provide further validation via two large-scale studies of Apache Hive and Hadoop to illustrate that our recommender system predicts tactics that are actually implemented by developers in later releases.

Source code comprehension affects software development — especially its maintenance — where code reading is one of the most time-consuming activities. A programming language, together with the programming paradigm it supports, is a strong factor that profoundly impacts how programmers comprehend code. We conducted a human-subject controlled experiment to evaluate comprehension of code written using the Data Context Interaction (DCI) paradigm relative to code written with commonly used Object-Oriented (OO) programming. We used a new research-level language called Trygve which implements DCI concepts, and Java, a pervasive OO language in the industry. DCI revisits lost roots of the OO paradigm to address problems that are inherent to Java and most other contemporary OO languages. We observed correctness, time consumption, and locality of reference during reading comprehension tasks. We present a method which relies on the Eigenvector Centrality metric from Social Network Analysis to study the locality of reference in programmers by inspecting their sequencing of reading language element declarations and their permanence time in the code. Results indicate that DCI code in Trygve supports more comprehensible code regarding correctness and improves the locality of reference, reducing context switching during the software discovery process. Regarding reading time consumption, we found no statistically significant differences between both approaches.

Test case maintainability is an important concern, especially in open source and distributed development environments where projects typically have high contributor turn-over with varying backgrounds and experience, and where code ownership changes often. Similar to design patterns, patterns for unit testing promote maintainability quality attributes such as ease of diagnoses, modifiability, and comprehension. In this paper, we report the results of a large-scale study on the usage of four xUnit testing patterns which can be used to satisfy these maintainability attributes. This is a first-of-its-kind study which developed automated techniques to investigate these issues across 82,447 open source projects, and the findings provide more insight into testing practices in open source projects. Our results indicate that only 17% of projects had test cases, and from the 251 testing frameworks we studied, 93 of them were being used. We found 24% of projects with test files implemented patterns that could help with maintainability, while the remaining did not use these patterns. Multiple qualitative analyses indicate that usage of patterns was an ad-hoc decision by individual developers, rather than motivated by the characteristics of the project, and that developers sometimes used alternative techniques to address maintainability concerns.

Software datasets play a crucial role in advancing automated software traceability research. They can be used by researchers in different ways to develop or validate new automated approaches. The diversity and quality of the datasets within a research community have a significant impact on the accuracy, generalizability, and reproducibility of the results and consequently on the usefulness and practicality of the techniques under study. Collecting and assessing the quality of such datasets are not trivial tasks and have been reported as an obstacle by many researchers in the domain of software engineering. This paper presents a first-of-its-kind study to review and assess the datasets that have been used in software traceability research over the last fifteen years. It presents and articulates the current status of these datasets, their characteristics, and their threats to validity. Furthermore, this paper introduces a Traceability-Dataset Quality Assessment (T-DQA) framework to categorize software traceability datasets and assist researchers to select appropriate datasets for their research based on different characteristics of the datasets and the context in which those datasets will be used.

Automated traceability techniques based on supervised machine learning algorithms can significantly reduce the cost and effort needed to create and maintain traceability links between requirements, architecture and source code. However, the upfront cost to train these algorithms is the main bottleneck for expanding, and validating these traceability techniques as well as applying them to complex industrial systems. In this tool demo, we present our Web-based tool named BUDGET, as a solution to automate creation of training data for the problem of tracing architectural concerns. BUDGET uses Automated Web-Mining, and Big-Data Analysis techniques to generate training data for supervised architecture-traceability techniques. It uses several sampling strategies and mines ultra-large scale code repositories to generate datasets of tactical code snippets. The BUDGET falls in the research tool category and supports researchers in the area of software architecture and requirements engineering.

Software architectures are often constructed through a series of design decisions. In particular, architectural tactics are selected to satisfy specific quality concerns such as reliability, performance, and security. However, the knowledge of these tactical decisions is often lost, resulting in a gradual degradation of architectural quality as developers modify the code without fully understanding the underlying architectural decisions. In this paper we present a machine learning approach for discovering and visualizing architectural tactics in code, mapping these code segments to tactic traceability patterns, and monitoring sensitive areas of the code for modification events in order to provide users with up-to-date information about underlying architectural concerns. Our approach utilizes a customized classifier which is trained using code extracted from fifty performance-centric and safety-critical open source software systems. Its performance is compared against seven off-the-shelf classifiers. In a controlled experiment all classifiers performed well; however our tactic detector outperformed the other classifiers when used within the larger context of the Hadoop Distributed File System. We further demonstrate the viability of our approach for using the automatically detected tactics to generate viable and informative messages in a simulation of maintenance events mined from Hadoop's change management system.

The relationships and interdependencies between software requirements and the architectures of software-intensive systems are described in the Twin Peaks model. The fundamental idea of the Twin Peaks model is that Requirements Engineering and Software Architecture should not be treated in isolation. Instead, we need to progressively discover and specify requirements while concurrently exploring alternative architectural solutions. However, bridging the gap between Requirements Engineering and Software Architecture has mainly been discussed independently in the respective communities. Therefore, this ICSE workshop aims at bringing together researchers, practitioners and educators from the Requirements Engineering and Software Architecture fields to jointly explore the strong interdependencies between requirements and architecture. Based on the results from previous editions of the workshop, this edition focuses on agile software development contexts and on exploring lightweight techniques for integrating requirements and architectural thinking.

"Big Data" analytics has become the next hot topic for most companies - from financial institutions to technology companies to service providers. Likewise in software engineering, data collected about the development of software, the operation of the software in the field, and the users feedback on software have been used before. However, collecting and analyzing this information across hundreds of thousands or millions of software projects gives us the unique ability to reason about the ecosystem at large, and software in general. At no time in history has there been easier access to extremely powerful computational resources as it is today, thanks to the advances in cloud computing, both from the technology and business perspectives. In this technical briefing, we will present the state-of-the-art with respect to the research carried out in the area of big data analytics in software engineering research.

Conventional wisdom suggests that a software system's architecture has a significant impact on its evolution. Prior research has studied the evolution of software using the information of how its files have changed together in their revision history. No prior study, however, has investigated the impact of architecture on the evolution of software from its change history. This is mainly because most open-source software systems do not document their architectures. We have overcome this challenge using several architecture recovery techniques. We used the recovered models to examine if co-changes spanning multiple architecture modules are more likely to introduce bugs than co-changes that are within modules. The results show that the co-changes that cross architectural module boundaries are more correlated with defects than co-changes within modules, implying that, to improve accuracy, bug predictors should also take the software architecture of the system into consideration.

Android has grown to be the world's most popular mobile platform with apps that are capable of doing everything from checking sports scores to purchasing stocks. In order to assist researchers and developers in better understanding the development process as well as the current state of the apps themselves, we present a large dataset of analyzed open-source Android applications and provide a brief analysis of the data, demonstrating potential usefulness. This dataset contains 1,179 applications, including 4,416 different versions of these apps and 435,680 total commits. Furthermore, for each app we include the analytical results obtained from several static analysis tools including Androguard, Sonar, and Stowaway. In order to better support the community in conducting research on the security characteristics of the apps, our large analytical dataset comes with the detailed information including various versions of AndroidManifest.xml files and synthesized information such as permissions, intents, and minimum SDK. We collected 13,036 commits of the manifest files and recorded over 69,707 total permissions used. The results and a brief set of analytics are presented on our website: http://androsec.rit.edu.

Architectural qualities such as reliability, performance, and security, are often realized in a software system through the adoption of tactical design decisions such as the decision to use redundant processes, a heartbeat monitor, or a specific authentication mechanism. Such decisions are critical for delivering a system that meets its quality requirements. Despite the stability of high-level decisions, our analysis has shown that tactic-related classes tend to be modified more frequently than other classes and are therefore stronger predictors of change than traditional Object-Oriented coupling and cohesion metrics. In this paper we present the results from this initial study, including an analysis of why tactic-related classes are changed, and a discussion of the implications of these findings for maintaining architectural quality over the lifetime of a software system.

Research projects tend to evolve through multiple phases of incubation and experimentation, before maturing to levels of full industry adoption. Practice has shown that successful research solutions often take over 20 years to achieve full technology transfer. However, many projects never leave the incubation phase either because the new technique fails to perform well, or because researchers lack the knowledge, skills, or time to transition the idea to practice. A healthy research community could be expected to produce a steady stream of innovative solutions that positively impact industrial practice. To achieve these goals, we need ongoing, rigorous, and mutually beneficial conversations between academics and practitioners. Such exchanges are a desirable part of the research process, and will help the requirements engineering community to integrate technology transfer plans into the ongoing research plans. In this interactive panel, teams of researchers, representing different requirements engineering research areas, will present their research solutions to a panel of seasoned industrial practitioners. The practitioners provide insightful feedback that can help with the transition to practice. While Ready-Set-Transfer is presented as an interactive game-show, it has the serious goal of fostering collaboration and conversations between practitioners and researchers in the requirements engineering community.

Software architecture reconstruction plays an increasingly essential role in software engineering tasks such as architecture renovation, program comprehension, and change impact analysis. Various methods have been developed which use a software system's implementation-level artifacts to recover the architecture of the software. This tutorial will answer three fundamental questions about software architecture recovery: Why? What? and How? Through several examples it articulates and synthesizes technical forces and financial motivations that make software companies to invest in software architecture recovery. It discusses what are the pieces of design knowledge that can be recovered and lastly demonstrates a methodology as well as required tools for answering how to reconstruct architecture from implementation artifacts.

Software requirements specifications often focus on functionality and fail to adequately capture quality concerns such as security, performance, and usability. In many projects, quality-related requirements are either entirely lacking from the specification or intermingled with functional concerns. This makes it difficult for stakeholders to fully understand the quality concerns of the system and to evaluate their scope of impact. In this paper we present a data mining approach for automating the extraction and subsequent modeling of quality concerns from requirements, feature requests, and online forums. We extend our prior work in mining quality concerns from textual documents and apply a sequence of machine learning steps to detect quality-related requirements, generate goal graphs contextualized by project-level information, and ultimately to visualize the results. We illustrate and evaluate our approach against two industrial health-care related systems.

The quality of a software architecture is largely dependent upon the underlying architectural decisions at the framework, tactic, and pattern levels. Decisions to adopt certain solutions determine the extent to which desired qualities such as security, availability, and performance are achieved in the delivered system. In this tool demo, we present our Eclipse plug-in named Archie as a solution for maintaining architectural qualities in the design and code despite long-term maintenance and evolution activities. Archie detects architectural tactics such as heartbeat, resource pooling, and role-based access control (RBAC) in the source code of a project; constructs traceability links between the tactics, design models, rationales and source code; and then uses these to monitor the environment for architecturally significant changes and to keep developers informed of underlying design decisions and their associated rationales.

The disciplines of requirements engineering (RE) and software architecture (SA) are fundamental to the success of software projects. Even though RE and SA are often considered separately, it has been argued that drawing a line between RE and SA is neither feasible nor reasonable as requirements and architectural design processes impact each other. Requirements are constrained by what is feasible technically and also by time and budget restrictions. On the other hand, feedback from the architecture leads to renegotiating architecturally significant requirements with stakeholders. The topic of bridging RE and SA has been discussed in both the RE and SA communities, but mostly independently. Therefore, the motivation for this ICSE workshop is to bring both communities together in order to identify key issues, explore the state of the art in research and practice, identify emerging trends, and define challenges related to the transition and the relationship between RE and SA.

Nowadays, a successful software production is increasingly dependent on how the final deployed system addresses customers' and users' quality concerns such as security, reliability, availability, interoperability, performance and many other types of such requirements. In order to satisfy such quality concerns, software architects are accountable for devising and comparing various alternate solutions, assessing the trade-offs, and finally adopting strategic design decisions which optimize the degree to which each of the quality concerns is satisfied. Although designing and implementing a good architecture is necessary, it is not usually enough. Even a good architecture can deteriorate in subsequent releases and then fail to address those concerns for which it was initially designed. In this work, we present a novel traceability approach for automating the construction of traceabilty links for architectural tactics and utilizing those links to implement a change impact analysis infrastructure to mitigate the problem of architecture degradation. Our approach utilizes machine learning methods to detect tactic-related classes. The detected tactic-related classes are then mapped to a Tactic Traceability Pattern. We train our trace algorithm using code extracted from fifty performance-centric and safety-critical open source software systems and then evaluate it against a real case study.

Domain analysis is a labor-intensive task in which related software systems are analyzed to discover their common and variable parts. Many software projects include extensive domain analysis activities, intended to jumpstart the requirements process through identifying potential features. In this paper, we present a recommender system that is designed to reduce the human effort of performing domain analysis. Our approach relies on data mining techniques to discover common features across products as well as relationships among those features. We use a novel incremental diffusive algorithm to extract features from online product descriptions, and then employ association rule mining and the (k)-nearest neighbor machine learning method to make feature recommendations during the domain analysis process. Our feature mining and feature recommendation algorithms are quantitatively evaluated and the results are presented. Also, the performance of the recommender system is illustrated and evaluated within the context of a case study for an enterprise-level collaborative software suite. The results clearly highlight the benefits and limitations of our approach, as well as the necessary preconditions for its success.

The software architectures of business, mission, or safety critical systems must be carefully designed to balance an exacting set of quality concerns describing characteristics such as security, reliability, and performance. Unfortunately, software architectures tend to degrade over time as maintainers modify the system without understanding the underlying architectural decisions. Although this problem can be mitigated by manually tracing architectural decisions into the code, the cost and effort required to do this can be prohibitively expensive. In this paper we therefore present a novel approach for automating the construction of traceability links for architectural tactics. Our approach utilizes machine learning methods and lightweight structural analysis to detect tactic-related classes. The detected tactic-related classes are then mapped to a Tactic Traceability Information Model. We train our trace algorithm using code extracted from fifteen performance-centric and safety-critical open source software systems and then evaluate it against the Apache Hadoop framework. Our results show that automatically generated traceability links can support software maintenance activities while helping to preserve architectural qualities.

In many software intensive systems traceability is used to support a variety of software engineering activities such as impact analysis, compliance verification, and requirements validation. However, in practice, traceability links are often created towards the end of the project specifically for approval or certification purposes. This practice can result in inaccurate and incomplete traces, and also means that traceability links are not available to support early development efforts. We address these problems by presenting a trace recommender system which pushes recommendations to project stakeholders as they create or modify traceable artifacts. We also introduce the novel concept of a trace obligation, which is used to track satisfaction relations between a target artifact and a set of source artifacts. We model traceability events and subsequent actions, including user recommendations, using the Business Process Modeling Notation (BPMN). We demonstrate and evaluate the efficacy of our approach through an illustrative example and a simulation conducted using the software engineering artifacts of a robotic system for supporting arm rehabilitation. Our results show that tracking trace obligations and generating trace recommendations throughout the active phases of a project can lead to early construction of traceability knowledge.

Architectural tactics are important building blocks of software architecture. Tactics come in many shapes and sizes, describe solutions for addressing specific quality concerns, and are prevalent across high-performance fault-tolerant systems. Once a decision is made to utilize a tactic, the developer must generate a concrete plan for realizing the tactic in the design and code. Unfortunately, the variability points found in individual tactics can make this a challenging task. To address this knowledge gap, we conducted a study to investigate how design patterns were used to implement various tactics. Data mining techniques were used to identify potential pattern instances within tactic implementations. Our manual analysis of the retrieved data identified a distinct set of variability points for each tactic, as well as corresponding design patterns used to address them. From these observations we construct tactic-level decision trees depicting variability points of a tactic and generate a reference model which provides implementation guidance.

We present a recommender system that models and recommends product features for a given domain. Our approach mines product descriptions from publicly available online specifications, utilizes text mining and a novel incremental diffusive clustering algorithm to discover domain-specific features, generates a probabilistic feature model that represents commonalities, variants, and cross-category features, and then uses association rule mining and the k-Nearest-Neighbor machine learning strategy to generate product specific feature recommendations. Our recommender system supports the relatively labor-intensive task of domain analysis, potentially increasing opportunities for re-use, reducing time-to-market, and delivering more competitive software products. The approach is empirically validated against 20 different product categories using thousands of product descriptions mined from a repository of free software applications.

This thesis describes a Decision-Centric traceability framework that supports software engineering activities such as architectural preservation, impact analysis, and visualization of design intent. We present a set of traceability patterns, derived from studying real-world architectural designs in high-assurance and high-performance systems. We further present a trace-retrieval approach that reverse engineers design decisions and their associated traceability links by training a classifier to recognize fragments of design decisions and then using the traceability patterns to reconstitute the decisions from their individual parts.

Software architecture is shaped by a diverse set of interacting and competing quality concerns, each of which may have broad-reaching impacts across multiple architectural views. Without traceability support, it is easy for developers to inadvertently change critical architectural elements during ongoing system maintenance and evolution, leading to architectural erosion. Unfortunately, existing traceability practices, tend to result in the proliferation of traceability links, which can be difficult to create, maintain, and understand. We therefore present a decision-centric approach that focuses traceability links around the architectural decisions that have shaped the delivered system. Our approach, which is informed through an extensive investigation of architectural decisions made in real-world safety-critical and performance-critical applications, provides enhanced support for advanced software engineering tasks.

The software architectures of safety and mission-critical systems are designed to satisfy and balance an exacting set of quality concerns describing characteristics such as performance, reliability, and safety. Unfortunately, practice has shown that long-term maintenance activities can erode these architectural qualities. In this paper we present a novel solution for preserving architectural qualities through the use of Tactic Traceability Information Models (tTIMs). A tTIM provides a reusable infrastructure of traceability links focused around a commonly implemented architectural tactic, as well as a set of mapping points for tracing the tactic into the architectural design and the implemented code. The use of tTIMs significantly reduces the effort needed to create and maintain traceability links, provides support for visualizing the rationale behind various architectural components, and delivers timely information to maintainers so that they can preserve critical architectural qualities while implementing modifications. Our approach is described and evaluated within the context of a mission-critical software-intensive system.

A software architecture is carefully designed to satisfy the quality concerns of its stakeholders, and as such, represents a systematic and intricately balanced set of design decisions which deliver required qualities such as performance, reliability, and safety. In practice, architectural degradation tends to occur over the lifetime of the software system, as developers make ongoing and incremental maintenance changes to the system without knowledge of its underlying design decisions. Fortunately, this problem can be alleviated by establishing traceability between concrete elements in the architecture and their associated design decisions, and then using these traceability links to keep developers informed of relevant architectural tactics, styles, and design patterns throughout the development process. This paper focuses on the task of creating and using such traceability links. We present six trace creation patterns describing techniques and supporting structures for creating architecturally significant traceability links, and two usage patterns describing techniques for using the created links to help preserve qualities in the architectural design. The patterns described in this paper emerged from our experiences and observations of tracing architectural concerns in safety critical systems.