Large scale software systems are decomposed into gross-level components to reduce maintenance efforts. Software Architecture Recovery (SAR) approaches have been introduced to automatically cluster software modules for identifying modular components. In our previous work, we analyzed and hybridized various algorithms for clustering software modules. In addition to the employed clustering algorithm, SAR approaches are subject to many variations from multiple perspectives like the types of input and recovered artifacts. In this paper, we systematically review these variations. We aim at extending our work towards a unified approach for SAR in the future.

Several quality attributes like maintainability, reliability and security tend to degrade as software evolves. We aim at monitoring the impact of changes on software systems and identify potential vulnerabilities that are introduced by these changes. We apply static analysis on successive versions of source code to extract call graphs throughout its evolution. These graph models are analyzed and compared with each other to quantify the impact of software evolution and the risk for potential vulnerabilities. Graph edit distance metric is used for quantifying delta between graph models. The risk for security vulnerabilities is evaluated based on the dependencies of the source code on a set of functions that are known to be vulnerable, and distances of these functions to the entry points of the program in the call graph. We apply our approach on a set of open source projects. We show that versions with drastic changes and potential vulnerabilities can be highlighted.

Early effort estimation is both essential and challenging in software development projects. Accurate effort estimation mainly relies on accurate software sizing. COSMIC Function Point (CFP) is a means for measuring the functional size of software. It is a standardized method that can be applied to obtain size measurements based on early artifacts such as the software architecture design. In this paper, we present an industrial case study conducted to evaluate the correlation between CFP measurements performed at the architecture design phase and the actual development efforts. The study is performed in the context of a FinTech company that develops a system comprising a backend, a native iOS application, and a native Android application. We collect CFP calculations that are performed at the architecture design phase of the corresponding project. We also collect measurements regarding the actual effort for 12 months. Then, we investigate the correlation between CFP calculations and the measured effort. The actual effort can vary for some development tasks that are associated with the same CFP calculations. We analyze the relation of these variations with the level of developer experience and the type of target platform. Our results show that there is a significant correlation between size estimations made with CFP and the actual effort. The variance in effort per CFP among varying types of platforms is statistically significant. We also observe variance in effort per CFP among developers of varying expertise levels.

Abstract There have been success stories reported regarding the adoption of agile software development methods in the industry. There also exist observations on their limitations. One of these limitations is scalability since agile methods like Scrum were originally designed for small software teams. Scalable agile frameworks were introduced to address this limitation. We conducted an industrial case study on the adoption of such a framework, called Nexus. Our study involves quantitative and qualitative evaluation based on observations within a product development organization over a period of 12 months. Scrum is used for the development of a product during the first 6 months of this period. Nexus is used in the remaining 6 months. Data are collected throughout the whole period for measuring productivity, quality, and team member motivation. Results suggest a significant increase in productivity and product quality after switching to Nexus. Team motivation was slightly improved as well.

This paper proposes a new metric, namely the coevolution index (CEI), for measuring the relative evolutionary coupling of modules of a software system. CEI is inspired by the h-index, which is a popular metric used for measuring the productivity and citation impact of scholars and scientists. CEI of a module is equal to n, which is the number of times it is modified together with at least n other modules of the system. We develop a script that can calculate CEI for source files in a code repository. We analyze the repository of 4 software systems. Source files that are subject to a high number of changes to address issues tend to have high CEI scores. CEI also reflects a relative footprint in maintenance efforts by definition. Hence, it can help in tracking technical debt interest and focusing the refactoring efforts for improving maintainability and reusability.

Software continuously changes to address new requirements and to fix defects. Regression testing is performed to ensure that the applied changes do not adversely affect existing functionality. The increasing number of test cases makes it infeasible to execute the whole regression test suite. Test case selection is adopted to select a subset of the test suite, which is associated with the changed parts of the software. These parts are assumed to be error-prone. We present and evaluate a test case selection approach in the context of black-box regression testing of embedded systems. In this context, it is challenging to relate test cases with a set of distinct source code elements to be able to select those test cases associated with the modified parts of the source code. We analyze previously fixed defects for this purpose. We relate test cases that detect these defects with the source files that are previously modified for fixing them. Then, we select test cases related with source code files that are modified in the subsequent revision. The strength of this relation is determined as the number of changes associated with fixed defects previously detected by the same test cases. We conduct a case study on 3 real projects from the consumer electronics domain. Results show that it is possible to detect from 65% up to 85% of the defects detected by the whole test suite by selecting from 30% up to 70% of the test cases.

Accurate prediction of defect fix time is important for estimating and coordinating software maintenance efforts. Likewise, it is useful to predict whether or not the initially estimated defect fix time will be exceeded during the maintenance process. We present an empirical evaluation on the use of machine learning for predicting defect fix time violations. We conduct an industrial case study based on real projects from the telecommunications domain. We prepare a dataset with 69,000 defect reports regarding 293 projects being maintained between 2015 and 2021. We employ 7 machine learning algorithms. We experiment with 3 subsets of 25 features derived from defects as well as the corresponding projects. Gradient boosted classifiers perform the best by reaching up to 72% accuracy.

Architectural refactoring requires substantial effort. We introduce an approach and a tool to predict this effort prior to refactoring. We focus on PL/SQL programs that are developed as data access tiers of business software. There are two types of common refactoring needs for these programs. First, some of the modules might need to be migrated to a separate database. Second, some of the modules in the data tier might need to be migrated to the application tier. In both cases, the refactoring effort is proportional to the amount of coupling between the migrated modules and the rest of the modules in the database. Our tool can parse PL/SQL programs to reveal this coupling based on an analysis of SQL queries. Unlike prior studies, our tool can analyze queries that are created dynamically and that use multiple tables as well as PL/SQL-specific features. We evaluate our approach with an industrial PL/SQL program from the telecommunications domain. Our results are approved to be accurate by domain experts.

This is an extended abstract of the article: Okan Orsan Ozener and Hasan Sozer, “An Effective Formulation of the Multi-Criteria Test Suite Minimization Problem”, published in the Journal of Systems and Software, Vol. 168, pp. 110632, 2020. https://doi.org/10.1016/j.jss.2020.110632.

Test oracles differentiate between the correct and incorrect system behavior. Automation of test oracles for visual output systems mainly involves image comparison, where a snapshot of the output is compared with respect to a reference image. Hereby, the captured snapshot can be subject to variations such as scaling and shifting. These variations lead to incorrect evaluations. Existing approaches employ computer vision techniques to address a specific set of variations. In this article, we introduce ADVISOR, an adjustable framework for test oracle automation of visual output systems. It allows the use of a flexible combination and configuration of computer vision techniques. We evaluated a set of valid configurations with a benchmark dataset collected during the tests of commercial digital TV systems. Some of these configurations achieved up to 3% better overall accuracy with respect to state-of-the-art tools. Further, we observed that there is no configuration that reaches the best accuracy for all types of image variations. We also empirically investigated the impact of significant parameters. One of them is a threshold regarding image matching score that determines the final verdict. This parameter is automatically tuned by offline training. We evaluated runtime performance as well. Results showed that differences among the ADVISOR configurations and state-of-the-art tools are in the order of seconds per image comparison.

Anomalies during system execution can be detected by automated analysis of logs generated by the system. However, large scale systems can generate tens of millions of lines of logs within days. Centralized implementations of traditional machine learning algorithms are not scalable for such data. Therefore, we recently introduced a distributed log analysis framework for anomaly detection. In this paper, we introduce an extension of this framework, which can detect anomalies earlier via incremental analysis instead of the existing offline analysis approach. In the extended version, we periodically process the log data that is accumulated so far. We conducted controlled experiments based on a benchmark dataset to evaluate the effectiveness of this approach. We repeated our experiments with various periods that determine the frequency of analysis as well as the size of the data processed each time. Results showed that our online analysis can improve anomaly detection time significantly while keeping the accuracy level same as that is obtained with the offline approach. The only exceptional case, where the accuracy is compromised, rarely occurs when the analysis is triggered before all the log data associated with a particular session of events are collected.

Summary System logs constitute a rich source of information for detection and prediction of anomalies. However, they can include a huge volume of data, which is usually unstructured or semistructured. We introduce DILAF, a framework for distributed analysis of large‐scale system logs for anomaly detection. DILAF is comprised of several processes to facilitate log parsing, feature extraction, and machine learning activities. It has two distinguishing features with respect to the existing tools. First, it does not require the availability of source code of the analyzed system. Second, it is designed to perform all the processes in a distributed manner to support scalable analysis in the context of large‐scale distributed systems. We discuss the software architecture of DILAF and we introduce an implementation of it. We conducted controlled experiments based on two datasets to evaluate the effectiveness of the framework. In particular, we evaluated the performance and scalability attributes under various degrees of parallelism. Results showed that DILAF can maintain the same accuracy levels while achieving more than 30% performance improvement on average as the system scales, compared to baseline approaches that do not employ fully distributed processing.

Anomaly detection is a valuable feature for detecting and diagnosing faults in large-scale, distributed systems. These systems usually provide tens of millions of lines of logs that can be exploited for this purpose. However, centralized implementations of traditional machine learning algorithms fall short to analyze this data in a scalable manner. One way to address this challenge is to employ distributed systems to analyze the immense amount of logs generated by other distributed systems. We conducted a case study to evaluate two unsupervised machine learning algorithms for this purpose on a benchmark dataset. In particular, we evaluated distributed implementations of PCA and K-means algorithms. We compared the accuracy and performance of these algorithms both with respect to each other and with respect to their centralized implementations. Results showed that the distributed versions can achieve the same accuracy and provide a performance improvement by orders of magnitude when compared to their centralized versions. The performance of PCA turns out to be better than K-means, although we observed that the difference between the two tends to decrease as the degree of parallelism increases.

We introduce a prototype testing framework as an extension of JUnit for testing actor-based systems. Our framework runs a given JUnit test in various schedules and records the execution trace for each run. In case a failure is observed during one of the test runs, the corresponding execution trace is compared with those traces associated with passed test runs. The sequence of exchanged messages that are different with respect to these runs are extracted. A UML sequence diagram is automatically generated to depict this sequence. Hence, the developer can observe the interleaving of messages that caused the failure. We illustrated the application of our framework on two sample actor-based software systems.

We introduce a tool that facilitates the reuse of test models for a family of systems. Test models are defined as hierarchical Markov chains for specifying possible usage of features for these systems. Variability among these features is documented with a feature model. Optional and alternative features in the feature model are mapped to a set of states in test models. Transition probabilities for these states are then modified according to the selected features so that the generated test cases focus only on these features. We conducted a controlled experiment in an industrial setting to evaluate the effectiveness of the tool. We observed that the tool reduces costs significantly and the cost of product line engineering adoption can be amortized for a small product family with 13 products.

Summary Static code analysis tools automatically generate alerts for potential software faults that can lead to failures. However, these tools usually generate a very large number of alerts, some of which are subject to false positives. Because of limited resources, it is usually hard to inspect all the alerts. As a complementary approach, runtime verification techniques verify dynamic system behavior with respect to a set of specifications. However, these specifications are usually created manually based on system requirements and constraints. In this paper, we introduce a noval approach and a toolchain for integrated static code analysis and runtime verification. Alerts that are generated by static code analysis tools are utilized for automatically generating runtime verification specifications. On the other hand, runtime verification results are used for automatically generating filters for static code analysis tools to eliminate false positives. The approach is illustrated for the static analysis and runtime verification of an open‐source bibliography reference manager software. Copyright © 2014 John Wiley & Sons, Ltd.

SUMMARY In SOA, composite services depend on a set of partner services to perform their tasks. These partner services may become unavailable because of system and/or network faults, leading to an increased error rate for the composite service. In this paper, we propose an approach to prevent the occurrence of errors that result from the unavailability of partner services. We introduce an external Web service, dubbed Fault Avoidance Service (FAS), to which composite services can register at will. After registration, FAS periodically checks the partner links, detects unavailable partner services, and updates the composite service with available alternatives. Thus, in case of a partner service error, the composite service will have been updated before attempting an ill‐destined request. We provide mathematical analysis regarding the error rate and the false positive rate with respect to the monitoring frequency of FAS for two models. We obtained empirical results by conducting several tests on the Amazon Elastic Compute Cloud to evaluate our mathematical analyses. We also introduce an industrial case study for improving the quality of a service‐oriented system from the broadcasting and content delivery domain. Copyright © 2014 John Wiley & Sons, Ltd.

Due to copyright restrictions, the access to the full text of this article is only available via subscription.

In practice, inconsistencies between architectural documentation and the code might arise due to improper implementation of the architecture or the separate, uncontrolled evolution of the code. Several approaches have been proposed to detect inconsistencies between the architecture and the code but these tend to be limited for capturing inconsistencies that might occur at runtime. We present a runtime verification approach for detecting inconsistencies between the dynamic behavior of the documented architecture and the actual runtime behavior of the system. The approach is supported by a set of tools that implement the architecture and the code patterns in Prolog, and automatically generate runtime monitors for detecting inconsistencies. We illustrate the approach and the toolset for a Crisis Management System case study.

Each software architecture design is the result of a broad set of design decisions and their justifications, that is, the design rationale. Capturing the design rationale is important for a variety of reasons such as enhancing communication, reuse and maintenance. Unfortunately, it appears that there is still a lack of appropriate methods and tools for effectively capturing and managing the architecture design rationale. In this paper we present a feature-based rationale management approach and the corresponding tool environment ArchiRationale for supporting software architecture adaptation. The approach takes as input an existing architecture and captures the design rationale for adapting the architecture for a given quality concern. For this we define a feature model that includes the possible set of architectural tactics to realize the quality concern. The presented approach captures the rationale for deciding on feature selections and for selecting the corresponding architecture design alternatives. ArchiRationale customizes and integrates the Eclipse plugin tools XFeature, ArchStudio and XQuery to provide tool support for capturing, storing and accessing the design rationale. We illustrate the approach for adapting a software architecture for fault tolerance.

Abstract The decomposition of software architecture into modular units is usually driven by the required quality concerns. In this paper we focus on the impact of local recovery concern on the decomposition of the software system. For achieving local recovery, the system needs to be decomposed into separate units that can be recovered in isolation. However, it appears that this required decomposition for recovery is usually not aligned with the decomposition based on functional concerns. Moreover, introducing local recovery to a software system, while preserving the existing decomposition, is not trivial and requires substantial development and maintenance effort. To reduce this effort we propose a framework that supports the decomposition and implementation of software architecture for local recovery. The framework provides reusable abstractions for defining recoverable units and the necessary coordination and communication protocols for recovery. We discuss our experiences in the application and evaluation of the framework for introducing local recovery to the open‐source media player called MPlayer. Copyright © 2009 John Wiley & Sons, Ltd.

An analysis of the current practice for representing architectural views reveals that they focus mainly on functional concerns and are limited when considering quality concerns. We introduce the recovery style for modeling the structure of the system related to the recovery concern. The recovery style is a specialization of the module viewtype in the Views&Beyond approach. It is used to communicate and analyze architectural design decisions and to support detailed design with respect to recovery. We illustrate the style for modeling the recovery views for the open-source software, MPlayer.

Bluetooth networks can be constructed as piconets or scatternets depending on the number of nodes in the network, Although piconet construction is a well-defined process specified in Bluetooth standards, scatternet formation policies and algorithms are not well specified. Among many solution proposals for this problem, only a few of them focus on efficient usage of bandwidth in the resulting scatternets. In this paper, we propose a distributed algorithm for the scatternet formation problem that dynamically constructs and maintains a scatternet based on estimated traffic flow rates between nodes. The algorithm is adaptive to changes and maintains a constructed scatternet for bandwidth-efficiency when nodes come and go or when traffic flow rates change. Based on simulations, the paper also presents the improvements in bandwidth-efficiency and reduction in energy consumption provided by the proposed algorithm

Bluetooth networks can be constructed as piconets or scatternets depending on the number of nodes in the network. Although piconet construction is a well-defined process specified in Bluetooth standards, scatternet construction policies and algorithms are not well specified. Among many solution proposals for this problem, only a few of them focus on efficient usage of bandwidth in the resulting scatternets. In this paper, we propose a distributed algorithm for the scatternet construction problem, that dynamically constructs and maintains a scatternet based on estimated traffic flow rates between nodes. The algorithm is adaptive to changes and maintains a constructed scatternet for bandwidth-efficiency when nodes come and go or when traffic flow rates change. Based on simulations, the paper also presents the improvements in bandwidth-efficiency provided by the proposed algorithm.

We propose a Software Architecture Reliability Analysis (SARA) approach that benefits from both reliability engineering and scenario-based software architecture analysis to provide an early reliability analysis of the software architecture. SARA makes use of failure scenarios that are prioritized with respect to the user-perception in order to provide a severity analysis for the software architecture and the individual components.