• We introduced a fully automated framework that quantifies the relationship between microservice architecture complexity, derived from multiple types of statically detected dependencies, and performance-related quality attributes. • We analyzed five complex benchmark systems, including four structurally distinct releases of the Train-Ticket microservice benchmark and one instance of the DeathStarBench suite. • Regarding the relationship between microservice-architecture complexity and overall performance, our analysis demonstrated that systems with poor structural complexity, reflected in high propagation cost and high Clique ratios, exhibited reduced scalability and supported fewer user requests. • Regarding the relationship between individual service complexity and their performance, our results show that endpoints with higher coupling scores tend to have longer response times. Using Spearman’s rank correlation to assess the correlation between each endpoint’s coupling score and its performance score, we find a statistically significant positive correlation. A microservice architecture is intended to promote modularity and evolvability. In this paper, we present an automated framework for assessing the relationship between microservice architecture complexity and performance-related quality attributes. In this framework, we use PPTAM, a performance testing tool, to evaluate system response time under varying user loads, and DV8, an architecture analysis tool, to assess architectural complexity and the complexity of individual services using coupling scores, propagation cost, and architectural antipatterns derived from various types of dependency relations. Using this approach, we evaluated five benchmark systems, including four releases of a microservice system that share similar functionalities but differ in structural design. The results show that microservice architectures with poor complexity scores also exhibited degraded performance outcomes. This automated framework, for the first time, enables a comprehensive measurement of microservice architecture complexity, formed through multiple types of statically extracted dependencies, and its correlation with dynamically obtained performance metrics.

The microservice architecture structures an application as a collection of small autonomous services, enhancing development practices and maintainability. It impacts system performance and scalability, which are influenced by the architectural design, the system deployment, and usage. In this paper, we present a systematic literature review (SLR) on the assessment of performance and scalability in the microservice architecture, aiming to identify research gaps and assess the current state of knowledge. The review protocol screened 801 studies, selecting 29 primary studies for detailed analysis. We perform comparisons of studies and try to identify the main gaps in current research and suggest areas for further investigation. The main conclusion is that the current research partially addresses the dimensions associated with the performance and scalability qualities of microservice systems. In particular, there is a lack of comparative studies of architectural patterns and styles or comparable assessment models and methods. Therefore, a systematic approach is lacking and the paper reports a set of open research trends to guide researchers in the area of microservice-based software architecture quality assessment.

Context. Software Performance Antipatterns (SPAs) research has focused on algorithms for the characterization, detection, and solution of antipatterns. However, existing algorithms are based on the analysis of runtime behavior to detect trends on several monitored variables (e.g., response time, CPU utilization, and number of threads) using pre-defined thresholds. Objective. In this paper, we introduce a new approach for SPA characterization and detection designed to support continuous integration/delivery/deployment (CI/CDD) pipelines, with the goal of addressing the lack of computationally efficient algorithms.

The Workshop on Education and Practice of Performance Engineering, in its 3rd edition, brings together University researchers and Industry Performance Engineers to share education and practice experiences. The recommendations from previous WEPPE workshops pointed to the need to form a team of critical thinkers that also have good communication skills.

Microservices have emerged as an architectural style for developing distributed applications. Assessing the performance of architecture deployment configurations — e.g., with respect to deployment alternatives — is challenging and must be aligned with the system usage in the production environment. In this paper, we introduce an approach for using operational profiles to generate load tests to automatically assess scalability pass/fail criteria of microservice configuration alternatives. The approach provides a Domain-based metric for each alternative that can, for instance, be applied to make informed decisions about the selection of alternatives and to conduct production monitoring regarding performance-related system properties, e.g., anomaly detection. We have evaluated our approach using extensive experiments in a large bare metal host environment and a virtualized environment. First, the data presented in this paper supports the need to carefully evaluate the impact of increasing the level of computing resources on performance. Specifically, for the experiments presented in this paper, we observed that the evaluated Domain-based metric is a non-increasing function of the number of CPU resources for one of the environments under study. In a subsequent series of experiments, we investigate the application of the approach to assess the impact of security attacks on the performance of architecture deployment configurations.

This paper provides a summary of the First International Workshop on Model Based Design for Cyber- Physical Systems (MB4CP 2015) in conjunction with DSN 2015 conference in Rio de Janeiro, Brazil.

In this paper we propose a new framework to assess survivability of software projects accounting for media capability details as introduced in Media Synchronicity Theory (MST). Specifically, we add to our global engineering framework the assessment of the impact of inadequate conveyance and convergence available in the communication infrastructure selected to be used by the project, on the system ability to recover from project disasters. We propose an analytical model to assess how the project recovers from project disasters related to process and communication failures. Our model is based on media synchronicity theory to account for how information exchange impacts recovery. Then, using the proposed model we evaluate how different interventions impact communication effectiveness. Finally, we parameterize and instantiate the proposed survivability model based on a data gathering campaign comprising thirty surveys collected from senior global software development experts at ICGSE'2014 and GSD'2015.

Ensuring system survivability in the wake of advanced persistent threats is a big challenge that the security community is facing to ensure critical infrastructure protection. In this paper, we define metrics and models for the assessment of coordinated massive malware campaigns targeting critical infrastructure sectors. First, we develop an analytical model that allows us to capture the effect of neighborhood on different metrics (infection probability and contagion probability). Then, we assess the impact of putting operational but possibly infected nodes into quarantine. Finally, we study the implications of scanning nodes for early detection of malware (e.g., worms), accounting for false positives and false negatives. Evaluating our methodology using a small four-node topology, we find that malware infections can be effectively contained by using quarantine and appropriate rates of scanning for soft impacts.

Software projects that are engineered using global software development techniques are required to deploy processes and tools to support collaboration over large geographies. Specifically, these projects involve the deployment of processes and tools to support project management, communication, and risk management. Whereas a traditional software development effort can use standard processes and tools to support communication and collaboration, global software development efforts require a unified and holistic project management, development process, collaboration, and communication approach taking into consideration the interplay of time zones, number of sites and cultural diversity. In this paper, we introduce a novel approach for modeling and quantification of global software engineering frameworks. In our approach, we apply transient survivability metrics to support the design of global software engineering projects. Therefore, our approach combines survivability analysis and global software engineering frameworks analysis. The survivability metric used in this paper is the time required to recover from a software project disaster (e.g., break down in communication between remote teams) for a given global software engineering framework. The global software engineering modeling framework we use is composed of models supporting the evaluation of communication tools, software development processes and cultural diversity management. We illustrate the application of our approach by applying it to the analysis of an example derived from a real global software engineering project. Our results indicate that the combination of survivability analysis and modeling of global software engineering frameworks can provide meaningful insights when designing global software engineering frameworks.

SUMMARY Smart grids are fostering a paradigm shift in the realm of power distribution systems. Whereas traditionally different components of the power distribution system have been provided and analyzed by different teams through different lenses, smart grids require a unified and holistic approach that takes into consideration the interplay of communication reliability, energy backup, distribution automation topology, energy storage, and intelligent features such as automated fault detection, isolation, and restoration (FDIR) and demand response. In this paper, we present an analytical model and metrics for the survivability assessment of the distribution power grid network. The proposed metrics extend the system average interruption duration index, accounting for the fact that after a failure, the energy demand and supply will vary over time during a multi‐step recovery process. The analytical model used to compute the proposed metrics is built on top of three design principles: state space factorization, state aggregation, and initial state conditioning. Using these principles, we reduce a Markov chain model with large state space cardinality to a set of much simpler models that are amenable to analytical treatment and efficient numerical solution. In case demand response is not integrated with FDIR, we provide closed form solutions to the metrics of interest, such as the mean time to repair a given set of sections. Under specific independence assumptions, we show how the proposed methodology can be adapted to account for multiple failures. We have evaluated the presented model using data from a real power distribution grid, and we have found that survivability of distribution power grids can be improved by the integration of the demand response feature with automated FDIR approaches. Our empirical results indicate the importance of quantifying survivability to support investment decisions at different parts of the power grid distribution network. Copyright © 2014 John Wiley & Sons, Ltd.

No abstract available.

Smart grids are fostering a paradigm shift in the realm of power distribution systems. Whereas traditionally different components of the power distribution system have been provided and analyzed by different teams, smart grids require a unified and holistic approach taking into consideration the interplay of distributed generation, distribution automation topology, intelligent features, and others. In this paper, we use transient survivability metrics to create better distribution automation network designs. Our approach combines survivability analysis and power flow analysis to assess the survivability of the distribution power grid network. Additionally, we present an initial approach to automatically optimize available investment decisions with respect to survivability and investment costs. We have evaluated the feasibility of this approach by applying it to the design of a real distribution automation circuit. Our empirical results indicate that the combination of survivability analysis and power flow can provide meaningful investment decision support for power systems engineers.

Smart grids are fostering a paradigm shift in the realm of power distribution systems. Whereas traditionally different components of the power distribution system have been provided and analyzed by different teams through different lenses, smart grids require a unified and holistic approach that takes into consideration the interplay of communication reliability, energy backup, distribution automation topology, energy storage and intelligent features such as automated failure detection, isolation and restoration (FDIR) and demand response.

Smart grids are fostering a paradigm shift in the realm of power distribution systems. Whereas traditionally different components of the power distribution system have been provided and analyzed by different teams, smart grids require a unified and holistic approach taking into consideration the interplay of distributed generation, distribution automation topology, intelligent features, and others.

Special issue on global software engineeringOver the past 10 years, Global Software Engineering (GSE) and Development has traversed through several stages commonly experienced by new practices.Industry began the practice first as a way to manage development costs and then later with the hope that it could provide workforce flexibility and better allocation of expertise.Researchers began by categorizing different versions of the practice and exploring whether GSE could generally be successful or not.As the practice continued, successes and failures were studied to determine the conditions that led to each, and these results were melded with practical, Industry experience into guidelines for successful use.The utilization of GSE is now relatively mature, and the study of it has evolved into a more sophisticated and nuanced discipline.Although once, the big research question was: does GSE work?-the questions now are much more varied and utilize a much wider set of approaches and viewpoints.Moreover, current research sits atop and draws from a much larger history of established empirical findings.The seven papers in this Special Issue on GSE provide a good representation of the current state of GSE practice and research and demonstrate recent trends.Earlier and more abbreviated versions of these articles appeared in the 5th Annual International Conference on Global Software Engineering, held in Princeton, USA in August, 2010.As a discipline matures, it often finds important general principles by studying unique, specialpurpose instances; the unique instance can free the researcher for important insights that might seem too commonplace if discovered in the mainstream process.The paper by Smite and Wohlin entitled "Lessons Learned from Transferring Software Products to India" is a good example of this research methodology.The paper focuses on projects that begin in one country and then are moved to another, typically for reasons of cost and expertise-flexibility.However, from this specific practice comes a set of guidelines that apply to many aspects of GSE, for example, adjusting processes to people and transferring people to maintain work continuity.The paper by da Silva, Prikladnicki, França, Monteiro, Costa and Rocha entitled "An Evidencebased Model of Distributed Software Development Project Management: Results from a Systematic Mapping Study" demonstrates two recent methodological trends in GSE research.First, with a more substantial literature base from which to draw conclusions, GSE researchers can now analyze trends across whole groupings of studies.This paper provides a systematic and in-depth analytic review of 70 research papers in the field with an eye toward understanding the unique challenges faced by project managers in a distributed software development context as well as the tools and practices most often used to meet those challenges.A second current trend in GSE research has to do with the desire to generalize findings from individual studies into a broader conceptual model, and this paper attempts to model global project management by casting challenges, tools, and practices into a cause and effect relationships.These authors' model proposes how the effectiveness of project management practices might be influenced by geographic, temporal, social, and personal distances through mediators, such as trust, cooperation, and information sharing.The methodological trend of systematic review is again demonstrated in the paper by Jalali and Wohlin entitled "Global Software Engineering and Agile Practices: A Systematic Review".In this case, the topical focus is the employment of Agile methods in a globally distributed context.It is worth noting that this combination at one point seemed nearly impossible because of Agile method's emphasis on frequent face-to-face communication.But like most engineering methods, Agile represents a broad suite of practices, and this paper shows that some can be effective in a distributed environment, whereas others are often not.One advantage of the systematic review methodology is that it can analyze what kinds of studies contribute to conclusions.In this case, the

In this paper, we report on our experience with the application of validated models to assess performance, reliability, and adaptability of a complex mission critical system that is being developed to dynamically monitor and control the position of an oil-drilling platform. We present real-time modeling results that show that all tasks are schedulable. We performed stochastic analysis of the distribution of tasks execution time as a function of the number of system interfaces. We report on the variability of task execution times for the expected system configurations. In addition, we have executed a system library for an important task inside the performance model simulator. We report on the measured algorithm convergence as a function of the number of vessel thrusters. We have also studied the system architecture adaptability by comparing the documented system architecture and the implemented source code. We report on the adaptability findings and the recommendations we were able to provide to the system's architect. Finally, we have developed models of hardware and software reliability. We report on hardware reliability results based on the evaluation of the system architecture. As a topic for future work, we report on an approach that we recommend be applied to evaluate the system under study software reliability.

We describe an empirical approach for the assessment of the scheduling risk in a large globally distributed industrial software project. We analyzed two and half years of project data and we identified some factors that are likely to impact team productivity and that could affect the teampsilas ability to meet its schedule objective. We built an analytical model to capture teampsilas productivity variability, which can be used to approximate the total time required to complete the project. Our approach is designed to be used by project management groups to perform risk assessment early in the software development process.

We report on our experiences using agile practices in a global software development project. Specifically, we report on the communication and collaboration patterns that were discovered using the social network analysis methodology. We used as a case study the Global Studio Project Version 3.0, where Extreme Programming practices were applied to one of the remote software development teams. We summarize our conclusions about Extreme Programming usage in global software development projects by presenting ten lessons learned from the application of Extreme Programming practices to the Global Studio Project Version 3.0.

In this paper, we report on our experience assessing the relationship between the dependency structure of a software architecture and the coordination needs among distributed development teams. We use as a case study for global software development the Global Studio Project Version 3.0, where matrix models were used to represent both architectural dependencies and the coordination structure among the team members. Analysis of data gathered during the Global Studio Project Version 3.0 revealed that design structure matrix (DSM) models representing the modular structure of the software architecture are highly consistent with the social network matrix models that represent the actual coordination structure. We conclude that DSM modeling can help guide the task assignments in global software development projects.

In this paper we present an overview of current plans for the global studio project version 3.0. version 3.0 applies a new experimental process with a higher degree of collaboration and that relies less on the central team, as compared to the extended workbench model. We expect that the use of an architecture for component integration and the identification of domain experts for each component will significantly reduce the amount of upfront architecture and requirements work usually required from the central team in the extended workbench model.

We present three algorithms for detecting the need for software rejuvenation by monitoring the changing values of a customer-affecting performance metric, such as response time. Applying these algorithms can improve the values of this customer-affecting metric by triggering rejuvenation before performance degradation becomes severe. The algorithms differ in the way they gather and use sample values to arrive at a rejuvenation decision. Their effectiveness is evaluated for different sets of control parameters, including sample size, using simulation. The results show that applying the algorithms with suitable choices of control parameters can significantly improve system performance as measured by the response time

High performing teams are composed of members who are committed to team success and to each other [1]. These high-performance teams develop over a period of time and are capable of delivering outstanding performance results. Theirmembers often have complementary problemsolving, technical, and interpersonal skills [1]. These teams have learned how to leverage the strengths of its members and usually have developed very sophisticated styles of communication. Teams that are formed to support global software development are usually brought together to support a specific mission. These global software development teams are usually composed of members that come from several continents, speak several languages, subscribe to different cultural accepted management structures, and may have a different perceptions of the global software development team ultimate mission. In addition, each team member brings to these global software development teams different expectations of acceptable team behaviors, and may be used to different levels of formality in their communication styles.

Industrial experience assessing the stability of a large mission-critical software project is reported. We observed that the project incurred significant additional delays in resolving the types of problems usually uncovered when assessing mission-critical software stability. We present plausible hypotheses about the possible causes of these additional delays

An e-commerce scalability case study is presented in which both traditional performance testing and performance modeling were used to help tune the application for high performance. This involved the creation of a system simulation model as well as the development of an approach for test case generation and execution. We describe our experience using a simulation model to help diagnose production system problems, and discuss ways that the effectiveness of performance testing efforts was improved by its use.

The rapid changes in the telecommunications environment demand that new services are developed and deployed quickly, and that legacy systems are integrated seamlessly. While emerging technologies and standards such as CORBA allow rapid solution integration via shared reusable components, new issues arise for interoperability and performance assurance in highly heterogeneous environments. In this paper, we first identify these issues in a multi-vendor and multi-ORB telecommunications environment. We then propose a systematic approach for interoperability verification using a reference ORB and industry-standard test suites. We extend our interoperability testing framework to performance verification, where the scope of the tests is detailed and the design strategy is analyzed.

Three algorithms designed to enforce different quality of service criteria are presented, as well as empirical assessments of the algorithms for three large industrial telecommunications systems. These assessments are made in terms of the simulated performance of each system on average loads selected from operational distributions collected during beta release and field use. In addition, synthetic heavy loads designed to cause the overall CPU utilization rates to exceed 90% of capacity were run. The algorithms build on previously defined load testing algorithms, and use parameters and operational distributions computed for that purpose. This makes the quality of service enforcement algorithms particularly efficient. The primary bases for the assessment of the algorithms were the overall deviation of the response time from the average, and the fraction of service requests that were throttled from clients under varying conditions.

An approach is presented to compare the performance of an existing production platform and a proposed replacement architecture. The traditional approach to such a comparison is to develop software for the proposed platform, build the new architecture, and collect performance measurements on both the existing system in production and the new system in the development environment. In this paper we propose a new way to design an application-independent workload for doing such a performance evaluation. We demonstrate the applicability of our approach by describing our experience using it to help an industrial organization determine whether or not a proposed architecture would be adequate to meet their organization's performance requirements.

Three automatic test case generation algorithms intended to test the resource allocation mechanisms of telecommunications software systems are introduced. Although these techniques were specifically designed for testing telecommunications software, they can be used to generate test cases for any software system that is modelable by a Markov chain provided operational profile data can either be collected or estimated. These algorithms have been used successfully to perform load testing for several real industrial software systems. Experience generating test suites for five such systems is presented. Early experience with the algorithms indicate that they are highly effective at detecting subtle faults that would have been likely to be missed if load testing had been done in the more traditional way, using hand-crafted test cases. A domain-based reliability measure is applied to systems after the load testing algorithms have been used to generate test data. Data are presented for the same five industrial telecommunications systems in order to track the reliability as a function of the degree of system degradation experienced.>

Presents the application of a domain-based reliability measure to systems that can be represented by Markov chains. A load testing algorithm is presented, and the measure is applied to assess the reliability of these systems after they have been tested. Data are presented for three industrial telecommunications systems that had been tested using the load testing algorithm, tracking the reliability as a function of the degree of system degradation experienced.>

In this paper we introduce a new load testing technique called Deterministic Markov State Testing and report on its application. Our approach is called “deterministic” because the sequence of test case execution is set at planning time, and “state testing” because each test case certifies a unique software state. There are four main advantages of Deterministic Markov State Testing for system testers: provision of precise software state information for root cause analysis in load test, accommodation for limitations of the system test lab configuration, higher acceleration ratios in system test, and simple management of distributed execution of test cases. System testers using the proposed method have great flexibility in dealing with common system test problems: limited access to the system test environment, unstable software, or changing operational conditions. Because each test case verifies correct execution on a path from the idle state to the software state under test, our method does not require the continuous execution of all test cases. Deterministic Markov State Testing is operational-profile-based, and allows for measurement of software reliability robustness when the operational profile changes.

An analytical performance modeling approach is presented for load sharing policies that schedule jobs based on a global system state. The main contributions of this work are: the derivation of analytical bounds to compute the average delay in large asymmetric systems under dynamic load sharing control, when job routing is based on the global system state; the experimental derivation of parameters which affect the implementation of load sharing policies at the user level; the demonstration that load sharing is effective even in the presence of high overheads; and the demonstration that in the asymmetric environment under study, carefully tuned algorithms for load sharing provide a significant improvement in performance over simpler algorithms.>

The design is discussed of the Locus family of networks rendering PC interface (PCI) service. Given are the expected user workload, the hardware costs, and the performance constraints. The workload consists of three classes of users: interactive, highly-interactive, and communication-intensive. The number of PCs is given and is equal to the number of users. It is shown that the problem can be reduced to a discrete capacity and flow assignment (CFA) problem. The backbone capacity assignment is inspired by the Lagrangian decomposition approach. It starts from a backbone capacity assignment which matches the initial flow and chooses a backbone server upgrade that gives the greatest response-time reduction per dollar. At this point, the flow deviation method is applied and iteration is continuous until the constraints are met. Several computational examples of network configuration synthesis are presented that emphasize the significance and generality of the results obtained.>