Security vulnerabilities originating from insecure information flows can violate the confidentiality of data, thereby negatively impacting individuals and service providers. This challenge gave rise to design-level analyses and source code analyses investigating information flow-related vulnerabilities. Architectural analysis, a type of design-level analysis, can detect security vulnerabilities by inspecting architectural models enriched with specifications of security-relevant information. However, the implementation may not comply with the architectural specification during software evolution. This non-compliance can result in the architectural analysis missing vulnerabilities. Consequently, vulnerabilities in the deployed system can be exploited, but the software engineers are left assuming the system to be secure. In this article, we address this problem of specification-related non-compliance by proposing a coupling approach that enables architectural analyses to use the values of security characteristics which are supplied from the implementation and retrieved by static source code analysis. Our coupling approach makes two contributions: a coupling process and the conditions necessary for the coupling (called integration conditions). In our coupling process, each process step performs transformations between the involved input and output models of the analyses. To enable the coupling, we define necessary integration conditions that must hold between the (meta)models of the analyses in the coupling. We generalize from specific analyses by specifying the integration conditions based on reference metamodels. In our evaluation, we inspect (1) the coverage of the reference metamodels by the metamodels of coupled analyses, (2) the coverage of the integration conditions by successful couplings, and (3) the accuracy of the coupled analysis in finding architectural vulnerabilities originating from a non-compliant implementation. The results of our case study show that the reference metamodels and the integration conditions are covered. We detect 60 true positive vulnerabilities and 5 false positive vulnerabilities. Upon this evidence, we conclude that the architectural analysis in the coupling is accurate in detecting vulnerabilities originating from non-compliant information flows in the implementation.

Although enormous progress has been made in Artificial Intelligence (AI), it entails new challenges. The growing complexity of learning tasks requires more complex AI models, which increasingly exhibit unreliable behaviour. This is of particular concern in safety-critical systems where AI is commonly used. Therefore, well-known architectural approaches have been adopted such as N-Version Programming or Simplex Architectures (denoted as Architectural Safeguards) to deal with the unreliable nature of AI. At design-time, however, it is difficult to identify an architectural safeguard that satisfies the reliability (or other quality) requirements of the system. In this paper, we present a model-based reliability analysis of AI-enabled systems to assess architectural safeguards at design-time taking into account the predictive uncertainty of AI components. We have validated our approach in a case study from the field of autonomous driving. Our results show that our approach not only enables the analysis of the impact of architectural safeguards on the overall reliability of an AI system but supports software engineers in decision-making.

Today’s software systems are neither built nor operated in isolation and have to adapt to their environment. Uncertainty in the software and its context is inherently unavoidable and should be actively analyzed and managed already at design time. This includes analyzing the impact of uncertainty on a system’s quality properties, which quickly becomes critical, e.g., regarding confidentiality. When not handled comprehensively, confidentiality violations can occur due to uncertainty that void previous analysis results. There exist many approaches to classify and handle uncertainty. However, without locating the impact of uncertainty, precise mitigation is often impossible. In this paper, we present an uncertainty impact analysis that shows potential confidentiality violations induced by different uncertainty types like structural, behavioral, or environmental uncertainty. This is achieved by combining software-architectural and data flow-based propagation of uncertainty. Our tool-supported approach is a first step towards predicting the impact of uncertainty without laborious modeling and testing of what-if scenarios. The case study-based evaluation shows that our impact analysis accurately predicts confidentiality violations with a high F1-score of 0.94 while reducing the effort of manual analysis by 82%.

Coping with different and changing requirements leads to concurrent products (variability in space) and subsequent revisions (variability in time). Moreover, products consist of interrelated models that represent different kinds of artifacts. Dependencies and redundancies between interrelated models within a product and across products can quickly lead to inconsistencies during evolution. Thus, dealing with both variability dimensions uniformly while preserving consistency of interrelated models is a major challenge when developing large and long-living variable systems. Recent research addresses uniform management of variability in space and time by unifying concepts and operations from software product line engineering and software configuration management. However, consistency preservation for interrelated models, which is a major research topic in model-driven software development, has hardly been considered in variability management. We propose an approach that builds on recent efforts for unifying variability in space and time and leverages view-based consistency preservation for systems comprised of different kinds of interrelated models. We evaluate our approach by applying it to two real-world case studies: the well-known ArgoUML-SPL, that is based on the UML modeling tool ArgoUML, and MobileMedia, a mobile application for media management. Our results show that, by manually evolving only the Java models of products, other interrelated models (i.e.,~UML class diagrams) and the remaining affected products can be kept consistent fully automatically.

Context: Software architecture (SA) as research area experienced an increase in empirical research, as identified by Galster and Weyns in 2016 [1]. Empirical research builds a sound foundation for the validity and comparability of the research. A current overview on the evaluation and replicability of SA research objects could help to discuss our empirical standards as a community. However, no such current overview exists.Objective: We aim at assessing the current state of practice of evaluating SA research objects and replication artifact provision in full technical conference papers from 2017 to 2021.Method: We first create a categorization of papers regarding their evaluation and provision of replication artifacts. In a systematic literature review (SLR) with 153 papers we then investigate how SA research objects are evaluated and how artifacts are made available.Results: We found that technical experiments (28%) and case studies (29%) are the most frequently used evaluation methods over all research objects. Functional suitability (46% of evaluated properties) and performance (29%) are the most evaluated properties. 17 papers (11%) provide replication packages and 97 papers (63%) explicitly state threats to validity. 17% of papers reference guidelines for evaluations and 14% of papers reference guidelines for threats to validity.Conclusions: Our results indicate that the generalizability and repeatability of evaluations could be improved to enhance the maturity of the field; although, there are valid reasons for contributions to not publish their data. We derive from our findings a set of four proposals for improving the state of practice in evaluating software architecture research objects. Researchers can use our results to find recommendations on relevant properties to evaluate and evaluation methods to use and to identify reusable evaluation artifacts to compare their novel ideas with other research. Reviewers can use our results to compare the evaluation and replicability of submissions with the state of the practice.

Exchanging data between different systems enables us to build new smart services and digitise various areas of our daily life. This digitalisation leads to more efficient usage of resources, and an increased monetary value. However, the connection of different systems also increases the number of potential vulnerabilities. The vulnerabilities on their own might be harmless, but attackers could build attack paths based on the combination of different vulnerabilities. Additionally, attackers might exploit existing access control policies to further propagate through the system. For analysing this dependency between vulnerabilities and access control policies, we extended an architecture description language (ADL) to model access control policies and specify vulnerabilities. We developed an attack propagation analysis operating on the extended ADL, which can help to determine confidentiality violations in a system. We evaluated our approach by analysing the accuracy and the effort compared to a manual analysis using different scenarios in three case studies. The results indicate that our analysis is capable of identifying attack paths and reducing the effort compared to manual detection.

Abstract With the increasing demand for customized systems and rapidly evolving technology, software engineering faces many challenges. A particular challenge is the development and maintenance of systems that are highly variable both in space (concurrent variations of the system at one point in time) and time (sequential variations of the system, due to its evolution). Recent research aims to address this challenge by managing variability in space and time simultaneously. However, this research originates from two different areas, software product line engineering and software configuration management, resulting in non-uniform terminologies and a varying understanding of concepts. These problems hamper the communication and understanding of involved concepts, as well as the development of techniques that unify variability in space and time. To tackle these problems, we performed an iterative, expert-driven analysis of existing tools from both research areas to derive a conceptual model that integrates and unifies concepts of both dimensions of variability. In this article, we first explain the construction process and present the resulting conceptual model. We validate the model and discuss its coverage and granularity with respect to established concepts of variability in space and time. Furthermore, we perform a formal concept analysis to discuss the commonalities and differences among the tools we considered. Finally, we show illustrative applications to explain how the conceptual model can be used in practice to derive conforming tools. The conceptual model unifies concepts and relations used in software product line engineering and software configuration management, provides a unified terminology and common ground for researchers and developers for comparing their works, clarifies communication, and prevents redundant developments.

The security of software-intensive systems is frequently attacked. High fines or loss in reputation are potential consequences of not maintaining confidentiality, which is an important security objective. Detecting confidentiality issues in early software designs enables cost-efficient fixes. A Data Flow Diagram (DFD) is a modeling notation, which focuses on essential, functional aspects of such early software designs. Existing confidentiality analyses on DFDs support either information flow control or access control, which are the most common confidentiality mechanisms. Combining both mechanisms can be beneficial but existing DFD analyses do not support this. This lack of expressiveness requires designers to switch modeling languages to consider both mechanisms, which can lead to inconsistencies. In this article, we present an extended DFD syntax that supports modeling both, information flow and access control, in the same language. This improves expressiveness compared to related work and avoids inconsistencies. We define the semantics of extended DFDs by clauses in first-order logic. A logic program made of these clauses enables the automated detection of confidentiality violations by querying it. We evaluate the expressiveness of the syntax in a case study. We attempt to model nine information flow cases and six access control cases. We successfully modeled fourteen out of these fifteen cases, which indicates good expressiveness. We evaluate the reusability of models when switching confidentiality mechanisms by comparing the cases that share the same system design, which are three pairs of cases. We successfully show improved reusability compared to the state of the art. We evaluated the accuracy of confidentiality analyses by executing them for the fourteen cases that we could model. We experienced good accuracy.

During the development of large software-intensive systems, developers use several modeling languages and tools to describe a system from different viewpoints. Model-driven and view-based technologies have made it easier to define domain-specific languages and transformations. Nevertheless, using several languages leads to fragmentation of information, to redundancies in the system description, and eventually to inconsistencies. Inconsistencies have negative impacts on the system’s quality and are costly to fix. Often, there is no support for consistency management across multiple languages. Using a single language is no practicable solution either, as it is overly complex to define, use, and evolve such a language. View-based development is a suitable approach to deal with complex systems, and is widely used in other engineering disciplines. Still, we need to cope with the problems of fragmentation and consistency. In this paper, we present the Vitruvius approach for consistency in view-based modeling. We describe the approach by formalizing the notion of consistency, presenting languages for consistency preservation, and defining a model-driven development process. Furthermore, we show how existing models can be integrated. We have evaluated our approach at two case studies from component-based and embedded automotive software development, using our prototypical implementation based on the Eclipse Modeling Framework.

Nearly all facets of our everyday life strongly depend on software-intensive systems. Besides correctness, highly relevant quality properties of these systems include performance, as directly perceived by the user, and maintainability, as an important decision factor for evolution. These quality properties strongly depend on architectural design decisions. Hence, to ensure high quality, research and practice is interested in approaches to analyze the system architecture for quality properties. Therefore, models of the system architecture are created and used for analysis. Many different languages (often defined by metamodels) exist to model the systems and reason on their quality. Such languages are mostly specific to quality properties, tools or development paradigms. Unfortunately, the creation of a specific model for any quality property of interest and any different tool used is simply infeasible. Current metamodels for quality modeling and analysis are often not designed to be extensible and reusable. Experience from generalizing and extending metamodels result in hard to evolve and overly complex metamodels. A systematic way of creating, extending and reusing metamodels for quality modeling and analysis, or parts of them, does not exist yet. When comparing metamodels for different quality properties, however, substantial parts show quite similar language features. This leads to our approach to define the first reference architecture for metamodels for quality modeling and analysis. A reference architecture in software engineering provides a general architecture for a given application domain. In this paper, we investigate the applicability of modularization concepts from object-oriented design and the idea of a reference architecture to metamodels for quality modeling and analysis to systematically create, extend and reuse metamodel parts. Thus, the reference architecture allows to tailor metamodels. Requirements on the reference architecture are gathered from a historically grown metamodel. We specify modularization concepts as a foundation of the reference architecture. Detailed application guidelines are described. We argue the reference architecture supports instance compatibility and non-intrusive, independent extension of metamodels. In four case studies, we refactor historically grown metamodels and compare them to the original metamodels. The study results show the reference architecture significantly improves evolvability as well as need-specific use and reuse of metamodels.

Preservation of confidentiality has become a crucial quality property of software systems that software vendors have to consider in each development phase. Especially, neglecting confidentiality constraints in the software architecture leads to severe issues in later phases that often are hard to correct. In contrast to the implementation phase, there is no support for systematically considering confidentiality in architectural design phases by means of data processing descriptions. To fill this gap, we introduce data flows in an architectural description language to enable simple definition of confidentiality constraints. Afterwards, we transform the software architecture specification to a logic program to find violated confidentiality constraints. In a case study-based evaluation, we apply the analysis to sixteen scenarios to show the accuracy of the approach.

Multi-view environments provide different views of software systems optimized for different stakeholders. One way of ensuring consistency of overlapping and inter-dependent information contained in such views is to project them “on demand” from a Single Underlying Model (SUM). However, there are various ways of building and evolving such SUMs. This paper presents criteria to distinguish them, describes three archetypical approaches for building SUMs, and analyzes their advantages and disadvantages. From these criteria, guidelines for choosing which approach to use in specific application areas are derived.

One of the most common usage scenarios for Cloud-IoT applications is Sensing-as-a-Service, which focuses on the processing of sensor data in order to make it available for other applications. Auto-scaling is a popular runtime management technique for cloud applications to cope with a varying resource demand by provisioning resources in an autonomous manner. However, if an auto-scaling system cannot provide the required resources, e.g., due to cost constraints, the cloud application is overloaded, which impacts its performance and availability. We present a feedback control mechanism to mitigate and recover from overload situations by adapting the send rate of smart devices in consideration of the current processing rate of the cloud application. This mechanism supports a coupling with the widely used threshold-based auto-scaling systems. In a case study, we demonstrate the capability of the approach to cope with overload scenarios in a realistic environment. Overall, we consider this approach as a novel tool for runtime managing cloud applications.

As long-living systems, automated Production Systems (aPS) have to be adapted due to optimization and inclusion of new features in their life cycle over decades. aPS consist of electrical, mechanical, and software components, which have a complex interaction and mutual dependencies. Consequently, these heterogeneous components have to be maintained together. Thus, the change propagation analysis in aPS is a challenging task. Existing approaches to change impact analysis lack tool-support and require expert knowledge in the aPS, as well as in the machine under study and its environment. This paper presents a tool-supported approach to change propagation analysis in aPS based on initial change requests. Our approach calculates a list of maintainability tasks to implement change requests in control programs deployed on Programmable Logic Controllers (PLC). To evaluate the quality and coverage of the generated task lists, we applied our approach to a community case study.

A CloudIoT solution typically connects thousands of IoT things with cloud applications in order to store or process sensor data. In this environment, the cloud applications often consist of microservices which are connected to each other via message queues and must reliably handle a large number of messages produced by the IoT things. The state of a message queue in such a system can be a challenge if the rate of incoming messages continuously exceeds the rate of outgoing messages. This can lead to performance and reliability degradations due to overloaded queues and result in the unavailability of the cloud application. In this paper we present a case study to investigate which performance metrics to be used by a threshold-based auto-scaler for scaling consuming microservices of a message queue in order to prevent overloaded queues and to avoid SLA violations. We evaluate the suitability of each metric for scaling I/O-intensive and compute-intensive microservices with constant and varying characteristics, such as service time. We show, that scaling decisions based on message queue metrics are much more resilient to microservice characteristics variations. In this case, relying on the CPU utilization may result in massive overprovisioning or no scaling decision at all which could lead to an overloaded queue and SLA violations. We underline the benefits of using message queue metrics for scaling decisions instead of the more traditional CPU utilization particularly for I/O-intensive microservices due to the vulnerability to variations in the microservice characteristics.

Infrastructure as a Service (IaaS) Cloud services allow users to deploy distributed applications in a virtualized environment without having to customize their applications to a specific Platform as a Service (PaaS) stack. It is common practice to host multiple Virtual Machines (VMs) on the same server to save resources. Traditionally, IaaS data center management required manual effort for optimization, e.g. by consolidating VM placement based on changes in usage patterns. Many resource management algorithms and frameworks have been developed to automate this process. Resource management algorithms are typically tested via experimentation or using simulation. The main drawback of both approaches is the high effort required to conduct the testing. Existing Cloud or IaaS simulators require the algorithm engineer to reimplement their algorithm against the simulator's API. Furthermore, the engineer manually needs to define the workload model used for algorithm testing. We propose an approach for the simulative analysis of IaaS Cloud infrastructure that allows algorithm engineers and data center operators to evaluate optimization algorithms without investing additional effort to reimplement them in a simulation environment. By leveraging runtime monitoring data, we automatically construct the simulation models used to test the algorithms. Our validation shows that algorithm tests conducted using our IaaS Cloud simulator match the measured behavior on actual hardware.

Business processes as well as software systems face various changes during their lifetime. As they mutually influence each other, business processes and software systems have to be modified in co-evolution. Thus, to adequately predict the change impact, it is important to consider the complex mutual dependencies of both domains. However, existing approaches are limited to analyzing the change propagation in software systems or business processes in isolation. In this paper, we present a tool-supported approach to estimate the change propagation caused by a change request in business processes or software systems based on the software architecture and the process design. We focus on the mutual dependencies regarding the change propagation between both domains. In the evaluation, we apply our approach to a community case study to demonstrate the quality of results in terms of precision, recall, and coverage.

Models are used in software engineering to describe parts of a system that are relevant for the computation of specific analyses, or the provision of specific functionality. Metamodeling languages such as Ecore make it possible to realize analyses and functionality with model-driven technology, such as transformation engines. If models conform to a metamodel that was expressed using Ecore, numerous Eclipse-based tools can be reused to directly analyze, display, or transform models. In many software projects, models are, however, realized with objects of plain-old Java classes rather than an explicit metamodel, so these popular toolscannot be used.In this new ideas paper, we present an Ecoreification approach, which can be used to automatically extract Ecore-conforming metamodels from Java code, and a code generator that combines the benefits of both worlds. The resulting code can be used exactly as before, but it also uses the modeling infrastructure and implements all interfaces for Ecore-based tooling. This way, arbitrary non-standard models can be displayed and modified, for example using graphical Sirius editors, or transformed with well-proven transformation languages, such as QVT-O or ATL.

Both information and automated production systems (aPS) evolve during their lifetime, e.g. due to changes in requirements and infrastructure. In order to estimate maintenance effort in information systems the KAMP method is applied. This paper discusses the necessary classification of changes as a prerequisite to apply such a method. Aggravating aPS consist not only of software but also include mechanics and electric/automation hardware. Therefore, the classification has to be enlarged to a multi-disciplinary one. The limitations of this approach for aPS are discussed in detail and demonstrated using three scenarios of a lab size pick and place unit. The paper closes delivering first ideas to cope with these.

Reverse engineering a software system is challenged by the typically very limited information available about existing systems. Useful reverse engineering tasks include recovering a system's architectural, behavioral, and usage models, which can then be leveraged to answer important questions about a system. For example, using such models to analyze and predict a system's non-functional properties would help to efficiently assess the system's current state, planned adaptations, scalability issues, etc. Existing approaches typically only extract a system's static architecture, omitting the dynamic information that is needed for such analyses. The contribution of this paper is an automated technique that extracts a system's static architecture, behavior, and usage models from very limited, but readily available information: source code and test cases. These models can then be fed into known performance, reliability, and cost prediction techniques. We evaluated our approach for accuracy against systems with already established usage models, and observed that our approach finds the correct, but more detailed usage models. We also analyzed 14 open source software systems spanning over 2 million lines of code to evaluate the scalability of our approach.

Software-intensive systems are subject to continuous change due to modification of the systems themselves and their environment.Methods for supporting evolution are a competitive edge in software engineering as software is operated over decades.Empirical research is useful to validate the effectiveness of these methods.However, empirical studies on software evolution are rarely comprehensive and hardly replicable.Collaboration in empirical studies may prevent these shortcomings.We analyzed the support for such collaboration and examined existing studies in a literature review.Based on our findings, we designed CoCoMEP-a platform for supporting collaboration in empirical research on software evolution by shared knowledge.We report lessons learned from the application of the platform in a large research programme.

Information and automated production systems are long-living, evolvable systems. Consequently, modifications are performed to correct, improve or adapt the respective system. We introduce different approaches to analyze maintainability of software-intensive systems and propose two different case studies from the information and from the automated production systems domain as a basis to validate approaches on system evolution.

Next generation IT infrastructures are highly driven by virtualization technology. The latter enables flexible and efficient resource sharing allowing to improve system agility and reduce costs for IT services. Due to the sharing of resources and the increasing requirements of modern applications on I/O processing, the performance of storage systems is becoming a crucial factor. In particular, when migrating or consolidating different applications the impact on their performance behavior is often an open question. Performance modeling approaches help to answer such questions, a prerequisite, however, is to find an appropriate workload characterization that is both easy to obtain from applications as well as sufficient to capture the important characteristics of the application. In this paper, we present an automated workload characterization approach that extracts a workload model to represent the main aspects of I/O-intensive applications using relevant workload parameters, e.g., request size, read-write ratio, in virtualized environments. Once extracted, workload models can be used to emulate the workload performance behavior in real-world scenarios like migration and consolidation scenarios. We demonstrate our approach in the context of two case studies of representative system environments. We present an in-depth evaluation of our workload characterization approach showing its effectiveness in workload migration and consolidation scenarios. We use an IBM System z equipped with an IBM DS8700 and a Sun Fire system as state-of-the-art virtualized environments. Overall, the evaluation of our workload characterization approach shows promising results to capture the relevant factors of I/O-intensive applications.

The ever-increasing I/O resource demands pose significant challenges for today's system environments to meet performance requirements. The resource demand effects are even magnified in modern virtualized environments where workloads are consolidated to save hardware and operating costs. Tool-supported analysis approaches can help to understand I/O performance characteristics and avoid I/O performance and interference issues. In this demo paper, we present the Storage Performance Analyzer (SPA) - a tool for automated I/O performance analysis. SPA is equipped with tailored features for virtualized environments allowing to measure, monitor, and model both I/O performance and interference effects in modern environments. SPA is open-source and available for common operating systems.

Methods for supporting evolution of software-intensive systems are a competitive edge in software engineering as software is often operated over decades. Empirical research is useful to validate the effectiveness of these methods. However, empirical studies on software evolution are rarely comprehensive and hardly replicable. Collaboration may prevent these shortcomings. We designed CoCoMEP — a platform for supporting collaboration in empirical research on software evolution by shared knowledge. We report lessons learned from the application of the platform in a large research programme.

Virtualization technology allows to share the physical resources used in IT infrastructures for efficient and flexible system operation. Sharing of physical resources, however, comes usually at the cost of performance and poses significant challenges to respect the Quality-of-Service of consolidated data-intensive applications due to the mutual performance interference among the applications. The non-trivial impact of workload consolidation on the I/O performance can be anticipated using explicit performance analysis techniques. In current practice, however, explicit modeling of I/O performance interference effects in virtualized environments is usually avoided due to their complexity. In this paper, we present an explicit performance modeling approach of I/O performance interference in virtualized environments with queueing Petri nets (QPNs). More specifically, we first highlight major challenges when modeling I/O performance in virtualized environments. Then, we create a single-VM I/O performance model calibrated with response time measurements to capture the complex behavior of a representative, real-world environment based on IBM System z and IBM DS8700 server hardware. Finally, we use the I/O performance model to evaluate the I/O performance when the workload is distributed heterogeneously on colocated virtual machines. Overall, we effectively create an I/O performance interference model capturing the I/O performance effects in a multi-VM environment with less than 10% prediction error on average.

Middleware performance models are useful building blocks in the performance models of distributed software applications. We focus on performance models of messaging middleware implementing the Java Message Service standard, showing how certain system design properties -- including pipelined processing and message coalescing -- interact to create performance behavior that the existing models do not capture accurately. We construct a performance model of the ActiveMQ messaging middleware that addresses the outlined issues and discuss how the approach extends to other middleware implementations.

Server virtualization is a key technology to share physical resources efficiently and flexibly. With the increasing popularity of I/O-intensive applications, however, the virtualized storage used in shared environments can easily become a bottleneck and cause performance and scalability issues. Performance modeling and evaluation techniques applied prior to system deployment help to avoid such issues. In current practice, however, virtualized storage and its effects on the overall system performance are often neglected or treated as a black-box. In this paper, we present a systematic I/O performance modeling approach for virtualized storage systems based on queueing theory. We first propose a general performance model building methodology. Then, we demonstrate our methodology creating I/O queueing models of a real-world representative environment based on IBM System z and IBM DS8700 server hardware. Finally, we present an in-depth evaluation of our models considering both interpolation and extrapolation scenarios as well as scenarios with multiple virtual machines. Overall, we effectively create performance models with less than 11% mean prediction error in the worst case and less than 5% prediction error on average.

Modern virtualized environments are key for reducing the operating costs of data centers. By enabling the sharing of physical resources, virtualization promises increased resource efficiency with decreased administration costs. With the increasing popularity of I/O-intensive applications, however, the virtualized storage used in such environments can quickly become a bottleneck and lead to performance and scalability issues. Performance modeling and evaluation techniques applied prior to system deployment help to avoid such issues. In current practice, however, virtualized storage and its performance-influencing factors are often neglected or treated as a black-box. In this paper, we present a measurement-based performance prediction approach for virtualized storage systems based on optimized statistical regression techniques. We first propose a general heuristic search algorithm to optimize the parameters of regression techniques. Then, we apply our optimization approach and create performance models using four regression techniques. Finally, we present an in-depth evaluation of our approach in a real-world representative environment based on IBM System z and IBM DS8700 server hardware. Using our optimized techniques, we effectively create performance models with less than 7% prediction error in the most typical scenario. Furthermore, our optimization approach reduces the prediction error by up to 74%.

Virtualized environments introduce an additional abstraction layer on top of physical resources to enable the collective resource usage by multiple systems. With the rise of I/O-intensive applications, however, the virtualized storage of such shared environments can quickly become a bottleneck and lead to performance and scalability issues. The latter can be avoided through careful design of the application architecture and systematic capacity planning throughout the system life cycle. In current practice, however, virtualized storage and its performance-influencing design decisions are often neglected or treated as a black-box. In this work-in-progress paper, we propose a generic approach for performance modeling and prediction of virtualized storage systems at the software architecture level. More specifically, we propose two performance modeling approaches of virtualized systems. Furthermore, we propose two approaches how the performance models can be combined with architecture-level performance models. The goal is to cope with the increasing complexity of virtualized storage systems with the benefit of intuitive software architecture-level models.

The workshops of ICSE 2012 provide a forum for researchers and practitioners to exchange and discuss scientific ideas before they have matured to warrant conzference or journal publication. ICSE Workshops also serve as incubators for scientific communities that form and share a particular research agenda.

With the increasing importance of reliability in business and industrial software systems, new techniques of architecture-based reliability engineering are becoming an integral part of the development process. These techniques can assist system architects in evaluating the reliability impact of their design decisions. Architecture-based reliability engineering is only effective if the involved reliability models reflect the interaction and usage of software components and their deployment to potentially unreliable hardware. However, existing approaches either neglect individual impact factors on reliability or hard-code them into formal models, which limits their applicability in component-based development processes. This paper introduces a reliability modeling and prediction technique that considers the relevant architectural factors of software systems by explicitly modeling the system usage profile and execution environment and automatically deriving component usage profiles. The technique offers a UML-like modeling notation whose models are automatically transformed into a formal analytical model. Our work builds upon the Palladio Component Model (PCM), employing novel techniques of information propagation and reliability assessment. We validate our technique with sensitivity analyses and simulation in two case studies. The case studies demonstrate effective support of usage profile analysis and architectural configuration ranking, together with the employment of reliability-improving architecture tactics.

Performance measurements are often concerned with accurate recording of timing values, which requires timer methods of high quality. Evaluating the quality of a given timer method or performance counter involves analysing several properties, such as accuracy, invocation cost and timer stability. These properties are metrics with platform-dependent values, and ranking and selecting timer methods requires comparisons using multidimensional metric sets, which make the comparisons ambiguous and unnecessary complex. To solve this problem, this paper proposes a new unified metric that allows for a simpler comparison. The one-dimensional metric is designed to capture fine-granular differences between timer methods, and normalises accuracy and other quality attributes by using CPU cycles instead of time units. The proposed metric is evaluated on all timer methods provided by Java and .NET platform APIs.

Performance evaluation requires accurate and dependable measurements of timing values. Such measurements are usually made using timer methods, but these methods are often too coarse-grained and too inaccurate. Thus, direct usage of hardware performance counters is frequently used for fine-granular measurements due to higher accuracy. However, direct access to these counters may be misleading on multicore computers because cores can be paused or core affinity changed by the operating system, resulting in misleading counter values. The contribution of this paper is the demonstration of an additional, significant flaw arising from the direct use of hardware performance counters. We demonstrate that using JNI and assembler instructions to access the Timestamp Counter from Java applications can result in grossly wrong values, even in single-threaded scenarios.

In software engineering, performance and the integration of performance analysis methodologies gain increasing importance, especially for complex systems. Well-developed methods and tools can predict non-functional performance properties like response time or resource utilization in early design stages, thus promising time and cost savings. However, as performance modeling and performance prediction is still a young research area, the methods are not yet well-established and in wide-spread industrial use. This work is a case study of the applicability of the Palladio Component Model as a performance prediction method in an industrial environment. We model and analyze different design alternatives for storage virtualization on an IBM* system. The model calibration, validation and evaluation is based on data measured on a System z9* as a proof of concept. The results show that performance predictions can identify performance bottlenecks and evaluate design alternatives in early stages of system development. The experiences gained were that performance modeling helps to understand and analyze a system. Hence, this case study substantiates that performance modeling is applicable in industry and a valuable method for evaluating design decisions.

In symmetric multiprocessing environments, the performance of a software system heavily depends on the application's parallelism, the scheduling and load-balancing policies of the operating system, and the infrastructure it is running on. The scheduling of tasks can influence the response time of an application by several orders of magnitude. Thus, detailed models of the operating system scheduler are essential for accurate performance predictions. However, building such models for schedulers and including them into performance prediction models involves a lot of effort. For this reason, simplified scheduler models are used for the performance evaluation of business information systems in general. In this work, we present an approach to derive load-balancing properties of general-purpose operating system (GPOS) schedulers automatically. Our approach uses goal-oriented measurements to derive performance models based on observations. Furthermore, the derived performance model is plugged into the Palladio Component Model (PCM), a model-based performance prediction approach. We validated the applicability of the approach and its prediction accuracy in a case study on different operating systems.

In component-based software engineering, existing components are often reused in new applications. Correspondingly, the response time of an entire component-based application can be predicted from the execution durations of individual component services. These execution durations depend on the runtime behavior of a component which itself is influenced by three factors: the execution platform, the usage profile, and the component wiring. To cover all relevant combinations of these influencing factors, conventional prediction of response times requires repeated deployment and measurements of component services for all such combinations, incurring a substantial effort. This paper presents a novel comprehensive approach for reverse engineering and performance prediction of components. In it, genetic programming is utilized for reconstructing a behavior model from monitoring data, runtime bytecode counts, and static bytecode analysis. The resulting behavior model is parameterized over all three performance-influencing factors, which are specified separately. This results in significantly fewer measurements: The behavior model is reconstructed only once per component service, and one application-independent bytecode benchmark run is sufficient to characterize an execution platform. To predict the execution durations for a concrete platform, our approach combines the behavior model with platform-specific benchmarking results. We validate our approach by predicting the performance of a file sharing application.

The shift of hardware architecture towards parallel execution led to a broad usage of multi-core processors in desktop systems and in server systems. The benefit of additional processor cores for software performance depends on the software's parallelism as well as the operating system scheduler's capabilities. Especially, the load on the available processors (or cores) strongly influences response times and throughput of software applications. Hence, a sophisticated understanding of the mutual influence of software behaviour and operating system schedulers is essential for accurate performance evaluations. Multi-core systems pose new challenges for performance analysis and developers of operating systems. For example, an optimal scheduling policy for multi-server systems, such as shortest remaining processing time (SRPT) for single-server systems, is not yet known in queueing theory. In this paper, we present a detailed experimental evaluation of general purpose operating system (GPOS) schedulers in symmetric multiprocessing (SMP) environments. In particular, we are interested in the influence of multiprocessor load balancing on software performance. Additionally, the evaluation includes effects of GPOS schedulers that can also occur in single-processor environments, such as I/O-boundedness of tasks and different prioritisation strategies. The results presented in this paper provide the basis for the future development of more accurate performance models of today's software systems.

Efforts for software evolution supersede any other part of the software life cycle. Technological decisions have a major impact on the maintainability, but are not well reflected by existing code or architecture based metrics. The way the persistency of object structures with relational databases is solved affects the maintainability of the overall system. Besides maintainability other quality attributes of the software are of interest, in particular performance metrics. However, a systematic evaluation of the benefits and drawback of different persistency frameworks is lacking. In this paper we systematically evaluate the maintainability and performance of different technological approaches for this mapping. The paper presents a testbed and an evaluation process with specifically designed metrics to evaluate persistency techniques regarding their maintainability and performance. In the second part we present and discuss the results of the case study.

While the importance of multi-tier architectures for enterprise information systems is widely accepted and their benefits are well published, the systematic migration from monolithic legacy systems toward multi-tier architectures is known to a much lesser extent. In this paper we present a pattern on how to re-use elements of legacy systems within multi-tier architectures, which also allows for a smooth migration path. We report on experience we made with migrating existing municipal information systems towards a multitier architecture. The experience is generalized by describing the underlying pattern such that it can be re-used for similar architectural migration tasks. The emerged Dublo pattern is based on the partial duplication of business logic among legacy system and newly deployed application server. While this somehow contradicts the separation-of-concerns principle, it offers a high degree of flexibility in the migration process and allows for a smooth transition. Experience with the combination of outdated database technology with modern server-side component and Web services technologies is discussed. In this context, we also report on technology and architecture selection processes.

This paper discusses the role of patterns in product line design by introducing a pattern based product line architecture for statistical analysis software. Associated to the architecture is a pattern language describing the instantiation of concrete software products from the product line. As patterns document design decisions better than mere code, patterns hinder the architectural drift. Since stability and organized evolution is of high importance for long-term assets, the use of patterns and pattern languages is particularly interesting for product line architectures.