Self-adaptation equips a software system with a feedback loop that resolves uncertainties during operation and adapts the system to deal with them when necessary. Most self-adaptation approaches today use decision-making mechanisms that select for execution the adaptation option with the best-estimated benefit expressed as a set of adaptation goals. A few approaches also consider the estimated (one-off) cost of executing the candidate adaptation options. We argue that besides benefit and cost, decision-making in self-adaptive systems should also consider the estimated risk the system or its users would be exposed to if an adaptation option were selected for execution. Balancing all three concerns when evaluating the options for adaptation to mitigate uncertainty is essential for satisfying stakeholders’ concerns and ensuring the safety and public acceptance of self-adaptive systems. In this article, we present a reference model for decision-making in self-adaptation that considers the estimated benefit, cost, and risk as core concerns of each adaptation option. Leveraging this model, we then present an ISO/IEC/IEEE 42010 compatible architectural viewpoint that aims at supporting software architects responsible for designing robust decision-making mechanisms for self-adaptive systems. We demonstrate the applicability, usefulness, and understandability of the viewpoint through a case study where participants with experience in the engineering of self-adaptive systems performed a set of design tasks in DeltaIoT, an Internet-of-Things exemplar for research on self-adaptive systems.

A/B testing, also referred to as online controlled experimentation or continuous experimentation, is a form of hypothesis testing where two variants of a piece of software are compared in the field from an end user’s point of view. A/B testing is widely used in practice to enable data-driven decision making for software development. While a few studies have explored different facets of research on A/B testing, no comprehensive study has been conducted on the state-of-the-art in A/B testing. Such a study is crucial to provide a systematic overview of the field of A/B testing driving future research forward. To address this gap and provide an overview of the state-of-the-art in A/B testing, this paper reports the results of a systematic literature review that analyzed primary studies. The research questions focused on the subject of A/B testing, how A/B tests are designed and executed, what roles stakeholders have in this process, and the open challenges in the area. Analysis of the extracted data shows that the main targets of A/B testing are algorithms, visual elements, and workflow and processes. Single classic A/B tests are the dominating type of tests, primarily based in hypothesis tests. Stakeholders have three main roles in the design of A/B tests: concept designer, experiment architect, and setup technician. The primary types of data collected during the execution of A/B tests are product/system data, user-centric data, and spatio-temporal data. The dominating use of the test results are feature selection, feature rollout, continued feature development, and subsequent A/B test design. Stakeholders have two main roles during A/B test execution: experiment coordinator and experiment assessor. The main reported open problems are related to the enhancement of proposed approaches and their usability. From our study we derived three interesting lines for future research: strengthen the adoption of statistical methods in A/B testing, improving the process of A/B testing, and enhancing the automation of A/B testing.

Recently, machine learning (ML) has become a popular approach to support self-adaptation. ML has been used to deal with several problems in self-adaptation, such as maintaining an up-to-date runtime model under uncertainty and scalable decision-making. Yet, exploiting ML comes with inherent challenges. In this article, we focus on a particularly important challenge for learning-based self-adaptive systems: drift in adaptation spaces. With adaptation space, we refer to the set of adaptation options a self-adaptive system can select from to adapt at a given time based on the estimated quality properties of the adaptation options. A drift of adaptation spaces originates from uncertainties, affecting the quality properties of the adaptation options. Such drift may imply that the quality of the system may deteriorate, eventually, no adaptation option may satisfy the initial set of adaptation goals, or adaptation options may emerge that allow enhancing the adaptation goals. In ML, such a shift corresponds to a novel class appearance, a type of concept drift in target data that common ML techniques have problems dealing with. To tackle this problem, we present a novel approach to self-adaptation that enhances learning-based self-adaptive systems with a lifelong ML layer. We refer to this approach as lifelong self-adaptation . The lifelong ML layer tracks the system and its environment, associates this knowledge with the current learning tasks, identifies new tasks based on differences, and updates the learning models of the self-adaptive system accordingly. A human stakeholder may be involved to support the learning process and adjust the learning and goal models. We present a general architecture for lifelong self-adaptation and apply it to the case of drift of adaptation spaces that affects the decision-making in self-adaptation. We validate the approach for a series of scenarios with a drift of adaptation spaces using the DeltaIoT exemplar.

Self-adaptive systems (SASs) are designed to handle changes and uncertainties through a feedback loop with four core functionalities: monitoring, analyzing, planning, and execution. Recently, generative artificial intelligence (GenAI), especially the area of large language models, has shown impressive performance in data comprehension and logical reasoning. These capabilities are highly aligned with the functionalities required in SASs, suggesting a strong potential to employ GenAI to enhance SASs. However, the specific benefits and challenges of employing GenAI in SASs remain unclear. Yet, providing a comprehensive understanding of these benefits and challenges is complex due to several reasons: limited publications in the SAS field, the technological and application diversity within SASs, and the rapid evolution of GenAI technologies. To that end, this article aims to provide researchers and practitioners a comprehensive snapshot that outlines the potential benefits and challenges of employing GenAI’s within SAS. Specifically, we gather, filter, and analyze literature from four distinct research fields and organize them into two main categories to potential benefits: (i) enhancements to the autonomy of SASs centered around the specific functions of the MAPE-K feedback loop, and (ii) improvements in the interaction between humans and SASs within human-on-the-loop settings. From our study, we outline a research roadmap that highlights the challenges of integrating GenAI into SASs. The roadmap starts with outlining key research challenges that need to be tackled to exploit the potential for applying GenAI in the field of SAS. The roadmap concludes with a practical reflection, elaborating on current shortcomings of GenAI and proposing possible mitigation strategies. †

Engineering long-running computing systems that achieve their goals under ever-changing conditions pose significant challenges. Self-adaptation has shown to be a viable approach to dealing with changing conditions. Yet, the capabilities of a self-adaptive system are constrained by its operational design domain (ODD), i.e., the conditions for which the system was built (requirements, constraints, and context). Changes, such as adding new goals or dealing with new contexts, require system evolution. While the system evolution process has been automated substantially, it remains human-driven. Given the growing complexity of computing systems, human-driven evolution will eventually become unmanageable. In this paper, we provide a definition for ODD and apply it to a self-adaptive system. Next, we explain why conditions not covered by the ODD require system evolution. Then, we outline a new approach for self-evolution that leverages the concept of ODD, enabling a system to evolve autonomously to deal with conditions not anticipated by its initial ODD. We conclude with open challenges to realise self-evolution.

Despite the vast body of knowledge developed by the self-adaptive systems community and the wide use of self-adaptation in industry, it is unclear whether or to what extent industry leverages output of academics. Hence, it is important for the research community to answer the question: Are the solutions developed by the self-adaptive systems community mature enough for industrial adoption? Leveraging a set of empirically-grounded guidelines for industry-relevant artifacts in self-adaptation, we develop a position to answer this question from the angle of using artifacts for evaluating research results in self-adaptation, which is actively stimulated and applied by the community

Previous research highlighted concerns about empirical research in software engineering (e.g., reproducibility, applicability of findings). It is unclear how these concerns reflect views of those who conduct and evaluate research. Focusing on software architecture, one subfield of software engineering, we study perceptions of the research community on (1) how empirical research is applied, (2) human participants, (3) internal and external validity, and (4) replications. We collected responses from 105 key players in architecture research via a survey; we analyzed data quantitatively and qualitatively. Although respondents do generally not prefer either quantitative or qualitative research, around 40% express a preference for various reasons. Professionals are the preferred participants; there is no consensus on the value of student participants. Also, there is no consensus on when to focus on internal or external validity. Most respondents value replications, but acknowledge difficulties. A comparison with published research shows differences between how the community thinks research should be done. We provide evidence that consensus about empirical research is limited. Findings have implications for conducting and reviewing empirical research (e.g., training researchers and reviewers), and call for reflection on empirical research (e.g., to resolve conflicts). We outline actions for the future.

Computing systems form the backbone of many areas in our society, from manufacturing to traffic control, healthcare, and financial systems. When software plays a vital role in the design, construction, and operation, these systems are referred to as software-intensive systems. Self-adaptation equips a software-intensive system with a feedback loop that either automates tasks that otherwise need to be performed by human operators or deals with uncertain conditions. Such feedback loops have found their way to a variety of practical applications; typical examples are an elastic cloud to adapt computing resources and automated server management to respond quickly to business needs. To gain insight into the motivations for applying self-adaptation in practice, the problems solved using self-adaptation and how these problems are solved, and the difficulties and risks that industry faces in adopting self-adaptation, we performed a large-scale survey. We received 184 valid responses from practitioners spread over 21 countries. Based on the analysis of the survey data, we provide an empirically grounded overview the of state of the practice in the application of self-adaptation. From that, we derive insights for researchers to check their current research with industrial needs, and for practitioners to compare their current practice in applying self-adaptation. These insights also provide opportunities for applying self-adaptation in practice and pave the way for future industry-research collaborations.

Self-adaptation equips a computing system with a feedback loop that enables it to deal with change caused by uncertainties during operation, such as changing availability of resources and fluctuating workloads. To ensure that the system complies with the adaptation goals, recent research suggests the use of formal techniques at runtime. Yet, existing approaches have three limitations that affect their practical applicability: (i) they ignore correctness of the behavior of the feedback loop, (ii) they rely on exhaustive verification at runtime to select adaptation options to realize the adaptation goals, which is time- and resource-demanding, and (iii) they provide limited or no support for changing adaptation goals at runtime. To tackle these shortcomings, we present ActivFORMS (Active FORmal Models for Self-adaptation). ActivFORMS contributes an end-to-end approach for engineering self-adaptive systems, spanning four main stages of the life cycle of a feedback loop: design, deployment, runtime adaptation, and evolution. We also present ActivFORMS-ta, a tool-supported instance of ActivFORMS that leverages timed automata models and statistical model checking at runtime. We validate the research results using an IoT application for building security monitoring that is deployed in Leuven. The experimental results demonstrate that ActivFORMS supports correctness of the behavior of the feedback loop, achieves the adaptation goals in an efficient way, and supports changing adaptation goals at runtime.

In the past years, machine learning (ML) has become a popular approach to support self-adaptation. While ML techniques enable dealing with several problems in self-adaptation, such as scalable decision-making, they are also subject to inherent challenges. In this paper, we focus on one such challenge that is particularly important for self-adaptation: ML techniques are designed to deal with a set of predefined tasks associated with an operational domain; they have problems to deal with new emerging tasks, such as concept shift in input data that is used for learning. To tackle this challenge, we present lifelong self-adaptation: a novel approach to self-adaptation that enhances self-adaptive systems that use ML techniques with a lifelong ML layer. The lifelong ML layer tracks the running system and its environment, associates this knowledge with the current tasks, identifies new tasks based on differentiations, and updates the learning models of the self-adaptive system accordingly. We present a reusable architecture for lifelong self-adaptation and apply it to the case of concept drift caused by unforeseen changes of the input data of a learning model that is used for decision-making in self-adaptation. We validate lifelong self-adaptation for two types of concept drift using two cases.

Micro-services are a common architectural approach to software development today. An indispensable tool for evolving micro-service systems is A/B testing. In A/B testing, two variants, A and B, are applied in an experimental setting. By measuring the outcome of an evaluation criterion, developers can make evidence-based decisions to guide the evolution of their software. Recent studies highlight the need for enhancing the automation when such experiments are conducted in iterations. To that end, we contribute a novel artifact that aims at enhancing the automation of an experimentation pipeline of a micro-service system relying on the principles of self-adaptation. Concretely, we propose SEAByTE, an experimental framework for testing novel self-adaptation solutions to enhance the automation of continuous A/B testing of a micro-service based system. We illustrate the use of the SEAByTE artifact with a concrete example.

Self-adaptation equips a software system with a feedback loop that automates tasks that otherwise need to be performed by operators. Such feedback loops have found their way to a variety of practical applications, one typical example is an elastic cloud. Yet, the state of the practice in self-adaptation is currently not clear. To get insights into the use of self-adaptation in practice, we are running a large-scale survey with industry. This paper reports preliminary results based on survey data that we obtained from 113 practitioners spread over 16 countries, 62 of them work with concrete self-adaptive systems. We highlight the main insights obtained so far: motivations for self-adaptation, concrete use cases, and difficulties encountered when applying self-adaptation in practice. We conclude the paper with outlining our plans for the remainder of the study.

Modern software systems often have to cope with uncertain operation conditions, such as changing workloads or fluctuating interference in a wireless network. To ensure that these systems meet their goals these uncertainties have to be mitigated. One approach to realize this is self-adaptation that equips a system with a feedback loop. The feedback loop implements four core functions – monitor, analyze, plan, and execute – that share knowledge in the form of runtime models. For systems with a large number of adaptation options, i.e., large adaptation spaces, deciding which option to select for adaptation may be time consuming or even infeasible within the available time window to make an adaptation decision. This is particularly the case when rigorous analysis techniques are used to select adaptation options, such as formal verification at runtime, which is widely adopted. One technique to deal with the analysis of a large number of adaptation options is reducing the adaptation space using machine learning. State of the art has showed the effectiveness of this technique, yet, a systematic solution that is able to handle different types of goals is lacking. In this paper, we present ML2ASR+, short for Machine Learning to Adaptation Space Reduction Plus. Central to ML2ASR+ is a configurable machine learning pipeline that supports effective analysis of large adaptation spaces for threshold, optimization, and setpoint goals. We evaluate ML2ASR+ for two applications with different sizes of adaptation spaces: an Internet-of-Things application and a service-based system. The results demonstrate that ML2ASR+ can be applied to deal with different types of goals and is able to reduce the adaptation space and hence the time to make adaptation decisions with over 90%, with negligible effect on the realization of the adaptation goals.

Many software systems today face uncertain operating conditions, such as sudden changes in the availability of resources or unexpected user behavior. Without proper mitigation these uncertainties can jeopardize the system goals. Self-adaptation is a common approach to tackle such uncertainties. When the system goals may be compromised, the self-adaptive system has to select the best adaptation option to reconfigure by analyzing the possible adaptation options, i.e., the adaptation space. Yet, analyzing large adaptation spaces using rigorous methods can be resource- and time-consuming, or even be infeasible. One approach to tackle this problem is by using online machine learning to reduce adaptation spaces. However, existing approaches require domain expertise to perform feature engineering to define the learner and support online adaptation space reduction only for specific goals. To tackle these limitations, we present “Deep Learning for Adaptation Space Reduction Plus”—DLASeR+ for short. DLASeR+ offers an extendable learning framework for online adaptation space reduction that does not require feature engineering, while supporting three common types of adaptation goals: threshold, optimization, and set-point goals. We evaluate DLASeR+ on two instances of an Internet-of-Things application with increasing sizes of adaptation spaces for different combinations of adaptation goals. We compare DLASeR+ with a baseline that applies exhaustive analysis and two state-of-the-art approaches for adaptation space reduction that rely on learning. Results show that DLASeR+ is effective with a negligible effect on the realization of the adaptation goals compared to an exhaustive analysis approach and supports three common types of adaptation goals beyond the state-of-the-art approaches.

More than two decades of research have demonstrated an increasing need for software systems to be self-adaptive. Self-adaptation manages runtime dynamics, which are difficult to predict before deployment. A vast body of knowledge to develop Self-Adaptive Software Systems (SASS) has been established. However, we discovered a lack of process support to develop self-adaptive systems with reuse. The lack of process support may hinder knowledge transfer and quality design. To that end, we propose a domain-engineering based methodology, Autonomic Software Product Lines engineering (ASPLe), which provides step-by-step guidelines for developing families of SASS with systematic reuse. The evaluation results from a case study show positive effects on quality and reuse for self-adaptive systems designed using the ASPLe compared to state-of-the-art engineering practices.

Recently, we have been witnessing a rapid increase in the use of machine learning techniques in self-adaptive systems. Machine learning has been used for a variety of reasons, ranging from learning a model of the environment of a system during operation to filtering large sets of possible configurations before analyzing them. While a body of work on the use of machine learning in self-adaptive systems exists, there is currently no systematic overview of this area. Such an overview is important for researchers to understand the state of the art and direct future research efforts. This article reports the results of a systematic literature review that aims at providing such an overview. We focus on self-adaptive systems that are based on a traditional Monitor-Analyze-Plan-Execute (MAPE)-based feedback loop. The research questions are centered on the problems that motivate the use of machine learning in self-adaptive systems, the key engineering aspects of learning in self-adaptation, and open challenges in this area. The search resulted in 6,709 papers, of which 109 were retained for data collection. Analysis of the collected data shows that machine learning is mostly used for updating adaptation rules and policies to improve system qualities, and managing resources to better balance qualities and resources. These problems are primarily solved using supervised and interactive learning with classification, regression, and reinforcement learning as the dominant methods. Surprisingly, unsupervised learning that naturally fits automation is only applied in a small number of studies. Key open challenges in this area include the performance of learning, managing the effects of learning, and dealing with more complex types of goals. From the insights derived from this systematic literature review, we outline an initial design process for applying machine learning in self-adaptive systems that are based on MAPE feedback loops.

One of the primary drivers for self-adaptation is ensuring that systems achieve their goals regardless of the uncertainties they face during operation. Nevertheless, the concept of uncertainty in self-adaptive systems is still insufficiently understood. Several taxonomies of uncertainty have been proposed, and a substantial body of work exists on methods to tame uncertainty. Yet, these taxonomies and methods do not fully convey the research community’s perception on what constitutes uncertainty in self-adaptive systems and on the key characteristics of the approaches needed to tackle uncertainty. To understand this perception and learn from it, we conducted a survey comprising two complementary stages in which we collected the views of 54 and 51 participants, respectively. In the first stage, we focused on current research and development, exploring how the concept of uncertainty is understood in the community and how uncertainty is currently handled in the engineering of self-adaptive systems. In the second stage, we focused on directions for future research to identify potential approaches to dealing with unanticipated changes and other open challenges in handling uncertainty in self-adaptive systems. The key findings of the first stage are: (a) an overview of uncertainty sources considered in self-adaptive systems, (b) an overview of existing methods used to tackle uncertainty in concrete applications, (c) insights into the impact of uncertainty on non-functional requirements, (d) insights into different opinions in the perception of uncertainty within the community and the need for standardised uncertainty-handling processes to facilitate uncertainty management in self-adaptive systems. The key findings of the second stage are: (a) the insight that over 70% of the participants believe that self-adaptive systems can be engineered to cope with unanticipated change, (b) a set of potential approaches for dealing with unanticipated change, (c) a set of open challenges in mitigating uncertainty in self-adaptive systems, in particular in those with safety-critical requirements. From these findings, we outline an initial reference process to manage uncertainty in self-adaptive systems. We anticipate that the insights on uncertainty obtained from the community and our proposed reference process will inspire valuable future research on self-adaptive systems.

Self-adaptation provides a principled way to deal with software systems’ uncertainty during operation. Examples of such uncertainties are disturbances in the environment, variations in sensor readings, and changes in user requirements. As more systems with strict goals require self-adaptation, the need for formal guarantees in self-adaptive systems is becoming a high-priority concern. Designing self-adaptive software using principles from control theory has been identified as one of the approaches to provide guarantees. In general, self-adaptation covers a wide range of approaches to maintain system requirements under uncertainty, ranging from dynamic adaptation of system parameters to runtime architectural reconfiguration. Existing control-theoretic approaches have mainly focused on handling requirements in the form of setpoint values or as quantities to be optimized. Furthermore, existing research primarily focuses on handling uncertainty in the execution environment. This article presents SimCA*, which provides two contributions to the state-of-the-art in control-theoretic adaptation: (i) it supports requirements that keep a value above and below a required threshold, in addition to setpoint and optimization requirements; and (ii) it deals with uncertainty in system parameters, component interactions, system requirements, in addition to uncertainty in the environment. SimCA* provides guarantees for the three types of requirements of the system that is subject to different types of uncertainties. We evaluate SimCA* for two systems with strict requirements from different domains: an Unmanned Underwater Vehicle system used for oceanic surveillance and an Internet of Things application for monitoring a geographical area. The test results confirm that SimCA* can satisfy the three types of requirements in the presence of different types of uncertainty.

Ensuring the qualities of modern software systems, such as the Internet of Things, is challenging due to various uncertainties, such as dynamics in availability of resources or changes in the envir ...

Software systems are increasingly expected to cope with variable workloads, component failures and other uncertainties through self-adaptation. As such, self-adaptive software has been the subject of intense research over the past decade [3, 4, 9, 10].

The study presented in this paper aims to provide evidence for the hypothesis that software product line research has been changing and that the works in industry and academia have diverged over time. We analysed a subset (140) of all (593) papers published at the Software Product Line Conference (SPLC) until 2017. The subset was randomly selected to cover all years as well as types of papers. We assessed the research type of the papers (academic or industry), the kind of evaluation (application example, empirical, etc.), and the application domain. Also, we assessed which product line life-cycle phases, development practices, and topics the papers address. We present an analysis of the topics covered by academic vs. industry research and discuss the evolution of these topics and their relation over the years. We also discuss implications for researchers and practitioners. We conclude that even though several topics have received more attention than others, academic and industry research on software product lines are actually rather in line with each other.

Building on concepts drawn from control theory, self-adaptive software handles environmental and internal uncertainties by dynamically adjusting its architecture and parameters in response to events such as workload changes and component failures. Self-adaptive software is increasingly expected to meet strict functional and non-functional requirements in applications from areas as diverse as manufacturing, healthcare and finance. To address this need, we introduce a methodology for the systematic ENgineering of TRUstworthy Self-adaptive sofTware (ENTRUST). ENTRUST uses a combination of (1) design-time and runtime modelling and verification, and (2) industry-adopted assurance processes to develop trustworthy self-adaptive software and assurance cases arguing the suitability of the software for its intended application. To evaluate the effectiveness of our methodology, we present a tool-supported instance of ENTRUST and its use to develop proof-of-concept self-adaptive software for embedded and service-based systems from the oceanic monitoring and e-finance domains, respectively. The experimental results show that ENTRUST can be used to engineer self-adaptive software systems in different application domains and to generate dynamic assurance cases for these systems.

Modern software applications are subject to uncertain operating conditions, such as dynamics in the availability of services and variations of system goals. Consequently, runtime changes cannot be ignored, but often cannot be predicted at design time. Control theory has been identified as a principled way of addressing runtime changes and it has been applied successfully to modify the structure and behavior of software applications. Most of the times, however, the adaptation targeted the resources that the software has available for execution (CPU, storage, etc.) more than the software application itself. This paper investigates the research efforts that have been conducted to make software adaptable by modifying the software rather than the resource allocated to its execution. This paper aims to identify: the focus of research on control-theoretical software adaptation; how software is modeled and what control mechanisms are used to adapt software; what software qualities and controller guarantees are considered. To that end, we performed a systematic literature review in which we extracted data from 42 primary studies selected from 1,512 papers that resulted from an automatic search. The results of our investigation show that even though the behavior of software is considered non-linear, research efforts use linear models to represent it, with some success. Also, the control strategies that are most often considered are classic control, mostly in the form of Proportional and Integral controllers, and Model Predictive Control. The paper also discusses sensing and actuating strategies that are prominent for software adaptation and the (often neglected) proof of formal properties. Finally, we distill open challenges for control-theoretical software adaptation.

Abstract Cloud systems provide elastic execution environments of resources that link application and infrastructure/platform components, which are both exposed to uncertainties and change. Change appears in 2 forms: the evolution of architectural components under changing requirements and the adaptation of the infrastructure running applications. Cloud architecture continuity refers to the ability of a cloud system to change its architecture and maintain the validity of the goals that determine the architecture. Goal validity implies the satisfaction of goals in adapting or evolving systems. Architecture continuity aids technical sustainability, that is, the longevity of information, systems, and infrastructure and their adequate evolution with changing conditions. In a cloud setting that requires both steady alignment with technological evolution and availability, architecture continuity directly impacts economic sustainability. We investigate change models and change rules for managing change to support cloud architecture continuity. These models and rules define transformations of architectures to maintain system goals: Evolution is about unanticipated change of structural aspects of architectures, and adaptation is about anticipated change of architecture configurations. Both are driven by quality and cost, and both represent multidimensional decision problems under uncertainty. We have applied the models and rules for adaptation and evolution in research and industry consultancy projects.

An increasingly important concern of software engineers is handling uncertainties at design time, such as environment dynamics that may be difficult to predict or requirements that may change during operation. The idea of self-adaptation is to handle such uncertainties at runtime, when the knowledge becomes available. As more systems with strict requirements require self-adaptation, providing guarantees for adaptation has become a high-priority. Providing such guarantees with traditional architecture-based approaches has shown to be challenging. In response, researchers have studied the application of control theory to realize self-adaptation. However, existing control-theoretic approaches applied to adapt software systems have primarily focused on satisfying only a single adaptation goal at a time, which is often too restrictive for real applications. In this paper, we present Simplex Control Adaptation, SimCA, a new approach to self-adaptation that satisfies multiple goals, while being optimal with respect to an additional goal. SimCA offers robustness to measurement inaccuracy and environmental disturbances, and provides guarantees. We evaluate SimCA for two systems with strict requirements that have to deal with uncertainties: an underwater vehicle system used for oceanic surveillance, and a tele-assistance system for health care support.

Context: Empirical research helps gain well-founded insights about phenomena. Furthermore, empirical research creates evidence for the validity of research results. Objective: We aim at assessing the state-of-practice of empirical research in software architecture. Method: We conducted a comprehensive survey based on the systematic mapping method. We included all full technical research papers published at major software architecture conferences between 1999 and 2015. Results: 17% of papers report empirical work. The number of empirical studies in software architecture has started to increase in 2005. Looking at the number of papers, empirical studies are about equally frequently used to a) evaluate newly proposed approaches and b) to explore and describe phenomena to better understand software architecture practice. Case studies and experiments are the most frequently used empirical methods. Almost half of empirical studies involve human participants. The majority of these studies involve professionals rather than students. Conclusions: Our findings are meant to stimulate researchers in the community to think about their expectations and standards of empirical research. Our results indicate that software architecture has become a more mature domain with regards to applying empirical research. However, we also found issues in research practices that could be improved (e.g., when describing study objectives and acknowledging limitations).

Cyber-physical system (CPS) have been recognized as a top-priority in research and development. The innovations sought for CPS demand them to deal effectively with dynamicity of their environment, to be scalable, adaptive, tolerant to threats, etc. -- i.e. they have to be smart. Although approaches in software engineering (SE) exist that individually meet these demands, their synergy to address the challenges of smart CPS (sCPS) in a holistic manner remains an open challenge. The workshop focuses on software engineering challenges for sCPS. The goals are to increase the understanding of problems of SE for sCPS, study foundational principles for engineering sCPS, and identify promising SE solutions for sCPS. Based on these goals, the workshop aims to formulate a research agenda for SE of sCPS.

Collective intelligence systems (CIS), such as wikis, social networks and content sharing platforms, have dramatically improved knowledge creation and sharing at society level. There is a trend to exploit the stigmergic mechanisms of CIS also at organization/corporate level. However, despite the wide adoption of CIS, there is a lack of consolidated systematic knowledge of the architectural principles and practices that underlie CIS. Software architects lack guidance to design CIS for the application context of individual organizations. To address these challenges, we contribute with an architecture framework for CIS, aligned with ISO/IEC/IEEE 42010. The CIS-AF framework provides guidance for architects to describe key CIS elements and systematically model a CIS that is well-suited for an organization's context and goals. The framework is grounded in an in-depth analysis of existing CIS, workshops and interviews with key stakeholders, and experiences from developing a prototypical CIS. We evaluated the architecture framework in two cases in industry setting where CIS have been designed and implemented using the framework. Results show that the framework effectively supports stakeholders with providing a shared vocabulary of CIS concepts, guiding them to systematically apply the stigmergic principles of CIS, and supporting them with kick starting CIS in their organizations.

Designing software systems that have to deal with dynamic operating conditions, such as changing availability of resources and faults that are difficult to predict, is complex. A promising approach to handle such dynamics is self-adaptation that can be realized by a MAPE-K feedback loop (Monitor-Analyze-Plan-Execute plus Knowledge). To provide evidence that the system goals are satisfied, given the changing conditions, the state of the art advocates the use of formal methods. However, little research has been done on consolidating design knowledge of self-adaptive systems. To support designers, this paper contributes with a set of formally specified MAPE-K templates that encode design expertise for a family of self-adaptive systems. The templates comprise: (1) behavior specification templates for modeling the different components of a MAPE-K feedback loop (based on networks of timed automata), and (2) property specification templates that support verification of the correctness of the adaptation behaviors (based on timed computation tree logic). To demonstrate the reusability of the formal templates, we performed four case studies in which final-year Masters students used the templates to design different self-adaptive systems.

Every software program that interacts with a user requires a user interface. Model-View-Controller (MVC) is a common design pattern to integrate a user interface with the application domain logic. MVC separates the representation of the application domain (Model) from the display of the application's state (View) and user interaction control (Controller). However, studying the literature reveals that a variety of other related patterns exists, which we denote with Model-View- (MV) design patterns. This paper discusses existing MV patterns classified in three main families: Model-View-Controller (MVC), Model-View-View Model (MVVM), and Model-View-Presenter (MVP). We take a practitioners' point of view and emphasize the essentials of each family as well as the differences. The study shows that the selection of patterns should take into account the use cases and quality requirements at hand, and chosen technology. We illustrate the selection of a pattern with an example of our practice. The study results aim to bring more clarity in the variety of MV design patterns and help practitioners to make better grounded decisions when selecting patterns.

In today's volatile business environments, collaboration between information systems, both within and across company borders, has become essential to success. An efficient supply chain, for example, requires the collaboration of distributed and heterogeneous systems of multiple companies. Developing such collaborative applications and building the supporting information systems poses several engineering challenges. A key challenge is to manage the ever-growing design complexity. In this article, we argue that software architecture should play a more prominent role in the development of collaborative applications. This can help to better manage design complexity by modularizing collaborations and separating concerns. State-of-the-art solutions, however, often lack proper abstractions for modeling collaborations at architectural level or do not reify these abstractions at detailed design and implementation level. Developers, on the other hand, rely on middleware, business process management, and Web services, techniques that mainly focus on low-level infrastructure. To address the problem of managing the design complexity of collaborative applications, we present Macodo. Macodo consists of three complementary parts: (1) a set of abstractions for modeling adaptive collaborations, (2) a set of architectural views, the main contribution of this article, that reify these abstractions at architectural level, and (3) a proof-of-concept middleware infrastructure that supports the architectural abstractions at design and implementation level. We evaluate the architectural views in a controlled experiment. Results show that the use of Macodo can reduce fault density and design complexity, and improve reuse and productivity. The main contributions of this article are illustrated in a supply chain management case.

Context: Variability (i.e., the ability of software systems or artifacts to be adjusted for different contexts) became a key property of many systems. Objective: We analyze existing research on variability in software systems. We investigate variability handling in major software engineering phases (e.g., requirements engineering, architecting). Method: We performed a systematic literature review. A manual search covered 13 premium software engineering journals and 18 premium conferences, resulting in 15,430 papers searched and 196 papers considered for analysis. To improve reliability and to increase reproducibility, we complemented the manual search with a targeted automated search. Results: Software quality attributes have not received much attention in the context of variability. Variability is studied in all software engineering phases, but testing is underrepresented. Data to motivate the applicability of current approaches are often insufficient; research designs are vaguely described. Conclusions: Based on our findings we propose dimensions of variability in software engineering. This empirically grounded classification provides a step towards a unifying, integrated perspective of variability in software systems, spanning across disparate or loosely coupled research themes in the software engineering community. Finally, we provide recommendations to bridge the gap between research and practice and point to opportunities for future research.

The challenges of pervasive and mobile computing environments, which are highly dynamic and unpredictable, have motivated the development of self-adaptive software systems. Although noteworthy successes have been achieved on many fronts, the construction of such systems remains significantly more challenging than traditional systems. We argue this is partially because researchers and practitioners have been struggling with the lack of a precise vocabulary for describing and reasoning about the key architectural characteristics of self-adaptive systems. Further exacerbating the situation is the fact that existing frameworks and guidelines do not provide an encompassing perspective of the different types of concerns in this setting. In this article, we present a comprehensive reference model, entitled FOrmal Reference Model for Self-adaptation (FORMS), that targets both issues. FORMS provides rigor in the manner such systems can be described and reasoned about. It consists of a small number of formally specified modeling elements that correspond to the key concerns in the design of self-adaptive software systems, and a set of relationships that guide their composition. We demonstrate FORMS's ability to precisely describe and reason about the architectural characteristics of distributed self-adaptive software systems through its application to several existing systems. FORMS's expressive power gives it a potential for documenting reusable architectural solutions (e.g., architectural patterns) to commonly encountered problems in this area.

Abstract — Background: Case studies and experiments are research methods frequently applied in empirical software engineering. Experiments are well-understood and their value as an empirical method is recognized. On the other hand, there seem to be different opinions on what constitutes a case study, and about the value of case studies as a thorough research method. Aim: We aim at exploring the attitudes of software engineering researchers on case studies and experiments. Furthermore, we investigate how the perceptions of researchers vary along their views on what constitutes a case study. Method: We performed an exploratory survey involving 26 software engineering researchers. We collected data using a paper-based questionnaire. Results: We found that participants slightly prefer experiments over case studies. Moreover, participants believe there is more useful literature on experiments, than on case studies. By analyzing two different views on the nature of case studies, we found differences in the perceived validity of case studies. Conclusions: The survey provided insights into the perceptions of researchers on case studies and experiments. Moreover, the results help reconcile different views on case studies. Keywords- empirical software engineering; case studies; experiments; survey I.

The evolution of Software Product Lines (SPL) is challenging because stakeholders have to deal with both regular evolution and the co-existence of different products. Our focus of product evolution is on the tasks integrators have to perform to update deployed SPL products with minimal interruption of services. In case of Egemin, our industrial partner, the updates of SPL products is further hampered as a consequence of outdated and imprecise architectural knowledge of deployed products. To facilitate the updates of products, we have developed the architecture-centric approach which comprises two complementary parts: an update viewpoint and a supporting tool. In this paper we present an evaluation of the architecture-centric approach. The approach is compared with the Egemin's current update approach in a controlled experiment. In the experiment 17 professionals were asked to perform 68 updates of logistic systems. The results obtained from the experiment show that the architecture-centric approach significantly improves the correctness of updates and reduces the interruption of services during updates of Egemin's SPL products.

Variability is the ability of a software artifact to be changed for a specific context. Mechanisms to accommodate variability include software product lines, configuration wizards and tools in commercial software, configuration interfaces of software components, or the dynamic runtime composition of web services. Variability is primarily reflected in and facilitated through the software architecture. Also, the software architecture is the centerpiece of software systems and acts as reference point for many development activities, and many of today's software systems are built to accommodate variability. Thus, variability in software architecture should be well-understood and be treated as a first-class concern. The software architecture community acknowledges that variability is a concern of different stakeholders, and in turn affects other concerns. Nevertheless, treating variability related to the architecture and all architecture aspects, as a cross-cutting concern, is currently not well understood. Therefore, VARSA 2011 aims at identifying critical challenges and progressing the state-of-the-art on variability in software architecture.

Despite the fact that some practitioners and researchers report successful stories on Software Product Lines (SPL) adaptation, the evolution of SPL remains challenging. In our research we study a specific aspect of SPL adaptation, namely on updating of deployed products. Our particular focus is on the correct execution of updates and minimal interruption of services during the updates. The update process has two stages. First, the products affected by the evolution must be identified. We call this stage SPL-wide change impact analysis. In the second stage, each of the affected products has to be updated. In our previous work we have addressed the second stage of the update process. In this paper we report on our early results of the first stage: change impact analysis. We discuss how existing variability models can be employed to support automated identification of the products that require an update. The discussion is illustrated with the examples from an educational SPL that we are developing at K.U. Leuven.

Despite the successes of software product lines (SPL), managing the evolution of a SPL remains difficult and error-prone. Our focus of evolution is on the concrete tasks integrators have to perform to update deployed SPL products, in particular products that require runtime updates with minimal interruption. The complexity of updating a deployed SPL product is caused by multiple interdependent concerns, including variability, traceability, versioning, availability, and correctness. Existing approaches typically focus on particular concerns while making abstraction of others, thus offering only partial solutions. An integrated approach that takes into account the different stakeholder concerns is lacking. In this paper, we present an architectural approach for updating SPL products that supports multiple concerns. The approach comprises of two complementary parts: (1) an update viewpoint that defines the conventions for constructing and using architecture views to deal with multiple update concerns, and (2) a supporting framework that provides an extensible infrastructure supporting integrators of a SPL. We evaluated the approach for an industrial SPL for logistic systems providing empirical evidence for its benefits and recommendations.

Advanced vehicle guidance systems use real-time traffic information to route traffic and to avoid congestion. Unfortunately, these systems can only react upon the presence of traffic jams and not to prevent the creation of unnecessary congestion. Anticipatory vehicle routing is promising in that respect, because this approach allows directing vehicle routing by accounting for traffic forecast information. This paper presents a decentralized approach for anticipatory vehicle routing that is particularly useful in large-scale dynamic environments. The approach is based on delegate multiagent systems, i.e., an environment-centric coordination mechanism that is, in part, inspired by ant behavior. Antlike agents explore the environment on behalf of vehicles and detect a congestion forecast, allowing vehicles to reroute. The approach is explained in depth and is evaluated by comparison with three alternative routing strategies. The experiments are done in simulation of a real-world traffic environment. The experiments indicate a considerable performance gain compared with the most advanced strategy under test, i.e., a traffic-message-channel-based routing strategy.

Today's distributed applications such as sensor networks, mobile multimedia applications, and intelligent transportation systems pose huge engineering challenges. Such systems often comprise different components that interact with each other as peers, as such forming a decentralized system. The system components and collaborations change over time, often in unanticipated ways. Multiagent systems belong to a class of decentralized systems that are known for realizing qualities such as adaptability, robustness, and scalability in such environments. A typical way to structure and manage interactions among agents is by means of organizations. Existing approaches usually endow agents with a dual responsibility: on the one hand agents have to play roles providing the associated functionality in the organization, on the other hand agents are responsible for setting up organizations and managing organization dynamics. Engineering realistic multiagent systems in which agents encapsulate this dual responsibility is a complex task. In this article, we present an organization model for context-driven dynamic agent organizations. The model defines abstractions that support application developers to describe dynamic organizations. The organization model is part of an integrated approach, called MACODO: Middleware Architecture for COntext-driven Dynamic agent Organizations. The complementary part of the MACODO approach is a middleware platform that supports the distributed execution of dynamic organizations specified using the abstractions, as described in Weyns et al. [2009]. In the model, the life-cycle management of dynamic organizations is separated from the agents: organizations are first-class citizens, and their dynamics are governed by laws. The laws specify how changes in the system (e.g., an agent joins an organization) and changes in the context (e.g., information observed in the environment) lead to dynamic reorganizations. As such, the model makes it easier to understand and specify dynamic organizations in multiagent systems, and promotes reusing the life-cycle management of dynamic organizations. The organization model is formally described to specify the semantics of the abstractions, and ensure its type safety. We apply the organization model to specify dynamic organizations for a traffic monitoring application.

One of the major challenges in engineering distributed multiagent systems is the coordination necessary to align the behavior of different agents. Decentralization of control implies a style of coordination in which the agents cooperate as peers with respect to each other and no agent has global control over the system, or global knowledge about the system. The dynamic interactions and collaborations among agents are usually structured and managed by means of roles and organizations. In existing approaches agents typically have a dual responsibility: on the one hand playing roles within the organization, on the other hand managing the life-cycle of the organization itself, for example, setting up the organization and managing organization dynamics. Engineering realistic multiagent systems in which agents encapsulate this dual responsibility is a complex task. In this article, we present a middleware for context-driven dynamic agent organizations. The middleware is part of an integrated approach, called MACODO: Middleware Architecture for COntext-driven Dynamic agent Organizations. The complementary part of the MACODO approach is an organization model that defines abstractions to support application developers in describing dynamic organizations, as described in Weyns et al. [2010]. The MACODO middleware offers the life-cycle management of dynamic organizations as a reusable service separated from the agents, which makes it easier to understand, design, and manage dynamic organizations in multiagent systems. We give a detailed description of the software architecture of the MADOCO middleware. The software architecture describes the essential building blocks of a distributed middleware platform that supports the MACODO organization model. We used the middleware architecture to develop a prototype middleware platform for a traffic monitoring application. We evaluate the MACODO middeware architecture by assessing the adaptability, scalability, and robustness of the prototype platform.