This paper proposes a cloud-based framework and platform for end-to-end development and lifecycle management of artificial intelligence (AI) applications. We build on our previous work on platform-level support for cloud-managed deep learning services, and show how the principles of software lifecycle management can be leveraged and extended to enable automation, trust, reliability, traceability, quality control, and reproducibility of AI pipelines. Based on a discussion of use cases and current challenges, we describe a framework for managingAI application lifecycles and its key components. We also show concrete examples that illustrate how this framework enables managing and executing model training and continuous learning pipelines while infusing trusted AI principles.

Continuous Integration (CI) has become a common practice in both industrial and open-source software development. While CI has evidently improved aspects of the software development process, errors during CI builds pose a threat to development efficiency. As an increasing amount of time goes into fixing such errors, failing builds can significantly impair the development process and become very costly. We perform an indepth analysis of build failures in CI environments. Our approach links repository commits to data of corresponding CI builds. Using data from 14 open-source Java projects, we first identify 14 common error categories. Besides test failures, which are by far the most common error category (up to >80% per project), we also identify noisy build data, e.g., induced by transient Git interaction errors, or general infrastructure flakiness. Second, we analyze which factors impact the build results, taking into account general process and specific CI metrics. Our results indicate that process metrics have a significant impact on the build outcome in 8 of the 14 projects on average, but the strongest influencing factor across all projects is overall stability in the recent build history. For 10 projects, more than 50% (up to 80%) of all failed builds follow a previous build failure. Moreover, the fail ratio of the last k=10 builds has a significant impact on build results for all projects in our dataset.

The rise of elastically scaling applications that frequently deploy new machines has led to the adoption of DevOps practices across the cloud engineering stack. So-called configuration management tools utilize scripts that are based on declarative resource descriptions and make the system converge to the desired state. It is crucial for convergent configurations to be able to gracefully handle transient faults, e.g., network outages when downloading and installing software packages. In this paper we introduce a conceptual framework for asserting reliable convergence in configuration management. Based on a formal definition of configuration scripts and their resources, we utilize state transition graphs to test whether a script makes the system converge to the desired state under different conditions. In our generalized model, configuration actions are partially ordered, often resulting in prohibitively many possible execution orders. To reduce this problem space, we define and analyze a property called preservation, and we show that if preservation holds for all pairs of resources, then convergence holds for the entire configuration. Our implementation builds on Puppet, but the approach is equally applicable to other frameworks like Chef, Ansible, etc. We perform a comprehensive evaluation based on real world Puppet scripts and show the effectiveness of the approach. Our tool is able to detect all idempotence and convergence related issues in a set of existing Puppet scripts with known issues as well as some hitherto undiscovered bugs in a large random sample of scripts.

The advent of the Smart City domain has led to the creation of massive amounts of diverse data. Stakeholders in this domain need to be able to analyze this data in order to make informed planning decisions. To address this complex task, Distributed Analytical Environments (DAEs) have emerged. These environments consist of different distributed analytical and data services, which are composed in a dynamic way to deliver insights that are crucial for stakeholders. Since these environments deal with business critical and sensitive information, strict compliance constraints apply. These constraints lead to situations where certain concrete services are not allowed to exchange data, even though their interaction is necessary to produce the desired results. Finding a valid solution in the space of possible instantiations is a non-trivial problem. In this paper we introduce Nomads, a framework that enables service mobility in such constrained dynamic composition environments to overcome aforementioned restrictions. The framework improves the overall satisfiability and therefore also the quality of constrained DAEs. We outline the requirements of a representative DAE scenario, provide a detailed problem formulation, and then discuss the service mobility framework along with our solution finding algorithm. The evaluation demonstrates that the Nomads framework considerably increases the number of successfully performed compositions even in highly constrained environments.

Building Infrastructure-as-a-Service (IaaS) applications today is a complex, repetitive, and error-prone endeavor, as IaaS does not provide abstractions on top of virtual machines. This article presents JC loud S cale , a Java-based middleware for moving elastic applications to IaaS clouds, with minimal adjustments to the application code. We discuss the architecture and technical features, as well as evaluate our system with regard to user acceptance and performance overhead. Our user study reveals that JC loud S cale allows many participants to build IaaS applications more efficiently, compared to industrial Platform-as-a-Service (PaaS) solutions. Additionally, unlike PaaS, JC loud S cale does not lead to a control loss and vendor lock-in.

Platform-as-a-Service (PaaS) should support the design, deployment, execution, test and monitoring of native elastic systems constructed from elastic service units based on multi-dimensional elasticity requirements. In this paper, we discuss fundamental building blocks for enabling multi-dimensional elasticity programming of software-defined elastic systems. We describe CoMoT, a novel PaaS for elasticity in the cloud that is developed based on these fundamental building blocks.

SUMMARY The Cloud computing paradigm provides the basis for a class of platforms and applications that face novel challenges related to multi‐tenancy, adaptivity, and elasticity. To account for service delivery guarantees in the face of ever increasing levels of heterogeneity, scale, and dynamism, service provisioning in the Cloud has raised the demand for systematic and flexible approaches to monitoring and adaptation of applications. In this paper, we tackle this issue and present a framework for efficient runtime management of Cloud environments and distributed heterogeneous systems in general. A novel domain‐specific language termed MONINA is introduced that allows to define integrated monitoring and adaptation functionality for controlling such systems. We propose a mechanism for optimal deployment of the defined control operators onto available computing resources. Deployment is based on solving a quadratic programming problem, which aims at achieving minimized reaction times, low overhead, and scalable monitoring and adaptation. The monitoring infrastructure is based on a distributed messaging middleware, providing high level of decoupling and allowing new monitoring nodes to join the system dynamically. We provide a detailed formalization of the problem domain, discuss architectural details, highlight the implementation of the developed prototype, and put our work into perspective with existing work in the field. Copyright © 2014 John Wiley & Sons, Ltd.

The process of engineering and provisioning service-based systems (SBS) follows a complex and dynamic lifecycle with different phases and levels of abstraction. We tackle the problem of making this lifecycle explicit, providing development time and runtime support for evolutionary changes in such systems. SBSs are modeled as integrated ecosystems consisting of four conceptual layers (or phases): design, implementation, deployment, and runtime. Our work is driven by the notion that identifying the right changes (monitoring) and effecting of these changes (adaptation) usually takes place individually on each layer. While considering changes on a single layer (e.g., runtime adaptation) is often sufficient, some cases require systematic escalation to adjacent layers. We present a generic lifecycle model that provides an abstracted view of the problem domain and can be mapped to concrete artifacts on each individual layer. We introduce a real-life scenario taken from the telecommunications domain, which serves as the basis for discussion of the challenges and our solution. Based on the scenario and our experience from a research project on Virtual Service Platforms, we evaluate three concrete use cases which illustrate the diversity of evolutionary changes supported by the approach.

Cloud-based elastic computing systems dynamically change their resources allocation to provide consistent quality of service and minimal usage of resources in the face of workload fluctuations. As elastic systems are increasingly adopted to implement business critical functions in a cost-efficient way, their reliability is becoming a key concern for developers. Without proper testing, cloud-based systems might fail to provide the required functionalities with the expected service level and costs. Using system testing techniques, developers can expose problems that escaped the previous quality assurance activities and have a last chance to fix bugs before releasing the system in production. System testing of cloud-based systems accounts for a series of complex and time demanding activities, from the deployment and configuration of the elastic system, to the execution of synthetic clients, and the collection and persistence of execution data. Furthermore, clouds enable parallel executions of the same elastic system that can reduce the overall test execution time. However, manually managing the concurrent testing of multiple system instances might quickly overwhelm developers' capabilities, and automatic support for test generation, system test execution, and management of execution data is needed. In this demo we showcase AUToCLES, our tool for automatic testing of cloud-based elastic systems. Given specifications of the test suite and the system under test, AUToCLES implements testing as a service (TaaS): It automatically instantiates the SUT, configures the testing scaffoldings, and automatically executes test suites. If required, AUToCLES can generate new test inputs. Designers can inspect executions both during and after the tests.

In this paper we propose a provider-managed, model-based adaptation approach for cloud computing applications, allowing customers to easily specify application behavior goals or adaptation rules. Delegating control over corrective actions to the cloud provider will pose advantages for both, customers and providers. Customers are relieved of effort and expertise requirements necessary to build sophisticated adaptation solutions, while providers can incorporate and analyze data from a multitude of customers to improve adaptation decisions. The envisioned approach will enable increased application performance, as well as cost savings for customers, whereas providers can manage their infrastructure more efficiently.

CONTEXT: A distributed business process is executed in a distributed computing environment. The service-oriented architecture (SOA) paradigm is a popular option for the integration of software services and execution of distributed business processes. Entailment constraints, such as mutual exclusion and binding constraints, are important means to control process execution. Mutually exclusive tasks result from the division of powerful rights and responsibilities to prevent fraud and abuse. In contrast, binding constraints define that a subject who performed one task must also perform the corresponding bound task(s). OBJECTIVE: We aim to provide a model-driven approach for the specification and enforcement of task-based entailment constraints in distributed service-based business processes. METHOD: Based on a generic metamodel, we define a domain-specific language (DSL) that maps the different modeling-level artifacts to the implementation-level. The DSL integrates elements from role-based access control (RBAC) with the tasks that are performed in a business process. Process definitions are annotated using the DSL, and our software platform uses automated model transformations to produce executable WS-BPEL specifications which enforce the entailment constraints. We evaluate the impact of constraint enforcement on runtime performance for five selected service-based processes from existing literature. RESULTS: Our evaluation demonstrates that the approach correctly enforces task-based entailment constraints at runtime. The performance experiments illustrate that the runtime enforcement operates with an overhead that scales well up to the order of several ten thousand logged invocations. Using our DSL annotations, the user-defined process definition remains declarative and clean of security enforcement code. CONCLUSION: Our approach decouples the concerns of (non-technical) domain experts from technical details of entailment constraint enforcement. The developed framework integrates seamlessly with WS-BPEL and the Web services technology stack. Our prototype implementation shows the feasibility of the approach, and the evaluation points to future work and further performance optimizations.

SUMMARY This paper addresses integration testing of data‐centric and event‐based dynamic service compositions. The compositions under test define abstract services that are replaced by concrete candidate services at runtime. Testing all possible instantiations of a composition leads to combinatorial explosion and is often infeasible. We consider data dependencies between services as potential points of failure and introduce the k‐node data flow test coverage metric, which helps to significantly reduce the number of test combinations. We formulate a combinatorial optimization problem for generating minimal sets of test cases. On the basis of this formalization, we present a mapping to the model of FoCuS, a coverage analysis tool. FoCuS efficiently computes near‐optimal solutions, which are used to automatically generate test instances. The proposed approach is applicable to various composition paradigms. We illustrate the end‐to‐end practicability based on an integrated scenario, which uses two diverse composition techniques: on the one hand, the Web Services Business Process Execution Language and on the other hand, WS‐Aggregation, a platform for event‐based service composition. Copyright © 2013 John Wiley & Sons, Ltd.

For providers of composite services, preventing cases of SLA violations is crucial. Previous work has established runtime adaptation of compositions as a promising tool to achieve SLA conformance. However, to get a realistic and complete view of the decision process of service providers, the costs of adaptation need to be taken into account. In this paper, we formalize the problem of finding the optimal set of adaptations, which minimizes the total costs arising from SLA violations and the adaptations to prevent them. We present possible algorithms to solve this complex optimization problem, and detail an end-to-end system based on our earlier work on the PREvent (prediction and prevention based on event monitoring) framework, which clearly indicates the usefulness of our model. We discuss experimental results that show how the application of our approach leads to reduced costs for the service provider, and explain the circumstances in which different algorithms lead to more or less satisfactory results.

Providers of applications deployed in an Infrastructure-as-a-Service cloud permanently face the decision of whether it is more cost-efficient to scale up(i.e., rent more resources from the cloud) or to delay incoming requests, even though doing so may lead to dissatisfied customers and broken service level agreements. This decision is further complicated by the fact that not all customers have the same agreements, and not all requests require the same amount of resources devoted to them. In this paper, we present an approach for optimally scheduling incoming requests to virtual computing resources in the cloud, so that the sum of payments for resources and loss incurred by service level agreement violations is minimized. We discuss our approach based on an illustrative use case. Furthermore, we present a numerical evaluation based on real-life request data, which shows that our agreement-aware algorithm improves upon earlier work, which does not take service level agreements into account.

In this paper we study fault localization techniques for identification of incompatible configurations and implementations in service-based applications. We propose an approach using pooled decision trees for localization of faulty service parameter and binding configurations, explicitly addressing temporary and changing fault conditions.

Today, most tools for processing big data are batch-oriented. However, many scenarios require continuous, online processing of data streams and events. We present ESC, a new stream computing engine. It is designed for computations with real-time demands, such as online data mining. It offers a simple programming model in which programs are specified by directed acyclic graphs (DAGs). The DAG defines the data flow of a program, vertices represent operations applied to the data. The data which are streaming through the graph are expressed as key/value pairs. ESC allows programmers to focus on the problem at hand and deals with distribution and fault tolerance. Furthermore, it is able to adapt to changing computational demands. In the cloud, ESC can dynamically attach and release machines to adjust the computational capacities to the current needs. This is crucial for stream computing since the amount of data fed into the system is not under the platform's control. We substantiate the concepts we propose in this paper with an evaluation based on a high-frequency trading scenario.

This paper addresses the problem of integration testing of data-centric dynamic compositions in service-based systems. These compositions define abstract services, which are replaced by invocations to concrete candidate services at runtime. Testing all possible runtime instances of a composition is often unfeasible. We regard data dependencies between services as potential points of failure, and introduce the k-node data flow test coverage metric. Limiting the level of desired coverage helps to significantly reduce the search space of service combinations. We formulate the problem of generating a minimum set of test cases as a combinatorial optimization problem. Based on the formalization we present a mapping of the problem to the data model of FoCuS, a coverage analysis tool developed at IBM. FoCuS can efficiently compute near-optimal solutions, which we then use to automatically generate and execute test instances of the composition. We evaluate our prototype implementation using an illustrative scenario to show the end-to-end practicability of the approach.

In this paper, we present an approach for identity and access management (IAM) in the context of (cross-organizational) service-oriented architectures (SOA). In particular, we defined a domain-specific language (DSL) for role-based access control (RBAC) that allows for the definition of IAM policies for SOAs. For the application in a SOA context, our DSL environment automatically produces WS-BPEL (Business Process Execution Language for Web services) specifications from the RBAC models defined in our DSL. We use the WS-BPEL extension mechanism to annotate parts of the process definition with directives concerning the IAM policies. At deployment time, the WS-BPEL process is instrumented with special activities which are executed at runtime to ensure its compliance to the IAM policies. The algorithm that produces extended WS-BPEL specifications from DSL models is described in detail. Thereby, policies defined via our DSL are automatically mapped to the implementation level of a SOA-based business process. This way, the DSL decouples domain experts' concerns from the technical details of IAM policy specification and enforcement. Our approach thus enables (non-technical) domain experts, such as physicians or hospital clerks, to participate in defining and maintaining IAM policies in a SOA context. Based on a prototype implementation we also discuss several performance aspects of our approach.