The widespread adoption of Large Language Models (LLMs) in enterprise applications has created a critical need for systematic evaluation of Generative AI Systems (GenAIS) that integrate orchestration frameworks, foundation models, and deployment optimizations. This paper presents the first comprehensive evaluation study specifically designed to evaluate the performance trade-offs between orchestration frameworks (LangChain, LlamaIndex), foundation models, and deployment constraints across multiple application domains. Our study systematically evaluates eight foundation models across question-answering with Retrieval-Augmented Generation (RAG) and mathematical reasoning tasks, measuring latency, accuracy, resource utilization, and power consumption under various optimization strategies including adaptive context windowing and concurrent request processing. Through extensive empirical analysis, we demonstrate that framework selection significantly impacts system performance independent of model choice, with task-specific trade-offs emerging across workload types. For retrieval-heavy RAG workloads, LlamaIndex delivers 3-15% lower latency, around 60% lower peak memory usage, and 2-4% more energy consumption. For reasoning-intensive mathematical tasks, Langchain achieves higher accuracy across all tested models and provides more predictable latency profiles and reasonable resource consumption, making it preferable for strict SLA environments. Adaptive context windowing does not yield uniform gains and offers only modest, model-dependent improvements over fixed policies, whereas increasing concurrency improves aggregate throughput but, beyond moderate loads, consistently inflates tail latency and degrades SLA predictability. These findings underscore the decisive role of orchestration design in GenAIS performance and provide empirical guidance for balancing efficiency, accuracy, and scalability in real-world deployments.

Generative Adversarial Networks (GANs) have shown promise in enabling intelligent and privacy-preserving data synthesis within decentralized environments like the Internet of Vehicles (IoV). However, applying GANs in such dynamic, resource-constrained, and latency-sensitive settings poses significant challenges, particularly due to intermittent connectivity, mobility-induced disruptions, and communication overhead. In this paper, we propose a novel collaborative Multi-Discriminator GAN (MD-GAN) approach tailored for IoV, where Road Side Units (RSUs) act as generators and vehicles function as mobile discriminators providing distributed feedback. Unlike traditional centralized or synchronous GAN setups, our proposed approach leverages an early update strategy that allows generators to proceed once a minimum threshold of feedback is received, along with a feedback selection mechanism that considers only discriminators offering improved loss scores. We adopt the Wasserstein GAN (WGAN) formulation to ensure stable convergence under sparse and asynchronous feedback conditions. The proposed approach is implemented through a hybrid simulation architecture combining NS3 and PyTorch to model both networklevel interactions and learning dynamics. Experimental results demonstrate that the proposed approach significantly reduces training latency and network overhead compared to the baselines, while maintaining competitive generator accuracy. Our findings highlight the effectiveness of communication-efficient GAN training in highly dynamic vehicular networks.

Modern cloud-native applications are adopting the microservice architecture in which applications are deployed in lightweight containers that run inside a virtual machine (VM). Containers running different services are often co-located inside the same virtual machine. While this enables better resource optimization, it can cause interference among applications. This can lead to performance degradation. Detecting the cause of performance degradation at runtime is crucial to decide the correct remediation action such as, but not limited to, scaling or migrating. We propose a non-intrusive detection technique that differentiates between degradation caused by load and by interference. First, we define an operational zone for the application. Then we define a disambiguation method that uses models to classify interference and normal load. In contrast to previous work, our proposed detection technique does not require intrusive application instrumentation and incurs minimal performance overhead. We demonstrate how we can design effective Machine Learning models that can be generalized to detect interference from different types of applications. We evaluate our technique using realistic microservice benchmarks on AWS EC2. The results show that our approach outperforms existing interference detection techniques in F_1 score by at least 2.75% and at most 53.86%.

As microservice and cloud computing operations increasingly adopt automation, the importance of models for fostering resilient and efficient adaptive architectures becomes paramount. This paper presents InstantOps, a novel approach to system failure prediction and root cause analysis leveraging a three-fold modality of IT observability data: logs, metrics, and traces. The proposed methodology integrates Graph Neural Networks (GNN) to capture spatial information and Gated Recurrent Units (GRU) to encapsulate the temporal aspects within the data. A key emphasis lies in utilizing a stitched representation derived from logs, microservices events(e.g. Image Pull Back Off, PVC Pending), and resource metrics to predict system failures proactively. The traces are aggregated to construct a comprehensive service call flow graph and represented as a dynamic graph. Furthermore, permutation testing is applied to harness node scores, aiding in the identification of root causes behind these failures. To evaluate the efficiency of InstantOps, we utilized in-house data from the open-source application Quote of the Day (QoTD) as well as two publicly available datasets, MicroSS and Train Ticket. The F1 scores obtained in predicting the system failures from these data sets were 0.96, 0.98, and 0.97, respectively, beating the stateof-the-art. Additionally, we further evaluated the efficiency of root cause analysis using MAR and MFR. These results also outperform the state of the art.

With the advent of 5G networks and the rise of the Internet of Things (IoT), Content Delivery Networks (CDNs) are increasingly extending into the network edge. This shift introduces unique challenges, particularly due to the limited cache storage and the diverse request patterns at the edge. These edge environments can host traffic classes characterized by varied object-size distributions and object-access patterns. Such complexity makes it difficult for traditional caching strategies, which often rely on metrics like request frequency or time intervals, to be effective. Despite these complexities, the optimization of edge caching is crucial. Improved byte hit rates at the edge not only alleviate the load on the network backbone but also minimize operational costs and expedite content delivery to end-users. In this paper, we introduce HR-Cache, a comprehensive learning-based caching framework grounded in the principles of Hazard Rate (HR) ordering, a rule originally formulated to compute an upper bound on cache performance. HR-Cache leverages this rule to guide future object eviction decisions. It employs a lightweight machine learning model to learn from caching decisions made based on HR ordering, subsequently predicting the "cache-friendliness'' of incoming requests. Objects deemed "cache-averse'' are placed into cache as priority candidates for eviction. Through extensive experimentation, we demonstrate that HR-Cache not only consistently enhances byte hit rates compared to existing state-of-the-art methods but also achieves this with minimal prediction overhead. Our experimental results, using three real-world traces and one synthetic trace, indicate that HR-Cache consistently achieves 2.2-14.6% greater WAN traffic savings than LRU. It outperforms not only heuristic caching strategies but also the state-of-the-art learning-based algorithm.

In self-adaptive systems, model-based control assumes decisions are taken based on a model that is identified at run-time. The model is built by measuring the control inputs, disturbances, and outputs of the controlled system and fitting the data into a function. Models can be accurate locally, that is, for data already seen by the system and by the model identification method. However, many times an Autonomic Manager (AM) needs to move the cloud-native applications into new operational points, e.g. by adding new applications to the shared environment, scaling applications or consolidating resources. There are no data points yet for these new operational regions to have any certainty that the prediction models are accurate. In this paper, we propose a method to identify a model that predicts metrics at any unexplored operational point of a cloud-native application. The method is based on a lightweight Look-Ahead Scanner (LAS) mechanism that explores different operational points by injecting controlled short-lived load. We evaluate our method on realistic applications deployed on public clouds. We show that the proposed method can build models that outperform the state of the art ML models by 42%.

Computing systems form the backbone of many areas in our society, from manufacturing to traffic control, healthcare, and financial systems. When software plays a vital role in the design, construction, and operation, these systems are referred to as software-intensive systems. Self-adaptation equips a software-intensive system with a feedback loop that either automates tasks that otherwise need to be performed by human operators or deals with uncertain conditions. Such feedback loops have found their way to a variety of practical applications; typical examples are an elastic cloud to adapt computing resources and automated server management to respond quickly to business needs. To gain insight into the motivations for applying self-adaptation in practice, the problems solved using self-adaptation and how these problems are solved, and the difficulties and risks that industry faces in adopting self-adaptation, we performed a large-scale survey. We received 184 valid responses from practitioners spread over 21 countries. Based on the analysis of the survey data, we provide an empirically grounded overview the of state of the practice in the application of self-adaptation. From that, we derive insights for researchers to check their current research with industrial needs, and for practitioners to compare their current practice in applying self-adaptation. These insights also provide opportunities for applying self-adaptation in practice and pave the way for future industry-research collaborations.

Serverless computing has emerged in recent years as the new computing paradigm adopted by key players in the industry for software development. This new paradigm has seen rapid growth in adoption due to its unique billing model and scaling characteristics. Public cloud providers such as Amazon Web Services (AWS) offer several configurations and language runtimes for their serverless functions. Although extensively explored by the research community, this field still lacks current studies that address the many challenges developers face when leveraging serverless functions for real-world applications. One of these challenges that are often overseen by many programmers is the cold start problem which is present in any serverless application. For this reason, we propose the first study to characterize the underlying cold start impacts caused by the choice of language runtime, application size, memory size and deployment type on AWS Lambda. In this paper, we analyze the performance of the container-based deployment and ZIP-based deployment of AWS Lambda using a variety of language runtimes and applications running with different function configurations; then we propose guidelines for developers and cloud managers to consider when deploying/managing the workloads on the cloud.

Cloud-native applications are often composed of lightweight containers and conform to the microservice architecture. Cloud providers offer platforms for container hosting and orchestration. These platforms reduce the level of support required from the application owner as operational tasks are delegated to the platform. Furthermore, containers belonging to different applications can be co-located on the same virtual machine to utilize resources more efficiently. Given that there are underlying shared resources and consequently potential performance interference, predicting the level of interference before deciding to share virtual machines can avoid undesirable performance deterioration. We propose a lightweight performance interference modelling technique for cloud-native microservices. The technique constructs ML models for response time prediction and can dynamically account for changing runtime conditions through the use of a sliding window method. We evaluate our technique against realistic microservices on AWS EC2. Our technique outperforms baseline and competing techniques in MAPE by at least 1.45% and at most 92.04%.

Function as a Service (FaaS) is a new software technology with promising features such as automated resource management and auto-scaling. Since these operational aspects are transparent, software engineers may not fully understand the scaling characteristics as well as limitations of this technology and this lack of information can lead to undesired performance results. To address these concerns, we perform a study to characterize FaaS' scalability with intensive workloads on three popular FaaS cloud platforms, namely Amazon AWS Lambda, IBM and Azure Cloud Function. We also study a workload smoother design pattern to examine if it enhances FaaS overall performance. The results show that different FaaS platforms adopt distinct scaling strategies and by applying a workload smoother, software engineers can achieve 99 - 100% success rates compared to 60 - 80% when FaaS' system is saturated.

In this article, we propose and implement a distributed autonomic manager that maintains service level agreements (SLA) for each application scenario. The proposed autonomic manager supports SLAs by configuring the bandwidth ratios for each application scenario and uses an overlay network as an infrastructure. The most important aspect of the proposed autonomic manager is its scalability which allows us to deal with geographically distributed cloud-based applications and a large volume of computation. This can be useful in look ahead optimization and in adaptations using complex models, such as machine learning. We formally prove the safety and liveness properties of the implemented distributed algorithms. Through experiments on the Amazon AWS cloud, using two different use cases, we demonstrate the elasticity and flexibility of the autonomic manager as a measure of its applicability to different cloud applications with different types of workloads. Experiments also demonstrate that increasing the size of a look ahead window, up to a certain size, improves the accuracy of the adaptation decisions by up to 50%.

Microservice architecture has transformed the way developers are building and deploying applications in the nowadays cloud computing centers. This new approach provides increased scalability, flexibility, manageability, and performance while reducing the complexity of the whole software development life cycle. The increase in cloud resource utilization also benefits microservice providers. Various microservice platforms have emerged to facilitate the DevOps of containerized services by enabling continuous integration and delivery. Microservice platforms deploy application containers on virtual or physical machines provided by public/private cloud infrastructures in a seamless manner. In this article, we study and evaluate the provisioning performance of microservice platforms by incorporating the details of all layers (i.e., both micro and macro layers) in the modeling process. To this end, we first build a microservice platform on top of Amazon EC2 cloud and then leverage it to develop a comprehensive performance model to perform what-if analysis and capacity planning for microservice platforms at scale. In other words, the proposed performance model provides a systematic approach to measure the elasticity of the microservice platform by analyzing the provisioning performance at both the microservice platform and the back-end macroservice infrastructures.

DevOps is an emerging paradigm that integrates the development and operations teams to enable fast and efficient continuous delivery of software. Applications and services deployed on cloud platforms can benefit from implementing the DevOps practice. This involves using different tools for enabling end-to-end automation to ensure continuous deployment and maintain good Quality-of-Service. Self-Adaptive systems can support the DevOps process by automating service deployment and maintenance without manual intervention by employing a MAPE-K (Monitoring, Analysis, Planning, Execution- Knowledge) framework. While industrial MAPE-K tools are robust and built for production environments, they lack the flexibility to adapt large applications on multi-cloud environments. Academic models are more flexible and can be used to perform sophisticated self-adaption, but can lack the robustness to be used in production environments. In this paper, we present a MAPE-K framework that is built with existing Components-off-the-Shelf (COTS) that interacts with each other to perform self-adaptive actions on multi-cloud environments. By integrating existing COTS, we are able to deploy a MAPE-K framework efficiently to support DevOps for applications running on a multi-cloud environment. We validate our framework with a prototype implementation and demonstrate its practical feasibility by a detailed case study done on a real industrial platform.

Web services hosted on public cloud platforms are often subjected to performance anomalies. Runtime detection of such anomalies is crucial for operations in cloud data centers. With ever-increasing data center size, complexities in software applications and dynamic traffic workload patterns, automatically detecting performance anomalies is a challenging task. In this paper, we propose RAD, a lightweight runtime anomaly detection technique that does not require application level instrumentation and can be easily implemented for detecting anomalies in multi-tier cloud-based Web services. In particular, we focus on anomalies that are difficult to detect by simply monitoring system level metrics alone, such as anomalies that are caused by contention from within a service and also those caused by shared resource contention by other services running on the cloud. RAD continuously monitors service resource metrics and uses a queuing network model to detect performance anomalies at runtime. Additionally, RAD uses historical data and implements a statistical methodology to diagnose the root cause of an anomaly. We evaluate RAD on a private cloud and also on the EC2 public cloud platform to show that RAD incurs extremely low levels of performance overhead on the service and is effective for detecting anomalies in both multi-tier monolithic services and microservices.

Auto-scalability is a common approach for management of cloud applications where resources are provisioned and de-provisioned on demand. Because of its automatic nature, auto-scaling can be exploited for various reasons that can hugely reduce the overall profit. Therefore, it becomes vital to consider the trade-off between the added revenue as a result of auto-scaling and its corresponding cost. To this end, we propose a novel autonomic solution to the management of cloud Web applications whose goal is to optimize the profit. At the core of the solution, there is an optimization module that considers revenue model as well as cost model and uses a number of run-time performance models to derive the best course of action if there is a need for adaptation. Using software defined features that include compute and network programmability, our proposed solution helps applications to optimize their resource allocation to best meet their business requirements. Our experiment results on a hybrid cloud validate the applicability of the proposed solution and demonstrate that it can increase the profit substantially higher compared to a number of baseline approaches.

In this paper, we propose and implement a distributed autonomic manager to maintain service level agreements (SLA) for each application' scenario. The proposed autonomic manager seeks to support SLAs by configuring bandwidth ratios for each application scenario using overlay network before provisioning more computing resources. The most important aspect of the proposed autonomic manager is scalability which allows us to deal with geographically distributed cloud-based applications and large volume of computation. This can be useful in look ahead optimization and when using complex models, such as machine learning. Through experiments on Amazon AWS cloud, we demonstrate the elasticity of the autonomic manager.

Adaptability is an expected property of modern software systems in order to cope with changes in the environment by self-adjusting their structure and behaviour. Robustness is a crucial component of adaptability and it refers to the ability of the systems to deal with uncertainty, i.e. perturbations or unmodelled system dynamics that can affect the quality of the adaptation. Cost is another important property to ensure that resources are used prudently and frugally, whenever possible. Engineering robust and cost-effective adaptive systems can be accomplished using a control theory approach. In this paper, we show how to implement a model identification adaptive controller (MIAC) using a combination of performance and control models and how such a system satisfies the goals for robustness and cost-effectiveness. The controller we employ is multi-input, meaning that it can issue a variety of commands to adapt the system and multi-output, meaning it can regulate multiple performance indicators simultaneously. We show that such a solution can account for uncertainty and modelling errors and efficiently adapt a web application with multiple tiers of functionality spanning multiple layers of deployment, software and virtual machines, on Amazon EC2, an actual cloud environment.

Virtualization of resources in cloud computing has enabled developers to commission and recommission resources at will and on demand. This virtualization is a coin with two sides. On one hand, the flexibility in managing virtual resources has enabled developers to efficiently manage their costs; they can easily remove unnecessary resources or add resources temporarily when the demand increases. On the other hand, the volatility of such environment and the velocity with which changes can occur may have a greater impact on the economic position of a stakeholder and the business balance of the overall ecosystem. In this work, we recognise the business ecosystem of cloud computing as an economy of scale and explore the effect of this fact on decisions concerning scaling the infrastructure of web applications to account for fluctuations in demand. The goal is to reveal and formalize opportunities for economically optimal scaling that takes into account not only the cost of infrastructure but also the revenue from service delivery and eventually the profit of the service provider. The end product is a scaling mechanism that makes decisions based on both performance and economic criteria and takes adaptive actions to optimize both performance and profitability for the system.

Web software systems generally reside in highly volatile environments, their incoming traffic may be subject to sharp fluctuations from reasons that cannot always be captured or predicted. Cloud computing provides a solution to this problem by offering flexible resources, like containers, which can be quickly and easily scaled according to the current workload needs. Automating this process is a key aspect for the management of modern web software systems, and there is a plethora of methods to implement autonomic management systems. In this work, we review three of these methods, a threshold-based approach, a control-based approach and a model-based approach. We design and run a number of experiments for all three systems with different workloads to evaluate their ability to manage the software system and how well they do so. Our experiments were conducted on the Amazon EC2 cloud with Docker containers.

In this paper, we propose and implement a machine learning based application autonomic management system that controls the bandwidth rates allocated to each scenario of a web application to postpone scaling out for as long as possible. Through experiments on Amazon AWS cloud, we demonstrate that the autonomic manager is able to quickly meet Service level Agreement (SLA) and reduce the SLA violations by 56% compared to a previous heuristic-based approach.

Self-adaptive applications are engineered to adapt to operation conditions to continuously meet application requirements at run-time. Before the emergence of Software Defined Networking (SDN) and new network virtualization technologies, the common run-time adaptation actions were limited to changes in computing and storage resources (i.e., scaling in/out). However, SDN creates new avenues for adaptation strategies that complement or solves issues associated with computing adaptations. Hence, in this thesis, we propose to use SDN and network virtualization techniques to build self-adaptive applications. The research goal is to build applications that are self-protecting, self-managing and self-optimizing using both computing and networking adaptations. We design, implement and verify such self-adaptive applications on real cloud environment.

This paper presents an adaptive service management mechanism that maintains service level agreement through use of overlay networks that are deployed over the cloud provider network. The application autonomic manager strives to maintain the SLA without provisioning new resources for as long as possible. Through continuous monitoring and analysis, autonomic manager uses software defined networking (SDN) to dynamically apply policies to the flows of requests that travel through the application components. We implement and evaluate the proposed method on a hybrid cloud environment. Through extensive experiments, we show that the management mechanism can successfully maintain the SLA of services while it avoids provisioning extra resources which is the common approach in cloud.

In this paper, we present an architecture and implementation for self-managing cloud application using overlay networks and software defined networking (SDN). Through real world experiments on Amazon EC2 and Smart Applications on Virtual Infrastructure (SAVI) cloud, we demonstrate how our management mechanism autonomously maintains SLAs of application scenarios without provisioning extra resources.

Management of large service centers and clouds requires adaptation to changing conditions and workload. Optimal deployments are possible on private clouds if there is full knowledge of the applications. Previous work,CloudOpt,combines large-scale linear optimization with a detailed non-linear performance model of every application to determine such deployments. However, when conditions change the new solution is independent of the previous deployment, and often requires excessive changes. Practical adaptive management needs a controlled compromise between the effort and cost of changing the current deployment, and the value lost by not making a change. This paper enhances CloudOpt by incorporating adjustablepersistencetechniques to constrain the deployment changes in each update. Three such techniques are compared, and the best ones reduce the reconfiguration effort at each adaptation step by about 90 percent, with about a 5-10 percent penalty in running costs. Persistence techniques impose no reduction in the scalability of CloudOpt.

Web technologies along with the virtualization of computation and storage resources have allowed extensive flexibility in the development, deployment and delivery of software solutions. On one hand, this flexibility has allowed engineers to commission and re purpose resources on demand and at will. On the other hand, the flexibility has given rise to new business models and interesting implications in the economics of software ecosystems. In this work, we propose a model to capture the technical and economic transactions within a web software ecosystem deployed on a dynamic cloud environment. Next, we demonstrate the use of such model to define rules for a more economically efficient adaptation strategy for web applications as their incoming traffic fluctuates.

Microservice architecture has started a new trend for application development/deployment in cloud due to its flexibility, scalability, manageability and performance. Various microservice platforms have emerged to facilitate the whole software engineering cycle for cloud applications from design, development, test, deployment to maintenance. In this paper, we propose a performance analytical model and validate it by experiments to study the provisioning performance of microservice platforms. We design and develop a microservice platform on Amazon EC2 cloud using Docker technology family to identify important elements contributing to the performance of microservice platforms. We leverage the results and insights from experiments to build a tractable analytical performance model that can be used to perform what-if analysis and capacity planning in a systematic manner for large scale microservices with minimum amount of time and cost.

Distributed Denial of Service (DDoS) attacks are one of the main concerns for online service providers because of their impact on cost/revenue and reputation. This paper presents Completely Automated DDoS Attack Mitigation Platform (CAAMP), a novel platform to mitigate DDoS attacks on public cloud applications using capabilities of software defined infrastructure and network function virtualization techniques. When suspicious traffic is identified, CAAMP deploys a copy of the application's topology on-the-fly (a shark tank) on an isolated environment in a private cloud. It then creates a virtual network that will host the shark tank. Software defined networking (SDN) controller programs the virtual switches dynamically to redirect the suspicious traffic to the shark tank until final decision is made. If traffic is proved to be non-malicious, SDN controller installs flow rules on the switches to redirect the traffic back to the original application. Thus, CAAMP autonomically protects applications against potential DDoS threats and lowers the false positives associated with common detection mechanisms by leveraging resources from a private cloud.

Large-scale software systems like Amazon and eBay must be load tested to ensure they can handle hundreds and millions of current requests in the field. Load testing usually lasts for a few hours or even days and generates large volumes of system behavior data (execution logs and counters). This data must be properly analyzed to check whether there are any performance problems in a load test. However, the sheer size of the data prevents effective manual analysis. In addition, unlike functional tests, there is usually no test oracle associated with a load test. To cope with these challenges, there have been many analysis techniques proposed to automatically detect problems in a load test by comparing the behavior of the current test against previous test(s). Unfortunately, none of these techniques compare their performance against each other. In this paper, we have proposed a framework, which evaluates and compares the effectiveness of different test analysis techniques. We have evaluated a total of 23 test analysis techniques using load testing data from three open source systems. Based on our experiments, we have found that all the test analysis techniques can effectively build performance models using data from both buggy or non-buggy tests and flag the performance deviations between them. It is more cost-effective to compare the current test against two recent previous test(s), while using testing data collected under longer sampling intervals (≥ 180 seconds). Among all the test analysis techniques, Control Chart, Descriptive Statistics and Regression Tree yield the best performance. Our evaluation framework and findings can be very useful for load testing practitioners and researchers. To encourage further research on this topic, we have made our testing data publicity available to download.

Designing an adaptive system to meet its quality constraints in the face of environmental uncertainties can be a challenging task. In a cloud environment, a designer has to consider and evaluate different control points, that is, those variables that affect the quality of the software system. This article presents a methodology for designing adaptive systems in cloud environments. The proposed methodology consists of several phases that take high-level stakeholders’ adaptation goals and transform them into lower-level MAPE-K loop control points. The MAPE-K loops are then activated at runtime using search-based algorithms. Our methodology includes the elicitation, ranking, and evaluation of control points, all meant to enable a runtime search-based adaptation. We conducted several experiments to evaluate the different phases of our methodology and to validate the runtime adaptation efficiency.

This paper presents K-Feed (Knowledge Feed), a platform for real-time application performance analysis and provisioning in the cloud. K-Feed can perform at scale monitoring, analysis, and provisioning of cloud applications. We explain the components, the implementation and the validation of the K-feed platform. To illustrate its feasibility, we use it to monitor and build a performance model of a clustered web application. To model the application, we use off the shelf components.

This paper presents a model and an algorithm for optimal service placement (OSP) of a set of N-tier software systems, subject to dynamic changes in the workload, Service Level Agreements (SLA), and administrator preferences. The objective function models the resources' cost, the service level agreements and the trashing cost. The optimization algorithm is predictive: its allocation or reallocation decisions are based not only on the current metrics but also on predicted evolution of the system. The solution of the optimization, in each step, is a set some service replicas to be added or removed from the available hosts. These deployment changes are optimal with regards to overall objectives defined over time. In addition, the optimization considers the restrictions imposed on the number of possible service migrations at each time interval. We present experimental results that show the effectiveness of our approach.

In this paper, we introduce a decentralized autonomic architecture for multi-tier applications deployed in cloud environments. The architecture maintains the application's service level objective at a predefined level and, implicitly, reduces the cost. The architecture uses a series of autonomic controllers, in which each controller independently regulates a tier of the application. The architecture utilizes feedback loops and implements Proportional, Integrative and Derivative control laws at each autonomic controller. A prototype is described and an initial set of experiments is conducted on a public commercial cloud. The experiments demonstrate the effectiveness of this approach at maintaining a service level objective through the decomposition of an application's aggregate performance into its set of discretely managed component tiers.

Distributed Denial of Service attacks are a growing threat to organizations and, as defense mechanisms are becoming more advanced, hackers are aiming at the application layer. For example, application layer Low and Slow Distributed Denial of Service attacks are becoming a serious issue because, due to low resource consumption, they are hard to detect. In this position paper, we propose a reference architecture that mitigates the Low and Slow Distributed Denial of Service attacks by utilizing Software Defined Infrastructure capabilities. We also propose two concrete architectures based on the reference architecture: a Performance Model-Based and Off-The-Shelf Components based architecture, respectively. We introduce the Shark Tank concept, a cluster under detailed monitoring that has full application capabilities and where suspicious requests are redirected for further filtering.

Current cloud management systems have limited awareness of the user application, and application managers have no awareness of the state of the cloud. For applications with strong real-time requirements, distributed across new multi-cloud environments, this lack of awareness hampers response-time assurance, efficient deployment and rapid adaptation to changing workloads. This paper considers what forms this awareness may take, how it can be exploited in managing the applications and the clouds, and how it can influence cloud architecture.

Denial of Service (DoS) attacks overwhelm online services, preventing legitimate users from accessing a service, often with impact on revenue or consumer trust. Approaches exist to filter network-level attacks, but application-level attacks are harder to detect at the firewall. Filtering at this level can be computationally expensive and difficult to scale, while still producing false positives that block legitimate users. This article presents a model-based adaptive architecture and algorithm for detecting DoS attacks at the web application level and mitigating them. Using a performance model to predict the impact of arriving requests, a decision engine adaptively generates rules for filtering traffic and sending suspicious traffic for further review, where the end user is given the opportunity to demonstrate they are a legitimate user. If no legitimate user responds to the challenge, the request is dropped. Experiments performed on a scalable implementation demonstrate effective mitigation of attacks launched using a real-world DoS attack tool.

As the amount of data created and stored by organizations continues to increase, attention is turning to extracting knowledge from that raw data, including making some data available outside of the organization to enable crowd analytics. The adoption of the MapReduce paradigm has made processing Big Data more accessible, but is still limited to data that is currently available, often only within an organization. Fine-grained control over what information is shared outside an organization is difficult to achieve with Big Data, particularly in the MapReduce model. We introduce a novel approach to sharing that enables fine-grained control over what data is shared. Users submit analytics tasks that run on infrastructure near the actual data, reducing network bottlenecks. Organizations allow access to a logical version of their data created at runtime by filtering and transforming the actual data without creating storage-intensive stale copies, and resellers can further segment or augment this data to provide added value to analytics tasks. A loosely-coupled ecosystem driven by web services allows for discovery and sharing with a flexible, secure environment that limits the knowledge those running analytics need to have about the actual provider of the data. We describe a proof-of-concept implementation of the various components required to realize this ecosystem, and present a set of experiments to demonstrate feasibility, showing advantageous performance versus storage trade-offs.

To facilitate software development for multiple, federated cloud systems, abstraction layers have been introduced to mask the differences in the offerings, APIs, and terminology of various cloud providers. Such layers rely on a common ontology, which a) is difficult to create, and b) requires developers to understand both the common ontology and how various providers deviate from it. In this paper we propose and describe a structured query language for the cloud, Cloud SQL, along with a system and methodology for acquiring and organizing information from cloud providers and other entities in the cloud ecosystem such that it can be queried. It allows developers to run queries on data organized based on their semantic understanding of the cloud. Like the original SQL, we believe the use of a declarative query language will reduce development costs and make the multi-cloud accessible to a broader set of developers.

This paper presents a flexible deployment service for cloud computing. The service facilitates the specification and the execution of cloud deployment plans for applications. An application is described through a pattern, an abstract view that captures the logical view of the application and its mapping into cloud resources. The services instantiate the pattern in the cloud and allows for runtime updates of the deployment. The service is accessible through a RESTful interface. We identify the requirements for the service, describe its interfaces and show several case studies that capture the main features of the service.

The sharing of large and interesting Big Data in cloud environments can be achieved using data-as-a-service, where a provider offers data to interested users. In enhanced data-as-a-service, the data provider also supplies compute infrastructure, allowing users to run analytics tasks local to the data and reducing the (expensive and slow) transmission of data over networks. This paper describes a services-based ecosystem that allows providers to precisely share portions of their data with users, using a model where users submit MapReduce jobs that run on the provider's Hadoop infrastructure. Providers are given mechanisms to filter, segment, and/or transform data before it reaches the user's task. The ecosystem also allows for intermediaries who offer value-added filtrations, segmentations, or transformations of the data (for example, pre-filtering a dataset to only include high-income users). We describe the RESTful services required to enable this ecosystem, introduce a prototype to demonstrate the concept, and present experiments using this ecosystem to both provide and analyze different segments of a single large data set.

ABSTRACT Performance engineering for Web applications must take into account both the development and runtime information about the target system and its environment. At development time, the architects have to choose from many architecture styles and consider all performance requirements across a multitude of workloads. At runtime, an Autonomic Manager has to compensate for the changing operating and environment conditions not accounted for at the design time and make decisions about changes in the architecture so the performance requirements are met. This paper proposes a formal framework called Software Performance for Autonomic Computing for making decisions with regard to a possible set of candidate architectures: usage scenarios are criteria according to which architectures are evaluated; actual performance metrics, such as response time or throughput, are obtained by solving performance models and then matched against the performance requirements; performance requirements are defined by modeling user satisfaction with a utility function. Criteria can be weighted to reflect their importance. The framework can be used both at design and run time. Copyright © 2012 John Wiley & Sons, Ltd.

This paper introduces a cloud broker service (STRATOS) which facilitates the deployment and runtime management of cloud application topologies using cloud elements/services sourced on the fly from multiple providers, based on requirements specified in higher level objectives. Its implementation and use is evaluated in a set of experiments.

Descriptive information about available cloud services (i.e., metadata) is required in order to make good decisions about which cloud service provider(s) to utilize when deploying an application topology to the cloud. Presently, there are no uniform mechanisms for describing these services. Further, there is no unifying process that aggregates this metadata from the set of cloud providers and makes it available to a user in a programmatic fashion from a single location. This paper presents a methodology for and an implementation of a service-oriented application that provides relevant metadata information describing offered cloud services via a uniform RESTful web service. The data provided by this service is automatically acquired and mapped to a standard ontology. Community members can submit performance benchmarks using a metrics agent that submits metrics via a web service. Several example applications using this API to help users select resources are presented.

An elasticity policy governs how and when resources (e.g., application server instances at the PaaS layer) are added to and/or removed from a cloud environment. The elasticity policy can be implemented as a conventional control loop or as a set of heuristic rules. In the control-theoretic approach, complex constructs such as tracking filters, estimators, regulators, and controllers are utilized. In the heuristic, rule-based approach, various alerts(e.g., events) are defined on instance metrics (e.g., CPU utilization), which are then aggregated at a global scale in order to make provisioning decisions for a given application tier. This work provides an overview of our experiences designing and working with both approaches to construct an auto scaler for simple applications. We enumerate different criteria such as design complexity, ease of comprehension, and maintenance upon which we form an informal comparison between the different methods. We conclude with a brief discussion of how these approaches can be used in the governance of resources to better meet a high-level goal over time.

In this paper, we present a method for performance testing of transactional systems. The methods models the system under test, finds the software and hardware bottlenecks and generate the workloads that saturate them. The framework is adaptive, the model and workloads are determined during the performance test execution by measuring the system performance, fitting a performance model and by analytically computing the number and mix of users that will saturate the bottlenecks.

Estimation techniques have been largely applied to track hidden performance parameters (e.g. service demands) of web based software systems. In this paper we investigate dynamic multiclass modeling of such systems, with variable classes of service, aiming at finding a low complexity model yet with enough accuracy. We propose a combination of clustering algorithm and tracking filter for effective grouping of classes of services. The tracking estimator is based on a layered queuing model with parameters for CPU demands and the user load intensity of each class of service. Clustering uses the K-means algorithm. The target application is autonomic control of web clusters, where changes occur at different rates and amplitudes and at random time instants. Experiments show that the tracking is effective, and reveal good filter settings for different variations.

Adaptive management of a software service system can take advantage of a performance model which can predict the effect of proposed changes, before they are deployed. As the system varies over time the model parameters can be tracked by an estimator such as a Kalman Filter, so that decisions can be updated. The filter is valuable when parameters are 'hidden' and cannot be directly measured without excessive cost (as is usually the case for the CPU time of a service). Because there may be significant delays in some management control actions (especially in deploying a new replica of a service), it is also important to be able to predict the changes ahead somewhat in time, that is, to predict the trends. The trend predictor itself needs to be estimated from observed trends in the model parameters. This work uses an autoregressive model for trend prediction and integrates it with the parameter estimator, in a single Kalman Filter, using auxiliary states for the parameter evolution process. This paper describes how the trend model is constructed, and evaluates its effectiveness. It compares the overall performance predictions to a simpler trend predictor using linear extrapolation of the fitted parameter time-series, which turns out to be almost as good. The approach is validated on a real system running a benchmark web application.

The Software Engineering for Adaptive and Self-managing Systems (SEAMS) workshop has consolidated the interest in the software engineering community on self-adaptive and self-managing systems. SEAMS provides a forum for researchers and practitioners to share new results, discuss challenging issues, raise awareness, and promote collaboration within the community. The SEAMS 2010 workshop aims to continue the success of previous ICSE SEAMS workshops: in Shanghai in 2006, in Minneapolis in 2007, in Leipzig in 2008, and in Vancouver in 2009.

When deploying services in a cloud, a balance must be found between performance and capacity of the service, and the memory available on nodes. This is further complicated if the number of replicas of an application is limited, for instance by the available number of licenses. The analysis of interference between services must scale to large numbers of host nodes, applications, replicas of applications, and classes of users. This paper combines a multi-dimensional packing heuristic and network flow optimization to satisfy simultaneous constraints on throughputs, processor utilizations, memory availability and license availability, at a minimum cost and with a minimum of host processors.

Autonomic systems received lately a great deal of interest from the research and industrial communities due to their ability to configure, optimize, heal and protect themselves with little to no human intervention. Such systems must be able to analyze themselves and their environment in order to determine how best they can achieve the high-level goals and policies given to them by system managers. Among many approaches to this subject, real-time control loops have imposed themselves due to the direct mapping of their components onto the components of the generic autonomic computing architecture. In this paper we explore the model-driven approach to the development of real-time architecture for autonomic computing for a self-optimization scenario. This leads to the synthesis of a platform independent model (PIM) for generic autonomic computing systems and a platform specific model (PMS) for a specific application of it. Both the PIM and the PMS are then detailed up to the implementation level. The Platform Specific Model was considered for a Web service based implementation of the PIM.

To update a performance model, its parameter values must be updated, and in some applications (such as autonomic systems) tracked continuously over time. Direct measurement of many parameters during system operation requires instrumentation which is impractical. Kalman filter estimators can track such parameters using other data such as response times and utilizations, which are readily observable. This paper adapts Kalman filter estimators for performance model parameters, evaluates the approximations which must be made, and develops a systematic approach to setting up an estimator. The estimator converges under easily verified conditions. Different queueing-based models are considered here, and the extension for state-based models (such as stochastic Petri nets) is straightforward.

In an autonomic computing system, an autonomic manager makes tuning, load balancing, or provisioning decisions based on a predictive model of the system. This article investigates performance analysis techniques used by the autonomic manager. It looks at the complexity of the workloads and presents algorithms for computing the bounds of performance metrics for distributed systems under asymptotic and nonasymptotic conditions, that is, with saturated and nonsaturated resources. The techniques used are hybrid in nature, making use of performance evaluation and linear and nonlinear programming models. The workloads are characterized by the workload intensity , which represents the total number of users in the system, and by the workload mixes , which depict the number of users in each class of service. The results presented in this article can be applied to distributed transactional systems. Such systems serve a large number of users with many classes of services and can thus be considered as representative of a large class of autonomic computing systems.

Program comprehension tools are a valuable resource for navigating and understanding large software systems. Package explorers, fan-in/fan-out views, dependency graphs and coverage analysis are example contributions from the program comprehension community. While many of these research projects have lead to exciting enhancements in our field, many other projects have failed to be adopted because of poor interface design or lack of integration with existing tools. Designing, building, integrating and evaluating interfaces is a challenge to software engineering researchers. In this paper we borrow from the field of model driven engineering (MDE) to assist with the creation of highly customizable interfaces for software visualization. MDE moves the level of abstraction from implementation to design, and help improve the efficiency of building software visualizations. By moving away from implementation details, and providing researchers with the ability to customize their visualizations in an efficient manner, software engineers have more resources to design and evaluate their ideas

Understanding software engineering issues for autonomic computing systems is critical for the software and information technology sectors, which are continually challenged to reduce the complexity of their systems. To be autonomic, a system must know itself as well as its boundaries and its environment, configure and reconfigure itself, continually optimize itself, recover or heal from malfunction, protect itself, and function in a heterogeneous world-while keeping its complexity hidden from the user. The goal of this workshop is to bring together researchers and practitioners, who investigate concepts, methodologies, techniques, technologies, and tools to design and evolve autonomic software.

Abstract In this paper we look at several performance pitfalls that Web services are facing today and at the performance penalties that have to be paid when exposing a legacy application as a Web service. We investigate two performance metrics of Web services, latency and scalability, and compare them with those of legacy middleware. The goal of the paper is to show how the performance penalties can be mitigated by following the principles and methods of performance engineering. Performance models can help the migration decisions, especially when new architectures or new deployment topologies are sought. The paper shows the mechanisms of building and solving a performance model involving Web services. An example is presented throughout the paper. Copyright © 2004 John Wiley & Sons, Ltd.

The key objective of this workshop is to explore innovative approaches to the adoption of software engineering tools and practices-in particular by embedding them in extensions of Commercial Off-The-Shelf (COTS) software products and/or middleware technologies. The workshop aims to advance the understanding and evaluation of adoption of software engineering tools and practices by bringing together researchers and practitioners who investigate novel solutions to software engineering adoption issues.

Distributed application systems are composed of classes of objects with instances that interact to accomplish common goals. Such systems can have many classes of users with many types of requests. Furthermore, the relative load of these classes can shift throughout the day, causing changes to system behavior and bottlenecks. When designing and deploying such systems, it is necessary to determine a process replication and threading policy for the server processes that contain the objects, as well as process activation policies. To avoid bottlenecks, the policy must support all possible workload conditions. Licensing, implementation or resource constraints can limit the number of permitted replicas or threads of a server process. Process activation policies determine whether a server is persistent or should be created and terminated with each call. This paper describes quantitative techniques for choosing process replication or threading levels and process activation policies. Inappropriate policies can lead to unnecessary queuing delays for callers or unnecessarily high consumption of memory resources. The algorithms presented consider all workload conditions, are iterative in nature and are hybrid mathematical programming and analytic performance evaluation methods. An example is given to demonstrate the technique and describe how the results can be applied during software design and deployment.