Resource-constrained smart eyewear (SEW) devices face significant challenges when deploying deep neural networks due to limited computational capacity and battery life. Computational offloading to companion devices like smartphones and cloud servers addresses processing limitations, but data transmission becomes a critical bottleneck, consuming over 50% of total energy in some scenarios. Although lossless compression methods provide limited data reduction for intermediate tensors, lossy techniques such as Vector Quantization (VQ) offer higher compression ratios (requiring only 3.3 bits per float) at the expense of inference accuracy degradation. This paper presents an adaptive multi-stage compression framework that dynamically balances these trade-offs across the SEW-phone-cloud continuum. We employ VQ at the SEW-phone interface where aggressive compression is essential (achieving 89.6% tensor size reduction with 90% retained accuracy), followed by adaptive selection between quantization and run-length encoding for phone-to-cloud transmission based on network conditions. A Deep Q-Network (DQN) agent jointly optimizes network partitioning points and compression strategies to minimize energy consumption while preserving accuracy and meeting latency constraints. A large simulation campaign considering object detection and human pose estimation tasks demonstrate that our method achieves 55--70% energy savings and 86--91% violation reduction compared to Neurosurgeon (a dynamic partitioning baseline without compression), 45.8% energy savings versus local execution, and 61.1% savings over uncompressed offloading, with latency violation rates below 9% and acceptable accuracy loss (8.0--8.1%). These results enable practical deployment of AI applications on battery-limited SEW devices.

Virtual and Extended Reality technologies are increasingly adopted in fields such as healthcare, entertainment, and education. These applications heavily rely on Smart Eye-Wears (SEWs) and AI to provide users with new ways to perceive their environment. However, SEWs face limitations in computational power, memory, and battery life. Offloading computations to external servers is a prominent example of edge computation. However, this also presents considerable challenges due to delays caused by varying network conditions and server workloads. This article proposes self-adaptive techniques based on tabular reinforcement learning (RL) to optimize the offloading of Deep Neural Network tasks between the SEW, the user’s smartphone, and cloud servers. The goal is to maintain a high-quality user experience while minimizing energy consumption and 5G connection costs. We evaluated our framework under varying 5G and WiFi bandwidths and cloud latency. The results show that Q-learning, SARSA, and Expected SARSA achieve near-optimal policies, with Q-learning demonstrating superior performance in reducing execution time violations (approximately at 10%) and improving agent stability. Additionally, our approach offers a more favorable tradeoff between energy efficiency and execution time violations compared to two baseline methods. Real-system experiments reveal that the proposed solution can double SEW battery life with respect to local computation while maintaining a good quality of service, with only 11% execution time violations. These findings highlight the effectiveness of our approach in managing resources and enhancing the overall user experience in SEW AI applications.

High-Performance Computing (HPC) is a fundamental tool for tackling complex scientific and engineering problems. Optimizing applications for the heterogeneous and massively parallel nature of modern HPC hardware is essential for achieving timely and resource-efficient results. This paper introduces ABC-MLCO, a novel Artificial Bee Colony (ABC) algorithm enhanced with machine learning for constrained optimization in discrete configuration spaces, specifically targeting HPC scenarios with black-box performance metrics. ABC-MLCO extends the original ABC algorithm with mechanisms that define the feasible region, promote escape from local optima, prevent redundant evaluations, and intensify exploration and exploitation. We first evaluate the algorithm on benchmark functions, observing improvements in regret, feasibility rate, and convergence speed over the original ABC. Then, targeting a virtual screening application for drug discovery, ABC-MLCO outperforms well-known state-of-the-art methods in terms of final regret, achieving average improvements up to $93 \%$.

Virtual screening applications are highly parameterized to optimize the balance between quality and execution performance. While output quality is critical, the entire screening process must be completed within a reasonable time. In fact, a slight reduction in output accuracy may be acceptable when dealing with large datasets. Finding the optimal quality-throughput trade-off depends on the specific HPC system used and should be re-evaluated with each new deployment or significant code update. This paper presents two parallel autotuning techniques for constrained optimization in distributed High-Performance Computing (HPC) environments. These techniques extend sequential Bayesian Optimization (BO) with two parallel asynchronous approaches, and they integrate predictions from Machine Learning (ML) models to help comply with constraints. Our target application is LiGen, a real-world virtual screening software for drug discovery. The proposed methods address two relevant challenges: efficient exploration of the parameter space and performance measurement using domain-specific metrics and procedures. We conduct an experimental campaign comparing the two methods with a popular state-of-the-art autotuner. Results show that our methods find configurations that are, on average, up to 35–42% better than the ones found by the autotuner and the default expert-picked LiGen configuration. • We propose two parallel algorithms for black-box constrained optimization. • We integrate Bayesian Optimization and Machine Learning for constraint estimation. • We use a meta-scheduler to hinge on the resources available in HPC settings. • We evaluate the benefits of the proposed approach using a relevant case study.

This paper proposes an automated framework for efficient application profiling and training of Machine Learning (ML) performance models, composed of two parts: OSCAR-P and aMLLibrary. OSCAR-P is an auto-profiling tool designed to automatically test serverless application workflows running on multiple hardware and node combinations in cloud and edge environments. OSCAR-P obtains relevant profiling information on the execution time of the individual application components. These data are later used by aMLLibrary to train ML-based performance models. This makes it possible to predict the performance of applications on unseen configurations. We test our framework on clusters with different architectures (x86 and arm64) and workloads, considering multi-component use-case applications. This extensive experimental campaign proves the efficiency of OSCAR-P and aMLLibrary, significantly reducing the time needed for the application profiling, data collection, and data processing. The preliminary results obtained on the ML performance models accuracy show a Mean Absolute Percentage Error lower than 30% in all the considered scenarios.

Bayesian Optimization (BO) is a family of powerful algorithms designed to solve complex optimization problems involving expensive black-box functions. These sequential algorithms iteratively update a surrogate model of the objective function (OF), effectively balancing exploration and exploitation to identify near-optimal solutions within a limited number of iterations. Originally designed for continuous, unconstrained domains, its efficiency has inspired adaptations for discrete, constrained optimization problems. On the other hand, Machine Learning (ML) models allow accurate predictions for black-box functions, although they typically require large amounts of data for training. Leveraging the strengths of BO and ML, research tackles the challenge of identifying optimal configurations in the context of cloud computing. This paradigm has become pervasive due to its ability to provide flexible and scalable resources. Identifying the optimal hardware-software configuration is essential for minimizing costs while meeting Quality of Service constraints. This task involves solving complex optimization problems over multidimensional discrete domains and black-box objective functions and constraints, within a limited number of iterations. To address this challenge, this work introduces d-MALIBOO , a BO-based algorithm that integrates ML techniques to enhance the efficiency of finding near-optimal solutions in discrete and bounded domains. While BO builds the surrogate model of the OF, ML models determine the feasible region of the black-box constraints and guide the BO algorithm toward promising regions of the discrete domain. Furthermore, we introduce an ɛ ɛ -greedy approach to favor exploration in domains with multiple local optima. Experimental results show that our algorithm outperforms OpenTuner, a popular framework for constrained optimization, by reducing the average regret by 29%, and SVM-CBO, a BO-based algorithm that integrates SVM models to determine the feasible region, by 82%.

The growth, development, and commercialization of artificial intelligence-based technologies such as self-driving cars, augmented-reality viewers, chatbots, and virtual assistants are driving the need for increased computing power. Most of these applications rely on Deep Neural Networks (DNNs), which demand substantial computing capacity to meet user demands. However, this capacity cannot be fully provided by users’ local devices due to their limited processing power, nor by cloud data centers due to high transmission latency from long distances. Edge cloud computing addresses this issue by processing user requests through 5G, which reduces transmission latency from local devices to computing resources and allows the offloading of some computations to cloud back-ends. This paper introduces a model for a Mobile Edge Cloud system designed for an application based on a DNN. The interaction among multiple mobile users and the edge platform is formulated as a one-leader multi-follower Stackelberg game, resulting in a challenging non-convex mixed integer nonlinear programming (MINLP) problem. To tackle this, we propose a heuristic approach based on Karush-Kuhn-Tucker conditions, which solves the MINLP problem significantly faster than the commercial state-of-the-art solvers (up to 50,000 times). Furthermore, we present an algorithm to estimate optimal platform profit when sensitive user parameters are unknown. Comparing this with the full-knowledge scenario, we observe a profit loss of approximately 1%. Lastly, we analyze the advantages for an edge provider to engage in a Stackelberg game rather than setting a fixed price for its users, showing potential profit increases ranging from 16% to 66%.

The proliferation of the Internet of Things, artificial intelligence, and real-time data processing applications has driven the demand for distributed computing architectures that span cloud, fog, and edge layers in a computing continuum. These architectures must address critical challenges in component placement and resource optimization to ensure low latency, cost efficiency, and compliance with Quality of Service (QoS) constraints. This paper introduces a novel optimization framework for addressing the joint problem of component placement and resource optimization in computing continua. The framework employs a Mixed Integer Nonlinear Programming model, where application components are modeled as a Directed Acyclic Graph and their performance is predicted using analytical models. A method based on the Karush-Kuhn-Tucker conditions is employed to compute the optimal number of virtual machine instances for a given component placement. This optimization is embedded within a reinforcement learning loop that iteratively refines placement decisions in response to fluctuations in workload. This hybrid approach ensures cost effectiveness while adhering to QoS constraints. Extensive experimental evaluations demonstrate the superiority of our framework. It outperforms leading approaches, including BARON solver, SPACE4AI-D, PPO_DLX, and a minimum k-cut baseline, achieving average cost reductions of 19%, 60%, 11%, and 6%, respectively, under dynamic workload conditions. These results highlight the efficiency, scalability, and adaptability of our approach, making it a robust solution to the demands of modern distributed systems.

The AI-SPRINT project, launched in 2021 and funded by the European Commission, focuses on the development and implementation of AI applications across the computing continuum. This continuum ensures the coherent integration of computational resources and services from centralized data centers to edge devices, facilitating efficient and adaptive computation and application delivery. AI-SPRINT has achieved significant scientific advances, including streamlined processes, improved efficiency, and the ability to operate in real time, as evidenced by three practical use cases. This paper provides an in-depth examination of these applications – Personalized Healthcare, Maintenance and Inspection, and Farming 4.0 – highlighting their practical implementation and the objectives achieved with the integration of AI-SPRINT technologies. We analyze how the proposed toolchain effectively addresses a range of challenges and refines processes, discussing its relevance and impact in multiple domains. After a comprehensive overview of the main AI-SPRINT tools used in these scenarios, the paper summarizes of the findings and key lessons learned.

Accurate classification of medical images is essential for modern diagnostics.Deep learning advancements led clinicians to increasingly use sophisticated models to make faster and more accurate decisions, sometimes replacing human judgment.However, model development is costly and repetitive.Neural Architecture Search (NAS) provides solutions by automating the design of deep learning architectures.This paper presents ZO-DARTS+, a differentiable NAS algorithm that improves search efficiency through a novel method of generating sparse probabilities by bilevel optimization.Experiments on five public medical datasets show that ZO-DARTS+ matches the accuracy of state-of-the-art solutions while reducing search times by up to three times.

Cloud computing has become essential for delivering flexible and scalable resources. In this environment, finding the optimal hardware-software configuration is crucial and often involves solving constrained black-box problems on discrete multidimensional domains. These optimizations must be performed within a limited number of evaluations to contain costs. Bayesian Optimization (BO) addresses this problem by providing near-optimal solutions with few iterations. However, the original BO algorithm was designed for continuous and unbounded domains. On the other hand, Machine Learning (ML) models are powerful tools for predicting the values of a black-box function. In this work, we present d-MALIBOO, a framework that integrates BO and ML techniques to improve the efficiency of finding optimal solutions in discrete and bounded domains. While BO suggests the next point to be evaluated by effectively balancing exploration and exploitation, ML models are valuable for determining feasibility regions and focusing the search for the optimum. Experimental results show that our algorithm outperforms alternative methods from the literature in all tested environments, especially in complex optimization scenarios, resulting in an improvement in regret by approximately 2–8 times.

In this work, we optimize the scheduling of Deep Learning (DL) training jobs from the perspective of a Cloud Service Provider running a data center, which efficiently selects resources for the execution of each job to minimize the average energy consumption while satisfying time constraints. To model the problem, we first develop a Mixed-Integer Non-Linear Programming formulation. Unfortunately, the computation of an optimal solution is prohibitively expensive, and to overcome this difficulty, we design a heuristic STochastic Scheduler (STS). Exploiting the probability distribution of early termination, STS determines how to adapt the resource assignment during the execution of the jobs to minimize the expected energy cost while meeting the job due dates. The results of an extensive experimental evaluation show that STS guarantees significantly better results than other methods in the literature, effectively avoiding due date violations and yielding a percentage total cost reduction between 32% and 80% on average. We also prove the applicability of our method in real-world scenarios, as obtaining optimal schedules for systems of up to 100 nodes and 400 concurrent jobs requires less than 5 seconds. Finally, we evaluated the effectiveness of GPU sharing, i.e., running multiple jobs in a single GPU. The obtained results demonstrate that depending on the workload and GPU memory, this further reduces the energy cost by 17–29% on average.

Bayesian Optimization (BO) is an efficient method for finding optimal cloud configurations for several types of applications. On the other hand, Machine Learning (ML) can provide helpful knowledge about the application at hand thanks to its predicting capabilities. This work proposes a general approach based on BO, which integrates elements from ML techniques in multiple ways, to find an optimal configuration of recurring jobs running in public and private cloud environments, possibly subject to black-box constraints, e.g., application execution time or accuracy. We test our approach by considering several use cases, including edge computing, scientific computing, and Big Data applications. Results show that our solution outperforms other state-of-the-art black-box techniques, including classical autotuning and BO- and ML-based algorithms, reducing the number of unfeasible executions and corresponding costs up to 2–4 times.

Nowadays, Artificial Intelligence (AI) applications are becoming increasingly popular in a wide range of industries, mainly thanks to Deep Neural Networks (DNNs) that needs powerful resources. Cloud computing is a promising approach to serve AI applications thanks to its high processing power, but this sometimes results in an unacceptable latency because of long-distance communication. Vice versa, edge computing is close to where data are generated and therefore it is becoming crucial for their timely, flexible, and secure management. Given the more distributed nature of the edge and the heterogeneity of its resources, efficient component placement and resource allocation approaches become critical in orchestrating the application execution. In this paper, we formulate the resource selection and AI applications component placement problem in a computing continuum as a Mixed Integer Non-Linear Problem (MINLP), and we propose a design-time tool for its efficient solution. We first propose a Random Greedy algorithm to minimize the cost of the placement while guaranteeing some response time performance constraints. Then, we develop some heuristic methods such as Local Search, Tabu Search, Simulated Annealing and Genetic Algorithms, to improve the initial solutions provided by the Random Greedy. To evaluate our proposed approach, we designed an extensive experimental campaign, comparing the heuristics methods with one another and then the best heuristic against Best Cost Performance Constraint (BCPC) algorithm, a state-of-the-art approach. The results demonstrate that our proposed approach finds lower-cost solution than BCPC (27.6% on average) under the same time limit in large-scale systems. Finally, during the validation in a real edge system including FaaS resources our approach finds the globally optimal solution, suffering a deviation of around 12% between actual and predicted costs.

Today digital revolution is having a dramatic impact on the pharmaceutical industry and the entire healthcare system. The implementation of machine learning, extreme-scale computer simulations, and big data analytics in the drug design and development process offers an excellent opportunity to lower the risk of investment and reduce the time to the patient.

aMLLibrary is an open-source, high-level Python package that allows the parallel building of multiple Machine Learning (ML) regression models. It is focused on performance modeling and supports several methods for feature engineering/selection and hyperparameter tuning. The library implements fault tolerance mechanisms to recover from system crashes, and only a simple declarative text file is required to launch a full experimental campaign for all required models. Its modular structure allows users to implement their own plugins and model-building wrappers and easily add them to the library. We test aMLLibrary on building the performance models of neural networks and image processing applications, with the best model produced often having less than 20% prediction error.

Deep Learning applications are pervasive today, and efficient strategies are designed to reduce the computational time and resource demand of the training process. The Distributed Deep Learning (DDL) paradigm yields a significant speed-up by partitioning the training into multiple, parallel tasks. The Ray framework supports DDL applications exploiting data parallelism by enhancing the scalability with minimal user effort. This work aims at evaluating the performance of DDL training applications, by profiling their execution on a Ray cluster and developing Machine Learning-based models to predict the training time when changing the dataset size, the number of parallel workers and the amount of computational resources. Such performance-prediction models are crucial to forecast computational resources usage and costs in Cloud environments. Experimental results prove that our models achieve average prediction errors between 3 and 15% for both interpolation and extrapolation, thus demonstrating their applicability to unforeseen scenarios.

Companies depend on mining data to grow their business more than ever. To achieve optimal performance of Big Data analytics workloads, a careful configuration of the cluster and the employed software framework is required. The lack of flexible and accurate performance models, however, render this a challenging task. This article fills this gap by presenting accurate performance prediction models based on Stochastic Activity Networks (SANs). In contrast to existing work, the presented models consider multiple work queues, a critical feature to achieve high accuracy in realistic usage scenarios. We first introduce a monolithic analytical model for a multi-queue YARN cluster running DAG-based Big Data applications that models each queue individually. To overcome the limited scalability of the monolithic model, we then present a fixed-point model that iteratively computes the throughput of a single queue with respect to the rest of the system until a fixed-point is reached. The models are evaluated on a real-world cluster running the widely-used Apache Spark framework and the YARN scheduler. Experiments with the common transaction-based TPC-DS benchmark show that the proposed models achieve an average error of only$5.6\%$in predicting the execution time of the Spark jobs. The presented models enable businesses to optimize their cluster configuration for a given workload and thus to reduce their expenses and minimize service level agreement (SLA) violations. Makespan minimization and per-stage analysis are examined as representative efforts to further assess the applicability of our proposition.

The Deep Learning (DL) paradigm gained remarkable popularity in recent years. DL models are used to tackle increasingly complex problems, making the training process require considerable computational power. The parallel computing capabilities offered by modern GPUs partially fulfill this need, but the high costs related to GPU as a Service solutions in the cloud call for efficient capacity planning and job scheduling algorithms to reduce operational costs via resource sharing. In this work, we jointly address the online capacity planning and job scheduling problems from the perspective of cloud end-users. We present a Mixed Integer Linear Programming (MILP) formulation, and a path relinking-based method aiming at optimizing operational costs by (i) rightsizing Virtual Machine (VM) capacity at each node, (ii) partitioning the set of GPUs among multiple concurrent jobs on the same VM, and (iii) determining a due-date-aware job schedule. An extensive experimental campaign attests the effectiveness of the proposed approach in practical scenarios: costs savings up to 97% are attained compared with first-principle methods based on, e.g., Earliest Deadline First, cost reductions up to 20% are obtained with respect to a previously proposed Hierarchical Method and up to 95% against a dynamic programming-based method from the literature. Scalability analyses show that systems with up to 100 nodes and 450 concurrent jobs can be managed in less than 7 seconds. The validation in a prototype cloud environment shows a deviation below 5% between real and predicted costs.

Automating the research for the best neural network model is a task that has gained more and more relevance in the last few years. In this context, Neural Architecture Search (NAS) represents the most effective technique whose results rival the state of the art hand-crafted architectures. However, this approach requires a lot of computational capabilities as well as research time, which make prohibitive its usage in many real-world scenarios. With its sequential model-based optimization strategy, Progressive Neural Architecture Search (PNAS) represents a possible step forward to face this resources issue. Despite the quality of the found network architectures, this technique is still limited in research time. A significant step in this direction has been done by Pareto-Optimal Progressive Neural Architecture Search (POPNAS), which expand PNAS with a time predictor to enable a trade-off between search time and accuracy, considering a multi-objective optimization problem. This paper proposes a new version of the Pareto-Optimal Progressive Neural Architecture Search, called POPNASv2. Our approach enhances its first version and improves its performance. We expanded the search space by adding new operators and improved the quality of both predictors to build more accurate Pareto fronts. Moreover, we introduced cell equivalence checks and enriched the search strategy with an adaptive greedy exploration step. Our efforts allow POPNASv2 to achieve PNAS-like performance with an average 4x factor search time speed-up. Code: https://doi.org/10.5281/zenodo.6574040

Abstract Nowadays, Apache Hadoop and Apache Spark are two of the most prominent distributed solutions for processing big data applications on the market. Since in many cases these frameworks are adopted to support business critical activities, it is often important to predict with fair confidence the execution time of submitted applications, for instance when service-level agreements are established with end-users. In this work, we propose and validate a hybrid approach for the performance prediction of big data applications running on clouds, which exploits both analytical modeling and machine learning (ML) techniques and it is able to achieve a good accuracy without too many time consuming and costly experiments on a real setup. The experimental results show how the proposed approach attains improvement in accuracy, number of experiments to be run on the operational system and cost over applying ML techniques without any support from analytical models. Moreover, we compare our approach with Ernest, an ML-based technique proposed in the literature by the Spark inventors. Experiments show that Ernest can accurately estimate the performance in interpolating scenarios while it fails to predict the performance when configurations with increasing number of cores are considered. Finally, a comparison with a similar hybrid approach proposed in the literature demonstrates how our approach significantly reduce prediction errors especially when few experiments on the real system are performed.

Cloud Computing has assumed a relevant role in the ICT, profoundly influencing the life-cycle of modern applications in the manner they are designed, developed, and deployed and operated. In this article, we tackle the problem of supporting the design-time analysis of Cloud applications to identify a cost-optimized strategy for allocating components onto Cloud Virtual Machine infrastructural services, taking performance requirements into account. We present an approach and a tool, SPACE4Cloud, that supports users in modeling the architecture of an application, in defining performance requirements as well as deployment constraints, and then in mapping each architecture component into a corresponding VM service, minimizing total costs. An optimization algorithm supports the mapping and determines the Cloud configuration that minimizes the execution costs of the application over a daily time horizon. The benefits of this approach are demonstrated in the context of an industrial case study. Furthermore, we show that SPACE4Cloud leads to a cost reduction up to 60 percent, when compared to a first-principle technique based on utilization thresholds, like the ones typically used in practice, and that our solution is able to solve large problem instances within a time frame compatible with a fast-paced design process (less than half an hour in the worst case). Finally, we show that SPACE4Cloud is suitable to model even microservice-based applications and to compute the corresponding optimized deployment configuration which is compared with a state-of-the art meta-heuristic alternative method, achieving savings between 21 and 85 percent.

Nowadays, the big data paradigm is consolidating its central position in the industry, as well as in society at large. Lots of applications, across disparate domains, operate on huge amounts of data and offer great advantages both for business and research. According to analysts, cloud computing adoption is steadily increasing to support big data analyses and Spark is expected to take a prominent market position for the next decade. As big data applications gain more and more importance over time and given the dynamic nature of cloud resources, it is fundamental to develop an intelligent resource management system to provide Quality of Service guarantees to end-users. This article presents a set of run-time optimization-based resource management policies for advanced big data analytics. Users submit Spark applications characterized by a priority and by a hard or soft deadline. Optimization policies address two scenarios: i) identification of the minimum capacity to run a Spark application within the deadline; ii) re-balance of the cloud resources in case of heavy load, minimising the weighted soft deadline application tardiness. The solution relies on an initial non-linear programming model formulation and a search space exploration based on simulation-optimization procedures. Spark application execution times are estimated by relying on a gamut of techniques, including machine learning, approximated analyses, and simulation. The benefits of the approach are evaluated on Microsoft Azure HDInsight and on a private cloud cluster based on POWER8 by considering the TPC-DS industry benchmark and SparkBench. The results obtained in the first scenario demonstrate that the percentage error of the prediction of the optimal resource usage with respect to system measurement and exhaustive search is in the range 4–29 percent while literature-based techniques present an average error in the range 6–63 percent. Moreover, in the second scenario, the proposed algorithms can address complex problems like computing the optimal redistribution of resources among tens of applications in less than a minute with an error of 8 percent on average. On the same considered tests, literature-based approaches obtain an average error of about 57 percent.

The adoption of Artificial intelligence (AI) technologies is steadily increasing. However, to become fully pervasive, AI needs resources at the edge of the network. The cloud can provide the processing power needed for big data, but edge computing is close to where data are produced and therefore crucial to their timely, flexible, and secure management. In this paper, we introduce the AI-SPRINT project, which will provide solutions to seamlessly design, partition, and run AI applications in computing continuum environments. AI-SPRINT will offer novel tools for AI applications development, secure execution, easy deployment, as well as runtime management and optimization: AI-SPRINT design tools will allow trading-off application performance (in terms of end-to-end latency or throughput), energy efficiency, and AI models accuracy while providing security and privacy guarantees. The runtime environment will support live data protection, architecture enhancement, agile delivery, runtime optimization, and continuous adaptation.

Artificial Intelligence (AI) and Deep Learning (DL) are pervasive today, with applications spanning from personal assistants to healthcare. Nowadays, the accelerated migration towards mobile computing and Internet of Things, where a huge amount of data is generated by widespread end devices, is determining the rise of the edge computing paradigm, where computing resources are distributed among devices with highly heterogeneous capacities. In this fragmented scenario, efficient component placement and resource allocation algorithms are crucial to orchestrate at best the computing continuum resources. In this paper, we propose a tool to effectively address the component placement problem for AI applications at design time. Through a randomized greedy algorithm, it identifies the placement of minimum cost providing performance guarantees across heterogeneous resources including edge devices, cloud GPU-based Virtual Machines and Function as a Service solutions.

Cloud Computing is emerging as a major trend in ICT industry. However, as with any new technology, new major challenges lie ahead, one of them concerning the resource provisioning. Indeed, modern Cloud applications deal with a dynamic context that requires a continuous adaptation process in order to meet satisfactory Quality of Service (QoS) but even the most titled Cloud platform provide just simple rule-based tools; the rudimentary autoscaling mechanisms that can be carried out may be unsuitable in many situations as they do not prevent SLA violations, but only react to them. In addition, these approaches are inherently static and cannot catch the dynamic behavior of the application and are unsuitable to manage multi-Cloud/data center deployments required for mission critical services. This situation calls for advanced solutions designed to provide Cloud resources in a predictive and dynamic way. This work presents capacity allocation algorithms, whose goal is to minimize the total execution cost while satisfying some constraints on the average response time of multi-Cloud based applications. This paper proposes a joint load balancing and receding horizon capacity allocation techniques, which can be employed to handle multiple classes of requests. An extensive evaluation of the proposed solution against an Oracle with perfect knowledge of the future and well-known heuristics proposed in the literature is provided. The analysis shows that our solution outperforms the heuristics producing results very close to the optimal ones, and reducing the number of QoS violations (in the worst case QoS constraints violation rate is 4.26 percent versus up to 17.25 percent of other approaches and can easily reduced by roughly a factor of four by exploiting the receding horizon approach). Furthermore, a sensitivity analysis over two different time scales indicates that finer grained time scales are more appropriate for spiky workloads. Analytical results are validated through simulation, which also analyzes the impact of Cloud environment random perturbations. Finally, experiments on a prototype environment demonstrate the effectiveness of the proposed approach under real workloads.

The last years witnessed a steep rise in data generation worldwide and, consequently, the widespread adoption of software solutions able to support data-intensive application. Competitiveness and innovation have strongly benefited from these new platforms and methodologies, and there is a great deal of interest around the new possibilities that Big Data analytics promise to make reality. Many companies currently engage in data-intensive processes as part of their core businesses; however, fully embracing the data-driven paradigm is still cumbersome, and establishing a production-ready, fine-tuned deployment is time-consuming, expensive, and resource-intensive. This situation calls for innovative models and techniques to streamline the process of deployment configuration for Big Data applications. In particular, the focus in this paper is on the rightsizing of Cloud deployed clusters, which represent a cost-effective alternative to installation on premises. This paper proposes a novel tool, integrated in a wider DevOps-inspired approach, implementing a parallel and distributed simulation-optimization technique that efficiently and effectively explores the space of alternative Cloud configurations, seeking the minimum cost deployment that satisfies quality of service constraints. The soundness of the proposed solution has been thoroughly validated in a vast experimental campaign encompassing different applications and Big Data platforms.

Spark is an in-memory framework for implementing distributed applications of various types. Predicting the execution time of Spark applications is an important but challenging problem that has been tackled in the past few years by several studies; most of them achieving good prediction accuracy on simple applications (e.g. known ML algorithms or SQL-based applications). In this work, we consider complex data-driven workflow applications, in which the execution and data flow can be modeled by Directly Acyclic Graphs (DAGs). Workflows can be made of an arbitrary combination of known tasks, each applying a set of Spark operations to their input data. By adopting a hybrid approach, combining analytical and machine learning (ML) models, trained on small DAGs, we can predict, with good accuracy, the execution time of unseen workflows of higher complexity and size. We validate our approach through an extensive experimentation on real-world complex applications, comparing different ML models and choices of feature sets.

Big data applications and analytics are employed in many sectors for a variety of goals: improving customers satisfaction, predicting market behavior or improving processes in public health. These applications consist of complex software stacks that are often run on cloud systems. Predicting execution times is important for estimating the cost of cloud services and for effectively managing the underlying resources at runtime. Machine Learning (ML), providing black box solutions to model the relationship between application performance and system configuration without requiring in-detail knowledge of the system, has become a popular way of predicting the performance of big data applications. We investigate the cost-benefits of using supervised ML models for predicting the performance of applications on Spark, one of today's most widely used frameworks for big data analysis. We compare our approach with Ernest (an ML-based technique proposed in the literature by the Spark inventors) on a range of scenarios, application workloads, and cloud system configurations. Our experiments show that Ernest can accurately estimate the performance of very regular applications, but it fails when applications exhibit more irregular patterns and/or when extrapolating on bigger data set sizes. Results show that our models match or exceed Ernest's performance, sometimes enabling us to reduce the prediction error from 126-187% to only 5-19%.

Recent years saw an increasing success in the application of deep learning methods across various domains and for tackling different problems, ranging from image recognition and classification to text processing and speech recognition. In this paper we propose and validate an approach to model the execution time for training convolutional neural networks (CNNs) deployed on GPGPUs. We demonstrate that our approach is generally applicable to a variety of CNN models and different types of G PG PU s with high accuracy, aiming at the preliminary design phases for system sizing.

Big data applications are among the most suitable applications to be executed on cluster resources because of their high requirements of computational power and data storage.Correctly sizing the resources devoted to their execution does not guarantee they will be executed as expected.Nevertheless, their execution can be affected by perturbations which can change the expected execution time.Identifying when these types of issue occurred by comparing their actual execution time with the expected one is mandatory to identify potentially critical situations and to take the appropriate steps to prevent them.To fulfill this objective, accurate estimates are necessary.In this paper, machine learning techniques coupled with a posteriori knowledge are exploited to build performance estimation models.Experimental results show how the models built with the proposed approach are able to outperform a reference state-of-the-art method (i.e., Ernest method), reducing in some scenarios the error from the 221.09-167.07%to 13.15-30.58%.

Infrastructure as a Service (IaaS) is one of the most significant and fastest growing fields in cloud computing. To efficiently use the resources of an IaaS cloud, several important factors such as performance, availability, and power consumption need to be considered and evaluated carefully. Evaluation of these metrics is essential for cost-benefit prediction and quantification of different strategies which can be applied to cloud management. In this paper, analytical models based on Stochastic Reward Nets (SRNs) are proposed to model and evaluate an IaaS cloud system at different levels. To achieve this, an SRN is initially presented to model a group of physical machines which are controlled by a management layer. Afterwards, the SRN models presented for the groups of physical machines in the first stage are combined to capture a monolithic model representing an entire IaaS cloud. Since the monolithic model does not scale well for large cloud systems, two approximate SRN models using folding and fixed-point iteration techniques are proposed to evaluate the performance, availability, and power consumption of the IaaS cloud. The existence of a solution for the fixed-point approximate model is proved using Brouwer's fixed-point theorem. A validation of the proposed monolithic and approximate models against both an ad-hoc discrete-event simulator developed in Java and the CloudSim framework is presented. The analytic-numeric results obtained from applying the proposed models to sample cloud systems show that the errors introduced by approximate models are insignificant while an improvement of several orders of magnitude in the state space reduction of the monolithic model is obtained.

Recent analyses[2, 4, 5] report that many sectors of our economy and society are more and more guided by data-driven decision processes (e.g., health care, public administrations, etc.). As such, Data Intensive (DI) applications are becoming more and more important and critical. They must be fault-tolerant, they should scale with the amount of data, and be able to elastically leverage additional resources as and when these last ones are provided [3]. Moreover, they should be able to avoid data drops introduced in case of sudden overloads and should offer some Quality of Service (QoS) guarantees.

Recent years saw an increasing success in the application of deep learning methods across various domains and for tackling different problems, ranging from image recognition and classification to text processing and speech recognition. In this paper we propose and validate an approach to model the execution time for training convolutional neural networks (CNNs) deployed on GPGPUs. We demonstrate that our approach is generally applicable to a variety of CNN models and different types of G PG PU s with high accuracy, aiming at the preliminary design phases for system sizing.

Data heterogeneity and irregularity are key characteristics of big data applications that often overwhelm the existing software and hardware infrastructures. In such context, the exibility and elasticity provided by the cloud computing paradigm over a natural approach to cost-effectively adapting the allocated resources to the application's current needs. Yet, the same characteristics impose extra challenges to predicting the performance of cloud-based big data applications, a central step in proper management and planning. This paper explores two modeling approaches for performance prediction of cloud-based big data applications. We evaluate a queuing-based analytical model and a novel fast ad-hoc simulator in various scenarios based on different applications and infrastructure setups. Our results show that our approaches can predict average application execution times with 26% relative error in the very worst case and about 12% on average. Moreover, our simulator provides performance estimates 70 times faster than state of the art simulation tools.

Nowadays many companies have available large amounts of raw, unstructured data. Among Big Data enabling technologies, a central place is held by the MapReduce framework and, in particular, by its open source implementation, Apache Hadoop. For cost effectiveness considerations, a common approach entails sharing server clusters among multiple users. The underlying infrastructure should provide every user with a fair share of computational resources, ensuring that service level agreements (SLAs) are met and avoiding wastes. In this paper we consider mathematical models for the optimal allocation of computational resources in a Hadoop 2.x cluster with the aim to develop new capacity allocation techniques that guarantee better performance in shared data centers. Our goal is to get a substantial reduction of power consumption while respecting the deadlines stated in the SLAs and avoiding penalties associated with job rejections. The core of this approach is a distributed algorithm for runtime capacity allocation, based on Game Theory models and techniques, that mimics the MapReduce dynamics by means of interacting players, namely the central Resource Manager and Class Managers.

The adoption of cloud technologies is steadily increasing. In such systems, applications can benefit from nearly infinite virtual resources on a pay-per-use basis. However, being the cloud massively multi-tenant and characterized by highly variable workloads the development of more and more effective provisioning policies assumes paramount importance. Boosted by the success of the cloud, the application of Game Theory models and methodologies has also become popular, since they have been demonstrated to suit perfectly to cloud social, economic, and strategic structures. This paper aims to study, model and efficiently solve the cost minimization problem associated with the service provisioning of SaaS virtual machines in multiple IaaSs. We propose a game-theoretic approach for the runtime management of resources from multiple IaaS providers to be allocated to multiple competing SaaSs, along with a cost model including revenues and penalties for requests execution failures. A distributed algorithm for identifying Generalized Nash Equilibria has been developed and analysed in detail. The effectiveness of our approach has been assessed by performing a wide set of analyses under multiple workload conditions. Results show that our algorithm is scalable and provides significant cost savings with respect to alternative methods (80 percent on average). Furthermore, increasing the number of IaaS providers SaaSs can achieve 9-15 percent cost savings from the workload distribution on multiple IaaSs.

Spark has grown both in popularity and complexity in recent years. In order to use available resources in an efficient way, users need to understand how the behavior of their applications is affected by the size of the datasets and various configuration settings. Indeed, Spark allows users to specify many configuration parameters and understanding the impact of these choices with respect to the application execution time is not easy. An accurate estimate of application execution time is important for cluster capacity planning and/or runtime scheduling. In this work we propose a gray-box approach to analyze the performance of Spark applications deployed in public cloud infrastructures. The approach is divided into two phases: during application profiling, the application is executed multiple times against different subsets of the input datasets to understand the effect of the data size on the execution time and its dependency on the main configuration parameters. Next, during the estimation phase, we use the data gathered in the first step to predict the execution time of the application, run against the entire dataset. The prediction approach builds several models in order to estimate separately the growth of the time required to execute each stage within the application. Finally, the DAG used by Spark to schedule the execution of stages is analyzed to aggregate the predictions of the stages execution times into the overall application execution time. Both phases are supported by our SLAP open source tool. Experimental results show that our model can effectively and accurately predict application execution time. The approach outperforms pure black-box polynomial regression methods obtaining 1-3% relative error.

Cloud computing is a new emerging paradigm that aims to streamline the on-demand provisioning of resources as services, providing end users with flexible and scalable services accessible through the Internet on a pay-per-use basis. Because modern cloud systems operate in an open and dynamic world characterized by continuous changes, the development of efficient resource provisioning policies for cloud-based services becomes increasingly challenging. This paper aims to study the hourly basis service provisioning problem through a generalized Nash game model. We take the perspective of Software as a Service (SaaS) providers that want to minimize the costs associated with the virtual machine instances allocated in a multiple Infrastructures as a Service (IaaS) scenario while avoiding incurring penalties for execution failures and providing quality of service guarantees. SaaS providers compete and bid for the use of infrastructural resources, whereas the IaaSs want to maximize their revenues obtained providing virtualized resources. We propose a solution algorithm based on the best-reply dynamics, which is suitable for a distributed implementation. We demonstrate the effectiveness of our approach by performing numerical tests, considering multiple workloads and system configurations. Results show that our algorithm is scalable and provides significant cost savings with respect to alternative methods (5% on average but up to 260% for individual SaaS providers). Furthermore, varying the number of IaaS providers means an 8%–15% cost savings can be achieved from the workload distribution on multiple IaaSs.

Model-driven engineering (MDE) often features quality assurance (QA) techniques to help developers creating software that meets reliability, efficiency, and safety requirements. In this paper, we consider the question of how quality-aware MDE should support data-intensive software systems. This is a difficult challenge, since existing models and QA techniques largely ignore properties of data such as volumes, velocities, or data location. Furthermore, QA requires the ability to characterize the behavior of technologies such as Hadoop/MapReduce, NoSQL, and stream-based processing, which are poorly understood from a modeling standpoint. To foster a community response to these challenges, we present the research agenda of DICE, a quality-aware MDE methodology for data-intensive cloud applications. DICE aims at developing a quality engineering tool chain offering simulation, verification, and architectural optimization for Big Data applications. We overview some key challenges involved in developing these tools and the underpinning models.

The advent of cloud computing has offered to developers a new appealing paradigm to deploy their applications without capital investments. Resources can now be acquired on-demand in a flexible, scalable and rapid way. However, cloud providers lack of native mechanisms to guarantee the Quality of Service required by specific application domains. High availability can be achieved by replication of critical components. Since outages could affect the entire cloud provider, replication can be effective only by using multiple providers. In this paper we tackle the above problem and present an approach to guarantee availability requirements of cloud-based applications by exploiting replication on multiple clouds to reduce unavailability, still limiting costs. More precisely, we propose: i) an approach to model, at design time, the application, its availability requirements and the characteristics of the used clouds, and ii) a self-adaptive technique responsible, at runtime, of both in-cloud scaling policies and traffic routing among different cloud providers, by means of a control-theoretical approach. We integrated the modeling approach in the Palladio Bench IDE and developed a runtime self-adaptation controller in Matlab. The controller has been evaluated against different workload conditions, costs variations and service failures in simulated scenarios. The controller has been able to provide the desired availability minimizing costs.

Worldwide interest in the delivery of computing and storage capacity as a service continues to grow at a rapid pace. The complexities of such cloud computing centers require advanced resource management solutions that are capable of dynamically adapting the cloud platform while providing continuous service and performance guarantees. The goal of this paper is to devise resource allocation policies for virtualized cloud environments that satisfy performance and availability guarantees and minimize energy costs in very large cloud service centers. We present a scalable distributed hierarchical framework based on a mixed-integer nonlinear optimization of resource management acting at multiple timescales. Extensive experiments across a wide variety of configurations demonstrate the efficiency and effectiveness of our approach.

In recent years, the evolution and the widespread adoption of virtualization, service-oriented architectures, autonomic, and utility computing have converged letting a new paradigm to emerge: cloud computing. Clouds allow the on-demand delivering of software, hardware, and data as services. Currently, the cloud offer is becoming wider day by day because all the major IT companies and service providers, like Microsoft, Google, Amazon, HP, IBM, and VMWare, have started providing solutions involving this new technological paradigm. As cloud-based services are more numerous and dynamic, the development of efficient service provisioning policies becomes increasingly challenging. In this paper, we take the perspective of Software as a Service (SaaS) providers that host their applications at an Infrastructure as a Service (IaaS) provider. Each SaaS needs to comply with quality-of-service requirements, specified in service-level agreement (SLA) contracts with the end users, which determine the revenues and penalties on the basis of the achieved performance level. SaaS providers want to maximize their revenues from SLAs, while minimizing the cost of use of resources supplied by the IaaS provider. Moreover, SaaS providers compete and bid for the use of infrastructural resources. On the other hand, the IaaS wants to maximize the revenues obtained providing virtualized resources. In this paper, we model the service provisioning problem as a generalized Nash game and we show the existence of equilibria for such game. Moreover, we propose two solution methods based on the best-reply dynamics, and we prove their convergence in a finite number of iterations to a generalized Nash equilibrium. In particular, we develop an efficient distributed algorithm for the runtime allocation of IaaS resources among competing SaaS providers. We demonstrate the effectiveness of our approach by simulation and performing tests on a real prototype environment deployed on Amazon EC2. Results show that, compared to other state-of-the-art solutions, our model can improve the efficiency of the cloud system evaluated in terms of Price of Anarchy by 50-70 percent.

This paper describes a novel model for the service selection problem of workflow-based applications in the context of self-managing situated computing. In such systems, the execution environment includes different types of devices, from remote servers to personal notebooks, smartphones, and wireless sensors, which build an infrastructure that can dynamically change both its physical and logical architecture at runtime. We assume that workflows are defined abstractly; i.e., they invoke abstract services whose concrete counterparts can be selected dynamically. We also assume that concrete service implementations may possibly migrate on the nodes of the infrastructure. The selection problem we address is framed as an optimization problem of the quality of service (QoS), which evaluates at runtime the optimal binding to concrete services as well as the tradeoff between the remote execution of software fragments and their dynamic deployment on local nodes of the computational environment. The final deployment takes into account quality of service constraints, the capabilities of the physical devices involved, including their performance and energy consumption, and the characteristics of the networking links connecting them.

Cloud computing is emerging as a major trend in the ICT industry. While most of the attention of the research community is focused on considering the perspective of the Cloud providers, offering mechanisms to support scaling of resources and interoperability and federation between Clouds, the perspective of developers and operators willing to choose the Cloud without being strictly bound to a specific solution is mostly neglected. We argue that Model-Driven Development can be helpful in this context as it would allow developers to design software systems in a cloud-agnostic way and to be supported by model transformation techniques into the process of instantiating the system into specific, possibly, multiple Clouds. The MODAClouds (MOdel-Driven Approach for the design and execution of applications on multiple Clouds) approach we present here is based on these principles and aims at supporting system developers and operators in exploiting multiple Clouds for the same system and in migrating (part of) their systems from Cloud to Cloud as needed. MODAClouds offers a quality-driven design, development and operation method and features a Decision Support System to enable risk analysis for the selection of Cloud providers and for the evaluation of the Cloud adoption impact on internal business processes. Furthermore, MODAClouds offers a run-time environment for observing the system under execution and for enabling a feedback loop with the design environment. This allows system developers to react to performance fluctuations and to re-deploy applications on different Clouds on the long term.

With the increase of energy consumption associated with IT infrastructures, energy management is becoming a priority in the design and operation of complex service-based systems. At the same time, service providers need to comply with Service Level Agreement (SLA) contracts which determine the revenues and penalties on the basis of the achieved performance level. This paper focuses on the resource allocation problem in multitier virtualized systems with the goal of maximizing the SLAs revenue while minimizing energy costs. The main novelty of our approach is to address—in a unifying framework—service centers resource management by exploiting as actuation mechanisms allocation of virtual machines (VMs) to servers, load balancing, capacity allocation, server power state tuning, and dynamic voltage/frequency scaling. Resource management is modeled as an NP-hard mixed integer nonlinear programming problem, and solved by a local search procedure. To validate its effectiveness, the proposed model is compared to top-performing state-of-the-art techniques. The evaluation is based on simulation and on real experiments performed in a prototype environment. Synthetic as well as realistic workloads and a number of different scenarios of interest are considered. Results show that we are able to yield significant revenue gains for the provider when compared to alternative methods (up to 45 percent). Moreover, solutions are robust to service time and workload variations.

In Cloud computing systems, resource management is one of the main issues. Indeed, in any time instant resources have to be allocated to handle effectively workload fluctuations, while providing Quality of Service (QoS) guarantees to the end users. In such systems, workload prediction-based autonomic computing techniques have been developed. In this paper we propose capacity allocation techniques able to coordinate multiple distributed resource controllers working in geographically distributed cloud sites. Furthermore, capacity allocation solutions are integrated with a load redirection mechanism which forwards incoming requests between different domains. The overall goal is to minimize the costs of the allocated virtual machine instances, while guaranteeing QoS constraints expressed as a threshold on the average response time. We compare multiple heuristics which integrate workload prediction and distributed non-linear optimization techniques. Experimental results show how our solutions significantly improve other heuristics proposed in the literature (5-35% on average), without introducing significant QoS violations.

Cloud computing is an emerging paradigm which allows the on-demand delivering of software, hardware, and data as services. As cloud-based services are more numerous and dynamic, the development of efficient service provisioning policies become increasingly challenging. Game theoretic approaches have shown to gain a thorough analytical understanding of the service provisioning problem. In this paper we take the perspective of Software as a Service (SaaS) providers which host their applications at an Infrastructure as a Service (IaaS) provider. Each SaaS needs to comply with quality of service requirements, specified in Service Level Agreement (SLA) contracts with the end-users, which determine the revenues and penalties on the basis of the achieved performance level. SaaS providers want to maximize their revenues from SLAs, while minimizing the cost of use of resources supplied by the IaaS provider. Moreover, SaaS providers compete and bid for the use of infrastructural resources. On the other hand, the IaaS wants to maximize the revenues obtained providing virtualized resources. In this paper we model the service provisioning problem as a Generalized Nash game, and we propose an efficient algorithm for the run time management and allocation of IaaS resources to competing SaaSs.

Modern cloud infrastructures live in an open world, characterized by continuous changes in the environment and in the requirements they have to meet. Continuous changes occur autonomously and unpredictably, and they are out of control of the cloud provider. Therefore, advanced solutions have to be developed able to dynamically adapt the cloud infrastructure, while providing continuous service and performance guarantees. A number of autonomic computing solutions have been developed such that resources are dynamically allocated among running applications on the basis of short-term demand estimates. However, only performance and energy trade-off have been considered so far with a lower emphasis on the infrastructure dependability/availability which has been demonstrated to be the weakest link in the chain for early cloud providers. The aim of this paper is to fill this literature gap devising resource allocation policies for cloud virtualized environments able to identify performance and energy trade-offs, providing a priori availability guarantees for cloud end-users.

Multiple combinations of hardware and network components can be selected to design an information technology (IT) infrastructure that satisfies requirements. The professional criterion to deal with these degrees of freedom is cost minimization. However, a scientific approach has been rarely applied to cost minimization, particularly for the joint optimization of hardware and network systems. This paper provides an overall methodology for combining hardware and network designs in a single cost minimization problem for multisite computer systems. Costs are minimized by applying a heuristic optimization approach to a sound decomposition of the problem. We consider most of the design alternatives that are enabled by current hardware and network technologies, including server sizing, localization of mutitier applications, and reuse of legacy systems. The methodology is empirically verified with a database of costs that has also been built as part of this paper. Verifications consider several test cases with different computing and communication requirements. Cost reductions are evaluated by comparing the cost of methodological results with those of architectural solutions that are obtained by applying professional design guidelines. The quality of heuristic optimization results is evaluated through comparison with lower bounds.

In advanced service oriented systems, complex applications, described as abstract business processes, can be executed by invoking a number of available Web services. End users can specify different preferences and constraints and service selection can be performed dynamically identifying the best set of services available at runtime. In this paper, we introduce a new modeling approach to the Web service selection problem that is particularly effective for large processes and when QoS constraints are severe. In the model, the Web service selection problem is formalized as a mixed integer linear programming problem, loops peeling is adopted in the optimization, and constraints posed by stateful Web services are considered. Moreover, negotiation techniques are exploited to identify a feasible solution of the problem, if one does not exist. Experimental results compare our method with other solutions proposed in the literature and demonstrate the effectiveness of our approach toward the identification of an optimal solution to the QoS constrained Web service selection problem.

In service oriented architectures, complex applications are composed from a variety of functionally equivalent Web services, which may differ for quality parameters. Under this scenario, applications are defined as high level business processes and service composition can be implemented dynamically by identifying the best set of services available at run time. In this paper, we model the service composition problem as a mixed integer linear problem where both local constraints and global constraints can be specified.

Nowadays, large service centers provide computational capacity to many customers by sharing a pool of IT resources. The service providers and their customers negotiate utility based service level agreement (SLA) to determine the costs and penalties on the base of the achieved performance level. The system is often based on a multi-tier architecture to service requests. The service provider would like to maximize the SLA revenues, while minimizing its operating costs. The system we consider is based on a centralized network dispatcher which controls the allocation of applications to servers, the request volumes at various servers and the scheduling policy at each server. The dispatcher can also decide to turn ON or OFF servers depending on the system load. This paper designs a resource allocation scheduler for such multi-tier environments so as to maximize the profits associated with multiple class SLAs. The overall problem is NP-hard. We develop heuristic solutions by implementing a local-search algorithm. Results are presented to demonstrate the benefits of our approach

With the development of the Service Oriented Architecture (SOA), organizations are able to compose complex applications from distributed services supported by third party providers. Under this scenario, large data centers provide services to many customers by sharing available IT resources. This leads to the efficient use of resources and the reduction of operating costs. Service providers and their customers often negotiate utility based Service Level Agreements (SLAs) to determine costs and penalties based on the achieved performance levels. Data centers often employ an autonomic computing infrastructure and use a centralized dispatch and control component (a dispatcher) to distribute the user requests to backend servers, and to set the scheduling policies at each server. This dispatcher can also decide to turn ON or OFF servers depending on the system load. This paper designs a set of dispatching and control policies for the dispatcher in such service oriented environments. The objective is to maximize the provider's profits associated with multiple class of SLAs. We show that the overall problem is NP-hard, and develop meta-heuristic solutions based on the tabu-search algorithm. Experimental results are presented to show the benefits of our approach.