Organizations are increasingly adopting Cloud-based development to create services by integrating Cloud applications and Software as a Service (SaaS) APIs that meet performance standards set by Service Level Agreements (SLAs). However, optimizing such Cloud-based integrations through traditional load testing is challenging due to cost and Cloud application restrictions. Static analysis approaches like mean value analysis (MVA) can be used to minimize these difficulties. However, they require insights into invoked Cloud applications and make unrealistic assumptions. Alternatively, predictive performance modeling offers more accurate results, yet it necessitates the need for load testing the integration up to a certain level to obtain training data, which can be costly, especially when invoking third-party paid Cloud applications. To address these issues, PERF-Expert is introduced as a novel approach that provides dynamic and accurate performance feedback on Cloud-application integrations during development. PERF-Expert learns the performance characteristics of atomic Cloud applications within existing deployed integrations and combines them with mathematical models to determine the performance of new integrations without performance tests. PERF-Expert continuously monitors changes and provides dynamic feedback without human intervention. Compared to MVA, PERF-Expert is 34.75% more accurate for real-world deployments and can drill down to individual Cloud-application performance with high accuracy. PERF-Expert, backed up by a suitable Cloud observability platform, can be a powerful tool for developers and performance engineers. Illustrating its powerfulness, PERF-Expert is already integrated into a Cloud developer platform to provide development time performance feedback for Cloud-based integration.

Modern application servers run on multi-core hardware and these servers use multi-threading to achieve high performance. Performance of such servers is highly dependent on the number of processing threads in the worker thread pool. The optimal thread pool size (that would result in the best performance) depends on workload properties (both the application and the incoming arrival pattern). In this paper, we propose ADAPT-T, a lightweight, adaptive algorithm that can auto-adjust the thread pool size to optimize the latency. ADAPT-T exploits the concave upward property of system performance and optimizes the latency on-line by periodically monitoring the server's behaviour and then adjusts the thread pool size based on monitored statistics. It has a special locking mechanism to minimize the frequent fluctuations that may occur in latency while tuning the thread pool size. We perform extensive experiments under different workload patterns and show that ADAPT-T performs well under a wide range of workload scenarios compared to other policies.

Service time distribution of certain computing workloads such as static web content is well known. However, for many other computing workloads (e.g., dynamic web content, scientific workloads) the service time distribution is not well understood and it is not correct to assume that these tasks follow a particular distribution. In this paper, we consider task assignment in server farms when both the service time distribution of tasks and (actual) sizes of tasks are not known a priori. We propose an adaptive task assignment policy, called ADAPT-POLICY, which is based on the concept of multiple static-based task assignment policies. ADAPT-POLICY defines a set of policies for a given system taking into account the specific properties of the system. These policies are selected in such a way that they have different performance characteristics under different workload conditions (i.e., service time distributions, etc.). The objective is to use the task assignment policy with the best performance (i.e., the one with the least expected waiting time) to assign tasks. Which task assignment policy performs the best depends on the traffic conditions that vary over time. ADAPT-POLICY determines the best task assignment using the service time distribution of tasks (and various other traffic properties), which is estimated on-line and then it adaptively changes the task assignment policy to suit the most recent traffic conditions. The experimental results show that ADAPT-POLICY can result in significant performance improvements over both static and dynamic task assignment policies.

There is extensive evidence indicating that modern computer workloads exhibit highly variability in their processing requirements. Under such workloads, traditional task assignment policies do not perform well. Size-based policies perform significantly better than traditional policies under highly variable workloads. The main limitation of existing size-based policies though is that these have been targeted for batch computing systems. In this paper, we provide performance analysis of 3 novel task assignment policies that are based on multi-level time sharing policy, namely MLMS (Multi-level Multi-server Task Assignment Policy), MLMS-M (Multi-level Multi-server Task Assignment Policy with Task Migration) and MLMS-M* (Multi-tier Multi-level Multi-server Task Assignment policy with Task Migration). These policies attempt to improve the performance first by giving preferential treatment to small tasks and second by reducing the task size variability in host queues. MLMS only reduces the variability of tasks locally, while MLMS-M and MLMS-M* utilise both local and global variance reduction mechanisms. MLMS outperforms existing size-based policies such as TAGS under specific workload conditions. MLMS-M outperforms TAGS under all the scenarios considered. MLMS-M*outperforms TAGS and MLMS-M under specific workload conditions and vice versa.

Existing task assignment policies designed to schedule highly variable computer workloads are not scalable and they generate large amount of wasted processing. This paper proposes a scalable Multi-Tier Task Assignment Policy with Minimum Excess Load (MTTMEL) that can efficiently schedule tasks with highly variable processing requirements. Unlike existing policies, the proposed policy does not assign all incoming tasks to the first host in the system, rather it assigns incoming tasks to a subset of hosts in the system. The proposed policy has multiple tiers that consist of one or more hosts. The proposed policy scales well and it significantly reduces the amount wasted processing (excess load). For example, the TAGS policy produces 460% more wasted processing compared to MTTMEL under certain workloads. MTTMEL outperforms existing policies under a wide range of workload conditions. It outperforms RANDOM by a factor of 23 and TAGS by a factor of 2.6 under certain scenarios.

Size-based task assignment policies have shown significant performance improvements over traditional task assignment policies under highly variable workload conditions. However, these policies are not suitable to assign tasks in time sharing systems. Moreover, these policies are not scalable and they also generate significant amount of wasted processing. This paper proposes a Multi-Level-Multi-Server Task Assignment Policy with Work-Conserving Migration (MLMS-WC-M) that addresses these issues. MLMS-WC-M has three important features. First, it gives preferential treatment to tasks with short processing requirements. Second, it utilises a 2-level variance reduction mechanism. Third, it supports work-conserving migration. We evaluate the performance of MLMS-WC-M against the performance of several well known task assignment policies. The proposed policy outperforms existing policies significantly under a wide range of workload conditions.

Many existing works on multi-level time sharing policies have assumed infinitely small quanta, infinite levels or exponential service time distributions. In this paper, we investigate the performance of a multi-level time sharing policy under heavy-tailed workloads under finite levels when quanta are not infinitely small. Such a policy is consistent with those implemented on modern computer systems and these findings will enable system designers to better understand how various factors (e.g. system load, task size variability and number of levels) affect the overall performance of a given system. First, we obtain the performance metrics for a multi-level time sharing policy with finite number of levels. Second, for the case of 2 and 3 levels (queues), we show that optimal quantum multi-level time sharing policy (MLOQTP) can result in significant performance improvements over other policies under certain traffic and workload conditions. Finally, we investigate the impact of number of levels on the overall performance and propose a simple statistical regression model that can accurately estimate overall performance of a multi-level time sharing system.