Task offloading in distributed computing involves complex tradeoffs among delay, scalability, cost, and resource utilization. Cloud platforms face long communication delays, while edge nodes have constrained capacity. Static, rule-based schedulers cannot adapt to fluctuating loads or per-instance heterogeneity. Similarly, Reinforcement Learning (RL) schemes typically address only a single layer or assume homogeneous servers, overlooking the hierarchical and multi-instance nature of deployments. To address these challenges, we introduce a server-aware Proximal Policy Optimization (PPO) framework that performs fine-grained offloading across a three-tier (Edge, Regional, Cloud) multi-instance architecture. We formulate offloading as a Markov Decision Process whose state vector includes per-instance delay, CPU/memory utilization, network congestion, cost, and energy metrics. The PPO agent learns to offload tasks to the best server in real time. Our developed RegionalEdgeSimPy simulation shows that PPO agent makes optimal offloading choices for over 90% of tasks, keeping each server near; however, below a 70% utilization. This optimized decision making drives up to 66.9% delay reduction, 78.6% energy savings, and 47.8% cost reductions relative to cloud-only and edge-only baselines.

The workflows of the predominant datacenter services are underlaid by various Fork-Join structures. Due to the lack of good understanding of the performance of Fork-Join structures in general, today's datacenters often operate under low resource utilization to meet stringent service level objectives (SLOs), e.g., in terms of tail and/or mean latency, for such services. Hence, to achieve high resource utilization, while meeting stringent SLOs, it is of paramount importance to be able to accurately predict the tail and/or mean latency for a broad range of Fork-Join structures of practical interests. In this article, we propose a black-box Fork-Join model that covers a wide range of Fork-Join structures for the prediction of tail and mean latency, called ForkTail and ForkMean, respectively. We derive highly computational effective, empirical expressions for tail and mean latency as functions of means and variances of task response times. Our extensive testing results based on model-based and trace-driven simulations, as well as a real-world case study in a cloud environment demonstrate that the models can consistently predict the tail and mean latency within 20 and 15 percent prediction errors at 80 and 90 percent load levels, respectively, for heavy-tailed workloads, and at any load levels for light-tailed workloads. Moreover, our sensitivity analysis demonstrates that such errors can be well compensated for with no more than 7 percent resource overprovisioning. Consequently, the proposed prediction model can be used as a powerful tool to aid the design of tail-and-mean-latency guaranteed job scheduling and resource provisioning, especially at high load, for datacenter applications.

The workflows of the predominant user-facing datacenter services, including web searching and social networking, are underlaid by various Fork-Join structures. Due to the lack of understanding the performance of Fork-Join structures in general, today's datacenters often resort to resource overprovisioning, operating under low resource utilization, to meet stringent tail-latency service level objectives (SLOs) for such services. Hence, to achieve high resource utilization, while meeting stringent tail-latency SLOs, it is of paramount importance to be able to accurately predict the tail latency for a broad range of Fork-Join structures of practical interests.