This work presents a comprehensive evaluation of neural network graph compilers across heterogeneous hardware platforms, addressing the critical gap between theoretical optimization techniques and practical deployment scenarios. We demonstrate how vendor-specific optimizations can invalidate relative performance comparisons between architectural archetypes, with performance advantages sometimes completely reversing after compilation. Our systematic analysis reveals that graph compilers exhibit performance patterns highly dependent on both neural architecture and batch sizes. Through fine-grained block-level experimentation, we establish that vendor-specific compilers can leverage repeated patterns in simple architectures, yielding disproportionate throughput gains as model depth increases. We introduce novel metrics to quantify a compiler's ability to mitigate performance friction as batch size increases. Our methodology bridges the gap between academic research and practical deployment by incorporating compiler effects throughout the research process, providing actionable insights for practitioners navigating complex optimization landscapes across heterogeneous hardware environments.

Serverless edge computing promises autonomous function management across the heterogeneous edge-cloud continuum. Specifically, autoscaling of functions increases resource efficiency by creating and destroying instances on demand. However, reactive function instance creation can cause high end-to-end latency due to requests waiting for processing. Researchers and practitioners use proactive autoscaling mechanisms to overcome this by predicting the required number of function instances in advance. Proactive approaches pose challenges because prediction models must be fine-tuned for new environments and updated over time. Therefore, we propose a simulation-driven framework, EdgeCloudForge, that generates datasets to train prediction models for autoscaling in edge-cloud environments ahead of their deployment, thus preparing them in advance for new environments. We introduce the concept of Edge-Cloud Domain Space to describe different infrastructure attributes. EdgeCloudForge uses a two-step method based on initial sampling and Bayesian optimization to navigate this domain space. Simulation results show that our approach can reduce SLO violations by up to 79% across different request patterns compared to the reactive approach and is able to outperform approaches from related work by up to 91%. EdgeCloudForge also shows the capability of training models that can be used across different infrastructures.

Nanosatellite constellations equipped with sensors capturing large geographic regions provide unprecedented opportunities for Earth observation. As constellation sizes increase, network contention poses a downlink bottleneck. Orbital Edge Computing (OEC) leverages limited onboard compute resources to reduce transfer costs by processing the raw captures at the source. However, current solutions have limited practicability due to reliance on crude filtering methods or over-prioritizing particular downstream tasks. This work presents an OEC-native and task-agnostic feature compression method that preserves prediction performance and partitions high-resolution satellite imagery to maximize throughput. Further, it embeds context and leverages inter-tile dependencies to lower transfer costs with negligible overhead. While the encoding prioritizes features for downstream tasks, we can reliably recover images with competitive scores on quality measures at lower bitrates. We extensively evaluate transfer cost reduction by including the peculiarity of intermittently available network connections in low earth orbit. Finally, we test the feasibility of our system for standardized nanosatellite form factors. We demonstrate that the proposed approach permits downlinking over 100× the data volume without relying on prior information on the downstream tasks.

Serverless Edge Computing is growing in popularity, and while commercial providers are starting to offer edge-oriented products, much research is still being done on orches-trating functions (e.g., autoscaling). These approaches range from threshold- to AI-based strategies and support various Service Level Objectives (SLOs), such as Round-Trip-Time (RTT) and re-source usage. However, the Quality of Service (Qo$S$) continuously deteriorates due to the dynamic edge-cloud continuum and static parameterization of orchestration strategy parameters. Platforms must adapt the orchestration parameters during runtime to counteract this drift that causes SLO violations. To this end, we introduce the Orchestration Parameter Optimization Prob-lem (OPOP), which aims to find parameters for orchestration strategies to minimize SLO violations. We propose a novel self-adaptive Simulation-based Scaling (SimuScale) approach that uses co-simulation to solve OPOP for autoscalers during runtime. SimuScale uses live monitoring data to feed the simulation and perform parameter optimization. Our Proof of Concept is inte-grated with Kubernetes and evaluated on a real-world edge-cloud testbed. While this work focuses on a threshold-based autoscaler, it can be extended to optimize other orchestration components (e.g., schedulers). Our experimental results show that SimuScale finds parameters that decrease RTT SLO violations between 15% and 40%. SimuScale also can reduce resource usage by 29.87% while maintaining the target 95th RTT percentile. Moreover, it can reduce variance caused by different request patterns, making orchestration strategies more resilient in realistic scenarios.

Reducing the energy consumption of data centers is critical to meeting international climate goals and lowering operation costs. Container orchestration platforms can help counteract this trend by optimally placing applications across the infrastructure to increase resource utilization and reduce energy consumption. But platforms in use today are still energy-agnostic and do not offer any insights into energy consumption. In this paper, we present a monitoring framework and a new modeling approach for resource usage in data centers. The model captures heterogeneous hardware and software and acts as input for a Graph Neural Network (GNN) to predict power consumption. Based on this model, we derive a set of container scheduling algorithms that opportunistically schedule applications based on the estimated energy impact of incoming containers. Our results show that the GNN-based prediction model is very accurate and achieves an average RMSE (Root Mean Square Error) of 7.5%. We have implemented a custom scheduler to demonstrate the benefits of using our prediction, and our scheduler can decrease energy consumption on average by 6.2% without any code changes for the application and without increasing workload completion time compared to the default Kubernetes scheduler.

The rise of mobile AI accelerators allows latency-sensitive applications to execute lightweight Deep Neural Networks (DNNs) on the client side. However, critical applications require powerful models that edge devices cannot host and must therefore offload requests, where the high-dimensional data will compete for limited bandwidth. Split Computing (SC) alleviates resource inefficiency by partitioning DNN layers across devices, but current methods are overly specific and only marginally reduce bandwidth consumption. This work proposes shifting away from focusing on executing shallow layers of partitioned DNNs. Instead, it advocates concentrating the local resources on variational compression optimized for machine interpretability. We introduce a novel framework for resource-conscious compression models and extensively evaluate our method in an environment reflecting the asymmetric resource distribution between edge devices and servers. Our method achieves 60% lower bitrate than a state-of-the-art SC method without decreasing accuracy and is up to 16x faster than offloading with existing codec standards.

The complexity inherent in managing cloud computing systems calls for novel solutions that can effectively enforce high-level Service Level Objectives (SLOs) promptly. Unfortunately, most of the current SLO management solutions rely on reactive approaches, i.e., correcting SLO violations only after they have occurred. Further, the few methods that explore predictive techniques to prevent SLO violations focus solely on forecasting low-level system metrics, such as CPU and Memory utilization. Although valid in some cases, these metrics do not necessarily provide clear and actionable insights into application behavior. This paper presents a novel approach that directly predicts high-level SLOs using low-level system metrics. We target this goal by training and optimizing two state-of-the-art neural network models, a Short-Term Long Memory - LSTM, and a Transformer-based model. Our models provide actionable insights into application behavior by establishing proper connections between the evolution of low-level workload-related metrics and the high-level SLOs. We demonstrate our approach to selecting and preparing the data. We show in practice how to optimize LSTM and Transformer by targeting efficiency as a high-level SLO metric and performing a comparative analysis. We show how these models behave when the input workloads come from different distributions. Consequently, we demonstrate their ability to generalize in heterogeneous systems. Finally, we operationalize our two models by integrating them into the Polaris framework we have been developing to enable a performance-driven SLO-native approach to Cloud computing.

The amalgamation of multiple Edge and Cloud clusters into an Edge-Cloud continuum requires efficient scheduling techniques to cope with high numbers of infrastructure nodes and computing jobs. Since monolithic schedulers typically do not scale well beyond a certain cluster size, distributed scheduling approaches are usually employed to address such scalability issues. Distributed schedulers are often designed for Cloud environments and lack support for the Edge. Conversely, many Edge schedulers focus on single clusters and provide limited support to deal with the scale of the Edge-Cloud continuum. In this paper, we present the Vela Distributed Scheduler, a globally distributed scheduler, which is specifically tailored for the Edge-Cloud continuum. The main contributions of our work include: i) A novel, globally distributed and orchestrator-independent scheduler with a 3-phase scheduling workflow; ii) A two-level, informed sampling mechanism, which reduces latency for globally distributed sampling and leverages job requirements to produce high quality node samples; And iii) a MultiBind mechanism that significantly reduces job evictions and rescheduling due to scheduling conflicts. We implement Vela on top of Kubernetes and evaluate it in a realistic large-scale setup using multiple interconnected, globally distributed, and production-ready MicroK8s clusters with up to 20,000 total simulated nodes. Our results show that Vela’s performance scales linearly with infrastructure size and that it reduces scheduling conflicts by a factor of 10.

Abstract This paper presents faas‐sim , a simulation framework tailored to serverless edge computing platforms. In serverless computing, platform operators are tasked with efficiently managing distributed computing infrastructure completely abstracted from application developers. To that end, platform operators and researchers need tools to design, build, and evaluate resource management techniques that efficiently use of infrastructure while optimizing application performance. This challenge is exacerbated in edge computing scenarios, where, compared to cloud computing, there is a lack of reference architectures, design tools, or standardized benchmarks. faas‐sim bridges this gap by providing (a) a generalized model of serverless systems that builds on the function‐as‐a‐service abstraction, (b) a simulator that uses trace data from real‐world edge computing testbeds and representative workloads, and (c) a network topology generator to model and simulate distributed and heterogeneous edge‐cloud systems. We present the conceptual design, implementation, and a thorough evaluation of faas‐sim . By running experiments on both real‐world test beds and replicating them using faas‐sim , we show that the simulator provides accurate results and reasonable simulation performance. We have profiled a wide range of edge computing infrastructure and workloads, focusing on typical edge computing scenarios such as edge AI inference or data processing. Moreover, we present several instances where we have successfully used faas‐sim to either design, optimize, or evaluate serverless edge computing systems.

This paper presents a framework for defining, performing, and analyzing distributed load testing experiments for benchmarking edge-cloud clusters. This end-to-end workflow helps researchers build reproducible environments to evaluate cluster management techniques. Our implementation extends the open source tool Galileo by adding support for distributed execution on Kubernetes clusters, additional system monitoring instruments, as well as out-of-the box experiment workloads. We focus on providing tools that run across popular CPU architectures and provide a set of representative workloads, such as edge AI functions. We demonstrate our framework's capabilities in a set of experiments based on use cases commonly found in edge computing systems research. Additionally, we show that the resource usage of our system is minimal and that it can run on resource-constrained devices.