Federated learning, as a compelling machine learning framework, enables collaborative model training without exposing private data. However, the excessive communication overhead remains a major challenge. To tackle this challenge, hierarchical federated edge learning (HFEL) framework has been proposed for reducing the communication load via migrating the model aggregation partially from cloud to edge servers. Although HFEL has significant potential, it is still constrained by end-devices with limited computational capabilities and unfavorable network conditions. A common approach to reduce this effect is to involve only the fastest end-devices in the training process. But because only parts of end-devices' data samples can be selected by such means, it damages the diversity of training data, and hence greatly affects the model's quality. In addition, for further improving the training performance, a proper bandwidth allocation strategy is also needed to make full use of the shared network resource of edge servers. To this end, we proposeDAHFF, aDiversity-AwareHierarchicalFastFederated learning framework consisting ofVirtual Queue based Device Selectionphase andBinary Search based Bandwidth Allocation, which are responsible for selecting participated end-devices and allocating bandwidth for selected devices, respectively. Extensive experiments on different deep learning models show that our proposed framework can averagely speed up the training performance by$2.07\times$in comparison with state-of-the-art approaches.

Scientific research is increasingly relying on complex workflows that span multiple computing paradigms, including high-performance computing (HPC), high-throughput computing (HTC), and machine learning/artificial intelligence (ML/AI). Traditional monolithic computing infrastructures often struggle to accommodate these diverse and evolving demands. The Reconfigurable Advanced Platform for Transdisciplinary Open Research (RAPTOR) addresses this challenge by providing a dynamically reconfigurable computing environment that integrates with federated resources. RAPTOR's architecture enables dynamic provisioning between an HPC cluster and the Chameleon Cloud platform based on workload requirements, supporting bare-metal customization for specialized applications. This paper focuses on RAPTOR's reconfigurability features and demonstrates their effectiveness through quantitative performance evaluations across four scientific domains: computational proteomics, climate modeling, weather research, and hurricane risk assessment. Our results demonstrate that RAPTOR's reconfigurable design significantly enhances research productivity by providing an appropriate computing environment for diverse computational needs.

Serverless computing is increasingly being adopted to provision various on-demand services at the edge cloud, including inference tasks based on deep neural networks (DNNs) for the Internet of Things (IoT). This approach leverages the advantages of flexible resource allocation and fine-grained resource management. However, the provisioning of on-demand inference typically requires downloading the DNN model at runtime, which can introduce significant delays. In the edge cloud with heterogeneous network connections, the inevitable model downloading time and the inter-model data transmission impose high challenges to the QoS of inference tasks. In this paper, we investigate how to jointly consider both model downloading time and communication time to minimize inference time. We first formulate this problem into a nonlinear optimization form and proved it as NP-hard. We further propose a Priority-based Model Assignment (PASS) algorithm in polynomial time and trace-driven experimental results show that it reduces the average inference time by 23.6% compared to existing state-of-the-art solutions.

The rapid deployment of intelligent applications on edge cloud calls for efficient and responsive DNN inference, especially under the burst scenarios of inference request. Serverless inference offers a promising solution by enabling rapid and flexible activation of inference tasks to cope with peak request, but its achievable performance is highly influenced by the initialization overhead. Existing studies on inference acceleration mainly focuses on execution optimization, they usually overlook the fact that inference performance also heavily depends on the initialization. In this paper, we propose EdgePrios, a novel priority-based scheduling mechanism that jointly optimizes initialization and execution phases for serverless inference acceleration. EdgePrios dynamically prioritizes tasks by considering workloads, dependency relationships, and the current status of available resources. It enables precise assignment of tasks to computing resources while minimizing overall inference time in edge cloud. Extensive trace-driven evaluations demonstrate the efficiency of EdgePrios as it outperforms state-of-the-art methods, achieving 15.6%-38.8% reduction in inference time under varying resource configurations, network bandwidths, and application topologies.

Thanks to the fine-grained resource management capabilities, serverless computing has been extended to edge cloud environments to support diverse Artificial Intelligence of Things (AIoT) applications, particularly those involving complex workflows of interdependent deep neural network (DNN) inference tasks. However, the inherent on-demand provisioning nature of serverless computing imposes the fact that, in serverless inference processes, the DNN models are typically maintained in the remote storage cluster and retrieved as needed. This inevitably incurs substantial latency overhead, particularly in resource-constrained edge cloud. In this paper, we investigate how to accelerate the AI application with joint consideration of both the model downloading time and intermediate data transmission time. We first formulate this problem into a nonlinear optimization form and prove it as NP-hard. We further propose a Priority-Based Model Assignment (PASS) algorithm and theoretically analyze its upper bound. The trace-driven experimental results demonstrate that our proposed algorithm outperforms other sate-of-art solutions and reduces the average application completion time by 23.6%.

The booming development of artificial intelligence (AI) applications has greatly promoted edge intelligence technology. To support latency-sensitive Deep Neural Network (DNN) based applications, the integration of serverless inference paradigm into edge intelligence has become a widely recognized solution. However, the long DNN model downloading time from central clouds to edge servers hinders inference performance, and asks for establishing model repository within the edge cloud. This paper first identifies the inherent layer redundancy in DNN models, which is potentially beneficial to improve the storage efficiency of the model repository in the edge cloud. However, how to exploit the layer redundancy feature and allocate the DNN layers across different edge servers with capacitated storage resources to reduce the model downloading time remains challenging. To address this issue, we first formulate this problem in Quadratic Integer Programming (QIP) form, based on which a randomized rounding layer redundancy aware DNN model storage planning strategy is proposed. Our approach significantly reduces model downloading time by up to 63% compared to state-of-the-art methods, as demonstrated through extensive trace-driven experiments.

Distributed Deep Neural Network (DNN) inference is a promising technology to explore the distributed resources in edge cloud to realize edge intelligence. Meanwhile the inherent resource sharing nature of edge cloud infrastructure also raises serious concerns on security and privacy. Software Guard Ex-tensions (SGX) emerges as a potential hardware-level solution but its limited secure memory (i.e., enclave page cache) imposes new challenges, especially in contrast to memory-hungry DNN models. A task's performance will be severely affected when its memory footprint is beyond the enclave page cache size, due to expensive secure page swapping. In this case, how to appropriately partition a DNN model and assign the partitions to distributed edge servers to efficiently utilize edge resources for fast secure inference becomes a challenging problem. In this paper, we first show that this problem is NP-hard. We further propose a MEmory -aware Distributed Inference Acceleration (MEDIA) algorithm, whose guaranteed approximation ratio is also formally analyzed. We have implemented a prototype system and applied some well-known representative DNN models to evaluate MEDIA's performance. Through extensive experiments, we verify the efficiency of MEDIA by the fact that it reduces the inference time by 19.5%-38.1 % in comparison with state-of-the-art approaches.

Edge native applications usually consist of several dependent tasks encapsulated in containers and started on-demand in the edge cloud. Unfortunately, the application performance is deeply affected by the notorious cold startup problem of containers. Pre-warming Zygote container pre-imported certain common packages has been proven as an effective startup acceleration solution. Since a Zygote can be shared among colocated tasks that require identical common packages, not only the Zygote planning but also the task scheduling decisions shall be carefully made to maximize the benefit of the Zygotes pre-warmed in limited memory. Additionally, task dependency necessitates co-locating highly dependent tasks on the same server, naturally raising a dilemma in task scheduling. To this end, in this paper, we investigate the problem of how to plan Zygote and schedule tasks for application completion time minimization, which is proved to be NP-hard. We further propose a Priority and Popularity (P&P) based edge native application acceleration algorithm. Both theoretical analysis and extensive experiments demonstrate the effectiveness of our proposed algorithm. The experiment results show that P&P can reduce the application completion time by 11.7%.

Thanks to the capability of fine-grained resource allocation and fast task scheduling, serverless computing has been adopted into edge cloud to accommodate various applications, e.g., deep neural network (DNN) inference for Artificial Intelligence of Things (AIoT). In serverless edge cloud, the servers are started up on-demand. However, as a container-based architecture, the inherent sequential startup feature of container imposes high affection on the DNN inference performance in serverless edge clouds. In this article, we investigate the distributed DNN inference problem in serverless edge cloud with the consideration of such characteristics, aiming to eliminate the extra container startup time cost to minimize the DNN inference latency. We formulate this problem into a nonlinear optimization form and then linearize it into an integer programming problem, which is proved as NP-hard. To tackle the computation complexity, we propose a priority-based layer scheduling (PLAYS) algorithm. Extensive experiment results verify the effectiveness and the adaptability of our PLAYS algorithm in comparison with other state-of-art algorithms under several well known DNN models.

Using docker to encapsulate the task has been regarded as a potential way to achieve efficient task orchestration and management in edge computing. Despite the lightweight nature of containers, downloading a larger container image can still be resource-intensive, particularly in resource-constrained edge environments. Fortunately, the unique layered architecture of the container allows multiple containerized tasks to share the same layer, thereby offering an opportunity for reducing the image downloading overhead via sharing the common layers. To explore the potential of layer sharing on image downloading over-head reduction, we investigate a joint task scheduling and image routing problem in edge environment, aiming at minimizing the image downloading overhead. We first formulate the problem into an integer linear programming form, and then propose a layer-aware scheduling and routing (LSR) algorithm to tackle this problem. Finally, to evaluate the effectiveness of our proposed algorithm, we conduct a group of simulation experiments. The experimental results show that our proposed algorithm can reduce the download time by about 20% in comparison with other approaches.

Edge computing is considered a promising computing paradigm that can mitigate energy consumption and workload of end devices through task offloading to edge servers. Albeit with high potential, edge computing is still challenged by various forms of “system noise”, e.g., node failures, system failures, and poor network conditions. To this end, distributed coded computing has been proposed for alleviating such effects by introducing redundancy into the computation. However, traditional distributed coded computing only focuses on leveraging the unreliable computing resource, and this indeed increases the risk of task non-completion within the acceptable timeframe. To address this problem, in this paper, we propose a partial distributed coded computing framework that can leverage the reliable and unreliable resources in the edge environment. We further investigate the problem of how to offload the coded subtasks for energy reduction while meeting the task tolerant latency. To tackle the computation complexity, we then propose an Iterated Greedy Algorithm. The experimental results verify the efficiency of our proposed algorithm, and it can reduce the energy consumption by 20% compared with other algorithms.

The cold startup of the container is regarded as a crucial problem to the performance of serverless computing, especially to the resource-capacitated edge clouds. Pre-warming hot containers has been proved as an efficient solution but is at the expense of high memory consumption. Instead of pre-warming a complete container for a function, recent studies advocate Zygote container, which pre-imports some packages and is able to import the other dependent packages at runtime, so as to avoid the cold startup problem. However, as different functions have different package dependencies, how to plan the Zygote generation and pre-warming in a resource-capacitated edge cloud becomes a critical challenge. In this paper, aiming to minimize the overall function startup time and subjective to the resource capacity constraints, we formulate this problem into a Quadratic Integer Programming (QIP) form. We further propose a Randomized Rounding based Zygote Planning (RRZP) algorithm. The performance efficiency of our algorithm is proved via both theoretical analysis and trace-driven simulations. The results show that our algorithm can significantly reduce the startup time by 25.6%.

Task offloading to edge servers has become a promising solution to tackle the computation resource poverty of the end devices. However, the zero-trust edge computing platform is highly challenged by the growing concern on security and privacy. Thus, Trust Execution Environment (TEE), like TrustZone, is advocated to empower edge clouds to enable secure task offloading. To explore TrustZone, the inevitable involvement of data encryption and decryption operations makes existing offloading strategies not applicable any more, especially when the task dependency is considered. In addition, TrustZone has distinguishable task scheduling paradigm as one CPU core does not allow multitask coexist at the same time. Taking the above issues into consideration, we investigate a dependency-aware task offloading problem for makespan minimization in TrustZone empowered edge clouds. By inventing an extended graph to describe the task execution process, we provide a formal statement to the problem and prove its NP-hardness. We then propose a Customized List Scheduling (CLS) based approximate algorithm and theoretically analyze its achievable performance. Extensive testbed based experiment results show that our approximation algorithm can effectively reduce the makespan and significantly outperforms existing state-of-the-art offloading approaches in TrustZone empowered edge clouds.

GPU has been widely applied to accelerate the DNN based applications. However, single GPU is overwhelmed by the increasing computation requirement of large-scale DNN inference task. Although GPUs cluster alleviates the pressure of massive inference task, it still traps into low efficiency owing to the limited computing power and network bandwidth. Besides, the exclusiveness of GPU device may cause the straggler problem and hence long inference time. Reinforcement Learning (RL) has been widely used in such task scheduling problems. But the delayed reward during the training may slow the convergence speed or even result in non-convergence. To this end, in this paper, we design an improved reinforcement learning based algorithm, called DRM-DQL, to achieve a layer-aware DNN inference task scheduling. We first analyze and model the layer-wise inference task scheduling problem by deep Q-learning. Then, a delayed reward matching strategy is proposed for matching the global reward value to the immediate reward value, which help the algorithm to get the right experience in DNN layer scheduling. The experiment results demonstrate that our algorithm performs better than both heuristic algorithm and the vanilla DQL algorithm, and show the robustness in various network bandwidths, computing power, and DNN model structures.

Cloud native computing featured by microservice has been regarded as a compelling trend in cloud application development. Edge computing, as an alternative or complemen-tary to cloud computing, is potential to expand the microservice to edge computing, simplifying the development and deployment of edge applications. Despite that, there is still a challenge on how to manage the microservices efficiently in the open and heterogeneous distributed environment. To this end, service mesh provides a potential solution in efficient microservices management. However, as a traditional cloud-oriented architecture, it can not be applied into edge computing directly since the centralized controller policy. To address this problem, in this paper, we propose an edge service mesh architecture with distributively deployed controllers for edge native computing. We further inves-tigate the problem on how to deploy these distributive controllers in a cost efficient manner with the consideration of control cost and the synchronization cost. The problem is formulated into a non-linear optimization form and then linearized into an integer linear programming (ILP) problem. To tackle the computation complexity, we then come up with a customized k-means based algorithm (i.e., ck-means) in polynomial computation complexity. The experimental results verify the efficiency of our ck-means algorithm in comparison with the traditional k-means algorithm.

Serverless edge computing is booming as an efficient carrier of deploying complex applications composed of dependent functions, whose assignment decisions highly influence the application performance. Although similar problem has been widely studied, none of existing approaches considers the diversity of communication styles, which is specially introduced in serverless computing and also imposes high influence to the performance efficiency. We compare two communication styles, called direct-passing and remote-storage, to transmit intermediate data between functions. We find that there is no single communication style that can prevail under all scenarios and the optimal selection depends on several factors, such as fanout degree, data size, and network bandwidth. Hence, how to select the appropriate communication style for each inter-function communication link, together with the function assignment decision, is essential to the application performance. To this end, we propose a Priority-based ASsignment and Selection (PASS) algorithm with joint consideration of function assignment and communication style selection. We theoretically analyze the approximation ratio of PASS algorithm and extensive experiments on real-world applications show that PASS can averagely reduce the completion time by 24.1% in comparison with state-of-the-art approaches.

Edge intelligence has emerged as a prevalent enabling technology to support various intelligent applications. Along with the prosperity, it also raises great concern on the security and privacy since the edge servers are usually shared and untrusted. The security-sensitive code (i.e., the pre-trained model) and data may be easily stolen by malicious tenants, and even untrusted infrastructure providers. To this end, Software Guard Extensions (SGX) is proposed to provide an isolated Trust Execution Environment (TEE) for security and privacy guarantee. However, we find that running tasks in SGX suffer certain performance degradation due to the limited Enclave Page Cache (EPC) size. This further leads to frequent page swapping operations and the high enclave call overhead, which are also influenced by the task (i.e., DNN layer) dispatching and scheduling. To this end, in this paper, we designLasagna, as an SGX based secure DNN inference acceleration framework, which explores the layered-structure of DNN models to well balance the usage of the scarce EPC resources and the computation resources. Lasagna mainly consists of a global task balancer and a local task scheduler, responding for task dispatching across distributed edge servers and task scheduling in local server, respectively. We evaluate Lasagna over different well-known DNN models, and the results show that Lasagna effectively speeds up the inference performance by$1.11\times -1.51\times$.

Edge intelligence has already been widely regarded as a key enabling technology in a variety of domains. Along with the prosperity, increasing concern is raised on the security and privacy of intelligent applications. As these applications are usually deployed on shared and untrusted edge servers, malicious co-located attackers, or even untrustworthy infrastructure providers, may acquire highly security-sensitive data and code (i.e., the pre-trained model). Software Guard Extensions (SGX) provides an isolated Trust Execution Environment (TEE) for task security guarantee. However, we notice that DNN inference performance in SGX is severely affected by the limited enclave memory space due to the resultant frequent page swapping operations and the high enclave call overhead. To tackle this problem, we propose Lasagna, an SGX oriented DNN inference performance acceleration framework without compromising the task security. Lasagna consists of a local task scheduler and a global task balancer to optimize the system performance by exploring the layered-structure of DNN models. Our experiment results show that our layer-aware Lasagna effectively speeds up the well-known DNN inference in SGX by 1.31x-1.97x.

Summary Dataflow computing has become a promising computing paradigm as an alternative to traditional control‐centric computing paradigm to facilitate big data processing. Big data process often happens in cloud computing environment as the datacenter provisions a large amount of resource. Dataflow computing, as a data‐centric computing paradigm, requires the dataflows to be shuffled among different codelets (ie, data processing units) deployed in the datacenter servers. It is significant to well schedule the dataflow transferring for communication efficiency. It is highly regarded that the datacenter network shall be managed by software defined networking (SDN) technology for flexibility consideration. In SDN managed datacenter, a dataflow requires a forwarding rule in the forwarding table of each switch on its routing path. However, the SDN switches are limited in the forwarding table size. This introduces an unignorable issue in the codelet deployment problem. Therefore, we are motivated to take such forwarding table size constraints into the problem of dataflow codelet deployment in the datacenters managed by SDN. In particular, we aim at minimizing the communication cost efficiency while guarantee the dataflow computing performance at the same time. The communication cost minimization problem is formulated into an integer linear programming form, which is relaxed to design a heuristic algorithm. The experiment results show that our relaxation algorithm can significantly improve the communication cost efficiency via ingenious codelet placement.

To tackle the computation resource poorness on the end devices, task offloading is developed to reduce the task completion time and improve the Quality-of-Service (QoS). Edge cloud facilitates such offloading by provisioning resources at the proximity of the end devices. Modern applications are usually deployed as a chain of subtasks (e.g., microservices) where a special offloading strategy, referred as binary offloading, shall be applied. Binary offloading divides the chain into two parts, which will be executed on end device and the edge cloud, respectively. The offloading point in the chain therefore is critical to the QoS in terms of task completion time. Considering the system dynamics and algorithm sensitivity, we apply Q-learning to address this problem. In order to deal with the late feedback problem, a reward rewind match strategy is proposed to customize Q-learning. Trace-driven simulation results show that our customized Q-learning based approach is able to achieve significant reduction on the total execution time, outperforming traditional offloading strategies and non-customized Q-learning.

To tackle the computation resource poorness on the end devices, task offloading is developed to reduce the task completion time and improve the Quality-of-Service (QoS). Edge computing facilitates such offloading by provisioning resources at the proximity of the end devices. Nowadays, many tasks on end devices have an urgent demand for the security of execution environment. To address this problem, we introduce trusted execution environment (TEE) to empower edge computing for secure task offloading. To explore TEE, the offloading process should be redesigned with the introduction of data encryption and decryption. This makes traditional offloading optimization policy fail to be applied directly. To address this issue, we are motivated to take the data encryption and decryption into the offloading scheduling algorithm. In particular, we propose a Customized List Scheduling based Offloading (CLSO) algorithm, aiming at minimizing the total completion time with the consideration of energy budget limitations on the end devices. The experiment results show that our approximation algorithm can effectively reduce the total completion time and significantly outperforms existing state-of-the-art offloading strategy.