The EU-funded EMPYREAN project (empyrean-horizon.eu) aims to establish a hyper-distributed computing paradigm, leveraging collaborative, heterogeneous IoT devices and federated resources. EMPYREAN focuses on developing technologies for efficient AI workload processing, secure distributed edge storage and cloud-native application development. It will offer open and standardised APIs and use open-source platforms. EMPYREAN's capabilities will be demonstrated through three use cases: advanced manufacturing, smart agriculture, and warehouse automation.

We present an end-to-end solution to facilitate the seamless execution of hardware-accelerated compute-intensive tasks on heterogeneous hardware platforms spanning the Cloud-Edge continuum. Our approach includes a programming interface, orchestration, application management components, the vAccel framework, and a library of hardware-accelerated kernels. These components enable a Function-as-a-Service (FaaS) based operational flow that supports numerous diverse use cases while minimizing the time required for the developer to integrate their code and for the vendor to provide hardware acceleration capabilities to end users. Experimental results showcase the merits of our approach.

Distributed storage systems spanning across different cloud data centers have substantially improved availability and flexibility for data storage and retrieval operations. However, stringent latency requirements of emerging applications necessitate optimized selection of storage resources that exhibit smaller delay. Introducing edge resources into distributed storage systems enables data placement closer to its source, but simultaneously increases the complexity of decision-making and orchestration processes for optimal data placement. In this work, we develop mechanisms for storing data across an infrastructure that includes both edge and cloud resources. Our approach focuses on optimizing data integrity, longevity, security, and cost, while leveraging erasure coding when performing the resource allocation. We first present a comprehensive mixed integer linear programming formulation of the storage resource orchestration problem. As the search space for the optimal solution can be vast and the execution time prohibitively large for real size problems, we also propose an innovative multi-agent heuristic approach that uses the rollout, a reinforcement based policy, to balance performance and execution time efficiently. Through various simulation experiments, we evaluate the developed mechanisms and trade-offs involved in our approach. By incorporating data from a multi-cloud provider, we further enhance the validity of the simulations and the conclusions drawn.

The uncertain and non-dispatchable nature of renewable energy sources renders Demand Response (DR) a critical component of modern electricity distribution systems. Demand Response (DR) service provision takes place via aggregators and special distribution-level markets (e.g., flexibility markets), where small, distributed DR resources, such as building energy management systems, electric vehicle charging stations, micro-generation and storage, connected to the low-voltage distribution grid, offer DR services. In such systems, energy balancing (and thus, also DR decisions) have to be made close to real-time. Thus, market clearing algorithms for DR service provision must fulfill several requirements related to the efficiency of their operation. More specifically, a DR market clearing algorithm needs to be optimal in terms of cost-efficiency, scalable in terms of number of assets and locations, and able to satisfy real-time constraints. In order to cope with these challenges, this article presents a distributed DR market clearing algorithm based on Lagrangian decomposition, combined with an optimal cloud resource allocation algorithm for assigning the required computation power. A heuristic algorithm is also presented, able to achieve a near-optimal solution, within negligible computational time. Simulations, performed on a testbed, demonstrate the computational burden introduced by various DR models, as well as the heuristic algorithm's near-optimal performance. The resource allocation algorithm is able to service multiple DR requests (e.g., in multiple distribution networks), and minimize the cost of computational resources while respecting the execution time constraints of each request. This enables third parties to offer cost-efficient and competitive DR operation as a service.

The design and operation of any network are complex processes that require the evaluation, utilization and configuration of a variety of usually expensive network devices. Through the use of an emulation platform, network operators are able to examine different scenarios and network parameters and benefit from multi-objective decision mechanisms. These enable the decrease of the network design phase duration and the optimal operation of the network under different well examined conditions. In this work, we present an emulation platform for SDN-enabled 5G integrated Fiber-Wireless networks that provides a transparent view of the 5G infrastructure to any SDN-based control plane. We present the overall architecture and design of the emulator, along with the implementation details of its main components. Network devices are described through YANG models and are emulated using containerized processes, configured and managed through the Network Configuration (NETCONF) protocol. Finally, a number of emulation scenarios are described and evaluated, utilizing a joint fiber and wireless resource allocation algorithm that drives the SDN-enabled devices.

The shift of the radio processing to the cloud, through cloud Radio Access Networks (C-RAN) technologies and of the cloud processing to the edge, through edge computing, form the environment in which 5G systems are being implemented, fostered and transformed from a future technology to a mainstream one. By its nature, global optimization of the edge resource deployment cannot by easily performed considering the number and the diversity of the players that will be involved in the edge computing arena. As a result, building and maintaining more and more edge-located resources for serving radio and application data will eventually lead to increased cost and energy consumption and resource underutilization. One way to overcome this predicament, is through virtual resource consolidation, where separate but efficiently interconnected edge resources appear as a single computing entity, serving radio and application data. In this context, we present the Virtual Elastic Datacenters (VEDC) in the edge notion for 5G networks that can alleviate these issues. We also describe an Integer Linear Programming (ILP) based mechanism for the placement of baseband and application processing loads in a VEDC-based environment, and perform respective experiments. We show that through VEDC resource consolidation better quality services can be provided, while improving resource efficiency.

We present a methodology, called communication-aware virtual infrastructures (COMAVI), for the concurrent migration of multiple Virtual Machines (VMs) in cloud computing infrastructures, which aims at the optimum use of the available computational and network resources, by capturing the interdependencies between the communicating VMs. This methodology uses multiple criteria for selecting the VMs that will migrate, with different weights assigned to each of them. COMAVI also selects the computing sites/units where the migrating VMs will be hosted, by accounting for the way migration affects the logical (or virtual) topologies formed by the communicating VMs and viewing this selection as a logical topology reconfiguration problem. COMAVI resolves the maximum possible number of VM resource shortages, while tending to minimize the number of migrations performed, the induced network overhead, the logical topology reconfigurations required, and the corresponding service interruptions. We evaluate the proposed method through simulations, where we exhibit their performance benefits.

The monitoring and the analysis of public clouds gains momentum, due to their widespread exploitation by individual users, researchers and companies for their daily tasks. We propose an algorithm for optimizing the cost and the utilization of a set of running Amazon EC2 instances by resizing them appropriately. The algorithm, namely Cost and Utilization Optimization (CUO) algorithm, receives information regarding the current set of instances used (their number, type, utilization) and proposes a new set of instances for serving the same load, so as to minimize cost and maximize utilization, or increase performance efficiency. CUO is integrated in Smart cloud Monitoring (SuMo), an open-source tool we develop for collecting monitoring data from Amazon Web Services (AWS) and analyzing them. A number of experiments are performed, using input data that correspond to realist AWS configuration scenarios, which exhibit the benefits of the CUO algorithm.

SUMMARY Grid scheduling algorithms are usually implemented in a simulation environment using tools that hide the complexity of the Grid and assumptions that are not always realistic. In our work, we describe the steps followed, the difficulties encountered and the solutions provided to develop and evaluate a scheduling policy, initially implemented in a simulation environment, in the gLite Grid middleware. Our focus is on a scheduling algorithm that allocates in a fair way the available resources among the requested users or jobs. During the actual implementation of this algorithm in gLite, we observed that the validity of the information used by the scheduler for its decisions affects greatly its performance. To improve the accuracy of this information, we developed an internal feedback mechanism that operates along with the scheduling algorithm. Also, a Grid computation resource cannot be shared concurrently between different users or jobs, making it difficult to provide actual fairness. For this reason we investigated the use of virtualization technology in the gLite middleware. We did a proof‐of‐concept implementation and performed an experimental evaluation of our scheduling algorithm in a small gLite testbed that proves the validity and applicability of our solutions. Copyright © 2012 John Wiley & Sons, Ltd.

We describe our experiences from implementing and integrating a new job scheduling algorithm in the gLite Grid middleware and present experimental results that compare it to the existing gLite scheduling algorithms. It is the first time that gLite scheduling algorithms are put under test and compared with a new algorithm under the same conditions. We describe the problems that were encountered and solved, going from theory and simulations to practice and the actual implementation of our scheduling algorithm. In this work we also describe the steps one needs to follow in order to develop and test a new scheduling algorithm in gLite. We present the methodology followed and the testbed that was set up for the comparisons. Our research sheds light on some of the problems of the existing gLite scheduling algorithms and makes clear the need for the development of new.

In this work we examine a task scheduling and data migration problem for grid networks, which we refer to as the data consolidation (DC) problem. DC arises when a task needs for its execution two or more pieces of data, possibly scattered throughout the grid network. In such a case, the scheduler and the data manager must select the data replicas to be used and the site where these will accumulate for the task to be executed. The policies for selecting the data replicas and the data consolidating site comprise the data consolidation problem. We propose and experimentally evaluate a number of DC techniques. Our simulation results brace our belief that DC is an important technique for data grids since it can substantially improve task delay, network load and other performance related parameters.