CAPE is a European-funded project targeting to reshape edge-cloud computing by defining edge micro data centers as a new unit of computing. Fully committing to open source, CAPE develops a fully Composable Infrastructure (CI) for high-performance edge server hardware platforms grounded in open, forward-looking standards. Together with an open-source software stack covering the Edge-Cloud Continuum, this holistic approach boosts power and energy efficiency while reducing resource overprovisioning. Completely based on open standards, CAPE strengthens the digital sovereignty Europe needs in a challenging future. This work gives an overview of the current architectural blueprint of the project, focusing on integrating game-changing technologies like Compute Express Link (CXL) for compute and memory disaggregation, pushing open source cluster management, and AI-assisted deployment software stacks using Infrastructure from Code (IfC). The proposed approaches and benefits for future Edge-Cloud data centers are demonstrated within three use cases, ranging from Smart Grid and Edge-AI to Satellite Data Processing.

The EU-funded EMPYREAN project (empyrean-horizon.eu) aims to establish a hyper-distributed computing paradigm, leveraging collaborative, heterogeneous IoT devices and federated resources. EMPYREAN focuses on developing technologies for efficient AI workload processing, secure distributed edge storage and cloud-native application development. It will offer open and standardised APIs and use open-source platforms. EMPYREAN's capabilities will be demonstrated through three use cases: advanced manufacturing, smart agriculture, and warehouse automation.

The cloud is extended towards the edge to form a computing continuum while managing resources' heterogeneity. The serverless technology simplified how to build cloud applications and use resources, becoming a driving force in consolidating the continuum with the deployment of small functions with short execution. However, the adaptation of serverless to the edge-cloud continuum brings new challenges mainly related to resource management and scheduling. Standard cloud scheduling policies are based on greedy algorithms that do not efficiently handle platforms' heterogeneity nor deal with problems such as cold start delays. This work introduces a new scheduling policy that tries to address these issues. It is based on multi-objective optimization for data transfers and makespan while considering heterogeneity. Using simulations that vary workloads, platforms, and heterogeneity levels, we study the system utilization, the trade-offs between the targets, and the impacts of considering platforms' heterogeneity. We perform comparisons with a baseline inspired by a Kubernetes-based policy, representing greedy algorithms. Our experiments show considerable gaps between the efficiency of a greedy-based scheduling policy and a multi-objective-based one. The last outperforms the baseline by reducing makespan, data transfers, and system utilization by up to two orders of magnitudes in relevant cases for the edge-cloud continuum.

Containerisation demonstrates its efficiency in application deployment in cloud computing. Containers can encapsulate complex programs with their dependencies in isolated environments, hence are being adopted in HPC clusters. HPC workload managers lack micro-services support and deeply integrated container management, as opposed to container orchestrators (e.g. Kubernetes). We introduce Torque-Operator (a plugin) which serves as a bridge between HPC workload managers and container Orchestrators.

Localized air pollution need to be tackle in dense urban area. This paper presents a mobile sensor platform aiming at high quality air particle matter measurements in a metropolis. Its installation on a bus service enables a cost-effective sensor deployment and the integration of computing power at the edge.

According to McKinsey & Company, about a third of food produced is lost or wasted every year, amounting to a $940 billion economic hit. Inefficiencies in planting, harvesting, water use, reduced animal contributions, as well as uncertainty about weather, pests, consumer demand and other intangibles contribute to the loss. Precision Agriculture (PA) and Precision Livestock Farming (PLF) come to assist in optimizing agricultural and livestock production and minimizing the wastes and costs aforementioned. PA is a technology-enabled, data-driven approach to farming management that observes, measures, and analyzes the needs of individual fields and crops. PLF is also a technology-enabled, data-driven approach to livestock production management, which exploits technology to quantitatively measure the behavior, health and performance of animals. Big data delivered by a plethora of data sources related to these domains, has a multitude of payoffs including precision monitoring of fertilizer and fungicide levels to optimize crop yields, risk mitigation that results from monitoring when temperature and humidity levels reach dangerous levels for crops, increasing livestock production while minimizing the environmental footprint of livestock farming, ensuring high levels of welfare and health for animals, and more. By adding analytics to these sensor and image data, opportunities also exist to further optimize PA and PLF by having continuous data on how a field or the livestock is responding to a protocol. For these domains, two main challenges exist: 1) to exploit this multitude of data facilitating dedicated improvements in performance, and 2) to make available advanced infrastructure so as to harness the power of this information in order to benefit from the new insights, practices and products, efficiently time-wise, lowering responsiveness down to seconds so as to cater for time-critical decisions. The current paper aims to introduce CYBELE, a platform aspiring to safeguard that the stakeholders involved in the agri-food value chain (research community, SMEs, entrepreneurs, etc.) have integrated, unmediated access to a vast amount of very large scale datasets of diverse types and coming from a variety of sources, and that they are capable of actually generating value and extracting insights out of these data, by providing secure and unmediated access to large-scale High Performance Computing (HPC) infrastructures supporting advanced data discovery, processing, combination and visualization services, solving computationally-intensive challenges modelled as mathematical algorithms requiring very high computing power and capability.

Executing Big Data workloads upon High Performance Computing (HPC) infrastractures has become an attractive way to improve their performances. However, the collocation of HPC and Big Data workloads is not an easy task, mainly because of their core concepts' differences. This paper focuses on the challenges related to the scheduling of both Big Data and HPC workloads on the same computing platform. In classic HPC workloads, the rigidity of jobs tends to create holes in the schedule: we can use those idle resources as a dynamic pool for Big Data workloads. We propose a new idea based on Resource and Job Management System's (RJMS) configuration, that makes HPC and Big Data systems to communicate through a simple prolog/epilog mechanism. It leverages the built-in resilience of Big Data frameworks, while minimizing the disturbance on HPC workloads. We present the first study of this approach, using the production RJMS middleware OAR and Hadoop YARN from the HPC and Big Data ecosystems respectively. Our new technique is evaluated with real experiments upon the Grid5000 platform. Our experiments validate our assumptions and show promising results. The system is capable of running an HPC workload with 70% cluster utilization, with a Big Data workload that fills the schedule holes to reach a full 100% utilization. We observe a penalty on the mean waiting time for HPC jobs of less than 17% and a Big Data effectiveness of more than 68% in average.

Energy consumption has become one of the mostcritical issues in the evolution of High Performance Computingsystems (HPC). Controlling the energy consumption of HPCplatforms is not only a way to control the cost but also a stepforward on the road towards exaflops. Powercapping is a widelystudied technique that guarantees that the platform will notexceed a certain power threshold instantaneously but it givesno flexibility to adapt job scheduling to a longer term energybudget control. We propose a job scheduling mechanism that extends thebackfilling algorithm to become energy-aware. Simultaneously, we adapt resource management with a node shutdown technique to minimize energy consumption whenever needed. Thiscombination enables an efficient energy consumption budgetcontrol on a cluster during a period of time. The technique isexperimented, validated and compared with various alternativesthrough extensive simulations. Experimentation results show highsystem utilization and limited bounded slowdown along withinteresting outcomes in energy efficiency while respecting anenergy budget during a particular time period.

The ever-increasing energy demands of modern High Performance Computing (HPC) platforms is undeniably one of the most critical aspects for the future design and evolution of such systems. The capability of managing their energy consumption not only allows for significant reduction in electricity costs but is also a step forward on the road towards the exascale. Powercapping is a widely studied technique that contributes to address this challenge by instantaneously setting and maintaining a predefined power threshold (power cap) that cannot be exceeded. However, the lack of a centralized mechanism responsible for efficiently allocating the available power among resources and jobs may ultimately yield to fragmentation, low system utilization and increased user waiting times. Additionally, power cap violations can lead to high risk scenarios and/or increase operational costs. This paper proposes to prevent such issues with the introduction of the Enhanced Power Adaptive Scheduling (E-PAS) algorithm. The E-PAS algorithm combines scheduling and resource management mechanisms, correlating estimated and real power consumption data in order to optimize the resource utilization of the platform under a predefined power cap. The algorithm has been implemented in the widely used open-source resource and job management system SLURM and is planned to be pushed in a future mainstream version. Its effectiveness has been evaluated through real-scale experiments respectively on an ARM- and an Intel-based cluster of comparable size. All experiments have been performed using synthetic workloads from a set of mini-applications.

Energy consumption has become one of the most important factors in High Performance Computing platforms. However, while there are various algorithmic and programming techniques to save energy, a user has currently no incentive to employ them, as they might result in worse performance. We propose to manage the energy budget of a supercomputer through EnergyFairShare (EFS), a FairShare-like scheduling algorithm. FairShare is a classic scheduling rule that prioritizes jobs belonging to users who were assigned small amount of CPU-second in the past. Similarly, EFS keeps track of users 'consumption of Watt-seconds and prioritizes those whom jobs consumed less energy. Therefore, EFS incentives users to optimize their code for energy efficiency. Having higher priority, jobs have smaller queuing times and, thus, smaller turn-around time. To validate this principle, we implemented EFS in a scheduling simulator and processed workloads from various HPC centers. The results show that, by reducing it energy consumption, auser will reduce it stretch (slowdown), compared to increasing it energy consumption. To validate the general feasibility odour approach, we also implemented EFS as an extension forSLURM, a popular HPC resource and job management system.We validated our plugin both by emulating a large scale platform, and by experiments upon a real cluster with monitored energy consumption. We observed smaller waiting times for energy efficient users.

The question of energy savings has been a matter of concern since a long time in the mobile distributed systems and battery-constrained systems. However, for large-scale non-mobile distributed systems, which nowadays reach impressive sizes, the energy dimension (electrical consumption) just starts to be taken into account. In this paper, we present the GREEN-NET1framework which is based on 3 main components: an ON/OFF model based on an Energy Aware Resource Infrastructure (EARI), an adapted Resource Management System (OAR) for energy efficiency and a trust delegation component to assume network presence of sleeping nodes.

A widely used method for large scale experiment execution upon P2P or cluster computing platforms, is the exploitation of idle resources. Specifically in the case of clusters, administrators share their cluster's idle cycles into computational grids, for the execution of the so called bag-of-tasks applications. Fault-tolerance and scheduling are some of the important challenges that have arisen on the specific research field. On this paper, we present a simple, scalable, fault tolerant and user-transparent approach of harnessing idle cluster resources for executing grid bag-of-tasks applications. Our main interest lies on the large-scale deployment and evaluation of our lightweight grid computing approach, under real-life parameters. Under this context, we experiment with CIGRI and a fully transparent system-level checkpointing feature for scheduling and turnaround-time optimisation. We discuss the value of experimentation on computer science, we propose reproducible experiments based on real workload traces and we explain how our experimental methodology contributes on the development and evaluation of our grid platform.

Focused around the field of the exploitation and the administration of high performance large-scale parallel systems, this article describes the work carried out on the deployment of environment on high computing clusters and grids. We initially present the problems involved in the installation of an environment (OS, middleware, libraries, applications...) on a cluster or grid and how an effective deployment tool, Kadeploy2, can become a new form of exploitation of this type of infrastructures. We present the tool's design choices, its architecture and we describe the various stages of the deployment method, introduced by Kadeploy2. Moreover, we propose methods on the one hand, for the improvement of the deployment time of a new environment; and in addition, for the support of various operating systems. Finally, to validate our approach we present tests and evaluations realized on various clusters of the experimental grid Grid5000.

In this article we present the design choices and the evaluation of a batch scheduler for large clusters, named OAR. This batch scheduler is based upon an original design that emphasizes on low software complexity by using high level tools. The global architecture is built upon the scripting language Perl and the relational database engine Mysql. The goal of the project OAR is to prove that it is possible today to build a complex system for resource management using such tools without sacrificing efficiency and scalability. Currently, our system offers most of the important features implemented by other batch schedulers such as priority scheduling (by queues), reservations, backfilling and some global computing support. Despite the use of high level tools, our experiments show that our system has performances close to other systems. Furthermore, OAR is currently exploited for the management of 700 nodes (a metropolitan grid) and has shown good efficiency and robustness.