The EU-funded EMPYREAN project (empyrean-horizon.eu) aims to establish a hyper-distributed computing paradigm, leveraging collaborative, heterogeneous IoT devices and federated resources. EMPYREAN focuses on developing technologies for efficient AI workload processing, secure distributed edge storage and cloud-native application development. It will offer open and standardised APIs and use open-source platforms. EMPYREAN's capabilities will be demonstrated through three use cases: advanced manufacturing, smart agriculture, and warehouse automation.

The need for real-time analytics and faster decision-making mechanisms has led to the adoption of hardware accelerators such as GPUs and FPGAs within the edge cloud computing continuum. However, their programmability and lack of orchestration mechanisms for seamless deployment make them difficult to use efficiently. We address these challenges by presenting SERRANO, a project for transparent application deployment in a secure, accelerated, and cognitive cloud continuum. In this work, we introduce the SERRANO platform and its software, orchestration, and deployment services, focusing on its methods for automated GPU/FPGA acceleration and efficient, isolated, and secure deployments. By evaluating these services against representative use cases, we highlight SERRANO 's ability to simplify the development and deployment process without sacrificing performance.

We introduce Titchy, which is a compression method for time-series data generated by the Internet of Things. Our proposed method is flexible and has several advantages when applied in the IoT ecosystem: 1) it is able to compress even when only a small amount of memory can be allocated to it; 2) it compresses data in tiny chunks, so it introduces very little latency during online operation and thus, enables frequent and timely updates with compressed data; and 3) it facilitates efficient data storage and retrieval by enabling low-cost random access to the compressed data, eliminating the need to decompress large chunks when only a small amount of data is requested. To evaluate each of these advantages, we have implemented the compressor and conducted extensive experiments with long-term real-world data captured over days or weeks. We also present results for seven state-of-the-art compression methods to act as a baseline. Our evaluation shows that Titchy not only outperforms all seven on random access capability and for frequent transmissions but also provides great compression ratios as well as high compression and decompression speeds.

Although cloud systems provide a reliable and flexible storage solution, the use of a single cloud service constitutes a single point of failure, which can compromise data availability, download speed, and security. To address these challenges, we advocate for the use of multiple cloud storage providers simultaneously using network coding as the key enabling technology. Our goal is to study two challenges of network coded storage systems. First, the efficient update of the number of coded fragments per cloud in a system aggregating multiple clouds in order to boost the download speed of files. We developed a novel scheme using recoding with limited packets to trade-off storage space, reliability, and data retrieval speed. Implementation and measurements with commercial cloud providers show that up to 9x less network use is needed compared to other network coding schemes, while maintaining similar download speeds and reliability. Second, the ability to update coded fragments from a linear erasure code when the original file is modified. We exploit code structure to provide efficient representations of the evolution of the file. Evaluations using file changes on software library repositories show that a five-order of magnitude reduction in network and storage use is possible compared to state-of-the-art.

Video on Demand (VoD) streaming makes up an increasing portion of global IP traffic. Traditionally, data is served by content providers directly, putting a huge network load on the servers. We propose using our WebRTC-based protocols WebPeer and CodedWebPeer to create a P2P-assisted VoD streaming system with network coding. We introduce two metrics: network health to measure overall data saturation and network stability to show if the peers in the network are able to serve each other without the help of the server. We have implemented a testbed to emulate real-life network scenarios that includes a probabilistic model of peer behavior. We show through measurements that by applying network coding, network health is increased by up to 100% without changing the cache size or number of peers. Furthermore, network stability is achieved using up to half the cache compared to the uncoded approach, without increasing the servers load. Finally, to validate the scalability of our solution, we tested our protocols in a network with more than 100 tablets to deliver smooth video playback.

As faster storage devices become commercially viable alternatives to disk drives, the network is increasingly becoming the bottleneck in achieving good performance in distributed storage systems. This is especially true for erasure coded storage, where the reconstruction of lost data can significantly encumber the system. Thus, a significant amount of research has focused on reducing the amount of data transferred during this repair process. However, in most cases the network is assumed to have a uniform static structure. One reason behind this is that many of the state of the art codes have a fixed repair mechanism or are constrained in the choice of repair strategies, therefore in theory benefit less from being network aware. We propose a general mechanism that explores the space of possible repairs and examine how much different types of erasure codes benefit by being network aware. We show significant gains for three erasure codes using both theoretical modeling and simulation results. We also consider the practical applicability of our proposed mechanism by limiting the search space to repairs that have the potential to be minimal cost and present a case study for RLNC, a class of flexible codes.

Conventional cloud storage services offer relatively good reliability and performance in a cost-effective manner. However, they are typically structured in a centralized and highly controlled fashion. In more dynamic storage scenarios, these centralized approaches are unfeasible and developing decentralized storage approaches becomes critical. The novelty of this paper is the introduction of the highly dynamic distributed mobile cloud, which uses free resources on user devices to move storage to the edges of the network. At the core of our approach, lies the use of random linear network coding to provide an effective and flexible erasure correcting code. This paper identifies and answers key questions regarding the feasibility of such a system. We show that the mobile cloud has sufficient network resources to adapt to changes in node numbers and also study the redundancy level needed to maintain data availability. We have found that as little as 75% redundancy is enough to offer 99.28% availability for the examined period and essentially 100% availability is achieved when using 50% redundancy along with high-availability nodes. We have leveraged traces from a popular P2P mobile application to simulate the processes governing user behavior to show feasibility of mobile storage clouds in real scenarios.

Distributed storage solutions have become widespread due to their ability to store large amounts of data reliably across a network of unreliable nodes, by employing repair mechanisms to prevent data loss. Conventional systems rely on static designs with a central control entity to oversee and control the repair process. Given the large costs for maintaining and cooling large data centers, our work proposes and studies the feasibility of a fully decentralized systems that can store data even on unreliable and, sometimes, unavailable mobile devices. This imposes new challenges on the design as the number of available nodes varies greatly over time and keeping track of the system's state becomes unfeasible. As a consequence, conventional erasure correction approaches are ill-suited for maintaining data integrity. In this highly dynamic context, random linear network coding (RLNC) provides an interesting solution. Our goal is to characterize RLNC's guaranteed data integrity region in terms of the total number of storage devices that need to be available and stored data per device. We compare our fully distributed RLNC approach to centralized (genie aided) and fully decentralized replication and Reed-Solomon mechanisms. Our results use traces from a BitTorrent client for Android devices to show that RLNC outperforms the next best scheme (fully centralized Reed-Solomon) not only by having a much lower probability of data loss, but by reducing storage requirements by up to 50% and reconstruction traffic by up to 40%. Gains over decentralized schemes are even larger.