Data-intensive workloads like distributed deep learning generate significant temporary data between pipeline stages, requiring efficient temporary data storage solutions. Platforms like Kubernetes abstract infrastructure complexities by using loosely-coupled containers that leverage disaggregated storage for sharing data and storing state. This design facilitates workload deployment and scaling in the Cloud-Continuum, but also becomes a bottleneck, particularly in resource-constrained edge environments. A tiered storage system for temporary data can alleviate this issue by exploiting data locality: leveraging local storage attached to the physical hosts to share data between colocated containers without the need of accessing remote storage. This article serves as a preliminary work to explore how to efficiently and securely share data between co-located containers on the same physical host for workloads deployed on Kubernetes. Our experiments demonstrate that using CSI shared local volume mounts can be used not only to share files in an efficient and secure way but also to create shared memory regions and pass file descriptors, providing different viable approaches to data sharing for layered storage systems in Kubernetes.

Event streaming systems (e.g., Apache Kafka, Apache Pulsar) are a popular substrate for ingesting data with low latency from continuous data sources, such as sensors, cameras, or server logs. Due to the sheer amount of data being stored as data streams, several systems are incorporating storage tiering as a core feature. However, in some cases, the design of the streaming system assumes a reliable connection with the external storage to offload data. This may not be the case when deploying streaming pipelines in the Cloud-Edge Continuum. In this paper, we evaluate deploying a streaming storage system with integrated data tiering (Pravega) in the CloudEdge Continuum. We identify that while Pravega provides good 10 performance, extended unavailability of the long-term storage service may impact stream data ingestion. This can be problematic in Edge use cases with stringent streaming ingestion and processing requirements. To mitigate this problem, we explore the concept of multi-tier long-term storage in Pravega. We implement this concept by integrating an ephemeral tiered storage system (GEDS) to augment Pravega with advanced data tiering mechanisms. Our preliminary results show that GEDS can exploit multiple storage tiers that increase by$3.8 x$the tolerance of the streaming system to long-term storage unavailability.

Serverless data analytics generate a large amount of intermediate data during computation stages. However, serverless functions, which are short-lived and lack direct communication, face significant challenges in managing this data effectively. The traditional approach of using object storage to carry the data proves to be slow and costly, as it involves constant movement of data back and forth. Although specialized ephemeral storage solutions have been developed to address this issue, they fail to tackle the fundamental challenge of minimizing data movements. This work focuses on incorporating near-data computation into an ephemeral storage system to reduce the volume of transferred data in serverless analytics. We present Glider with the aim to enhance communication between serverless compute stages, allowing data to smoothly "glide" through the processing pipeline instead of bouncing between different services. Glider achieves this by leveraging stateful near-data execution of complex data-bound operations and an efficient I/O streaming interface. Under evaluation, it reduces data transfers by up to 99.7%, improves storage utilization by up to 99.8%, and enhances performance by up to 2.7×. In sum, Glider improves serverless data analytics by optimizing data movement, streamlining processing, and avoiding redundant transfers.

Recent high-performance storage devices have exposed software inefficiencies in existing storage stacks, leading to a new breed of I/O stacks. The newest storage API of the Linux kernel is io_uring. We perform one of the first in-depth studies of io_uring, and compare its performance and dis-/advantages with the established libaio and SPDK APIs. Our key findings reveal that (i) polling design significantly impacts performance; (ii) with enough CPU cores io_uring can deliver performance close to that of SPDK; and (iii) performance scalability over multiple CPU cores and devices requires careful consideration and necessitates a hybrid approach. Last, we provide design guidelines for developers of storage intensive applications.

During the past decade, network and storage devices have undergone rapid performance improvements, delivering ultra-low latency and several Gbps of bandwidth. Nevertheless, current network and storage stacks fail to deliver this hardware performance to the applications, often due to the loss of I/O efficiency from stalled CPU performance. While many efforts attempt to address this issue solely on either the network or the storage stack, achieving high-performance for networked-storage applications requires a holistic approach that considers both. In this article, we present FlashNet, a software I/O stack that unifies high-performance network properties with flash storage access and management. FlashNet builds on RDMA principles and abstractions to provide a direct, asynchronous, end-to-end data path between a client and remote flash storage. The key insight behind FlashNet is to co-design the stack’s components (an RDMA controller, a flash controller, and a file system) to enable cross-stack optimizations and maximize I/O efficiency. In micro-benchmarks, FlashNet improves 4kB network I/O operations per second (IOPS by 38.6% to 1.22M, decreases access latency by 43.5% to 50.4μs, and prolongs the flash lifetime by 1.6-5.9× for writes. We illustrate the capabilities of FlashNet by building a Key-Value store and porting a distributed data store that uses RDMA on it. The use of FlashNet’s RDMA API improves the performance of KV store by 2× and requires minimum changes for the ported data store to access remote flash devices.

During the past decade, network and storage devices have undergone rapid performance improvements, delivering ultra-low latency and several Gbps of bandwidth. Nevertheless, current network and storage stacks fail to deliver this hardware performance to the applications, often due to the loss of IO efficiency from stalled CPU performance. While many efforts attempt to address this issue solely on either the network or the storage stack, achieving high-performance for networked-storage applications requires a holistic approach that considers both.

Scientific workflows are often composed of compute-intensive simulations and data-intensive analysis and visualization, both equally important for productivity. High-performance computers run the compute-intensive phases efficiently, but data-intensive processing is still getting less attention. Dense non-volatile memory integrated into super-computers can help address this problem. In addition to density, it offers significantly finer-grained I/O than disk-based I/O systems. We present a way to exploit the fundamental capabilities of Storage-Class Memories (SCM), such as Flash, by using scalable key-value (KV) I/O methods instead of traditional file I/O calls commonly used in HPC systems. Our objective is to enable higher performance for on-line and near-line storage for analysis and visualization of very high resolution, but correspondingly transient, simulation results. In this paper, we describe 1) the adaptation of a scalable key-value store to a BlueGene/Q system with integrated Flash memory, 2) a novel key-value aggregation module which implements coalesced, function-shipped calls between the clients and the servers, and 3) the refactoring of a scientific workflow to use application-relevant keys for fine-grained data subsets. The resulting implementation is analogous to function-shipping of POSIX I/O calls but shows an order of magnitude increase in read and a factor 2.5x increase in write IOPS performance (11 million read IOPS, 2.5 million write IOPS from 4096 compute nodes) when compared to a classical file system on the same system. It represents an innovative approach for the integration of SCM within an HPC system at scale.

Distributed DRAM stores have become an attractive option for providing fast data accesses to analytics applications. To accelerate the performance of these stores, researchers have proposed using RDMA technology. RDMA offers high bandwidth and low latency data access by carefully separating resource setup from IO operations, and making IO operations fast by using rich network semantics and offloading. Despite recent interest, leveraging the full potential of RDMA in a distributed environment remains a challenging task. In this paper, we present RDMA Store or RStore, a DRAM-based data store that delivers high performance by extending RDMA's separation philosophy to a distributed setting. RStore achieves high aggregate bandwidth (705 Gb/s) and close-to-hardware latency on our 12-machine testbed. We developed a distributed graph processing framework and a Key-Value sorter using RStore's unique memory-like API. The graph processing framework, which relies on RStore for low-latency graph access, outperforms state-of-the-art systems by margins of 2.6 -- 4.2× when calculating Page Rank. The Key-Value sorter can sort 256 GB of data in 31.7 sec, which is 8× better than Hadoop TeraSort in a similar setting.

DMA-capable interconnects, providing ultra-low latency and high bandwidth, are increasingly being used in the context of distributed storage and data processing systems. However, the deployment of such systems in virtualized data centers is currently inhibited by the lack of a flexible and high-performance virtualization solution for RDMA network interfaces.

Remote Procedure Call (RPC) has been the cornerstone of distributed systems since the early 80s. Recently, new classes of large-scale distributed systems running in data centers are posing extra challenges for RPC systems in terms of scaling and latency. We find that existing RPC systems make very poor usage of resources (CPU, memory, network) and are not ready to handle these upcoming workloads.

Network latency has become increasingly important for data center applications. Accordingly, several efforts at both hardware and software level have been made to reduce the latency in data centers. Limited attention, however, has been paid to network latencies of distributed systems running inside an application container such as the Java Virtual Machine (JVM) or the .NET runtime.

The performance of large-scale data-intensive applications running on thousands of machines depends considerably on the performance of the network. To deliver better application performance on rapidly evolving high-bandwidth, low-latency interconnects, researchers have proposed the use of network accelerator devices. However, despite the initial enthusiasm, translating network accelerator's capabilities into high application performance remains a challenging issue.

Internet usage has changed dramatically in the past few years. Content is no longer dominated by static websites, but comprises an increasing number of multimedia streams. With the widespread availability of broadband connections, the quality of the media provided by video-on-demand as well as streaming services increases constantly. Even though today most videos are still encoded with a rather low bit rate, large Internet service providers already foresee high-definition media becoming the predominant format in the near future. However, a larger number of clients requesting media at high bit rates poses a challenge for the server infrastructure. Conventional stream dissemination methods, such as RTP over UDP or HTTP over TCP, result in high server loads due to excessive local data copy, context switching, and interrupt processing overhead. In this paper, we illustrate and discuss this problem in detail through extensive experiments with existing solutions. We then present a new approach based on zero-copy protocol stack implementations in software as well as dedicated RDMA hardware. Our performance experiments indicate that these optimizations allow servers to scale better and remove most of the overhead caused by current approaches.

AMnet is a framework for flexible and rapid service creation and provides primarily heterogeneous user-tailored group communication services, AMnet is based on Active and Programmable Networking technologies and uses active nodes (AMnodes) located within the network to execute so-called service modules for the provision of individual group communication services. It is, however not limited to group communication since it provides a rather generic framework. The AMnode design is shaped for the efficient execution of flexible loadable service modules, including dedicated hardware support.