Abstract As high performance computing (HPC) moves towards exascale, storage systems face core challenges such as data flooding, bandwidth bottlenecks, mixed load coordination, and performance cost balancing. This article systematically reviews the cutting-edge technologies of high performance storage systems, covering four aspects: storage architecture, hardware, software, and networking. At the architecture level, storage computing separation, distributed and hierarchical architectures decouple computing and storage resources, and optimize latency and scalability through high-speed networks. Typical cases include supercomputer systems such as Frontier and Fugaku. In terms of hardware, persistent memory, all flash array, and integrated storage and computing chips significantly improve throughput and reduce latency, while ZNS SSD and QLC technology optimize cost and lifespan. At the software level, distributed parallel file systems respond to massive small files and high concurrency access through burst buffering technology. In network communication, low latency protocols such as Slingshot, InfiniBand, and RoCE support TB level bandwidth, while CXL technology promotes storage resource pooling. In the future, photon interconnection, AI native architecture, and green energy-saving technologies will further promote the development of high performance storage towards efficiency and intelligence, to support ZB level storage requirements in scenarios such as Exascale computing and AI training.

Modern data-intensive applications increasingly demand efficient and scalable file systems that can operate across distributed and cross-domain environments. However, existing file systems are inefficient in metadata management, synchronization efficiency, and system scalability under high-concurrency and metadata-intensive workloads in cross-domain environments. To address these challenges, this article introduces CrossFS (CFS), a cross-domain distributed file system that enhances consistency guarantees and metadata indexing. Specifically, CFS leverages conflict-free replicated data types (CRDTs) to synchronize metadata, achieving strong eventual consistency with minimal synchronization overhead, even across network partitions. Furthermore, CFS employs a Hybrid Tree indexing structure, tailored for distributed environments, which optimizes metadata operations by reducing query latency by up to 33.4% and write amplification by 30.7%. Additionally, CFS achieves adaptive caching strategies and a hybrid synchronization model that effectively balances consistency latency with data availability. Extensive evaluations show that CFS outperforms CephFS and GlusterFS, achieving up to 33.9% higher metadata throughput, 36% lower latency, and 42% better data operation efficiency.

Virtual machines (VMs) impose increasing memory demands, exposing the capacity and cost limitations of traditional DRAM only memory architectures. To address this problem, heterogeneous DRAM+CXL tiered memory management systems have emerged as a promising solution. However, in virtualization environments, the semantic gap between guest and host abstraction layers, coupled with dynamic workload behaviors, hinders precise page tracking, classification, and efficient page migration across memory tiers.

It is increasingly important for the next-generation exascale supercomputers to extend its applications beyond traditional high-performance computing (HPC) scenarios, so as to achieve high social and economic benefit. Similar to Amazon Web Services (AWS) and Alibaba Cloud, cloud-style virtual HPC service is a promising application scenario on supercomputers, for which remote block storage is the key to provide tenants with supercomputers’ extremely high storage performance. Unfortunately, the state-of-the-art block storage software systems (such as URSA and Ceph) cannot adapt to the advanced hardware features of supercomputers. This article presents UrsaX, an efficient block storage service for our next-generation Tianhe exascale supercomputer that is equipped with the high-performance global express (GLEX) network and nonvolatile memory express (NVMe) SSDs. UrsaX’s virtual disks, which can be mounted like normal physical ones, enable not only traditional HPC applications but also supercomputer-oblivious POSIX applications to enjoy the high performance of supercomputers. At the core of UrsaX is with a novel design of the efficient integration of on-disk block I/O and in-network message transfer on supercomputers. UrsaX utilizes the NVMe Fabrics kernel module to expand the NVMe standard on the supercomputer network, and separates metadata I/O and data I/O of blocks, respectively, being handled over the mini packet (MP) and remote direct memory access (RDMA) protocols. We thoroughly explore the design space for remote block storage on supercomputers, including parallelism, scalability, fault tolerance, and consistency. We conduct an extensive evaluation on a subset of our exascale supercomputer consisting of 44 storage machines (each with four NVMe SSDs). The result shows that UrsaX achieves local-storage-level I/O latency (tens of microseconds) while being able to linearly increase the aggregate performance (IOPS and throughput) as the system scale increases, an order of magnitude higher than the state-of-the-art block storage systems.

Object-based storage systems have been widely used for various scenarios such as file storage, block storage, blob (e.g., large videos) storage, and so on, where the data is placed among a large number of object storage devices (OSDs). Data placement is critical for the scalability of decentralized object-based storage systems. The state-of-the-art CRUSH placement method is a decentralized algorithm that deterministically places object replicas onto storage devices without relying on a central directory. While enjoying the benefits of decentralization such as high scalability, robustness, and performance, CRUSH-based storage systems suffer from uncontrolled data migration when expanding the capacity of the storage clusters (i.e., adding new OSDs), which is determined by the nature of CRUSH and will cause significant performance degradation when the expansion is nontrivial. This article presents MapX , a novel extension to CRUSH that uses an extra time-dimension mapping (from object creation times to cluster expansion times) for controlling data migration after cluster expansions. Each expansion is viewed as a new layer of the CRUSH map represented by a virtual node beneath the CRUSH root. MapX controls the mapping from objects onto layers by manipulating the timestamps of the intermediate placement groups (PGs). MapX is applicable to a large variety of object-based storage scenarios where object timestamps can be maintained as higher-level metadata. We have applied MapX to the state-of-the-art Ceph-RBD (RADOS Block Device) to implement a migration-controllable, decentralized object-based block store (called Oasis ). Oasis extends the RBD metadata structure to maintain and retrieve approximate object creation times (for migration control) at the granularity of expansion layers. Experimental results show that the MapX -based Oasis block store outperforms the CRUSH-based Ceph-RBD (which is busy in migrating objects after expansions) by 3.17× ∼ 4.31× in tail latency, and 76.3% (respectively, 83.8%) in IOPS for reads (respectively, writes).

Modern Solid State Drives (SSDs) integrate more internal resources to get higher performance and capacity. Improving internal resource utilization by exploiting internal parallelism is important to enhance the performance of SSDs. Unfortunately, the internal resource utilization of SSDs is limited at runtime in practice because of the practical access conflicts to internal resources. In this paper, we propose a Conflict Eliminated Requests Based I/O Scheduler (CER-IOS) to better utilize internal parallelism of flash chips by scheduling I/O requests in a more fine-grained way. We introduce Conflict Eliminated Requests (CERs) in which parallelizable memory requests are grouped during the process of address translation in Flash Translation Layer. To schedule conflicting requests, we propose a small CER size prioritized resource distribution scheme, that ensures internal resources can always be distributed to valuable conflicting requests to further improve the efficiency of resource utilization. Our extensive experimental evaluation results show that CER-IOS provides significant improvement of resource utilization at runtime and reduces average I/O latency largely compared to state-of-the-art I/O schedulers implemented in operating systems.