Training Graph Neural Networks (GNNs) on large-scale data is essential in various applications, e.g., transportation, and molecular biology. As graph sizes increasingly surpass main memory capacities, the out-of-core (OOC) GNN training system (OOC-based) has been proposed, a scheme that storing graphs on external storage, such as SSD or HDD, sequentially loading and processing smaller partitions. However, existing OOC-based systems face the key challenges: excessive data migration between storage and memory, as well as reduced model accuracy. In this paper, we theoretically and empirically analyze the limitations of state-of-the-art OOC-based systems and identify opportunities for optimization. Guided by our theoretical insights, we propose VeloxGNN, a novel system to improve data migration efficiency while maintaining high model accuracy. First, we introduce a novel algorithm, named Delayed Gradient Propagation (DGP), which specifically designed for OOC-based system. DGP leverages both historical node embeddings and unbiased gradients to achieve two key objectives simultaneously: minimizing the data migration (by ensuring that dataset is read at most once) while maintaining model accuracy. To support DGP, we then propose system-level optimizations: dynamic memory management, DGP-aware loading order, and a new graph partitioning method that separates labeled and unlabeled data. Experimental results show that VeloxGNN achieves memory-based accuracy while reducing training time by 17.7% to 73.3% across various datasets and GNN models, outperforming state-of-the-art methods. This result highlights VeloxGNN's potential for efficient and scalable GNN training on large-scale graph data.

The rapid advancement of AI (from foundational machine learning to Large Language Models) and edge computing has placed unprecedented demands on computation, memory, and storage on resource-constrained edge devices. As AI models scale, the ability to efficiently manage computing resources, utilize memory and storage, and reduce energy consumption has become critical. This paper introduces contributions on 4 topics related to deploying AI on resource-constrained edge devices: 1) unlocking training of foundational machine learning algorithms on the edge, 2) exploring hardware-aware DNN architecture and mapping co-optimization for inference on heterogeneous systems, 3) scaling RAG by leveraging advanced memory, storage, and energy-efficient designs, and 4) investigating cost-effective and high-performance large-scale graph processing.

High-density NAND flash has revolutionized the storage ecosystem because of its rapidly decreasing per-bit costs and unprecedented capacities. However, the inherent large block size of modern high-density NAND flash inevitably aggravates the reclamation latency (i.e., the time required to reclaim the storage space occupied by the obsolete data), which subsequently prolongs the tail latency of flash-based storage devices. Inspired by the “erase duality” from the emerging bit-alterable NAND flash, this article proposes a reclamation latency suppressed (RLS) space management design to synergize the strengths of both block-level erase and page-level erase. Taking into account the data update frequency during runtime, RLS enables proactive adjustment of the dual-granularity erase. Moreover, RLS tightly couples the data cluster allocation strategy with a novel dual-granularity space reclamation design, thereby alleviating the reclamation latency. We extensively examine the benefits of RLS with real-world workloads. Our evaluation results reveal that, with the suppressed space reclamation latency, RLS achieves up to 37.51% improvement for both write and read tail latency (latency at the 99.9th percentile) compared with the state-of-the-art approaches.

The zoned storage has revolutionized the decades-old block storage in lowering the cost-per-gigabyte while enabling the host system to achieve better performance. With such benefit of cost and performance, we still require careful consideration on the endurance when deploying the applications on the zoned storage, since the modern storage tends to trade its endurance for larger capacity at lower cost. In this regard, although previous studies have deployed the log-structure merge (LSM-tree)-based key-value (KV) store on the zoned storage, the LSM-tree-based KV store can be suboptimal choice to build a cost-effective KV store on zoned storage, since LSM-tree has a well-known problem of write amplification (WA). Therefore, based on the key insight that the Fragmented Log-Structured Merge tree (FLSM-tree) substantially alleviates the notorious write amplification problem of the classical LSM-tree and inherently complies with the sequential write constraint of zoned storage, FLSM-tree would be a promising design choice to build a cost-effective KV store on zoned storage. However, based on our investigation, deploying an FLSM-tree-based KV store on zoned storage faces two challenges: The write amplification of the host-initiated garbage collection (GC) cancels out the low WA merit of FLSM-tree, and FLSM-tree results in high space amplification to increase the cost. In this regard, this article presents ZonesDB , a novel FLSM-tree-based KV store that comes with a series of innovative “zone-aware” techniques for pursuing write optimality and space adaptability. Our evaluations, based on two types of production-grade zoned storage (i.e., ZNS SSD and HM-SMR HDD), reveal that ZonesDB can bring into play the low WA merit of FLSM-tree, deliver outstanding write performance, and mitigate the space amplification problem of FLSM-tree on zoned storage.

Fully-external graph computation systems exhibit optimal scalability by computing the ever-growing, large-scale graph with a constant amount of memory on a single machine. In particular, they keep the entire massive graph data in storage and iteratively load parts of them into memory for computation. Nevertheless, despite the merit of optimal scalability, their unreasonably-low efficiency often makes them uncompetitive, and even unpractical, to the other types of graph computation systems. The key rationale is that most existing fully-external graph computation systems over-emphasize retrieving graph data from storage through sequential access. Although this principle achieves high storage bandwidth, it often causes reading excessive and irrelevant data, which can severely degrade their overall efficiency. Therefore, this work presents Seraph, a fully-external graph computation system that achieves optimal S calability while toward satisfactory E fficiency improvement. Particularly, inspired by the modern storage offering comparable sequential and random access speeds, Seraph adopts the principle of on-demand processing to access the necessary graph data for saving I/O while enjoying the decent speed in random access. On the basis of this principle, Seraph further devises three practical designs to bring excellent performance leap to fully-external graph computation: 1) the hybrid format to represent the graph data for striking a good balance between I/O amount and access locality, 2) the vertex passing to enable efficient vertex updates on top of hybrid format, and 3) the selective pre-computation to re-use the loaded data for I/O reduction. Our evaluations reveal that Seraph notably outperforms other state-of-the-art fully-external systems under all the evaluated billion-scale graphs and representative graph algorithms by up to two orders of magnitude.

As graphs exponentially grow recently, out-of-core graph systems have been invented to process large-scale graphs by keeping massive data in storage. Among them, many systems process the graphs iteration-by-iteration and provide synchronous semantics that allows easy programmability by forcing the computation dependency of vertex values between iterations. On the other hand, although future value computation is an effective IO optimization for out-of-core graph systems by computing vertex values of future iterations in advance, it is challenging to take full advantage of future value computation while guaranteeing iteration-based dependency. In fact, based on our investigation, even state-of-the-art work along this direction has a wide gap from optimality in IO reduction and further requires substantial overhead in computation as well as extra memory consumption.

Graph neural networks (GNN) one of the most popular neural network models, are extensively applied in graph-related fields, including drug discovery, recommendation systems, etc. Unsupervised graph learning as one type of GNN plays a crucial role in various graph-related missions like node classification and edge prediction. However, with the increasing size of real-world graph datasets, processing such massive graphs in host memory becomes impractical, and GNN training demands a substantial storage volume to accommodate the vast amount of graph data. Consequently, GNN training results in significant I/O migration between the host and storage. Although state-of-the-art frameworks have made strides in mitigating I/O overhead by considering embedding locality, their GNN frameworks still suffer from long training times. In this paper, we propose a fully out-of-core framework, called Celeritas, which speeds up the unsupervised GNN training on a single machine by co-designing the GNN algorithm and storage systems. First, based on the theoretical analysis, we propose a new partial combination operation to enable the embedding updates across GNN layers. This cross-layer computing achieves future computation for the embedding stored in memory to save data migration. Second, due to the dependency between embedding and edges, we consider their data locality together. Based on the cross-layer computing property, we propose a new loading order to fully utilize the data stored in the main memory to save I/O. Finally, a new sampling scheme called two-level sampling is proposed associated with a new partition algorithm to further reduce data migration and computation overhead while maintaining similar training accuracy. The real system experiments indicate that the proposed Celeritas can reduce the total training time of different G NN models from 44.76 % to 73.85 % compared to state-of-art schemes for different graph datasets.

Skyrmion racetrack memory (Sky-RM), as an emerging non-volatile memory technology, provides both high-density data storage and low-access latency, accompanying with a novel feature of using current to shift data along a racetrack. However, it is time-comsuing and energy-hungry to write/inject new particle-like Skyrimons on Sky-RM compared with other manipulations. Thus, the goal of this work is to presents a novel strategy to optimize write performance in Sky-RM, Permutation-Write (PW). Particuarly, PW reduces the number of Skyrmion injections when writing data, by “re-permuting" existing Skyrmion particles within a racetrack. Moreover, based on the flexible concept of re-permutation, this work further proposes PW+ strategy, which is an optimized PW strategy integrated with injection and shift optimizations to achieve better performance. Our evaluation results justify that PW+ can greatly reduce the amount of Skyrmion injections, which are considered highly expensive in Sky-RM, by at least 50%, compared with other state-of-the-art write-optimized strategies. Furthermore, PW+ only incurs 1.79 injections, averaged for every 64-bit-word write. We show that PW+ strategy can bring significant benefits over other state-of-the-art write-optimized strategies by about 16-50% for every 64-bit-word write, for both performance and energy efficiency.

Skyrmion racetrack memory (Sky-RM) is an emerging non-volatile memory technology that offers not only high-density data storage but also the feature of shifting data along a racetrack by current. However, compared to other shifting based manipulations, writing/injecting new particle-like Skyrmions is much more time-consuming and energy-hungry. To optimize the write performance and energy for Sky-RM, this paper introduces a new write strategy, called Permutation-Write. Specifically, this strategy circumvents new Skyrmion injections during data writes by "re-permuting" existing Skyrmion particles in the racetrack. Evaluation results show that Permutation-Write strategy can significantly reduce the number of expensive Skyrmion injections by at least 50% compared to other state-of-the-art write reduction strategies, and merely introduce 2.25 injections on average for every 64-bit-word write under realistic workloads.