In this study, we propose LWMalloc, a lightweight dynamic memory allocator designed for resource-constrained environments. LWMalloc incorporates a lightweight data structure, a deferred coalescing (DC) policy, and dedicated small chunk pools to optimize memory allocation. The lightweight data structure minimizes metadata overhead, ensuring a compact and efficient implementation. The DC policy reduces execution overhead by postponing redundant operations until allocation, maintaining both efficiency and low-response times. Dedicated small chunk pools enable$\mathcal {O}(1)$allocation for small memory requests, which are common in dynamic allocation patterns, by segregating them into fixed-size pools. Experimental results using real-world applications demonstrate that LWMalloc outperforms ptmalloc, the default allocator in Linux, achieving up to 53% faster execution time and 23% lower memory usage. With a compact implementation of only 530 lines of code and a 20-KB size, significantly smaller than ptmalloc’s 4838 lines and 116 KB, LWMalloc achieves an effective balance between performance and memory efficiency, making it highly suitable for resource-constrained environments.

A graph engine should possess adaptability to ensure efficient processing despite a variety of graph data and algorithms. In terms of out-of-core graph engines, which exploit a hierarchical memory structure, an adaptive caching scheme is necessary to sustain effectiveness of memory usage. A caching policy selectively stores data likely to be used in the upper-layer memory based on its own expectation about the future workload. However, the graph workload contains a complexity of memory access according to graph data, algorithm, and configurations. This makes it difficult for a static caching policy to respond to the changes in workload. In this paper, we propose a graph-adaptive caching scheme which ensures consistent effectiveness under the changing workloads. Our caching scheme employs an adaptive policy that responds to changes in real-time workloads. To detect the changes, we adopt the competition procedures between two contrasting properties-locality and regularity-that appear in graph workloads. In addition, we combine two window adjustment techniques to alleviate the overhead from competition procedures. The proposed caching scheme is applicable to different types of graph engines, achieving better efficiency in memory usage. Our experimental results prove that our scheme improves the performance of graph processing by up to 65% compared to existing schemes.

Parallel file systems (PFS) are used to distribute data processing and establish shared access to large-scale data. Despite being able to provide high I/O bandwidth on each node, PFS has difficulty utilizing the I/O bandwidth due to a single connection between the client and server nodes. To mitigate the performance bottleneck, users increase the number of connections between the nodes by modifying PFS or applications. However, it is difficult to modify PFS itself due to its complicated internal structure. Thus, PFS users manually increase the number of connections between the nodes by employing several methods. In this paper, we propose a user-transparent I/O subsystem, MulConn, to make users exploit high I/O bandwidth between nodes. To avoid the modifications of PFS and user applications, we have developed a horizontal mount procedure and two I/O scheduling policies, TtoS and TtoM, in the virtual file system (VFS) layer. We expose a single mount point that has multiple connections by modifying the mount path of VFS from vertical hierarchy to horizontal hierarchy. We also introduce two I/O scheduling policies to distribute I/O requests evenly to multiple connections. The experimental results show that MulConn improves write and read performance by up to 2.6x and 2.8x, respectively, compared with those of PFS using the existing kernel. In addition, we provide the best I/O performance that PFS can provide in the given experimental environments.

Many cloud services perform periodic database backup to keep the data safe from failures such as sudden system crashes. In the database system, two techniques are widely used for data backup and recovery: a physical backup and a logical backup. The physical backup uses raw data by copying the files in the database, whereas the logical backup extracts data from the database and dumps it into separated files as a sequence of query statements. Both techniques support a full backup strategy that contains data of the entire database and incremental backup strategy that contains changed data since a previous backup. However, both strategies require additional I/O operations to perform the backup and need a long time to restore a backup. In this paper, we propose an efficient backup and recovery scheme by exploiting write-ahead logging (WAL) in database systems. In the proposed scheme, for backup, we devise a backup system to use log data generated by the existing WAL to eliminate the additional I/O operations. To restore a backup, we utilize and optimize the existing crash recovery procedure of WAL to reduce recovery time. For example, we divide the recovery range and applying the backup data for each range independently via multiple threads. We implement our scheme in MySQL, a popular database management system. The experimental result demonstrates that the proposed scheme provides instant backup while reducing recovery time compared with the existing schemes.

Modern data centers aim to take advantage of high parallelism in storage devices for I/O intensive applications such as storage servers, cache systems, and key-value stores. Key-value stores are the most typical applications that should provide a highly reliable service with high-performance. To increase the I/O performance of key-value stores, many data centers have actively adopted next-generation storage devices such as Non-Volatile Memory Express (NVMe) based Solid State Devices (SSDs). NVMe SSDs and its protocol are characterized to provide a high degree of parallelism. However, they may not guarantee predictable performance while providing high performance and parallelism. For example, heavily mixed read and write requests can result in performance degradation of throughput and response time due to the interference between the requests and internal operations (e.g., Garbage Collection (GC)). To minimize the interference and provide higher performance, this paper presents IsoKV, an isolation scheme for key-value stores by exploiting internal parallelism in SSDs. IsoKV manages the level of parallelism of SSD directly by running application-driven flash management scheme. By storing data with different characteristics in each dedicated internal parallel units of SSD, IsoKV reduces interference between I/O requests. Also, IsoKV synchronizes the LSM-tree logic and data management in SSD to eliminate GC. We implement IsoKV on RocksDB and evaluate it using Open-Channel SSD. Our extensive experiments have shown that IsoKV improves overall throughput and response time on average 1.20× and 43% compared with the existing scheme, respectively.

As modern computer systems face the challenge of managing large data, filesystems must deal with a large number of files. This leads to amplified concerns of metadata and data operations. Filesystems in Linux manage the metadata of files by constructing in-memory structures such as directory entry (dentry) and inode. However, we found inefficiencies in metadata management mechanisms, especially in the path traversal mechanism of Linux file systems when searching for a dentry in the dentry cache.

With the growth of online services, a large amount of files have been generated by users or by the service itself. To make it easier to service users with different network environments and devices, online services usually keep different versions of the same file with various sizes. For users with high speed network and top of the line displays, a large size file with high precision can be supplied while users with mobile devices typically receive a smaller file with less precision. In some cases, a large file can be divided into small files to make it easier to transmit over the wide area networks. As a result, underlying filesystem should efficiently maintain a large number of small files. Providing such a huge number of files to applications is one of new challenges of existing filesystems. In this paper, we propose techniques to efficiently manage a large number of files in digital archives using data characteristics and access patterns of the application. Based on the knowledge we have of the upper layer applications, we have modified both in-memory and on-disk inode structure of the existing filesystem and were able to dramatically reduce the number of storage I/O operations to service the same files. Our experimental results show that the proposed methods significantly reduce the number of storage I/O operations both for reading and writing files, especially for small-sized ones. Moreover, we demonstrated that proposed techniques reduce the application-level latency as well as improve file operation throughput, using several synthetic- and microbenchmarks.