Key-Value Stores (KVS) based on log-structured merge-trees (LSM-trees) are widely used in storage systems but face significant challenges, such as high write amplification caused by compaction. KV-separated LSM-trees address write amplification but introduce significant space amplification, a critical concern in cost-sensitive scenarios. Garbage collection (GC) can reduce space amplification, but existing strategies are often inefficient and fail to account for workload characteristics. Moreover, current key-value (KV) separated LSM-trees overlook the space amplification caused by the index LSM-tree. In this paper, we systematically analyze the sources of space amplification in KV-separated LSM-trees and propose Scavenger+, which achieves a better performance-space tradeoff. Scavenger+ introduces (1) an I/O-efficient garbage collection scheme to reduce I/O overhead, (2) a space-aware compaction strategy based on compensated size to mitigate index-induced space amplification, and (3) a dynamic GC scheduler that adapts to system load to make better use of CPU and storage resources. Extensive experiments demonstrate that Scavenger+ significantly improves write performance and reduces space amplification compared to state-of-the-art KV-separated LSM-trees, including BlobDB, Titan, and TerarkDB.

Key- Value Stores (KVS) implemented with log- structured merge-tree (LSM-tree) have gained widespread ac-ceptance in storage systems. Nonetheless, a significant challenge arises in the form of high write amplification due to the compaction process. While KV-separated LSM-trees successfully tackle this issue, they also bring about substantial space am-plification problems, a concern that cannot be overlooked in cost-sensitive scenarios. Garbage collection (GC) holds significant promise for space amplification reduction, yet existing GC strategies often fall short in optimization performance, lacking thorough consideration of workload characteristics. Additionally, current KV-separated LSM-trees also ignore the adverse effect of the space amplification in the index LSM-tree. In this paper, we systematically analyze the sources of space amplification of KV- separated LSM-trees and introduce Scavenger, which achieves a better trade-off between performance and space amplification. Scavenger initially proposes an I/O-efficient garbage collection scheme to reduce I/O overhead and incorporates a space-aware compaction strategy based on compensated size to minimize the space amplification of index LSM-trees. Extensive experiments show that Scavenger significantly improves write performance and achieves lower space amplification than other KV-separated LSM-trees (including BlobDB, Titan, and TerarkDB).

Hierarchical Federated Learning (HFL) improves the scalability and communication efficiency of the system and achieves load balancing at each level. Incentive mechanisms enhance participant motivation and optimize resource allocation for HFL. However, existing mechanisms mainly focus on maximizing individual utility from the quantity of client data while neglecting to optimize social utility from the learning quality perspective. Meanwhile, strategic behavior and heterogeneous devices can significantly degrade the performance of incentive mechanisms. To this end, we propose a quality-aware incentive mechanism (QAIM) for HFL to improve training efficiency. Specifically, we first systematically evaluate the learning quality of clients based on their training loss and historical records, which allows us to recruit high-quality clients for model updating selectively. Then, we model the cloud-edge-end interaction and cooperation as a three-layer Stackelberg game to analyze the strategies of participants and utilize carefully designed algorithms to derive the solution of the unique Stackelberg Equilibrium (SE). Through the Pareto improvement of client association modeled as a coalition game, we can maximize social utility. Experimental results on both synthetic and real-world datasets demonstrate that our QAIM outperforms the state-of-the-art baselines, with an average increase in accuracy and social utility of 17% and 45%, respectively.

Persistent key-value (KV) stores are widely used by cloud services at ByteDance as local storage engines, and RocksDB used to be the de facto implementation since it can be tailored to a variety of workloads and requirements. In this paper, we provide key insights into local storage engine usage at ByteDance, explain why the combination of highly write-intensive workloads and stringent requirements on cost efficiency and point lookup tail latency may pose challenges to a general-purpose local storage engine such as RocksDB, and present the design and implementation of LavaStore , a high-performance cost-effective local storage engine purpose-built to address these challenges. LavaStore achieves its design goals by selectively customizing a few components of a RocksDB-based, general-purpose local storage engine, including a distinct KV separation design that decouples garbage collection from compaction, a specialized engine type for the commonly recurring Write-Ahead-Logging workload, and a customized user-space append-only filesystem. LavaStore has been deployed to production with hundreds of thousands of running instances, storing more than 100 PB of data and serving billions of requests per second, bringing significant performance improvements and cost reductions to customers over their original local storage engines. For example, a ByteDance proprietary distributed OLTP database service has experienced a reduction in average write and read latency by 61% and 16%, respectively, and a ByteDance proprietary caching service has gained an 87% increase in write throughput with no more than 6% space overhead.

Generative adversarial networks (GANs) have been widely used for creating diverse data such as images, audio, and videos. However, as the training data of GANs often contain sensitive information, they are vulnerable to privacy attacks on the training dataset, such as membership inference attacks (MIAs). To improve the resistance of GANs to MIA while ensuring their performance, we design a novel GAN framework PGAN-KD (member Privacy protection of GANs based on Knowledge Distillation). PGAN-KD prevents the discriminator from leaking membership information of the training data by introducing knowledge distillation and gradient clipping. Specifically, it adopts an extra teacher discriminator to distilling knowledge and then transfer it to a student discriminator, and thereby isolating the attacker from indirectly obtaining private information through the generator. In addition, the teacher discriminator prevents itself from MIAs through gradient clipping. To evaluate the performance of PGAN-KD, we conducted experiments on both real and simulated datasets. The results indicate that PGAN-KD achieves a 7.8% improvement in privacy protection levels while maintaining similar generation performance with the baselines.

Abstract As a usual component in virtual scenes, water surface plays an important role in various graphical applications, including special effects, video games, and virtual reality. Although recent years have witnessed significant progress based on Navier–Stokes equations and simplified water models, large‐scale water surface waves with high‐frequency visual details remain computationally expensive for interactive applications. This article proposes a novel frequency‐aware neural network to synthesize consistent and detailed water surface waves from low‐resolution input. At its core, our approach leverage the wavelet transformation theory over space, frequency and direction, and incremental supervision to decompose the 4D amplitude function into multiple smaller subproblems. Specifically, we first customize four subnetworks and corresponding loss functions for super‐resolution of spatial resolution, temporal evolution, wave direction subdivision, and wave number, respectively. Then, to enforce the upsampling along each dimension orthogonal to each other, we introduce a cooperative training scheme to fine‐tune and integrate the proposed subnetworks with carefully designed training dataset. Our method can visually enhance high‐resolution spatial details, temporal coherence, interactions with complex boundaries, and various wave patterns with flexible control along multiple dimensions. Through extensive experiments, our method arrives at 13 speedup for 32 upsampling of various simulation scenarios. We also validate the effectiveness and robustness of our method to produce realistic water surface waves toward artistic innovation.

File systems for mobile devices usually preserve data consistency by ordered I/Os. However, maintaining I/O ordering prevents applications from fully exploiting device parallelism and thus degrades the storage performance. In this paper, we propose NBStack to eliminate ordered I/Os without compromising data consistency. First, we augment the existing block interface to expose the Flash out-of-band area to file systems. Second, we build an enhanced block device prototype that supports the new interface. Third, we develop NBFS, a Linux file system, that leverages the new block interface to achieve atomic writes without enforcing I/O orderings. Experimental results show that NBStack doubles the performance of F2FS while providing strong consistency and durability guarantees. If applications are willing to trade-off durability, NBStack can further aggressively improve performance.

Abstract Despite the rapid development and proliferation of computer graphics hardware devices for scene capture in the most recent decade, the high‐resolution 3D/4D acquisition of gaseous scenes (e.g., smokes) in real time remains technically challenging in graphics research nowadays. In this paper, we explore a hybrid approach to simultaneously taking advantage of both the model‐centric method and the data‐driven method. Specifically, this paper develops a novel conditional generative model to rapidly reconstruct the temporal density and velocity fields of gaseous phenomena based on the sequence of two projection views. With the data‐driven method, we can achieve the strong coupling of density update and the estimation of flow motion, as a result, we can greatly improve the reconstruction performance for smoke scenes. First, we employ a conditional generative network to generate the initial density field from input projection views and estimate the flow motion based on the adjacent frames. Second, we utilize the differentiable advection layer and design a velocity estimation network with the long‐term mechanism to help achieve the end‐to‐end training and more stable graphics effects. Third, we can re‐simulate the input scene with flexible coupling effects based on the estimated velocity field subject to artists' guidance or user interaction. Moreover, our generative model could accommodate single projection view as input. In practice, more input projection views are enabling and facilitating the high‐fidelity reconstruction with more realistic and finer details. We have conducted extensive experiments to confirm the effectiveness, efficiency, and robustness of our new method compared with the previous state‐of‐the‐art techniques.

Reference-based color transfer between images has been a fundamental function in image editing. However, existing approaches pay less attention to high-dynamic-range (HDR) images. It is worth noting that designing an appropriate representation for HDR images to achieve satisfying color transfer is challenging. In this paper, we propose an innovative high-dynamic-range image color transfer generative adversarial network (HDRCTGAN) to encode the original image into fine representations that allow transfer of the color of the reference image to the target image. We propose to learn fine representations through a generative adversarial network (GAN) in a self-supervised way. Particularly, the proposed method is self-supervised learning that requires only unlabeled HDR images instead of supervised learning that requires lots of ground truth pairs. HDRCTGAN consists of a generator to transfer the color of the reference image to the target image over the feature domain and a discriminator to suppress the artifacts caused by the generator. We also design a loss function to ensure that HDRCTGAN possesses two required properties: (a) high fidelity and (b) self-identity. The proposed approach yields a pleasing visual result. We have carried out HDR specific evaluations including both objective quantitative experiments with HDR metrics and subjective user studies operated on HDR display devices to demonstrate the effectiveness of our method. Furthermore, we have verified the applicability of the proposed approach to several applications, such as color transfer of HDR images captured by smartphones, color transfer of fabric images, and reference-based grayscale image colorization.

This article articulates a novel learning framework for both parameter estimation and detail enhancement for Eulerian gas based on data guidance. The key motivation of this article is to devise a new hybrid, grid-based simulation that could inherit modeling and simulation advantages from both physically-correct simulation methods and powerful data-driven methods, while combating existing difficulties exhibited in both approaches. We first employ a convolutional neural network (CNN) to estimate the physical parameters of gaseous phenomena in Eulerian settings, then we can use the just-learnt parameters to re-simulate (with or without artists' guidance) for specific scenes with flexible coupling effects. Next, a second CNN is adopted to reconstruct the high-resolution velocity field to guide a fast re-simulation on the finer grid, achieving richer and more realistic details with little extra computational expense. From the perspective of physics-based simulation, our trained networks respect temporal coherence and physical constraints. From the perspective of the data-driven machine-learning approaches, our network design aims at extracting a meaningful parameters and reconstructing visually realistic details. Additionally, our implementation based on parallel acceleration could significantly enhance the computational performance of every involved module. Our comprehensive experiments confirm the controllability, effectiveness, and accuracy of our novel approach when producing various gaseous scenes with rich details for widespread graphics applications.

In this paper, we design a storage system consisting of Nonvolatile DIMMs (as NVRAM) and NAND-flash SSD. We propose a file system NVMFS to exploit the unique characteristics of these devices which simplifies and speeds up file system operations. We use the higher performance NVRAM as both a cache and permanent space for data. Hot data can be permanently stored on NVRAM without writing back to SSD, while relatively cold data can be temporarily cached by NVRAM with another copy on SSD. We also reduce the erase overhead of SSD by reorganizing writes on NVRAM before flushing to SSD. We have implemented a prototype NVMFS within a Linux Kernel and compared with several modern file systems such as ext3, btrfs and NILFS2. We also compared with another hybrid file system Conquest, which originally was designed for NVRAM and HDD. The experimental results show that NVMFS improves IO throughput by an average of 98.9 % when segment cleaning is not active, while improves throughput by an average of 19.6% under high disk utilization (over 85%) compared to other file systems. We also show that our file system can reduce the erase operations and overheads at SSD.

Decreasing the traffic from the CPU LLC to main memory is a very important issue in modern systems. Recent work focuses on cache misses, overlooking the impact of writebacks on the total memory traffic, energy consumption, IPC, and so forth. Policies that foster a balanced approach, between reducing write traffic to memory and improving miss rates, can increase overall performance and improve energy efficiency and memory system lifetime for NVM memory technology, such as phase-change memory (PCM). We propose Adaptive Replacement and Insertion (ARI), an adaptive approach to last-level CPU cache management, optimizing the two parameters (miss rate and writeback rate) simultaneously. Our specific focus is to reduce writebacks as much as possible while maintaining or improving the miss rate relative to conventional LRU replacement policy. ARI reduces LLC writebacks by 33%, on average, while also decreasing misses by 4.7%, on average. In a typical system, this boosts IPC by 4.9%, on average, while decreasing energy consumption by 8.9%. These results are achieved with minimal hardware overheads.

Modern computer systems have been built around the assumption that persistent storage is accessed via a slow, block-based interface. However, emerging nonvolatile memory technologies (sometimes referred to as storage class memory (SCM)), are poised to revolutionize storage systems. The SCM devices can be attached directly to the memory bus and offer fast, fine-grained access to persistent storage. In this article, we propose a new file system---SCMFS, which is specially designed for Storage Class Memory. SCMFS is implemented on the virtual address space and utilizes the existing memory management module of the operating system to help mange the file system space. As a result, we largely simplified the file system operations of SCMFS, which allowed us a better exploration of performance gain from SCM. We have implemented a prototype in Linux and evaluated its performance through multiple benchmarks. The experimental results show that SCMFS outperforms other memory resident file systems, tmpfs, ramfs and ext2 on ramdisk, and achieves about 70% of memory bandwidth for file read/write operations.

Emerging nonvolatile memory technologies (sometimes referred as Storage Class Memory (SCM)), are poised to close the enormous performance gap between persistent storage and main memory. The SCM devices can be attached directly to memory bus and accessed like normal DRAM. It becomes then possible to exploit memory management hardware resources to improve file system performance. However, in this case, SCM may share critical system resources such as the TLB, page table with DRAM which can potentially impact SCM's performance. In this paper, we propose to solve this problem by employing superpages to reduce the pressure on memory management resources such as the TLB. As a result, the file system performance is further improved. We also analyze the space utilization efficiency of superpages. We improve space efficiency of the file system by allocating normal pages (4KB) for small files while allocating super pages (2MB on ×86) for large files. We show that it is possible to achieve better performance without loss of space utilization efficiency of nonvolatile memory.