Cloud platforms host thousands of tenants that demand POSIX semantics, high throughput, and rapid evolution from their storage layer. Kernel-native distributed file systems supply raw speed, but their privileged code base couples every release to the kernel, widens the blast radius of crashes, and slows innovation. FUSE-based distributed file systems flip those trade-offs: they run in user space for fast deployment and strong fault isolation, yet the FUSE interface disables the kernel's write-back page cache whenever strong consistency is required. Practitioners must therefore choose between (i) weak consistency with fast write-back caching or (ii) strong consistency with slow write-through I/O, a limitation that has kept FUSE distributed file systems out of write-intensive cloud workloads.

In this work, we examine the relationship between rate-distortion theory and energy-based models (EBMs). We demonstrate that EBMs can be used to approximate the rate-distortion approaching posterior, as in the Blahut-Arimoto (BA) algorithm, and to solve batch denoising problems using the posterior distribution learned by EBMs. Our results highlight the potential of EBMs to enhance the efficiency of rate-distortion theory analysis and vice versa.

Rate-distortion theory provides a framework for understanding the limits of source coding. Energy-based models (EBMs), which have a broad range of applications in fields such as physics, statistics, and machine learning, can be used to estimate these limits. In this work, we demonstrate how EBMs can be used to estimate rate-distortion functions, and show that our empirical estimates agree with known closed-form expressions and bounds.

We explore the connections between rate distortion/lossy source coding and deep learning models, the Restricted Boltzmann Machines (RBMs) and Deep Belief Networks (DBNs). We show that rate distortion is a function of the RBM log partition function and that RBM/DBN can be used to learn the rate distortion approaching posterior as in the Blahut-Arimoto algorithm. We propose an algorithm for lossy compressing of binary sources. The algorithm consists of two stages, a training stage that learns the posterior with training data of the same class as the source, and a compression/reproduction stage that is comprised of a lossless compression and a lossless reproduction. Theoretical results show that the proposed algorithm achieves the optimum rate distortion function for stationary ergodic sources asymptotically. Numerical experiments show that the proposed algorithm outperforms the reported best results.

We answer two questions in this work: what Deep Boltzmann Machines (DBMs) can do for compression and vise versa. We show that (1) DBMs can be applied to learn the rate distortion approaching posterior as in the Blahut-Arimoto (BA) algorithm, and to construct a lossy source compression scheme based on the Deep AutoEncoder; (2) compression can improve DBMs’ training performances via compression-based denoising algorithms. The implementation of the BA algorithm in the form of DBMs is the foundation of the two applications.

Non-volatile memories suffer from two challenges due to their physical and system-level constraints. One challenge is limited memory lifetime, also called the endurance problem. The other is the difficulty in deleting data securely, called the insecure deletion problem. This paper proposes a coding scheme that addresses both challenges jointly. It studies the secure write-efficient memory (WEM) by analyzing its rewriting-rate equivocation region and secrecy rewriting capacity. It also presents an optimal code construction for a large family of secure WEM channels.

For nonvolatile memories such as flash memories and phase-change memories, endurance and reliability are both important challenges. Write-Efficient Memory (WEM) is an important rewriting model to solve the endurance problem. An optimal rewriting code has been proposed to approach the rewriting capacity of WEM. Aiming at jointly solving the endurance and the data reliability problem, this work focuses on a combined error correction and rewriting code for WEM. To that end, a new coding model, noisy WEM, is proposed here. Its noisy rewriting capacity is explored. An efficient coding scheme is constructed for a special case of noisy WEM. Its decoding and rewriting operations can be done in time O(N logN), with N as the length of the codeword, and it provides a lower bound to the noisy WEM's capacity.

Flash memories have become a significant storage technology. However, they have various types of error mechanisms, which are drastically different from traditional communication channels. Understanding the error models is necessary for developing better coding schemes in the complex practical settings. This paper endeavors to survey the noise and disturbs in NAND flash memories, and construct channel models for them. The capacity of flash memory under these models is analyzed, particularly regarding capacity degradation with flash operations, the trade-off of sub-thresholds for soft cell-level information, and the importance of dynamic thresholds.