This paper presents zBuffer, a zero-copy and metadata-free serialization library for high-performance and low-cost RPCs. At the core of zBuffer is scatter-gather reflection, a novel technique that collaboratively (i) leverages the NIC scatter-gather hardware feature to offload the costly data coalescing, and (ii) utilizes the static reflection mechanism of modern programming languages to enable type queries on complex data objects without requiring explicit metadata construction. We leverage C++ language features, mainly including template meta-programming and macros, to realize static reflection at compile time. Based on zBuffer, we design a fast RPC system (called zRPC) which eliminates all RPC memory copy overheads not only in (de)serialization but also in network transmission. Extensive evaluation shows that zBuffer/zRPC significantly outperforms state-of-the-art serialization/RPC mechanisms: zBuffer is approximately 7× faster than Cornflakes in serialization for complex data objects; and zRPC reduces 99th percentile latency by 21% and achieves 62% higher throughput than eRPC on the Masstree key-value (KV) store with the YCSB benchmark.

Serverless is a promising paradigm to provide scalable, cost-efficient, and easy-to-use model inference services. However, the cold start of model inference functions requires loading models to the devices, which incurs high latencies and undermines the benefits of serverless computing. In LLMs, things get even worse since two extra stages are introduced: a KV cache initialization stage that profiles and anticipates memory reservation for KV cache, and a capturing stage which dynamically constructs CUDA graphs for different batch sizes. Both stages are paramount to the inference performance, but become the main culprit of cold start latency.

The growing complexity of LLM usage today, e.g., multi-round conversation and retrieval-augmented generation (RAG), makes contextual states (i.e., KV cache) reusable across user requests. Given the capacity constraints of GPU memory, only a limited number of contexts can be cached on GPU for reusing. Existing inference systems typically evict part of the KV cache and restore it by recomputing it from the original tokens or offloading it to host storage for later retrieval, both of which introduce substantial computational or I/O overheads.

Current deep learning approaches for lithium-ion battery analysis are often specialized and limited to specific battery types or individual tasks. While recent advances in large language models (LLMs) highlight the potential of pretraining paradigms, existing time-series pretraining models inadequately address the physicochemical complexity and temporal irregularity inherent to battery operational data. We propose LiPM, a pretrained foundation model that unifies multi-dataset learning through physics-aware objectives and irregularity-tolerant temporal modeling. LiPM introduces three key innovations: (1) A Mix-Masked Autoencoder (MMAE) enforcing electrochemical consistency via joint reconstruction of temporally masked patches and cross-channel masked variables(2) A Coulombic Integration Regression (CIR) task explicitly encoding charge conservation laws, and (3) A dual-scale temporal encoder combining irregular intra-patch processing (preserving raw timestamps) with regular inter-patch attention (capturing macroscopic dynamics). Trained on eight heterogeneous battery datasets without cycle-label annotations, LiPM demonstrates universal applicability across partial charge-discharge segments and irregular sampling protocols. Extensive experiments show remarkable improvements over 9 state-of-the-art baselines in critical downstream tasks.

Retrieval-Augmented Generation (RAG) addresses large language model (LLM) hallucinations by grounding responses in external knowledge, but its effectiveness is compromised by poor-quality retrieved contexts containing irrelevant or noisy information. While existing approaches attempt to improve performance through context selection based on predefined context quality assessment metrics, they show limited gains over standard RAG. We attribute this limitation to their failure in holistically utilizing available information (query, context list, and generator) for comprehensive quality assessment. Inspired by recent advances in data selection, we reconceptualize context quality assessment as an inference-time data valuation problem and introduce the Contextual Influence Value (CI value). This novel metric quantifies context quality by measuring the performance degradation when removing each context from the list, effectively integrating query-aware relevance, list-aware uniqueness, and generator-aware alignment. Moreover, CI value eliminates complex selection hyperparameter tuning by simply retaining contexts with positive CI values. To address practical challenges of label dependency and computational overhead, we develop a parameterized surrogate model for CI value prediction during inference. The model employs a hierarchical architecture that captures both local query-context relevance and global inter-context interactions, trained through oracle CI value supervision and end-to-end generator feedback. Extensive experiments across 8 NLP tasks and multiple LLMs demonstrate that our context selection method significantly outperforms state-of-the-art baselines, effectively filtering poor-quality contexts while preserving critical information. Code is available at https://github.com/SJTU-DMTai/RAG-CSM.

This paper presents μTPS, a new thread architecture tailored for in-memory key-value stores (KVSs) that operate at tens of millions of operations per second. We show through analysis and demonstration that the widely used run-to-completion thread architecture, which executes monolithic functions from start to finish, often suffers from cache inefficiencies and contention issues. To address this, we revisit the once widely used thread-per-stage (TPS) architecture, but with a fresh perspective – separating cache-resident, contention-free stages and memory-resident, conflict-prone stages into distinct thread pools, and scheduling them with dedicated hardware resources (e.g., CPU cores, cache ways). This novel division enables independent optimization of each stage, significantly improving cache efficiency and mitigating contention. Additionally, μTPS incorporates reconfigurable RPC, resizable caching, and an auto-tuner to enhance its schedulability and performance. We implement two in-memory key-value stores, μTPS-H and μTPS-T, to demonstrate the effectiveness of this approach. Evaluation results show that μTPS achieves higher performance than the run-to-completion counterparts.

With the emergence of fast NVMe SSDs, key-value stores are becoming more CPU-efficient in order to reap their bandwidth. However, current CPU-optimized key-value stores adopt suboptimal intra- and inter-thread models, hence incurring memory-level stalling and load imbalance that hinder cores from realizing their full potential.We present STAGEWISE, an CPU-efficient key-value store on fast NVMe SSDs with high throughput. To achieve this, we introduce a new thread model for StageWise to process KV requests. Specifically, STAGEWISEconverts the processing of each KV request into multiple asynchronous stages, and thus enables pipelining across all stages. STAGEWISEfurther introduces a client-driven share-index architecture to ease inter-thread load imbalance and maximize the pipelining opportunity. Guided by Little’s Law, STAGEWISEimproves concurrency, and therefore efficiently uses CPU to reach higher throughput. Extensive experimental results show that STAGEWISEoutperforms CPUoptimized key-value stores (e.g., KVell) by up to 3.5× with writeintensive workloads, and storage-optimized ones (e.g., RocksDB) by over an order of magnitude. STAGEWISEalso shows higher read performance and excellent scalability under various workloads.

We introduce EasyIO, a new approach to explore asynchronous I/O on filesystems designed for (disaggregated) nonvolatile memories to improve CPU efficiency. EasyIO offloads expensive memory movement operations to the on-chip DMA engine and harvests the unleashed CPU cycles by transparently interleaving asynchronous I/Os with fine-grained application tasks. We further adopt a completion buffer-centric design to improve EasyIO's efficiency and schedulability; internally, orderless file operation and two-level locking are incorporated to break the serial order between file metadata and data, thus accelerating read and write operations and defusing deadlock risks. EasyIO also introduces a traffic-aware channel manager to fulfill the diverse performance goals of applications. Extensive experimental results show that, compared to conventional synchronous filesystems, EasyIO significantly reduces CPU consumption (using less than 88% of cores at most) while achieving comparable peak bandwidth; EasyIO also achieves 1.03-2.3× speedups across eight real-world applications. When achieving these goals, EasyIO exhibits higher but tolerable latencies for read operations due to the task interleaving.

We introduce uProcess, a pure userspace process abstraction that enables CPU cores to be rescheduled among applications at sub-microsecond timescale without trapping into the kernel. We achieve this by constructing a special privileged mode in userspace to achieve safe and efficient separation among uProcesses. The core idea is a careful combination of two emerging hardware features - userspace interrupts (Uintr) and memory protection keys (MPK). We materialize the uProcess abstraction by implementing Vessel, a userspace core scheduler that colocates latency-critical and best-effort applications with minimal switching overhead when they time-share CPU cores. Our experiment result shows that Vessel exhibits better overall performance and low latency when multiple applications are colocated.

We present the design and implementation of TH-iSSD, a near-data processing framework to address the data movement problem. TH-iSSD does not pose any restriction to the hardware selection and is highly reconfigurable—its core components, such as the on-device compute unit (e.g., FPGA, embedded CPUs) and data collectors (e.g., camera, sensors), can be easily replaced to adapt to different use cases. TH-iSSD achieves this goal by incorporating highly flexible computation and data paths. In the data path, TH-iSSD adopts an efficient device-level data switch that exchanges data with both host CPUs and peripheral sensors; it also enables direct accesses between the sensing, computation, and storage hardware components, which completely eliminates the redundant data movement overhead, and thus delivers both high performance and energy efficiency. In the computation path, TH-iSSD provides an abstraction offilestreamfor developers, which abstracts a collection of data along with the related computation task as a file. Since existing applications are familiar with POSIX-like interfaces, they can be ported on top of our platform with minimal code modification. Moreover, TH-iSSD also introduces mechanisms includingpipelined near-data processingandpriority-aware I/O schedulingto make TH-iSSD perform more effectively. We deploy TH-iSSD to accelerate two types of applications: the content-based information retrieval system and the edge zero-streaming system. Our experimental results show that TH-iSSD achieves up to 1.6× higher throughput and 36% lower latency thancompute-centricdesigns.

We present pessimistic locking and optimistic reading (PLOR), a hybrid concurrency control protocol for in-memory transaction systems that delivers high throughput and low tail latency. PLOR is especially designed for high-contention workloads: for high throughput, transactions are allowed to access records without being blocked by lock conflicts in the read phase; for low tail latency, conflict detection is delayed to the commit phase, where old transactions are always committed first using the timestamps in the lock. We demonstrate the efficacy of this approach under a variety of setups (e.g., stored-procedures, interactive mode, and persistent logging, etc.). Experiments show that PLOR delivers close or comparable throughput to that of Silo and TicToc in stored-procedures, while reducing 99.9th percentile latency by 8.8x to 14.5x. In the interactive processing mode, PLOR even achieves up to 2x higher throughput.

The recent advent of the hardware trusted execution environment (TEE), e.g., Intel SGX, enables encrypted and integrity-verified in-memory key-value (KV) stores. However, due to the architectural limitations of the hardware, it is non-trivial to build a secure in-memory KV store with SGX without compromising the performance. The reason comes from (i) the limited memory capacity the SGX TEE provides, and (ii) being unaware of the access patterns of skewed workloads, which are commonly seen in the real world.In this paper, we present Aria, a secure in-memory KV store based on SGX. Our goal is to utilize the limited resource while still achieving high performance. Aria places KV pairs and index structures directly in the untrusted memory and introduces the security metadata in the TEE to conduct protection. The core component of Aria is Secure Cache, a software-based cache layer, which uses the limited memory resource to guarantee the confidentiality and integrity (including freshness) of Aria. Secure Cache keeps the frequently accessed security metadata in the TEE memory at fine-granularity and evicts rarely-used ones to the untrusted memory. With Secure Cache, we have the opportunities to explore strategies that are impossible in SGX implementation. By decoupling the security metadata management from the index structure, Aria supports various index schemes. We implement Aria with the indexes of both a hash table and a B-tree. Experiments show that Aria improves throughput by up to 104% compared to the state-of-the-art system.

Non-volatile memory and remote direct memory access (RDMA) provide extremely high performance in storage and network hardware. However, existing distributed file systems strictly isolate file system and network layers, and the heavy layered software designs leave high-speed hardware under-exploited. In this article, we propose an RDMA-enabled distributed persistent memory file system, Octopus + , to redesign file system internal mechanisms by closely coupling non-volatile memory and RDMA features. For data operations, Octopus + directly accesses a shared persistent memory pool to reduce memory copying overhead, and actively fetches and pushes data all in clients to rebalance the load between the server and network. For metadata operations, Octopus + introduces self-identified remote procedure calls for immediate notification between file systems and networking, and an efficient distributed transaction mechanism for consistency. Octopus + is enabled with replication feature to provide better availability. Evaluations on Intel Optane DC Persistent Memory Modules show that Octopus + achieves nearly the raw bandwidth for large I/Os and orders of magnitude better performance than existing distributed file systems.

Emerging hardware like persistent memory (PM) and high-speed NICs are promising to build efficient key-value stores. However, we observe that the small-sized access pattern in key-value stores doesn't match with the persistence granularity in PMs, leaving the PM bandwidth underutilized. This paper proposes an efficient PM-based key-value storage engine named FlatStore. Specifically, it decouples the role of a KV store into a persistent log structure for efficient storage and a volatile index for fast indexing. Upon it, FlatStore further incorporates two techniques: 1) compacted log format to maximize the batching opportunity in the log; 2) pipelined horizontal batching to steal log entries from other cores when creating a batch, thus delivering low-latency and high-throughput performance. We implement FlatStore with the volatile index of both a hash table and Masstree. We deploy FlatStore on Optane DC Persistent Memory, and our experiments show that FlatStore achieves up to 35 Mops/s with a single server node, 2.5 - 6.3 times faster than existing systems.

The rapidly increasing data in recent years requires the datacenter infrastructure to store and process data with extremely high throughput and low latency. Fortunately, persistent memory (PM) and RDMA technologies bring new opportunities towards this goal. Both of them are capable of delivering more than 10 GB/s of bandwidth and sub-microsecond latency. However, our past experiences and recent studies show that it is non-trivial to build an efficient and distributed storage system with such new hardware. In this article, we design and implement TH-DPMS (TsingHua Distributed Persistent Memory System) based on persistent memory and RDMA, which unifies the memory, file system, and key-value interface in a single system. TH-DPMS is designed based on a unified distributed persistent memory abstract, pDSM. pDSM acts as a generic layer to connect the PMs of different storage nodes via high-speed RDMA network and organizes them into a global shared address space. It provides the fundamental functionalities, including global address management, space management, fault tolerance, and crash consistency guarantees. Applications are enabled to access pDSM with a group of flexible and easy-to-use APIs by using either raw read/write interfaces or the transactional ones with ACID guarantees. Based on pDSM, we implement a distributed file system and a key-value store named pDFS and pDKVS, respectively. Together, they uphold TH-DPMS with high-performance, low-latency, and fault-tolerant data storage. We evaluate TH-DPMS with both micro-benchmarks and real-world memory-intensive workloads. Experimental results show that TH-DPMS is capable of delivering an aggregated bandwidth of 120 GB/s with 6 nodes. When processing memory-intensive workloads such as YCSB and Graph500, TH-DPMS improves the performance by one order of magnitude compared to existing systems and keeps consistent high efficiency when the workload size grows to multiple terabytes.

Data encryption and authentication are essential for secure non-volatile memory (NVM). However, the introduced security metadata needs to be atomically written back to NVM along with data, so as to provide crash consistency, which unfortunately incurs high overhead. To support fine-grained data protection and fast recovery for a secure NVM system without compromising the performance, we propose ShieldNVM. It first proposes an epoch-based mechanism to aggressively cache the security metadata in the metadata cache while retaining the consistency of them in NVM. Deferred spreading is also introduced to reduce the calculating overhead for data authentication. Leveraging the ability of data hash message authentication codes, we can always recover the consistent but old security metadata to its newest version. By recording a limited number of dirty addresses of the security metadata, ShieldNVM achieves fast recovering the secure NVM system after crashes. Compared to Osiris, a state-of-the-art secure NVM, ShieldNVM reduces system runtime by 39.1% and hash message authentication code computation overhead by 80.5% on average over NVM workloads. When system crashes happen, ShieldNVM’s recovery time is orders of magnitude faster than Osiris. In addition, ShieldNVM also recovers faster than AGIT, which is the Osiris-based state-of-the-art mechanism addressing the recovery time of the secure NVM system. Once the recovery process fails, instead of dropping all data due to malicious attacks, ShieldNVM is able to detect and locate the area of the tampered data with the help of the tracked addresses.

Data encryption and authentication are essential for secure NVM. However, the introduced security metadata needs to be atomically written back to NVM along with data, so as to provide crash consistency, which unfortunately incurs high overhead. To support fine-grained data protection without compromising the performance, we propose cc-NVM. It firstly proposes an epoch-based mechanism to aggressively cache the security metadata in CPU cache while retaining the consistency of them in NVM. Deferred spreading is also introduced to reduce the calculating overhead for data authentication. Leveraging the hidden ability of data HMACs, we can always recover the consistent but old security metadata to its newest version. Compared to Osiris, a state-of-the-art secure NVM, cc-NVM improves performance by 20.4% on average. When the system crashes, instead of dropping all the data due to malicious attacks, cc-NVM is able to detect and locate the exact tampered data while only incurring extra write traffic by 29.6% on average.

RDMA provides extremely low latency and high bandwidth to distributed systems. Unfortunately, it fails to scale and suffers from performance degradation when transferring data to an increasing number of targets on Reliable Connection (RC). We observe that the above scalability issue has its root in the resource contention in the NIC cache, the CPU cache and the memory of each server. In this paper, we propose ScaleRPC, an efficient RPC primitive using one-sided RDMA verbs on reliable connection to provide scalable performance. To effectively alleviate the resource contention, ScaleRPC introduces 1) connection grouping to organize the network connections into groups, so as to balance the saturation and thrashing of the NIC cache; 2) virtualized mapping to enable a single message pool to be shared by different groups of connections, which reduces CPU cache misses and improve memory utilization. Such scalable connection management provides substantial performance benefits: By deploying ScaleRPC both in a distributed file system and a distributed transactional system, we observe that it achieves high scalability and respectively improves performance by up to 90% and 160% for metadata accessing and SmallBank transaction processing.

Buffer caching is an effective approach to improve the system performance and extend the lifetime of SSDs. However, the frequent synchronization operations in most real-world applications limit such advantages. This paper proposes to adopt emerging non-volatile main memories (NVMMs) to relieve the above problems while achieving both efficient and consistent cache management. To this end, an adaptive fine-grained cache (AFCM) scheme is proposed, which is motivated by our observation that the file data in many synchronized pages is partially updated for a wide range of workloads, implying that fine-grained cache management can save the NVMM cache space wasted by the clean parts. To reduce the cache index overhead introduced by fine-grained cache management, AFCM employs a Hybrid Cache based on DRAM and NVMM, with which the normal read and write operations are served without performance penalty. We also propose the Transactional Copy-on-Write mechanism to guarantee the crash consistency of both NVMM cache space and file system image. Our experimental results show that AFCM provides up to 84 percent performance improvement and 63 percent SSD write reduction on average compared to the conventional coarse-grained cache management scheme.

No abstract available.

Persistent memory provides data persistence at main memory with emerging non-volatile main memories (NVMMs). Recent persistent memory file systems aggressively use direct access , which directly copy data between user buffer and the storage layer, to avoid the double-copy overheads through the OS page cache. However, we observe they all suffer from slow writes due to NVMMs’ asymmetric read-write performance and much slower performance than DRAM. In this article, we propose HiNFS, a high-performance file system for non-volatile main memory, to combine both buffering and direct access for fine-grained file system operations. HiNFS uses an NVMM-aware Write Buffer to buffer the lazy-persistent file writes in DRAM, while performing direct access to NVMM for eager-persistent file writes. It directly reads file data from both DRAM and NVMM, by ensuring read consistency with a combination of the DRAM Block Index and Cacheline Bitmap to track the latest data between DRAM and NVMM. HiNFS also employs a Buffer Benefit Model to identify the eager-persistent file writes before issuing I/Os. Evaluations show that HiNFS significantly improves throughput by up to 184% and reduces execution time by up to 64%comparing with state-of-the-art persistent memory file systems PMFS and EXT4-DAX.