In this work, we solve a long-standing open problem in coding theory with broad applications in networking and systems: designing an erasure code that simultaneously satisfies three requirements: (1) high coding efficiency, (2) low coding complexity, and (3) being a streaming code (defined as one with low decoding latency). We propose METTLE (Multi-Edge Type with Touch-less Leading Edge), the first erasure code to meet all three requirements. Compared to "streaming RaptorQ" (RaptorQ configured with a small source block size to ensure a low decoding latency), METTLE is only slightly worse in coding efficiency, but 47.7 to 84.6 times faster to decode.

Set reconciliation (SetR) is an important research problem that has been studied for over two decades. In this problem, two large sets A and B of objects (tokens, files, records, etc.) are stored respectively at two different network-connected hosts, which we name Alice and Bob respectively. Alice and Bob need to communicate with each other to learn the set union A ∪ B (which then becomes their reconciled state), at low communication and computation costs. In this work, we study a different problem intricately related to SetR: Alice and Bob collaboratively compute A ∩ B . We call this problem SetX (set intersection). Although SetX is just as important as SetR, it has never been properly studied in its own right. Rather, there is an unspoken perception by the research community that SetR and SetX are equally difficult (in costs), and hence “roughly equivalent.” Our first contribution is to show that SetX is fundamentally a much “cheaper” problem than SetR, debunking this long-standing perception. Our second contribution is to develop a novel SetX solution, the communication cost of which handily beats the information-theoretic lower bound of SetR. This protocol is based on the idea of compressed sensing (CS), which we describe here only for the special case of A ⊆ B (We do have a more sophisticated protocol for the general case). Our protocol is for Alice to encode A into a CS sketch M 1 A and send it to Bob, where M is a CS matrix with l rows and 1 A is the binary vector representation of A . Our key innovation here is to make l (the sketch size) just large enough (for the sketch) to summarize B ∖ A (what Alice misses). In contrast, in existing protocols l needs to be large enough to summarize A (what Alice knows), which is typically much larger in cardinality. Our third contribution is to design a CS matrix M that is both “friendly” to (the performance of) applications and “compliant” with CS theory.

Data Shapley provides a principled approach to data valuation and plays a crucial role in data-centric machine learning (ML) research. Data selection is considered a standard application of Data Shapley. However, its data selection performance has shown to be inconsistent across settings in the literature. This study aims to deepen our understanding of this phenomenon. We introduce a hypothesis testing framework and show that Data Shapley’s performance can be no better than random selection without specific constraints on utility functions. We identify a class of utility functions, monotonically transformed modular functions, within which Data Shapley optimally selects data. Based on this insight, we propose a heuristic for predicting Data Shapley’s effectiveness in data selection tasks. Our experiments corroborate these findings, adding new insights into when Data Shapley may or may not succeed.

Today, blockchain ledgers utilize concurrent deterministic execution schemes to scale up. However, ordering fairness is not preserved in these schemes: although they ensure all replicas achieve the same serial order, this order does not always align with the fair, consensus-established order when executing smart contracts with runtime-determined accesses. To preserve ordering fairness, an intuitive method is to concurrently execute transactions and re-execute any order-violating ones. This in turn increases unforeseen conflicts, leading to scaling bottlenecks caused by numerous costly aborts under contention. To address these issues, we propose Spectrum, a novel deterministic execution scheme for smart contract execution on blockchain ledgers. Spectrum preserves the consensus-established serial order (so-called strict determinism) with high performance. Specifically, we leverage a speculative deterministic concurrency control to execute transactions in speculation and enforce an agreed-upon serial order by aborting and re-executing any mis-speculated ones. To overcome the scaling bottleneck, we present two key optimizations based on speculative processing: operation-level rollback and predictive scheduling, for reducing both the overhead and the number of mis-speculations. We evaluate Spectrum by executing EVM-based smart contracts on popular benchmarks, showing that it realizes fair smart contract execution by preserving ordering fairness and outperforms competitive schemes in contended workloads by 1.4x to 4.1x.