Today's modern hardware, with manycore servers, RDMA-capable networks and trusted execution environments, challenges the conventional wisdom about CFT protocols' design. We explore the synergy between modern hardware and the security and performance of strongly consistent replication protocols. Specifically, can we leverage (and how) modern cloud hardware to harden the security properties of a CFT protocol for Byzantine settings while achieving high performance?

Modern applications employ several heterogeneous memory types for improved performance, security, and reliability. To manage them, programmers must currently digress from the traditional load/store interface and rely on various custom libraries specific to each memory type, thus introducing programmability, performance, portability, and protection challenges.

Cloud infrastructure is experiencing a shift towards disaggregated setups, especially with the introduction of the Compute Express Link (CXL) technology, where byte-addressable ersistent memory (PM) is becoming prominent. To fully utilize the potential of such devices, it is a necessity to access them through network stacks with equivalently high levels of performance (e.g., kernel-bypass, RDMA). While, these advancements are enabling the development of high-performance data management systems, their deployment on untrusted cloud environments also increases the security threats. To this end, we present Anchor, a library for building secure PM systems. Anchor provides strong hardware-assisted security properties, while ensuring crash consistency. Anchor exposes APIs for secure data management within the realms of the established PM programming model, targeting byte-addressable storage devices. Anchor leverages trusted execution environments (TEE) and extends their security properties on PM. While TEE's protected memory region provides a strong foundation for building secure systems, the key challenge is that: TEEs are fundamentally incompatible with PM and kernel-bypass networking approaches-in particular, TEEs are neither designed to protect untrusted non-volatile PM, nor the protected region can be accessed via an untrusted DMA connection. To overcome this challenge, we design a PM engine that ensures strong security properties for the PM data, using confidential and authenticated PM data structures, while preserving crash consistency through a secure logging protocol. We further extend the PM engine to provide remote PM data operations via a secure network stack and a formally verified remote attestation protocol to form an end-to-end system. Our evaluation shows that Anchor incurs reasonable overheads, while providing strong security properties.

Distributed transaction processing is a fundamental building block for large-scale data management in the cloud. Given the threats of security violations in untrusted cloud environments, our work focuses on: How to design a distributed transactional KV store that achieves high-performance serializable transactions, while providing strong security properties?We introduce Treaty, a secure distributed transactional KV storage system that supports serializable ACID transactions while guaranteeing strong security properties: confidentiality, integrity, and freshness. Treaty leverages trusted execution environments (TEEs) to bootstrap its security properties, but it extends the trust provided by the limited enclave (volatile) memory region within a single node to build a secure (stateful) distributed transactional KV store over the untrusted storage, network and machines. To achieve this, Treaty embodies a secure two-phase commit protocol co-designed with a high-performance network library for TEEs. Further, Treaty ensures secure and crash-consistent persistency of committed transactions using a stabilization protocol. Our evaluation on a real hardware testbed based on the YCSB and TPC-C benchmarks shows that Treaty incurs reasonable overheads, while achieving strong security properties.

We introduce TEE-PERF, an architecture-and platform-independent performance measurement tool for trusted execution environments (TEEs). More specifically, TEE-PERF supports method-level profiling for unmodified multithreaded applications, without relying on any architecture-specific hardware features (e.g. Intel VTune Amplifier), or without requiring platform-dependent kernel features (e.g. Linux perf). Moreover, TEE-PERF provides accurate profiling measurements since it traces the entire process execution without employing instruction pointer sampling. Thus, TEE-PERF does not suffer from sampling frequency bias, which can occur with threads scheduled to align to the sampling frequency. We have implemented TEE-P ERF with an easy to use interface, and integrated it with Flame Graphs to visualize the performance bottlenecks. We have evaluated TEE-PERF based on the Phoenix multithreaded benchmark suite and real-world applications (RocksDB, SPDK, etc.), and compared it with Linux perf. Our experimental evaluation shows that TEE-PERF incurs low profiling overheads, while providing accurate profile measurements to identify and optimize the application bottlenecks in the context of TEEs. TEE-PERF is publicly available.