This paper presents the first hardware implementation of bittide, a decentralized clock synchronization mechanism for achieving logical synchrony in distributed systems. We de-tail the design and implementation of an 8-node bittide network using off-the-shelf FPGA boards and adjustable clock sources. Through experiments with various network topologies, including fully connected, hourglass, and cube, we demonstrate the ef-fectiveness of bittide in aligning node frequencies and bounding buffer excursions. We collect and analyze frequency, buffer oc-cupancy, and logical latency data, validating the hardware's per-formance against theoretical predictions and simulations. Our results show that bittide achieves tight frequency alignment, ro-bustly handles varying physical latencies, and establishes a con-sistent notion of logical time across the network, enabling pre-dictable distributed computation at scale with zero in-band over-head.

We introduce logical synchrony, a framework that allows distributed computing to be coordinated as tightly as in synchronous systems without the distribution of a global clock or any reference to universal time. We develop a model of events called a logical synchrony network, in which nodes correspond to processors and every node has an associated local clock which generates the events. We construct a measure of logical latency and develop its properties. A further model, called a multiclock network, is then analyzed and shown to be a refinement of the logical synchrony network. We present the bittide mechanism as an instantiation of multiclock networks, and discuss the clock control mechanism that ensures that buffers do not overflow or underflow. Finally we give conditions under which a logical synchrony network has an equivalent synchronous realization.

We discuss distributed reframing control of bittide systems. In a bittide system, multiple processors synchronize by monitoring communication over the network. Processors remain in logical synchrony by controlling the timing of frame transmissions. The protocol for doing this relies upon an underlying dynamic control system where each node makes only local observations and performs no direct coordination with other nodes. In this paper we develop a control algorithm based on the idea of buffer centering, which allows all nodes to maintain small buffer offsets while also requiring very little state information. We demonstrate that with buffer centering we can achieve separate control of frequency and phase, allowing frequencies to be syntonized and also buffers to maintain desired offsets rather than combining their control via a proportional-integral controller. The minimalism of this approach offers the potential to simplify both boot processes and failure handling.

In Software Defined Networking (SDN) the control plane is physically separate from the forwarding plane. Control software programs the forwarding plane (e.g., switches and routers) using an open interface, such as OpenFlow. This paper aims to overcomes two limitations in current switching chips and the OpenFlow protocol: i) current hardware switches are quite rigid, allowing ``Match-Action'' processing on only a fixed set of fields, and ii) the OpenFlow specification only defines a limited repertoire of packet processing actions. We propose the RMT (reconfigurable match tables) model, a new RISC-inspired pipelined architecture for switching chips, and we identify the essential minimal set of action primitives to specify how headers are processed in hardware. RMT allows the forwarding plane to be changed in the field without modifying hardware. As in OpenFlow, the programmer can specify multiple match tables of arbitrary width and depth, subject only to an overall resource limit, with each table configurable for matching on arbitrary fields. However, RMT allows the programmer to modify all header fields much more comprehensively than in OpenFlow. Our paper describes the design of a 64 port by 10 Gb/s switch chip implementing the RMT model. Our concrete design demonstrates, contrary to concerns within the community, that flexible OpenFlow hardware switch implementations are feasible at almost no additional cost or power.