In this paper, we demonstrate how Hyperledger Fabric, one of the most popular permissioned blockchains, can benefit from network-attached acceleration. The scalability and peak performance of Fabric is primarily limited by the bottlenecks present in its block validation/commit phase. We propose Blockchain Machine, a hardware accelerator coupled with a hardware-friendly communication protocol, to act as the validator peer. It can be adapted to applications and their smart contracts, and is targeted for a server with network-attached FPGA acceleration card. The Blockchain Machine retrieves blocks and transactions in hardware directly from the network interface, which are then validated through a configurable and efficient block-level and transaction-level pipeline. The validation results are then transferred to the host CPU where non-bottleneck operations are executed. From our implementation integrated with Fabric v1.4 LTS, we observed up to 12× speedup in block validation when compared to software-only validator peer, with commit throughput of up to 68,900 tps. Our work provides an acceleration platform that will foster further research on hardware acceleration of permissioned blockchains.

P4 has emerged as the de facto standard language for describing how network packets should be processed, and is becoming widely used by network owners, systems developers, researchers and in the classroom. The goal of the work presented here is to make it easier for engineers, researchers and students to learn how to program using P4, and to build prototypes running on real hardware. Our target is the NetFPGA SUME platform, a 4x10 Gb/s PCIe card designed for use in universities for teaching and research. Until now, NetFPGA users have needed to learn an HDL such as Verilog or VHDL, making it off limits to many software developers and students. Therefore, we developed the P4->NetFPGA workflow, allowing developers to describe how packets are to be processed in the high-level P4 language, then compile their P4 programs to run at line rate on the NetFPGA SUME board. The P4->NetFPGA workflow is built upon the Xilinx P4-SDNet compiler and the NetFPGA SUME open source code base. In this paper, we provide an overview of the P4 programming language and describe the P4->NetFPGA workflow. We also describe how the workflow is being used by the P4 community to build research prototypes, and to teach how network systems are built by providing students with hands-on experience working with real hardware.

The rise of programmable network devices and the P4 programming language has sparked an interest in developing new applications for packet processing data planes. Current data-plane programming models allow developers to express packet processing on a synchronous packet-by-packet basis, motivated by the goal of line rate processing in feed-forward pipelines. But some important data-plane operations do not naturally fit into this programming model. Sometimes we want to perform periodic tasks, or update the same state variables multiple times, or base a decision on state sitting at a different pipeline stage. While a P4-programmable device might contain special features to handle these tasks, such as packet generators and recirculation paths, there is currently no clean and consistent way to expose them to P4 programmers. We therefore propose a common, general way to express event processing using the P4 language, beyond just processing packet arrival and departure events. We believe that this more general notion of event processing can be supported without sacrificing line rate packet processing and we have developed a prototype event-driven architecture on the NetFPGA SUME platform to serve as an initial proof of concept.

Blockchain technologies are on the rise, and Hyperledger Fabric is one of the most popular permissioned blockchain platforms. In this paper, we re-architect the validation phase of Fabric based on our analysis from fine-grained breakdown of the validation phase's latency. Our optimized validation phase uses a chaincode cache during validation of transactions, initiates state database reads in parallel with validation of transactions, and writes to the ledger and databases in parallel. Our experiments reveal performance improvements of 2x for CouchDB and 1.3x for LevelDB. Notably, our optimizations can be adopted in a future release of Hyperledger Fabric.

The latest FPGA devices provide the headroom to implement large-scale and complex systems. A key requirement is the integration of modules from diverse sources to promote modular design and reuse. A contrary factor is that using dynamic partial reconfiguration typically requires low-level planning of the system implementation. In this article, we introduce ReShape: a high-level approach for designing reconfigurable systems by interconnecting modules, which gives a “plug and play” look and feel, is supported by tools that carry out implementation functions, and is carried through to support system reconfiguration during operation. The emphasis is on the inter-module connections and abstracting the communication patterns that are typical between modules: for example, the streaming of data, or the reading and writing of data to and from memory modules. The details of wiring and signaling are hidden from view, via metadata associated with individual modules. This setting allows system reconfiguration at the module level, both by supporting type checking of replacement modules and by managing the overall system implementation, via metadata associated with its FPGA floorplan. The methodology and tools have been implemented in a prototype targeted to a domain-specific setting---high-speed networking---and have been validated on real telecommunications design projects.

Software Defined Networking (SDN) has been described as the hope and hype for the future of networking. Definitions vary, but one research direction is to separate the control plane from the data plane, introducing abstractions that can provide a global network view, a description of required behavior, and a model of packet forwarding. While the worthy goal is to address ossification of the Internet, the "S" for "software" in SDN perhaps unintentionally ossifies views of the respective roles of hardware and software. Specifically, it introduces an inbuilt assumption that there is relatively dumb switching hardware for high-speed packet forwarding, and relatively intelligent software running on processors for lesser-speed networking control. Programmable logic technology offers scope for 'soft hardware', with the potential to blur the distinctions between traditional roles. However, such technology must prove both its ability to deliver the necessary high performance and its ability to be programmed in a high-level manner. This talk will overview research that has been addressing these issues successfully, and will discuss its potential impact on the evolving view of SDN.

The evolution of the Internet has transformed the simplistic Ethernet/IP based packet forwarding into a complex collection of lookup schemes. Depending on the location of the networking equipment (core, provider/customer edge, etc.,) a router/switch may potentially have to support several such complex lookup schemes. However, the hardware resources allocated to perform such operations are limited, especially in single chip implementations. In this paper, we propose techniques to map such complex lookup schemes on to hardware platforms under a limited resource budget and produce a design for a pipelined packet lookup architecture. An Integer Linear Programming (ILP) based technique is introduced to optimally allocate the limited hardware resources for a single lookup scheme. We extend our solution to multiple lookup schemes by proposing techniques to improve resource sharing, which results in a resource planning tool. Field Programmable Gate Array (FPGA) - a natural choice for high-speed packet processing applications - is used as the target platform. By using the proposed techniques, we show that up to 4 complex lookup schemes can be hosted on a single FPGA consuming only 20 Mbit on-chip memory and 750 pins for external memory communication.

Packet parsing is necessary at all points in the modern networking infrastructure, to support packet classification and security functions, as well as for protocol implementation. Increasingly high line rates call for advanced hardware packet processing solutions, while increasing rates of change call for high-level programmability of these solutions. This paper presents an approach for harnessing modern Field Programmable Gate Array (FPGA) devices, which are a natural technology for implementing the necessary high-speed programmable packet processing. The paper introduces PP: a simple high-level language for describing packet parsing algorithms in an implementation-independent manner. It demonstrates that this language can be compiled to give high-speed FPGA-based packet parsers that can be integrated alongside other packet processing components to build network nodes. Compilation involves generating virtual processing architectures tailored to specific packet parsing requirements. Scalability of these architectures allows parsing at line rates from 1 to 400 Gb/s as required in different network contexts. Run-time programmability of these architectures allows dynamic updating of parsing algorithms during operation in the field. Implementation results show that programmable packet parsing of 600 million small packets per second can be supported on a single Xilinx Virtex-7 FPGA device handling a 400 Gb/s line rate.

The latest generation of FPGA devices offers huge resource counts that provide the headroom to implement large-scale and complex systems. However, this poses increasing challenges for the designer, not just because of pure size and complexity, but also to harness effectively the flexibility and programmability of the FPGA. A central issue is the need to integrate modules (IP blocks) from diverse sources to promote modular design and reuse. In this paper, we introduce ShapeUp: a high-level approach for designing systems by interconnecting modules, which gives a `plug and play' look and feel to the designer and is supported by tools that carry out implementation and verification functions. The emphasis is on the inter-module connections and abstracting the communication patterns that are typical between modules - for example, the streaming of data that is common in many FPGA based DSP or networking systems, or the reading and writing of data to and from memory modules. The details of wiring and signaling are hidden from view, via metadata associated with individual modules. The ShapeUp tool suite includes an implementation capability that automatically generates wiring between blocks, possibly including additional bridging blocks, and a simulation capability that allows multi-level verification of systems of interconnected modules. The methodology and tools have been validated on Xilinx customer design projects.

Field programmable logic is increasingly used to provide the high performance and flexible acceleration needed for network processing functions at multiple gigabit/second rates. Almost all such functions feature the use of clocks and timers in control and/or data roles, and these are typically implemented in an ad hoc manner. This paper introduces a set of three configurable timing modules that are based on abstractions of the prevalent timing paradigms observed in network protocols. The modules fit within the experimental ShapeUp methodology for modular FPGA-based system design, and so can be easily integrated with other modules that are tailored for specific networking functions. The use and benefits of the new modular approach are demonstrated by an example of a flexible FPGA reference design that has been made available for real-life use by telecommunication equipment providers.

The packet is the atom of the digital revolution: the unit of data communication. The use of packets in networking was first proposed almost 50 years ago, leading ultimately to the Internet as we know it today. As networking has scaled down towards networks on chip, so packets feature for digital communication in the small. As applications have gone digital, so their data has become packetised. Streams of packets, and the processing of these packets, are characteristic of the digital age. It is becoming increasingly necessary to provide numerous packet processing solutions, in order to support diversity and innovation both in services and in underlying infrastructure. In turn, there is a requirement for supplying underlying high-performance architectures that support flexibility, scalability, concurrency, and diversity. These characteristics present a great opportunity for modern field programmable technologies, which offer a very natural basis for providing the necessary high-speed programmable packet processing capabilities. One major barrier for these technologies to overcome is that design, testing and maintenance is very often a low-level, hardware-oriented, experience, requiring specialist skills. A key question is whether the raw capabilities of field programmable devices can be made available to the architect and programmer in a higher-level manner, comparable to best practice in modern software engineering. To supply an initial positive answer, this paper introduces G, a high-level packet-centric language for describing packet processing specifications in an implementation-independent manner, and demonstrates that G can be compiled to give high-performance FPGA-based packet processing components. Compilation involves generating virtual processing architectures tailored to the particular requirements of a G specification. The results affirm the FPGA as a natural packet processing technology that can be programmed in a high-level manner appropriate to a packet-oriented mindset.

Concurrent programs are difficult to write, reason about, re-use, and maintain. In particular, for system-level descriptions that use a shared memory abstraction for thread or process synchronization, the current practice involves manual scheduling of processes, introduction of guard conditions, and clocking tricks, to enforce memory dependencies. This process is tedious, time consuming, and error-prone. At the same time, the need for a concurrent programming model is becoming ever essential to bridge the productivity gap that is widening with every manufacturing process generation. In this paper, we present two novel techniques to automatically enforce memory dependencies in platform FPGAs using on-chip memories, starting from a system-level description. Both the techniques utilize static analysis to generate circuits for enforcing these dependencies. This paper investigates these two techniques for their generality, overhead in implementation, and usefulness or otherwise for different application requirements

This paper considers the elaboration of custom pipelines for network packet processing, built upon flexible programmability of pipeline stage granularity. A systematic procedure for accurately characterizing throughput, latency, and FPGA resource requirements, of different programmed pipeline variants is presented. This procedure may be exploited at design time, configuration time, or run time, to program pipeline architectures to meet specific networking application requirements. The procedure is illustrated using three case studies drawn from real-life packet processing at different levels of networking protocol. Detailed results are presented, demonstrating that the procedure estimates pipeline characteristics well, thus allowing rapid architecture space exploration prior to elaboration

A fundamental challenge to successful deployment of DRAMs is the availability of a flexible and scalable DRAM interface. This is exacerbated by the application specific nature of the logic-side DRAM interface. This paper presents a study that attempts to overcome this challenge for networking application domain. We quantify the various challenges and present techniques that were implemented to build a flexible and scalable interface to an existing multi-port memory controller for DDR DRAM using a FPGA. We demonstrate the deployment of this new interface in two example applications. We present two novel techniques that enable us to reduce the latency of DRAM related memory accesses and improve throughput. We believe our techniques enable harnessing maximum throughput from existing memory controllers with least possible latency.

Domain-specific design flows can enable an efficient path to implementation, as well as making the design process intuitive and the designs reusable. When targeting FPGAs, there are few techniques in high level synthesis that enable thorough exploration of the inherent flexibility of the FPGA fabric as an implementation medium. In this paper, we propose a new methodology, based on micro-coded datapaths, that enables design space exploration of processing engine architectures implemented in programmable logic that range from a fixed finite state machine to a soft processor. As a use case, these processing engines can be embedded within programmable logic threads that are used to carry out network packet processing. We demonstrate the application of this methodology on a network address translation application, and show that micro-coded data paths indeed enable both human designers and automated tools to explore the design space in a structured way, thus exploiting the full potential of the FPGA technology.

This paper addresses exploitation of the capabilities of platform FPGAs to implement embedded networking for systems on chip. In particular, a methodology for exploring trade-offs between the placement of protocol handling functions in programmable logic and on an embedded processor is demonstrated. This is facilitated by two new design tool capabilities: first, being able to describe programmable logic based functions in a more software-like manner; and second, being able automatically to generate efficient interfaces between a programmable logic fabric and an embedded processor. The methodology is illustrated by an example of a simple Web server, targeted at Xilinx Virtex-II Pro and Virtex-4 platform FPGAs. Trade-offs both of complete protocol placement and of within-protocol placement are systematically investigated in terms of resources used and packet handling latency. The work points the way to highly fluid allocation of functions to implementations, beyond conventional static codesign.

This paper addresses exploitation of the capabilities of platform FPGAs to implement embedded networking for systems on chip. In particular, a methodology for exploring trade-offs between the placement of protocol handling functions in programmable logic and on an embedded processor is demonstrated. This is facilitated by two new design tool capabilities: first, being able to describe programmable logic based functions in a more software-like manner; and second, being able automatically to generate efficient interfaces between a programmable logic fabric and an embedded processor. The methodology is illustrated by an example of a simple web server, targeted at Xilinx Virtex-II Pro or Virtex-4 FX platform FPGAs. Trade-offs both of complete protocol placement and of within-protocol placement are systematically investigated in terms of resources used and packet handling latency. This provides an excellent range of service times, corresponding to differing logic fabric and memory resource requirements. The work points the way to highly fluid allocation of functions to implementations, beyond conventional static codesign.

A domain specific language (DSL) enables designers to rapidly specify and implement systems for a particular domain, yielding designs that are easy to understand, reason about, re-use and maintain. However, there is usually a significant overhead in the required infrastructure to map such a DSL on to a programmable logic device. In this paper, we present a mapping of an existing DSL for the networking domain on to a platform FPGA by embedding the DSL into an existing language infrastructure. In particular, we will show that, using few basic concepts, we are able to achieve a successful mapping of the DSL on to a platform FPGA and create a re-usable structure that also makes it easy to extend the DSL. Finally we will present some results of mapping the DSL on to a platform FPGA and comment on the resulting overhead.

In this paper, we explore two important latency issues associated with using an embedded processor as an assistant to programmable logic within a logic-centric system implemented on a platform FPGA. The context is that of the 'software decelerator' - a term introduced by the authors in 2003 to describe a logic-centric counterpart of the familiar hardware accelerator. We first focus on minimizing latency in the logic-processor interface, introducing an efficient interrupt-driven control mechanism. Then, in the context of a case study on packet address lookup, we focus on minimizing latency in memory interfaces, using the processor's hardware cache mechanism for assistance.

Modern programmable logic devices have capabilities that are well suited for them to assume a central role in the holistic implementation of networked systems. We have devised a highly flexible soft platform architecture abstracted from such physical devices, which may be viewed as a particularly configurable and programmable type of network processor. In this paper, we discuss a programming model for the architecture, and present an XML-based description language for expressing the programming information. This intermediate language is designed both to be an attractive compilation target for domain-specific languages used for describing networking applications, and also to have efficient mappings to programmable logic devices, harnessing to the full their high degree of concurrency, interconnectivity and programmability. We present a detailed example, where a high-speed remote procedure call (RPC) protocol server for gigabit Ethernet was described directly in the XML-based language, and automatically compiled to a working implementation on a platform FPGA device. The exercise was carried out by a non-hardware expert in only two weeks, thus demonstrating the unlocking of access to programmable logic technology.

Reconfigurable system-on-chip (SoC) platforms are now a physical reality. This gives a system substrate which, physically at least, is fairly neutral with respect to use models and system architectures. When embedded processors are present, the most obvious use model to follow is to consider the system on chip as a scaled-down version of a system on board: we call this a processor-centric model. To provide better access to many of the benefits of the new SoC devices, we introduce an alternative logic-centric model, where the environment of a system is the driving force behind its architecture. In particular, the role of embedded processors is to assist the majority processing being carried out in logic and in input/output interfaces. We see such 'eccentric' architectures as the norm for the future, particularly given the vision of the 'disappearing computer' and the rise of 'ambient intelligence'. We illustrate the general discussion with examples drawn from our current research into systems designed for message processing.

This paper is concerned with networking at the chip level. Networks on chip have become a convenient focus for discussing the architecture of systems on chip, and design methodologies for such systems. One central question for such a focus concerns the extent to which it is useful or realistic just to scale down approaches used conventionally in larger-area computer networking. Here, we discuss this question, in the context of the modern Platform FPGA device as a system on chip substrate. We illustrate the discussion with two design examples of networking on chip being implemented in an unconventional way.

This paper concerns novel single-chip system architecture options, based on the Xilinx Virtex-II Pro part, which includes up to four PowerPC cores and was launched in Spring 2002. The research described here was carried out pre-launch (i.e., prior to availability of real parts), so the paper focuses on initial architectural experiments based on simulation. The application is a Mixed-version IP Router, named MIR, servicing gigabit ethernet ports. This would be of use to organizations with several gigabit ethernets, with a mixture of IPv4 and IPv6 hosts and routers attached directly to the networks. A particular benefit of a programmable approach based on Virtex-II Pro is that the router's functions can evolve smoothly, maintaining router performance as the organization migrates from IPv4 to IPv6 internally, and also as the Internet migrates externally. The basic aim is to carry out more frequent, and less control intensive, functions in logic, and other functions in the processor. Two prototypes are described here. Both support four ethernet ports, but the designs are scalable upwards. The second one, the more ambitious of the two, instantiates a configuration appropriate when the bulk of the incoming packets are IPv4. Such packets are processed and switched entirely by logic, with no internal copying of packets between buffers and virtually no delay between packet receipt and onward forwarding. This involves a specially-tailored internal interconnection network between the four ports, and also processing performed in parallel with packet receipt, i.e. multi-threading in logic. IPv6 packets, or some rare IPv4 cases, are passed to a PowerPC core for processing. In essence, the PowerPC acts as a slave to the logic, rather than the more common opposite master-slave relationship.

Custom computing is concerned with deriving benefits from importing the flexibility of (software) programs into (conventionally, hardware) circuitry. A main aim is to gain speed-ups by programming at a level closer to the physical hardware, and in a medium that allows explicit parallelism. The paper is concerned with making the first steps towards another benefit: portability of circuitry in a network computing environment. That is, expressing applets in circuitry terms, rather than program terms. An implemented client server system is described, and then this is followed by discussion of how circlets-a term introduced in the paper to denote applets expressed as circuits-might best be represented in a portable form, and also how circlets may be supported by appropriate execution environments. At this point in history, it is apt to study the problems posed by circlets, since this should influence future directions for FPGA technological development.

Swappable Logic Units (SLUs) were introduced by the author previously (1996) to play a role in virtual hardware subsystems that is analogous to the role of pages or segments in virtual memory subsystems. The intention is that a conventional operating system can be extended to manage SLU circuitry implemented using FPGA real estate. In order to minimise operating system overheads, two particular SLU-based virtual hardware models were deemed practical: a "sea of accelerators" model and a "parallel harness" model. This paper looks in some detail at how SLUs will fit within the overall environment of a fairly conventional hardware/software system. First, there is a discussion of the FPGA-based hardware environment for SLUs, followed by a discussion of the software environment from which SLUs might be used. After this, there is a description of the operational properties that SLUs can have, and how these fit in with the two virtual hardware models. Finally, proposals for standard interfaces between SLUs and their environment are discussed. These interfaces can be regarded as constraints on the designers of SLU circuitry or, more positively, as suppliers of an enriched context within which such circuitry operates. The overall impact of the work presented in the paper is to show that it is feasible to incorporate configurable hardware within traditional computer systems that use high-level language programs and computer operating systems. That is, it should not always be necessary to devise special-purpose hardware/software systems to realise custom computing.

Abstract The networking software for a VAX/VMS computer system had been implemented as a collection of communicating processes. One night, an unusually high load on the electronic mail component of the software caused deadlock to occur between two of the processes. This paper describes how the deadlock was analysed by modelling the software using the Calculus of Communicating Systems (CCS) and then by investigating the behaviour of the model using the Edinburgh Concurrency Workbench (CWB). The analysis suggested how the software should be restructured to prevent the problem recurring; the new set of processes was analysed, and shown to be deadlock-free.