Present-day Internet protocol routers typically employ monolithic operating systems that are not easily upgradable and extensible. With the rapid rate of protocol development it is becoming increasingly important to dynamically upgrade router software in an incremental fashion. We have designed and implemented a high-performance, modular, extended services router software architecture in the Net BSD operating system kernel. This architecture allows code modules, called plugins, to be dynamically added and configured at run time. One of the novel features of our design is the ability to bind different plugins to individual flows; this allows for distinct plugin implementations to seamlessly coexist in the same runtime environment. We achieve high performance through a carefully designed modular architecture, an innovative packet classification algorithm that is highly efficient, and by caching that exploits the flow-like characteristics of Internet traffic. Compared to a monolithic best effort kernel, our implementation requires an average increase in packet processing overhead of only 8%, or 600 cycles per packet when running on an Intel Pentium Pro at 233 MHz. By shortcutting the forward loop based on the per-flow state we establish, we can forward packets up to three times faster than the best effort kernel.

Present day routers typically employ monolithic operating systems which are not easily upgradable and extensible. With the rapid rate of protocol development it is becoming increasingly important to dynamically upgrade router software in an incremental fashion. We have designed and implemented a high performance, modular, extended integrated services router software architecture in the NetBSD operating system kernel. This architecture allows code modules, called plugins, to be dynamically added and configured at run time. One of the novel features of our design is the ability to bind different plugins to individual flows; this allows for distinct plugin implementations to seamlessly coexist in the same runtime environment. High performance is achieved through a carefully designed modular architecture; an innovative packet classification algorithm that is both powerful and highly efficient; and by caching that exploits the flow-like characteristics of Internet traffic. Compared to a monolithic best-effort kernel, our implementation requires an average increase in packet processing overhead of only 8%, or 500 cycles/2.1ms per packet when running on a P6/233.

We are building a high performance 1.2 Gb/s ATM network interface chip called the APIC (ATM Port Interconnect Controller). In addition to borrowing useful ideas from a number of research and commercial prototypes, the APIC design embraces several innovative features, and integrates all of these pieces into a coherent whole. Some of the novel ideas incorporated in the APIC design include: protected DMA and protected I/O, which allow applications to queue data for transmission or reception directly from user-space, effectively bypassing the kernel. This argues for moving the entire protocol stack including the interface device driver into the user-space, thereby yielding better latency and throughput performance than kernel-resident implementations. Pool DMA when used with packet splitting, is a technique that can be used to build true zero-copy kernel-resident protocol stack implementations, using a page-remapping technique. Finally, orchestrated interrupts and interrupt demultiplexing are mechanisms used to reduce the frequency of interrupts issued by the APIC. Although many of these ideas have been developed in the context of an ATM network interface, we believe they are also applicable in other contexts. In particular, protected DMA and I/O are promising techniques for improving the performance of several different types of I/O devices.

We present the design of a high performance ATM host-network interface for multimedia workstations and servers. At Washington University, as part of an ARPA-sponsored gigabit local ATM testbed, we are building a prototype of this interface that can support a sustained aggregate bidirectional data rate of 2.4 Gbps. The centerpiece of our interface design is a custom chip called the APIC (ATM port interconnect controller). Multiple such chips can be interconnected to yield a desk-area network (DAN) which would serve as a high speed I/O interconnect for the host computer. This paper details the internal design of the APIC chip, and outlines some of its key features. Noteworthy among these are: connection caching, transmit pacing, cell batching, remote control, and support for AAL-0, AAL-5, multipoint, and loopback connections. We have chosen to defer to a later paper the details pertaining to several other features which provide support for zero-copy, improved interrupt handling, direct control of the chip from user-space, and efficient buffering and demultiplexing.