Getting the best performance from the ever-increasing number of hardware platforms has been a recurring challenge for data processing systems. In recent years, the advent of data science with its increasingly numerous and complex types of analytics has made this challenge even more difficult. In practice, system designers are overwhelmed by the number of combinations and typically implement a single analytics type on one platform, leading to repeated implementation effort---and a plethora of semi-compatible tools for data scientists.

This paper describes our effort to build extensible routers using a combination of general-purpose and network processors. We emphasize five overriding challenges that dictate our design decisions: (1) optimal resource allocation; (2) efficient but flexible scheduling of the CPU; (3) maintaining overall router robustness; (4) maximizing router performance; and (5) providing sufficient extensibility to enable the injection of new functionality into the router. We adopt a hierarchical architecture, in which packet flows traverse a range of processing/forwarding paths, thereby partitioning hardware and software in concert. This paper both presents the architecture, and describes our experiences implementing the architecture and addressing the five design challenges in a prototype built from Intel IXP 1200 and a Pentium. Copyright © 2005 John Wiley & Sons, Ltd.