Cyber-Physical Systems (CPSes) use computers to control and monitor physical processes in domains such as aviation, industrial automation, transportation, communication, water distribution, waste treatment, and power systems. These systems are increasingly connected with both private and public networks, making them susceptible to disruption or destruction by adversaries anywhere in the world. Attacks such as the Stuxnet worm have highlighted CPS vulnerabilities and motivated the development of CPS-specific defenses. The lab modules described in this paper arose from a project initially focused on technology to counter a Stuxnet-class attack on a CPS, and the suitability of this technology's commercial off-the-shelf components for undergraduate education was later realized. CPS security is not given much attention in undergraduate programs because networks and control systems are traditionally distinct areas of study. We have developed hands-on courseware to show how an adversary can compromise a CPS in a Stuxnet-like manner by tunneling through a layered sequence of network protocol and interface weaknesses. The courseware incorporates cyber-physical security concepts into effective learning modules that highlight real-world technical issues. This modular learning approach helps students understand CPS architectures and their vulnerabilities to cyber-attacks via experiential learning, and acquire practical skills through actively participating in the hands-on exercises. This courseware uses an economical and easily replicable hardware testbed, making this experience available as an adjunct to conventional embedded system, control system design, or cybersecurity courses. To assess the impact of this courseware, an evaluation survey is developed to measure the understanding of the unique aspects of CPS security. These modules leverage existing academic subjects, help students understand the sequence of steps taken by adversaries, and serve to bridge theory and practice.

We mitigate malicious software threats to industrial control systems, not by bolstering perimeter security, but rather by using application-specific configurable hardware to monitor and possibly override software operations in real time at the lowest (I/O pin) level of a system-on-chip platform containing a micro controller augmented with configurable logic. The process specifications, stability-preserving backup controller, and switchover logic are specified and formally verified as C code commonly used in control systems, but synthesized into hardware to resist software reconfiguration attacks. In addition, a copy of the production controller task is optionally implemented in an on-chip, isolated soft processor, connected to a model of the physical process, and accelerated to preview what the controller will attempt to do in the near future. This prediction provides greater assurance that the backup controller can be invoked before the physical process becomes unstable. Adding trusted, application-tailored, software-invisible, autonomic hardware is well-supported in a commercial system-on-chip platform.

Industrial control systems (ICSes) have the conflicting requirements of security and network access. In the event of large-scale hostilities, factories and infrastructure would more likely be targeted by computer viruses than the bomber squadrons used in WWII. ICS zero-day exploits are now a commodity sold on brokerages to interested parties including nations. We mitigate these threats not by bolstering perimeter security, but rather by assuming that potentially all layers of ICS software have already been compromised and are capable of launching a latent attack while reporting normal system status to human operators. In our approach, application-specific configurable hardware is the final authority for scrutinizing controller commands and process sensors, and can monitor and override operations at the lowest (I/O pin) level of a configurable system-on-chip platform. The process specifications, stability-preserving backup controller, and switchover logic are specified and formally verified as C code, and synthesized into hardware to resist software reconfiguration attacks. To provide greater assurance that the backup controller can be invoked before the physical process becomes unstable, copies of the production controller task and plant model are accelerated to preview the controller's behavior in the near future.

Design validation is the most time-consuming task in the FPGA design cycle. Although manufacturers and third-party vendors offer a range of tools that provide visibility and control of the different stages of a design, many require that the design be fully re-implemented for even simple parameter modifications or do not allow the design to be run at full speed. Designs are typically first modeled using a high-level language then later rewritten in a hardware description language, first for simulation and then later modified for synthesis. IP and third-party cores may differ during these final two stages complicating development and validation. The developed approach provides two means of directly validating synthesized hardware designs. The first allows the original high-level model written in C or C++ to be directly coupled to the synthesized hardware, abstracting away the traditional gate-level view of designs. A high-level programmatic interface allows the synthesized design to be validated directly by the software reference model. The second approach provides an alternative view to FPGAs within the scope of a traditional software debugger. This debug framework leverages partially reconfigurable regions to accelerate the modification of dynamic, software-like breakpoints for low-level analysis and provides a automatable, scriptable, command-line interface directly to a running design on an FPGA.

Security is difficult to achieve on general-purpose computing platforms due to their complexity, excess functionality, and resource sharing. An alternative is the creation of a Tailored Trustworthy Space for the system or application class of interest. We focus on data-intensive computing systems using reconfigurable hardware to implement streaming operations, and provide security assurances that are independent of application software, middleware, or operating system integrity and correctness. All interaction between software and the dataflow hardware passes through an automatically synthesized and formally verified hardware controller incorporating enforcement and real-time monitoring of application-specific rules. Abstractions provided by the Blue spec high-level language assist in the translation of domain-specific policy rules to synthesized logic. For the cognitive radio example used, hardware-enforced policies include physical layer rules such as sanctioned spectrum usage. Policy changes cause the secure generation and transfer of a new controller-wrapped datapath hardware plug-in. Datapath dynamic block swaps and cryptographic operations are managed entirely by the hardware controller rather than software drivers. Design for performance and design for security are therefore simultaneously addressed since the datapath is configured and monitored at hardware speeds, and software has no access to datapath configurations and cryptographic keys.

Rapidly increasing FPGA density and complexity has heightened the need for higher levels of abstraction in validation and more rapid, focused approaches for design inspection. We present two methods of validating and debugging active, implemented FPGA designs running at target speeds. The first binds high-level software reference models directly to hardware enabling complex, automated, software-controlled testing scenarios, reducing the reliance on simulation. The second approach provides direct interactivity and visibility into a running FPGA design, enabling software-controlled breakpoints and arbitrary access to design registers. In-circuit breakpoints can be modified without the need to re-implement the entire design.

FPGA implementation tool turnaround time has unfortunately not kept pace with FPGA density advances. It is difficult to parallelize place-and-route algorithms without sacrificing determinism or quality of results. As in many multithreaded applications, communication and synchronization incur significant overheads. Even if these challenges are overcome, the large graph data structures used can quickly exhaust memory bandwidth as more cores are employed. We approach the problem in a different way for development environments in which some circuit speed and area optimization may be sacrificed for improved implementation and debug turnaround. The PATIS automatic floorplanner enables dynamic modular design. While existing modular and incremental flows facilitate timing closure late in the design cycle by reusing the layout of unmodified blocks, dynamic modular design accelerates non-local changes to the physical layout arising from design exploration and the addition of debug circuitry. A PATIS floorplan consists of partial modules with structured physical interfaces observable through configuration readback, allowing module ports to be passively probed without disturbing the layout. Although PATIS supports incremental design, global changes are still rapid because each block's partial bitstream is produced by independent and concurrent invocations of the standard Xilinx tools running on separate cores or hosts. A continuous background task proactively generates floorplan variants to further accelerate layout changes.

Design validation is the most time-consuming task in the FPGA design cycle. Although manufacturers and third-party vendors offer a range of tools that provide different perspectives of a design, many require that the design be fully re-implemented for parameter modifications or do not allow the design to be run at full-speed. Designs are frequently first modeled using a high-level language then later rewritten in a hardware description language, first for simulation and then later modified for synthesis. IP and third-party cores may differ during these final two stages complicating development and validation. Our approach provides a means of directly validating synthesized hardware designs with the original high-level model, abstracting away the traditional bit-level view of designs. The Dynamic Modular Development (DMD) framework accelerates module turnaround by reusing floorplans and placing frequently changing modules into separate, partially reconfigurable regions. The debug framework also leverages these reconfigurable regions to accelerate the modification of dynamic, software-like breakpoints for low-level analysis.

The MicroBlaze architecture is a configurable "soft core" processor by Xilinx intended for use in embedded FPGA applications. Some embedded data processing algorithms benefit from the use of vectorized code and streaming I/O. Thus, configurable extensions that add these to the MicroBlaze architecture are developed. We discuss the current architecture, the necessary modifications, the methods employed, and analyze the theoretical gains. We then study the full synthesis of an example hardware platform with and without the extensions, comparing the expected performance of each. Following that, we discuss the work toward integrating with Xilinx's development tools, which allows simple and rapid deployment of the extended architecture. Modifications to the software toolchain are discussed as they pertain to extending the instruction set. Finally, we conclude with an assessment of the developed extensions and future work that could make them even more useful for reconfigurable, high-performance embedded applications.

More than a dozen Wireless @ Virginia Tech faculty are working to address the broad research agenda of cognitive radio and cognitive networks. Our core research team spans the protocol stack from radio and reconfigurable hardware to communications theory to the networking layer. Our work includes new analysis methods and the development of new software architectures and applications, in addition to work on the core concepts and architectures underlying cognitive radios and cognitive networks. This paper describes these contributions and points towards critical future work that remains to fulfill the promise of cognitive radio. We briefly describe the history of work on cognitive radios and networks at Virginia Tech and then discuss our contributions to the core cognitive processing underlying these systems, focusing on our cognitive engine. We also describe developments that support the cognitive engine and advances in radio technology that provide the flexibility desired in a cognitive radio node. We consider securing and verifying cognitive systems and examine the challenges of expanding the cognitive paradigm up the protocol stack to optimize end-to-end network performance. Lastly, we consider the analysis of cognitive systems using game theory and the application of cognitive techniques to problems in dynamic spectrum sharing and control of multiple-input multiple-output radios.

The difficult aspect of hardware reconfiguration is not creating the computational blocks, which are generally available from FPGA vendors and third parties, but linking the blocks in a manner that suits each application's unique connectivity, bandwidth, and latency requirements. Our approach uses the standard Xilinx implementation tools to generate dynamic module partial bitstreams, but choosing the module's coordinates and completing connections to other modules are runtime operations. Scripts automatically add interface wrappers to dynamic modules and generate a library of relocatable partial bitstreams. The library is used by an efficient runtime system that completes application requests for instancing and connecting modules, effectively insulating the designer from FPGA reconfiguration complexities. In this way, a large sandbox may be allocated to dynamic modules rather than fixed module slots and interconnect. Application engineers interact with the Wires on Demand (WoD) system through a runtime software API, and do not have to master hardware description languages and implementation tools.

Array-based, direct-sampling radio telescopes have computational and communication requirements unsuited to conventional computer and cluster architectures. Synchronization must be strictly maintained across a large number of parallel data streams, from A/D conversion, through operations such as beamforming, to dataset recording. FPGAs supporting multigigabit serial I/O are ideally suited to this application. We describe a recently-constructed radio telescope called ETA having all-sky observing capability for detecting low frequency pulses from transient events such as gamma ray bursts and primordial black hole explosions. Signals from 24 dipole antennas are processed by a tiered arrangement of 28 commercial FPGA boards and 4 PCs with FPGA-based data acquisition cards, connected with custom I/O adapter boards supporting InfiniBand and LVDS physical links. ETA is designed for unattended operation, allowing configuration and recording to be controlled remotely.

While there have been many reported implementations of networks-on-chip (NoCs) on FPGAs, they have not seen the same acceptance as NoCs on ASICs. One reason is that communication on an FPGA is already costly due to the die resources and time delays inherent in the reconfigurable structure. Layering another general-purpose network on top of the reconfigurable network simply incurs too many performance penalties. There is, however, already a largely unused, global network available in FPGAs. As a proof-of-concept, we demonstrate that the Xilinx FPGA configuration circuitry, which is normally idle during system operation, can function as a relatively high-performance NoC. MetaWire performs transfers through an overclocked Virtex-4 internal configuration access port (ICAP) and is shown to provide a bandwidth exceeding 200 MBytes/sec.

It is often desirable to change the logic and/or the connections within an FPGA design on-the-fly without the benefit of a workstation or vendor CAD software. This paper presents a dynamic router for Xilinx FPGAs, designed to run on stand-alone embedded systems. With information obtained from Xilinxpsilas XDL tool, a compact routing database for the Virtex-II/IIP/4 devices is built which only requires 96 KB of storage. A channel routing algorithm is used because of its deterministic execution time and because all routing resources in the channel are available. Sample channels are routed with the router and compared with the Xilinx PAR tool. Improvements in both execution time and in memory usage of several orders of magnitude are observed.

In systems typified by software defined radio, existing flows for run-time FPGA reconfiguration limit resource efficiency when constructing a variety of datapaths. Our approach allocates a sandbox region in which modules from a library can be flexibly placed and interconnected. An efficient run-time framework makes use of lightweight placement and routing techniques to respond on-demand to application requests. Compile time tools automate the task of adding interface wrappers to modules, insulating the designer from reconfiguration details.

This paper develops a formal model of process migration that describes programs, processes, and the migration of those processes within a migration realm. A migration realm is a group of processors modeled as finite state machines. The model is motivated by a migration application between software and Field Programmable Gate Array (FPGA) hardware, and the theorems of the model guide the use of FPGA resources while guaranteeing complete and correct execution of a process. By defining different types of migration realms this paper also develops a migration realm taxonomy.

This paper describes an execution cache that uses process migration between hardware and software contexts by way of run-time reconfiguration (RTR) of Field Programmable Gate Arrays (FPGAs). The feasibility of such a system is demonstrated using existing FPGAs by accelerating a cycle-based simulation of a Register Transfer Level (RTL) design description. Through the use of a common instruction set, each simulation process may be run in a software Virtual Machine (VM) or in a hardware Real Machine (RM). The implementation provides data for an empirical model used to examine the behavior of unimplemented parts of the system.

Array-based, direct-sampling radio telescopes have computational and communication requirements unsuited to conventional computer and cluster architectures. Synchronization must be strictly maintained across a large number of parallel data streams, from A/D conversion, through operations such as beamforming, to dataset recording. FPGAs supporting multi-gigabit serial I/O are ideally suited to this application. We describe a recently-constructed radio telescope called ETA having all-sky observing capability for detecting low frequency pulses from transient events such as gamma ray bursts and exploding primordial black holes. Signals from 24 dipole antennas are processed by a tiered arrangement of 28 commercial FPGA boards and 4 PCs with FPGA-based data acquisition cards, connected with custom I/O adapter boards supporting InfiniBand and LVDS physical links. ETA is designed for unattended operation, allowing configuration and recording to be controlled remotely.