FABRIC is a national and international scale research infrastructure built to enable cutting-edge research in a wide range of networking and computing areas and domain sciences that depend on advanced networking and computing capabilities. Since its full operation in October 2023, FABRIC has seen rapidly emerging experiments in different research topics. This paper provides a description of FABRIC’s hardware resources, software services, and an analysis of the characteristics of the emerging experiments that are leveraging FABRIC’s unique capabilities. The analysis showed a wide spectrum of topic areas, frequent emphasis on at-scale and high performance experiments that interact with real facilities and real people, and growing complexity and rigor in tools, data and measurement. Researchers are seen sharing well-developed artifacts with others to enable larger experiments and reproducible research.

The layered Internet architecture, while far from perfect, has provided a global and neutral platform for the development of a wide range of applications. However, this core architecture has been increasingly augmented with additional in-network functionality that improves the performance, security, and privacy of these applications. These additional in-network functions, which are typically implemented at the network edge, are consistent with the layering of the Internet architecture but deviate from two of the core tenets of the Internet: interconnection and end-to-end simplicity. In this paper, we propose an architecture for these edge networking services called the InterEdge that applies these two Internet tenets in a manner appropriate to edge services while not requiring changes to the underlying Internet architecture or infrastructure.

Maintaining a scalable high-performance virtual global address space using distributed memory hardware has proven to be challenging. In this paper we evaluate a new approach for such an active global address space that leverages the capabilities of the network fabric to manage addressing, rather than software at the endpoint hosts. We describe our overall approach, design alternatives, and present initial experimental results that demonstrate the effectiveness and limitations of existing network hardware.

The quality of data exchange in cloud computing applications relies on the connection performance between user clients and their cloud storage providers, and is often dependent on the wide area network (WAN) properties among data centers. For certain classes of applications, it can be crucial to provide an end-to-end solution that accelerates large data transfers and improves overall user experience. The development and deployment of WAN optimization technology has been investigated for improving application performance in heterogeneous, multi-domain environments. WAN optimization devices and services implement a number of approaches for performance improvement, and one key insight is that in contrast to traditional end-to-end TCP connections, middleboxes that segment and optimize transport-layer connections can improve the performance of wide area data transfers. In the context of dynamic cloud computing environments, there is an obvious target for implementations of WAN optimization as Network Function Virtualization (NFV), where the flexibility of virtualized cloud environments can be exploited. This paper describes recent developments and experimentation of our Phoebus WAN accelerator framework. We introduce a software suite that includes new Phoebus clients that operate with the Phoebus Gateway network. We test and discuss virtualizing Phoebus Gateways to provide acceleration services in cloud data transfers. Use cases and performance evaluations are conducted on FutureGrid and Internet2 testbeds, and we demonstrate the effectiveness of a virtualized Phoebus deployment.

Network resource measurement is a key functionality for large scale network management. Intelligent, network-aware applications may benefit from access to detailed representations of network resources, including multi-layer topologies and real-time traffic measurement, and shared resources may obtain better overall utilization by identifying performance bottlenecks. In this study, we describe a network measurement framework, which includes a network topology analysis mechanism as well as agent tools for running active probes and collecting data from end hosts. The system includes a centralized coordinator, which abstracts network elements into annotated network graphs and applies scheduling algorithms to calculate conflict free measurement probes over shared links. Our evaluation integrated perfSONAR services into our framework and included deployment scenarios on research and education networks such as Internet2 and ESnet. The data presented in this study offers compelling evidence that supports a method by which to measure the performance of real world networks.

Data set sizes are growing exponentially, so it is important to use data movement protocols that are the most efficient available. Most data movement tools today rely on TCP over sockets, which limits flows to around 20Gbps on today's hardware. RDMA over Converged Ethernet (RoCE) is a promising new technology for high-performance network data movement with minimal CPU impact over circuit-based infrastructures. We compare the performance of TCP, UDP, UDT, and RoCE over high latency 10Gbps and 40Gbps network paths, and show that RoCE-based data transfers can fill a 40Gbps path using much less CPU than other protocols. We also show that the Linux zero-copy system calls can improve TCP performance considerably, especially on current Intel “Sandy Bridge”-based PCI Express 3.0 (Gen3) hosts.

This paper presents the eXtensible Session Protocol (XSP), which provides a control plane for driving Software Defined Networks (SDNs). The XSP model supports proactive, application-driven configuration of dynamic network resources with support for authentication and authorization, within an extensible protocol framework. We describe XSP application use cases in SDNs using OpenFlow enabled network devices as well as dynamic forwarding rule management that can be implemented on existing router platforms.

Monitoring and managing multi-gigabit networks requires dynamic adaptation to end-to-end performance characteristics. This paper presents a measurement collection and analysis framework that automates the troubleshooting of end-to-end network bottlenecks. We integrate real-time host, application, and network measurements with a common representation (compatible with perfSONAR) within a flexible and scalable architecture. Our measurement architecture is supported by a light-weight eXtensible Session Protocol (XSP), which enables context-sensitive adaptive measurement collection. We evaluate the ability of our system to analyze and detect bottleneck conditions over a series of high-speed and I/O intensive bulk data transfer experiments and find that the overhead of the system is very low and that we are able to detect and understand a variety of bottlenecks.

IP spoofing exacerbates many security threats, and reducing it would greatly enhance Internet security. Seven defenses that filter spoofed traffic have been proposed to date; three are designed for end-network deployment, while four assume some collaboration with core routers for packet marking or filtering. Because each defense has been evaluated in a unique setting, the following important questions remain unanswered: 1) Can end networks effectively protect themselves or is core support necessary? 2) Which defense performs best assuming sparse deployment? 3) How to select core participants to achieve best protection with fewest deployment points? This paper answers the above questions by: 1) formalizing the problem of spoofed traffic filtering and defining novel effectiveness measures, 2) observing each defense as selfish (it helps its participants) or altruistic (it helps everyone) and differentiating their performance goals, 3) defining optimal core deployment points for defenses that need core support, and 4) evaluating all defenses in a common and realistic setting. Our results offer a valuable insight into advantages and limitations of the proposed defenses, and uncover the relationship between any spoofing defense's performance and the Internet's topology.

Large scale scientific data transfers are central to scientific processes. Data from large experimental facilities have to be moved to local institutions for analysis or often data needs to be moved between local clusters and large supercomputing centers. In this paper, we propose and evaluate a network overlay architecture to enable high-throughput, on-demand, coordinated data transfers over wide-area networks. Our work leverages Phoebus and On-demand Secure Circuits and Advance Reservation System (OSCARS) to provide high performance wide-area network connections. OSCARS enables dynamic provisioning of network paths with guaranteed bandwidth and Phoebus enables the coordination and effective utilization of the OSCARS network paths. Our evaluation shows that this approach leads to improved end-to-end data transfer throughput with minimal overheads. The achieved throughput using our overlay was limited only by the ability of the end hosts to sink the data.

Computational science is considered the third pillar of science but while we continue to make significant strides in high-performance computing, the ability to transfer the vast amounts of data generated and consumed by scientists has fallen behind. This divergence has significantly constrained scientific progress. This paper describes MNEMONIC, an end-to-end cyber-infrastructure system designed to improve scientists' productivity by automatically and transparently optimizing and tuning data movement over the network. MNEMONIC integrates advanced technologies, i.e., network interfaces, protocols, and measurement toolkits, to seamlessly manage high-performance data movement. We have integrated the Java implementation of GridFTP in MNEMONIC to provide a cross-platform data transfer capability. Preliminary results show that MNEMONIC+GridFTP outperforms the TCP+GridFTP implementation by over a factor of 10x, while remaining easy to install and operate.

Phoebus is an infrastructure for improving end-to-end throughput in high-bandwidth, long-distance networks by using a "session layer" protocol and "gateways" in the network. Phoebus has the ability to dynamically allocate network resources and to use segmentspecific transport protocols between gateways, as well as to apply other performance-improving techniques on behalf of the user.