FABRIC is a national and international scale research infrastructure built to enable cutting-edge research in a wide range of networking and computing areas and domain sciences that depend on advanced networking and computing capabilities. Since its full operation in October 2023, FABRIC has seen rapidly emerging experiments in different research topics. This paper provides a description of FABRIC’s hardware resources, software services, and an analysis of the characteristics of the emerging experiments that are leveraging FABRIC’s unique capabilities. The analysis showed a wide spectrum of topic areas, frequent emphasis on at-scale and high performance experiments that interact with real facilities and real people, and growing complexity and rigor in tools, data and measurement. Researchers are seen sharing well-developed artifacts with others to enable larger experiments and reproducible research.

Computational science today depends on complex, data-intensive applications operating on datasets from a variety of scientific instruments. A major challenge is the integration of data into the scientist's workflow. Recent advances in dynamic, networked cloud resources provide the building blocks to construct reconfigurable, end-to-end infrastructure that can increase scientific productivity. However, applications have not adequately taken advantage of these advanced capabilities. In this work, we have developed a novel network-centric platform that enables high-performance, adaptive data flows and coordinated access to distributed cloud resources and data repositories for atmospheric scientists. We demonstrate the effectiveness of our approach by evaluating time-critical, adaptive weather sensing workflows, which utilize advanced networked infrastructure to ingest live weather data from radars and compute data products used for timely response to weather events. The workflows are orchestrated by the Pegasus workflow management system and were chosen because of their diverse resource requirements. We show that our approach results in timely processing of Nowcast workflows under different infrastructure configurations and network conditions. We also show how workflow task clustering choices affect throughput of an ensemble of Nowcast workflows with improved turnaround times. Additionally, we find that using our network-centric platform powered by advanced layer2 networking techniques results in faster, more reliable data throughput, makes cloud resources easier to provision, and the workflows easier to configure for operational use and automation.

Computer Science experimental testbeds allow investigators to explore a broad range of different state-of-the-art hardware options, assess scalability of their systems, and provide conditions that allow deep reconfigurability and isolation so that one user does not impact the experiments of another. An experimental testbed is also in a unique position to support methods facilitating experiment analysis and improve repeatability and reproducibility of experiments. Providing these capabilities at least partially within a commodity framework improves the sustainability of systems experiments and thus makes them available to a broader range of experimenters.

Many scientific research communities and institutions are adopting the cloud as a primary platform to support their computing needs. Rapid adoption of the cloud for scientific computing is a result of the simplicity with which an individual researcher can obtain large amounts of customized compute and storage resources. At the same time, most public cloud providers have rolled out many advanced networking services. Many of these services, like their compute services, are simple to access by any cloud user (e.g. routing between regions and private networking spaces within a cloud). However, it is not possible for most researchers to access expensive low-level, externally facing cloud network services without complicated support by campus IT staff, as well as national and regional network providers. This paper describes how to use Chameleon, ExoGENI, and Internet2's Cloud Connect service to deploy research experiments that use AWS Direct Connect without requiring a privately owned Direct Connect endpoint or support from local campus IT staff.

A majority of today’s cloud services are independently operated by individual cloud service providers. In this approach, the locations of cloud resources are strictly constrained by the distribution of cloud service providers’ sites. As the popularity and scale of cloud services increase, we believe this traditional paradigm is about to change toward further federated services, a.k.a., multi-cloud, due to the improved performance, reduced cost of compute, storage and network resources, as well as increased user demands. In this paper, we present COMET, a lightweight, distributed storage system for managing metadata on large scale, federated cloud infrastructure providers, end users, and their applications (e.g. HTCondor Cluster or Hadoop Cluster). We showcase use case from NSF’s, Chameleon, ExoGENI and JetStream research cloud testbeds to show the effectiveness of COMET design and deployment.

Recent advances in cloud systems, on-demand circuits and software-defined networking have created new opportunities to enable complex, data-intensive scientific applications to run on dynamic networked cloud infrastructures. In this work, we present an end-to-end framework for autonomic adaptation for scientific workflows on networked cloud systems, which leverages novel network provisioning technologies. We present an application-independent controller framework called Mobius++ that includes dynamic network adaptation capabilities using Software-Defined Networking (SDN) mechanisms, which enables workflow management systems to address competing priorities of workflow operations, data movements in particular. We use a representative, data-intensive bioinformatics workflow as a driving use case to showcase the above capabilities. Experimental results show that the Mobius++ framework, in conjunction with a novel virtual Software Defined Exchange (SDX) platform, is able to dynamically prioritize bandwidths between different end-points, on-demand, and being driven by priority directives from a workflow management system. We show that data transfer jobs from two workflows with different priorities are accurately arbitrated as the relative priorities change.

Multi-tenant cloud infrastructures are increasingly used for high-performance and high-throughput domain science applications. In recent years, machine virtualization has come a long way toward supporting domain science applications. Various cloud platforms, such as Open Stack, Cloud Stack, and Amazon EC2 are attracting scientists to these platforms with the promise of customized environments with virtually infinite compute resources. At the same time, research efforts, such as NSF GENI are bringing together cloud computing with advanced network infrastructure provisioning. This paper presents work toward evaluating the use of GENI to support domain science applications. The evaluation involved two different domain science applications deployed on ExoGENI and Insta GENI. The first application is ADCIRC, a storm surge model that uses Message Passing Interface (MPI). The second is Motif network, a genomics application using the Pegasus workflow management system to manage a large data-intensive workflow.

Advanced networks are an essential element of data-driven science enabled by next generation cyberinfrastructure environments. Computational activities increasingly incorporate widely dispersed resources with linkages among software components spanning multiple sites and administrative domains. We have seen recent advances in enabling on-demand network circuits in the national and international backbones coupled with Software Defined Networking (SDN) advances like OpenFlow and programmable edge technologies like OpenStack. These advances have created an unprecedented opportunity to enable complex scientific applications to run on specially tailored, dynamic infrastructure that include compute, storage and network resources, combining the performance advantages of purpose-built infrastructures, but without the costs of a permanent infrastructure. This work presents an experience deploying scientific workflows on the ExoGENI national test bed that dynamically allocates computational resources with high-speed circuits from backbone providers. Dynamically allocated bandwidth-provisioned high-speed circuits increase the ability of scientific applications to access and stage large data sets from remote data repositories or to move computation to remote sites and access data stored locally. The remainder of this extended abstract is a brief description of the test bed and several scientific workflow applications that were deployed using bandwidth-provisioned high-speed circuits.

This paper presents the design, implementation, and evaluation of a new system for on-demand provisioning of Hadoop clusters across multiple cloud domains. The Hadoop clusters are created "on-demand" and are composed of virtual machines from multiple cloud sites linked with bandwidth-provisioned network pipes. The prototype uses an existing federated cloud control framework called Open Resource Control Architecture (ORCA), which orchestrates the leasing and configuration of virtual infrastructure from multiple autonomous cloud sites and network providers. ORCA enables computational and network resources from multiple clouds and network substrates to be aggregated into a single virtual "slice" of resources, built to order for the needs of the application. The experiments examine various provisioning alternatives by evaluating the performance of representative Hadoop benchmarks and applications on resource topologies with varying bandwidths. The evaluations examine conditions in which multi-cloud Hadoop deployments pose significant advantages or disadvantages during Map/Reduce/Shuffle operations. Further, the experiments compare multi-cloud Hadoop deployments with single-cloud deployments and investigate Hadoop Distributed File System (HDFS) performance under varying network configurations. The results show that networked clouds make cross-cloud Hadoop deployment feasible with high bandwidth network links between clouds. As expected, performance for some benchmarks degrades rapidly with constrained inter-cloud bandwidth. MapReduce shuffle patterns and certain Hadoop Distributed File System (HDFS) operations that span the constrained links are particularly sensitive to network performance. Hadoop's topology-awareness feature can mitigate these penalties to a modest degree in these hybrid bandwidth scenarios. Additional observations show that contention among co-located virtual machines is a source of irregular performance for Hadoop applications on virtual cloud infrastructure.

The Nexos Project is a joint effort between Marquette University, the University of Buffalo, and the University of Mississippi to build curriculum materials and a supporting experimental laboratory for hands-on projects in computer systems courses. The approach focuses on inexpensive, flexible, commodity embedded hardware, freely available development and debugging tools, and a fresh implementation of a classic operating system, Embedded Xinu, that is ideal for student exploration. This paper describes an extension to the Nexos laboratory that includes a new target platform composed of Qemu virtual machines. Virtual Xinu addresses two challenges that limit the effectiveness of Nexos. First, potential faculty adopters have clearly indicated that even with the current minimal monetary cost of installation, the hardware modifications, and time investment remain troublesome factors that scare off interested educators. Second, overcoming the inherent complications that arise due to the shared subnet that result in students' projects interfering with each other in ways that are difficult to recreate, debug, and understand. Specifically, this paper discusses porting the Xinu operating systems to Qemu virtual hardware, developing the virtual networking platform, and results showing success using Virtual Xinu in the classroom during one semester of Operating Systems at the University of Mississippi.

By federating resources from multiple domains, a shared infrastructure provides aggregated computation resources to a large number of users. With rapid advances in virtualization technologies, we propose the concept of virtual distributed environments as a new sharing paradigm for a multi-domain shared infrastructure. Such virtual environments provide users with confined, customized platforms to execute legacy parallel/distributed applications. Furthermore, we propose to support autonomic adaptation of virtual distributed environments, driven by both dynamic availability of infrastructure resources and dynamic application resource demand. We identify new research challenges and describe our on-going work and preliminary results. 1

A large organization, such as a university, commonly supplies computational power through multiple independently administered computational domains (e.g. clusters). Each computational domain faces the conflict between dynamic workload and static capacity. This is clearly inefficient at times when some clusters have idle nodes while others experience excessive workload. An opportunity arises to resolve this conflict by dynamically adapting the capacity of clusters by borrowing idle machines of peer domains. In this paper, we present the design, implementation, and evaluation of VioCluster, a virtualization based computational resource sharing platform. Through machine and network virtualization, VioCluster enables virtual computational domains that safely "trade" machines between them without infringing on the autonomy of either domain. Our performance evaluation results show that dynamic machine trading between virtual domains increases their resource utilization and decreases their job wait times

The initially unrestricted host-to-host communication model provided by the Internet Protocol has deteriorated due to political and technical changes caused by Internet growth. While this is not a problem for most client-server applications, peer-to-peer networks frequently struggle with peers that are only partially reachable. We describe how a peer-to-peer framework can hide diversity and obstacles in the underlying Internet and provide peer-to-peer applications with abstractions that hide transport specific details. We present the details of an implementation of a transport service based on SMTP. Small-scale benchmarks are used to compare transport services over UDP, TCP, and SMTP.