Data automation, monitoring, and management tools are reliant on being able to detect, report, and respond to file system events. Various data event reporting tools exist for specific operating systems and storage devices, such as inotify for Linux, kqueue for BSD, and FSEvents for macOS. However, these tools are not designed to monitor distributed file systems. Indeed, many cannot scale to monitor many thousands of directories, or simply cannot be applied to distributed file systems. Moreover, each tool implements a custom API and event representation, making the development of generalized and portable event-based applications challenging. As file systems grow in size and become increasingly diverse, there is a need for scalable monitoring solutions that can be applied to a wide range of both distributed and local systems. We present here a generic and scalable file system monitor and event reporting tool, FSMonitor, that provides a file-system-independent event representation and event capture interface. FSMonitor uses a modular Data Storage Interface (DSI) architecture to enable the selection and application of appropriate event monitoring tools to detect and report events from a target file system, and implements efficient and fault-tolerant mechanisms that can detect and report events even on large file systems. We describe and evaluate DSIs for common UNIX, macOS, and Windows storage systems, and for the Lustre distributed file system. Our experiments on a 897 TB Lustre file system show that FSMonitor can capture and process almost 38 000 events per second.

While the Machine Learning (ML) landscape is evolving rapidly, there has been a relative lag in the development of the “learning systems” needed to enable broad adoption. Furthermore, few such systems are designed to support the specialized requirements of scientific ML. Here we present the Data and Learning Hub for science (DLHub), a multi-tenant system that provides both model repository and serving capabilities with a focus on science applications. DLHub addresses two significant shortcomings in current systems. First, its self-service model repository allows users to share, publish, verify, reproduce, and reuse models, and addresses concerns related to model reproducibility by packaging and distributing models and all constituent components. Second, it implements scalable and low-latency serving capabilities that can leverage parallel and distributed computing resources to democratize access to published models through a simple web interface. Unlike other model serving frameworks, DLHub can store and serve any Python 3-compatible model or processing function, plus multiple-function pipelines. We show that relative to other model serving systems including TensorFlow Serving, SageMaker, and Clipper, DLHub provides greater capabilities, comparable performance without memoization and batching, and significantly better performance when the latter two techniques can be employed. We also describe early uses of DLHub for scientific applications.

Scientific research is data-centric, relying on the acquisition, management, movement, analysis, and sharing of data. Proficiently managing the end-to-end lifecycle of scientific data is non-trivial and comprises many time consuming and mundane tasks. While individual tasks are not prohibitive, when done repeatedly and frequently they represent a significant strain on researchers. We posit that a better approach is to automate these tasks through a Software Defined Cyberinfrastructure. We have developed R IPPLE to provide such capabilities by automating research data management activities via a programmable and event-based cyber-environment. Users specify high-level management policies, such as data movement and metadata extraction, using intuitive If-Trigger-Then-Action rules. These rules are then autonomously, and reliably, executed and managed by Ripple.

Globus Auth is a foundational identity and access management platform service designed to address unique needs of the science and engineering community. It serves to broker authentication and authorization interactions between end-users, identity providers, resource servers (services), and clients (including web, mobile, desktop, and command line applications, and other services). Globus Auth thus makes it easy, for example, for a researcher to authenticate with one credential, connect to a specific remote storage resource with another identity, and share data with colleagues based on another identity. By eliminating friction associated with the frequent need for multiple accounts, identities, credentials, and groups when using distributed cyberinfrastructure, Globus Auth streamlines the creation, integration, and use of advanced research applications and services. Globus Auth builds upon the OAuth 2 and OpenID Connect specifications to enable standards-compliant integration using existing client libraries. It supports identity federation models that enable diverse identities to be linked together, while also providing delegated access tokens via which client services can obtain short term delegated tokens to access other services. We describe the design and implementation of Globus Auth, and report on experiences integrating it with a range of research resources and services, including the JetStream cloud, XSEDE, NCAR's Research Data Archive, and FaceBase.

Modern science and engineering require increasingly sophisticated information technology (IT) for data analysis, simulation, and related tasks. Yet the small to medium laboratories (SMLs) in which the majority of research advances occur increasingly lack the human and financial capital needed to acquire and operate such IT. New methods are needed to provide all researchers with access to state-of-the-art scientific capabilities, regardless of their location and budget. Industry has demonstrated the value of cloud-hosted software-and platform-as-a-service approaches, small businesses that outsource their IT to third-party providers slash costs and accelerate innovation. However, few business cloud services are transferable to science. We thus propose the Discovery Cloud, an ecosystem of new, community-produced services to which SMLs can outsource common activities, from data management and analysis to collaboration and experiment automation. We explain the need for a Discovery Platform to streamline the creation and operation of new and interoperable services, and a Discovery Exchange to facilitate the use and sustainability of Discovery Cloud services. We report on our experiences building early elements of the Discovery Platform in the form of Globus services, and on the experiences of those who have applied those services in innovative applications.

XSEDE is the integration framework for national-scale, public HPC resources in the United States. XSEDE is used by thousands of researchers at hundreds of college and university campuses throughout the country, as well as many international collaborators. Over the past several program years, XSEDE has redefined its identity management, security, and service interfaces to bridge the gap between national-scale HPC resources and campus-based computing resources. These changes make it easier for research performed on campus to access our national computing resources and make them a part of the everyday research process. We report here on XSEDE's new identity management system and how it provides a smooth bridge between campus and national identity systems. We also describe how this federated security system supports two additional bridges between campuses and national HPC services, one involving data movement and another involving scientific workflows.

Broad access to the data on which scientific results are based is essential for verification, reproducibility, and extension. Scholarly publication has long been the means to this end. But as data volumes grow, new methods beyond traditional publications are needed for communicating, discovering, and accessing scientific data. We describe data publication capabilities within the Globus research data management service, which supports publication of large datasets, with customizable policies for different institutions and researchers, the ability to publish data directly from both locally owned storage and cloud storage, extensible metadata that can be customized to describe specific attributes of different research domains, flexible publication and curation workflows that can be easily tailored to meet institutional requirements, and public and restricted collections that give complete control over who may access published data. We describe the architecture and implementation of these new capabilities and review early results from pilot projects involving nine research communities that span a range of data sizes, data types, disciplines, and publication policies.

Globus, developed as Software-as-a-Service (SaaS) for research data management, also provides APIs that constitute a flexible and powerful Platform-as-a-Service (PaaS) to which developers can outsource data management activities such as transfer and sharing, as well as identity, profile and group management. By providing these frequently important but always challenging capabilities as a service, accessible over the network, Globus PaaS streamlines web application development and makes it easy for individuals, teams, and institutions to create collaborative applications such as science gateways for science communities. We introduce the capabilities of this platform and review representative applications.

Collaborative e-Science applications often need to manage large numbers of user identities, profiles, and groups. However, developing and maintaining such capabilities is often challenging given the plethora of security protocols available and requirements for scalable, robust, and highly available implementations. Globus Nexus is a professionally hosted Platform-as-a-Service that provides these capabilities for collaborative e-Science applications, with a particular focus on the needs of scientific communities. It provides features such as identity provisioning, identity federation, profile management, user-oriented group management, and branded web interfaces that are important to many e-Science applications. Globus Nexus implements best practices approaches for each of these features for example using delegated security protocols such as OAuth, provides sophisticated workflows for actions such as email validation, and implements complex user-defined policies regarding permissible actions. We present here Globus Nexus' capabilities, motivate design choices, and present results that characterize the scalability, reliability, and availability of its implementation and deployment.

Globus Nexus is a flexible and powerful Platform-as-a-Service to which developers can outsource identity, group, and profile management needs. By providing these frequently important but always challenging capabilities as a service, accessible over the network, Globus Nexus streamlines web application development and makes it easy for individuals, teams, and institutions to create collaborative web applications such as science gateways for the science community. We introduce the capabilities of this platform and review representative applications.

We often encounter in distributed systems the need to model, access, and manage state. This state may be, for example, data in a purchase order, service level agreements representing resource availability, or the current load on a computer. We introduce two closely related approaches to modeling and manipulating state within a Web services (WS) framework: the Open Grid Services Infrastructure (OGSI) and WS-Resource Framework (WSRF). Both approaches define conventions on the use of the Web service definition language schema that enable the modeling and management of state. OGSI introduces the idea of a stateful Web service and defines approaches for creating, naming, and managing the lifetime of instances of services; for declaring and inspecting service state data; for asynchronous notification of service state change; for representing and managing collections of service instances; and for common handling of service invocation faults. WSRF refactors and evolves OGSI to exploit new Web services standards, specifically WS-addressing, and to respond to early implementation and application experiences. WSRF retains essentially all of the functional capabilities present in OGSI, while changing some syntax (e.g., to exploit WS-addressing) and also adopting a different terminology in its presentation. In addition, WSRF partitions OGSI functionality into five distinct composable specifications. We explain the relationship between OGSI and WSRF and the related WS-notification specifications, explain the common requirements that both address, and compare and contrast the approaches taken to the realization of those requirements.

Grid computing is concerned with the sharing and coordinated use of diverse resources in distributed "virtual organizations." The dynamic and multiinstitutional nature of these environments introduces challenging security issues that demand new technical approaches. In particular, one must deal with diverse local mechanisms, support dynamic creation of services, and enable dynamic creation of trust domains. We describe how these issues are addressed in two generations of the Globus Toolkit/spl reg/. First, we review the Globus Toolkit version 2 (GT2) approach; then we describe new approaches developed to support the Globus Toolkit version 3 (GT3) implementation of the Open Grid Services Architecture, an initiative that is recasting Grid concepts within a service-oriented framework based on Web services. GT3's security implementation uses Web services security mechanisms for credential exchange and other purposes, and introduces a tight least-privilege model that avoids the need for any privileged network service.

The Globus Data Grid architecture (I. Foster and C. Kesselman, 1998), provides a scalable infrastructure for the management of storage resources and data that are distributed across Grid environments. These services are designed to support a variety of scientific applications, ranging from high-energy physics to computational genomics, that require access to large amounts of data (terabytes or even petabytes) with varied quality of service requirements. By layering on a set of core services, such as data transport, security, and replica cataloging, one can construct various higher-level services. We discuss the design and implementation of a high-level replica selection service that uses information regarding replica location and user preferences to guide selection from among storage replica alternatives. We first present a basic replica selection service design, then show how dynamic information collected using Globus information service capabilities concerning storage system properties can help improve and optimize the selection process. We demonstrate the use of Condor's ClassAds resource description and matchmaking mechanism as an efficient tool for representing and matching storage resource capabilities and policies against application requirements.

In recent years, there has been a dramatic increase in the amount of available computing and storage resources, yet few have been able to exploit these resources in an aggregated form. We present the Condor-G system, which leverages software from Globus and Condor to allow users to harness multi-domain resources as if they all belong to one personal domain. We describe the structure of Condor-G and how it handles job management, resource selection, security and fault tolerance.

Grid portals, based on standard Web technologies, are increasingly used to provide user interfaces for computational and data grids. However, such Grid portals do not integrate cleanly with existing Grid security systems such as the Grid Security Infrastructure (GSI), due to lack of delegation capabilities in Web security mechanisms. We solve this problem using an online credentials repository system, called MyProxy. MyProxy allows Grid portals to use the GSI to interact with Grid resources in a standard, secure manner. We examine the requirements of Grid portals, give an overview of the GSI, and demonstrate how MyProxy enables them to function together. The architecture and security of the MyProxy system are described in detail.

State-of-the-artand emerging scientific applications require fast access to large quantities of data and commensurately fast computational resources.Both resources and data are oflen distributed in a wide-area network with components administered locally and independently.Computations may involve hundreds of processes that must be able to acquire resources dynamically and communicate efficiently.This paper analyzes the unique security requirements of large-scale distributed (grid) computing and develops a security policy and a corresponding security architecture.An implementation of the architecture within the Globus metacomputing toolkit is discussed.

The development of applications and tools for high-performance "computational grids" is complicated by the heterogeneity and frequently dynamic behavior of the underlying resources; by the complexity of the applications themselves, which often combine aspects of supercomputing and distributed computing; and by the need to achieve high levels of performance. The Globus toolkit has been developed with the goal of simplifying this application development task, by providing implementations of various core services deemed essential for high-performance distributed computing. In this paper, we describe two large applications developed with this toolkit: a distributed interactive simulation and a teleimmersion system. We describe the process used to develop the applications, review the lessons learned and draw conclusions regarding the effectiveness of the toolkit approach.

High-speed wide-area networks are expected to enable innovative applications that integrate geographically distributed, high-performance computing, database, graphics and networking resources. However, there is as yet little understanding of the higher-level services required to support these applications, or of the techniques required to implement these services in a scalable, secure manner. We report on a large-scale prototyping effort that has yielded some insights into these issues. Building on the hardware base provided by the I-WAY, a national-scale asynchronous transfer mode (ATM) network, we developed an integrated management and application programming system, called I-Soft. This system was deployed at most of the 17 I-WAY sites and used by many of the 60 applications demonstrated on the I-WAY network. In this paper we describe the I-Soft design and report on lessons learned from application experiments. We focus on four novel concepts that we believe may have relevance to future, similar systems: point of presence machines as a means of simplifying implementation and management; scheduler proxies to integrate local schedulers to computational resource brokers; authentication proxies to provide a uniform authentication environment across multiple administrative domains; and network-aware parallel programming tools to hide heterogeneity and improve performance in heterogeneous environments. Lessons learned in building I-Soft have motivated subsequent research and development efforts in the Globus project. © 1998 John Wiley & Sons, Ltd.

High-performance execution in distributed computing environments often requires careful selection and configuration not only of computers, networks, and other resources but also of the protocols and algorithms used by applications. Selection and configuration in turn require access to accurate, up-to-date information on the structure and state of available resources. Unfortunately, no standard mechanism exists for organizing or accessing such information. Consequently, different tools and applications adopt ad hoc mechanisms, or they compromise their portability and performance by using default configurations. We propose a solution to this problem: a Metacomputing Directory Service that provides efficient and scalable access to diverse, dynamic, and distributed information about resource structure and state. We define an extensible data model to represent the information required for distributed computing, and we present a scalable, high-performance, distributed implementation. The dat...

Applications that use high-speed networks to connect geographically distributed supercomputers, databases, and scientific instruments may operate over open networks and access valuable resources. Hence, they can require mechanisms for ensuring integrity and confidentiality of communications and for authenticating both users and resources. Security solutions developed for traditional client-server applications do not provide direct support for the program structures, programming tools, and performance requirements encountered in these applications. We address these requirements via a security-enhanced version of the Nexus communication library, which we use to provide secure versions of parallel libraries and languages, including the Message Passing Interface. These tools permit a fine degree of control over what, where, and when security mechanisms are applied. In particular, a single application can mix secure and nonsecure communication allowing the programmer to make fine-grained security/performance tradeoffs. We present performance results that quantify the performance of our infrastructure.

We use the term ubiquitous supercomputing to refer to systems that integrate low- and mid-range computing systems, advanced networks and remote high-end computers with the goal of enhancing the computational power accessible from local environments. Such systems promise to enable new applications in areas as diverse as smart instruments and collaborative environments. However, they also demand tools for transporting code between computers and for establishing flexible, dynamic communication structures. In this article, we propose that these requirements be satisfied by introducing Java classes that implement the global pointer and remote service request mechanisms defined by a communication library called Nexus. Java supports transportable code; Nexus provides communication support and represents the core communication framework for Globus, a project building infrastructure for ubiquitous supercomputing. We explain how this NexusJava library is implemented and illustrate its use with examples. © 1997 John Wiley & Sons, Ltd.

High speed wide area networks are expected to enable innovative applications that integrate geographically distributed, high performance computing, database, graphics, and networking resources. However, there is as yet little understanding of the higher level services required to support these applications, or of the techniques required to implement these services in a scalable, secure manner. We report on a large scale prototyping effort that has yielded some insights into these issues. Building on the hardware base provided by the I-WAY, a national scale asynchronous transfer mode (ATM) network, we developed an integrated management and application programming system, called I-Soft. This system was deployed at most of the 17 I-WAY sites and used by many of the 60 applications demonstrated on the I-WAY network. We describe the I-Soft design and report on lessons learned from application experiments.

We use the term ubiquitous supercomputing to refer to systems that integrate low and mid range computing systems, advanced networks, and remote high end computers with the goal of enhancing the computational power accessible from local environments. Such systems promise to enable new applications in areas as diverse as smart instruments and collaborative environments. However, they also demand tools for transporting code between computers and for establishing flexible, dynamic communication structures. We propose that these requirements be satisfied by enhancing the Java programming language with global pointer and remote service request mechanisms from a communication library called Nexus. Java supports transportable code; Nexus provides communication support. We explain how this NexusJava library is implemented and illustrate its use with examples.

Metacomputing systems use high-speed networks to connect supercomputers, mass storage systems, scientific instruments, and display devices with the objective of enabling parallel applications to utilize geographically distributed computing resources. However, experience shows that high performance can often be achieved only if applications can integrate diverse communication substrates, transport mechanisms, and protocols, chosen according to where communication is directed, what is communicated, or when communication is performed. In this paper, we describe a software architecture that addresses this requirement. This architecture allows multiple communication methods to be supported transparently in a single application, with either automatic or user-specified selection criteria guiding the methods used for each communication. We describe an implementation of this architecture, based on the Nexus communication library, and use this implementation to evaluate performance issues. This implementation was used to support a wide variety of applications in the I-WAY metacomputing experiment at Supercomputing~95; we use one of these applications to provide a quantitative demonstration of the advantages of multimethod communication in a heterogeneous networked environment.