Summary The Pacific Rim Application and Grid Middleware Assembly (PRAGMA) is an international community of researchers that actively collaborate to address problems and challenges of common interest in eScience. The PRAGMA Experimental Network Testbed (PRAGMA‐ENT) was established with the goal of constructing an international software‐defined network (SDN) testbed to offer the necessary networking support to the PRAGMA cyberinfrastructure. PRAGMA‐ENT is isolated, and PRAGMA researchers have complete freedom to access network resources to develop, experiment, and evaluate new ideas without the concerns of interfering with production networks. In the first phase, PRAGMA‐ENT focused on establishing an international L2 backbone. With support from the Florida Lambda Rail, Internet2, PacificWave, Japan Gigabit Network, and TaiWan Advanced Research and Education Network, PRAGMA‐ENT backbone connects openflow‐enabled switches at University of Florida, University of California, San Diego, Nara Institute of Science and Technology (Japan), Osaka University (Japan), National Institute of Advanced Industrial Science and Technology (Japan), and National Applied Research Laboratories (Taiwan). The second phase of PRAGMA‐ENT consisted of an evaluation of technologies for the control plane that enables multiple experiments (ie, OpenFlow controllers) to coexist. Preliminary experiments with FlowVisor revealed some limitations leading to the development of a new approach, called AutoVFlow. This paper describes our experience in the establishment of PRAGMA‐ENT backbone (with international L2 links), its current status, and plans for the control plane. Discussion of preliminary application ideas, including optimization of routing control; multipath routing control; extending the backbone using overlay network; and remote visualization are also discussed.

Summary The Pacific Rim Application and Grid Middleware Assembly (PRAGMA) is a community of research scientists and institutions from around the Pacific Rim that work together to enable scientific expeditions in areas such as biodiversity distribution and lake ecology. PRAGMA's members collaborate on a testbed infrastructure, and over the past 4 years, the technology focus has shifted to cloud and software defined networking as enabling technologies. This paper describes the design of a web‐based cloud scheduler reservation system that enables users to easily run and manage virtual clusters for their science. Based on a client‐server architecture, the cloud scheduler is designed to require minimal installation and management effort for participating PRAGMA testbed sites. The PRAGMA cloud scheduler is built using a web‐based room reservation tool called Booked and leverages several PRAGMA technologies such as pragma_boot, Personal Cloud Controller, and virtual network overlays (eg, IPOP and ViNe). We discuss our implementation of the pilot cloud scheduler and then describe future work.

Summary One of the activities of the Pacific Rim Applications and Grid Middleware Assembly (PRAGMA) is fostering Virtual Biodiversity Expeditions by bringing domain scientists and cyber infrastructure specialists together as a team. Over the past few years, PRAGMA members have been collaborating on virtualizing the Lifemapper software. Virtualization and cloud computing have introduced great flexibility and efficiency into IT projects. Virtualization refers to the technologies that provide a layer of abstraction between server hardware system and software that runs on it. This abstraction enables a logical view of computing resources and allows multiple servers to run on the same hardware. With this project, we are virtualizing Lifemapper by enabling its installation and configuration on a virtual cluster. Virtualization provides application scalability, maximizes resources utilization, and creates a more efficient, agile, and automated infrastructure. However, there are downsides to the complexity inherent in these environments, including the need for special techniques to deploy cluster hosts, dependence on virtual environments, and challenging application installation, management, and configuration. In this study, we report on progress of the Lifemapper virtualization framework focused on a reproducible and highly configurable infrastructure capable of fast deployment. Lifemapper is a distributed software application developed by the Biodiversity Institute at The University of Kansas. Lifemapper creates and maintains a publicly accessible archive of species distribution maps calculated from public specimen data. Lifemapper software also provides a suite of tools for biodiversity researchers that calculate single and multispecies distribution predictions and macroecological analyses through application programming interfaces. Our goal is to create a viable solution that can be easily adopted and reused by scientists from multiple institutions or projects. This solution (1) allows fast deployment of ready‐made cluster images, (2) reproduces the complete Lifemapper processing pipeline on demand at multiple sites and in different hosting environments, and (3) enables scientists to perform Lifemapper‐facilitated data processing on restricted‐use data, very large datasets, or other unique data. A key contribution of this work is describing the practical experience in taking a complex, clustered, domain‐specific, data analysis, and simulation system and enabling its operation on a variety of system configurations. Uses of this portability range from whole cluster replication to teaching and experimentation on a single laptop. System virtualization is used to practically define and make portable the full application stack, including all of its complex set of supporting software and allows Lifemapper deployment in a variety of environments.

In this poster, we present a novel approach, called software fingerprinting, that captures application dependencies. Our Fingerprint tool enables the user to discover, track, display and save the dependencies of an application without modification to its source code. The tool can achieve this both through static and runtime dependency discovery and the result is stored in a separate file called a Swirl. A fingerprint swirl file can be used in a variety of modes including: determining if the local system has changed since the swirl was created; mapping of library dependencies to specific installed software packages; querying a foreign system to determine if it satisfies the application's requirements; and creating (within some reasonable limitations) a compatible software environment on a foreign machine to simplify porting the user's application to the new system.

Abstract Many production Grid and e‐Science infrastructures have begun to offer services to end‐users during the past several years with an increasing number of scientific applications that require access to a wide variety of resources and services in multiple Grids. Therefore, the Grid Interoperation Now—Community Group of the Open Grid Forum—organizes and manages interoperation efforts among those production Grid infrastructures to reach the goal of a world‐wide Grid vision on a technical level in the near future. This contribution highlights fundamental approaches of the group and discusses open standards in the context of production e‐Science infrastructures. Copyright © 2009 John Wiley & Sons, Ltd.

The aggregation of imaging, clinical, and behavioral data from multiple independent institutions and researchers presents both a great opportunity for biomedical research as well as a formidable challenge. Many research groups have well-established data collection and analysis procedures, as well as data and metadata format requirements that are particular to that group. Moreover, the types of data and metadata collected are quite diverse, including image, physiological, and behavioral data, as well as descriptions of experimental design, and preprocessing and analysis methods. Each of these types of data utilizes a variety of software tools for collection, storage, and processing. Furthermore sites are reluctant to release control over the distribution and access to the data and the tools. To address these needs, the Biomedical Informatics Research Network (BIRN) has developed a federated and distributed infrastructure for the storage, retrieval, analysis, and documentation of biomedical imaging data. The infrastructure consists of distributed data collections hosted on dedicated storage and computational resources located at each participating site, a federated data management system and data integration environment, an Extensible Markup Language (XML) schema for data exchange, and analysis pipelines, designed to leverage both the distributed data management environment and the available grid computing resources.

The Biomedical Informatics Research Network (BIRN) is creating, extending, and deploying a production grid to enable groups of biomedical researchers to share complex and large-scale data. In this talk we describe the BIRN, its structure and overall Information Technology Structure. In addition we will describe the details of how the complete software stack is integrated, managed and deployed. The Rocks toolkit, originally designed for clusters has been greatly influenced by the deployment needs of BIRN, GEON, the OptIPuter and other large science-based grids. BIRN is funded by the U.S. National Institutes of Health and integrates science teams across more than 20 institutions to share and federate 3- dimensional medical image data. We will describe, within the context of BIRN, the challenges of everincreasing data set sizes, software stack complexity, integration with other production grids, and overall system reproducibility.

In this paper we present 411, a password distribution system for high performance environments that provides security and scalability. We show that existing solutions such as NIS and Kerberos do not provide sufficient performance in large, tightly coupled systems such as computational clusters. Unlike existing single-signon services, the 411 design removes the need for communication during password lookup by using aggressive replication techniques. We demonstrate the use of shared keys to efficiently protect user information, and the careful management of system wide consistency and fault tolerance. A theoretical analysis of the behavior of 411 is matched with quantitative evidence of its performance and suitability to a clustered environment. We further show the system effectively responds to stress by simulating 50% message loss on a 60-node cluster. This protocol is currently used worldwide in hundreds of Rocks-based production systems to provide password and login information service.

The Rocks toolkit uses a graph-based framework to describe the configuration of all node types (termed appliances) that make up a complete cluster. With hundreds of deployed clusters, our turnkey systems approach has shown to be quite easily adapted to different hardware and logical node configurations. However, the Rocks architecture and implementation contains a significant asymmetry: the graph definition of all appliance types except the initial frontend can be modified and extended by the end-user before installation. However, frontends can be modified only afterward by hands-on system administration. To address this administrative discontinuity between nodes and frontends, we describe the design and implementation of Rolls. First and foremost, Rolls provide both the architecture and mechanisms that enable the end-user to incrementally and programmatically modify the graph description for all appliance types. New functionality can be added and any Rocks-supplied software component can be overwritten or removed simply by inserting the desired Roll CD(s) at installation time. This symmetric approach to cluster construction has allowed us to shrink the core of the Rocks implementation while increasing flexibility for the end-user. Rolls are optional, automatically configured, cluster-aware software systems. Current add-ons include: scheduling systems (SGE, PBS), grid support (based on NSF Middleware Initiative), database support (DB2), Condor, integrity checking (Tripwire) and the Intel compiler. Community-specific Rolls can be and are developed by groups outside of the Rocks core development group.

Abstract High‐performance computing clusters (commodity hardware with low‐latency, high‐bandwidth interconnects) based on Linux are rapidly becoming the dominant computing platform for a wide range of scientific disciplines. Yet, straightforward software installation, maintenance, and health monitoring for large‐scale clusters has been a consistent and nagging problem for non‐cluster experts. The NPACI Rocks distribution takes a fresh perspective on management and installation of clusters to dramatically simplify software version tracking and cluster integration. NPACI Rocks incorporates the latest Red Hat distribution (including security patches) with additional cluster‐specific software. Using the identical software tools used to create the base distribution, users can customize and localize Rocks for their site. Strong adherence to widely‐used (de facto) tools allows Rocks to move with the rapid pace of Linux development. Version 2.2.1 of the toolkit is available for download and installation. Over 100 Rocks clusters have been built by non‐cluster experts at multiple institutions (residing in various countries) providing a peak aggregate of 2 TFLOPS of clustered computing. Copyright © 2003 John Wiley & Sons, Ltd.

Clusters have made the jump from lab prototypes to full-fledged production computing platforms. The number variety, and specialized configurations of these machines are increasing dramatically with 32-128 node clusters being commonplace in science labs. The evolving nature of the platform is to target generic PC hardware to specialized functions such as login, compute, Web server file server and a visualization engine. This is the logical extension to the standard login/compute dichotomy of traditional Beowulf clusters. Clearly, these specialized nodes (henceforth "cluster appliances") share an immense amount of common configuration and software. What is lacking in many clustering toolkits is the ability to share configuration across appliances and specific hardware (where it should be shared) and differentiate only where needed In the NPACI Rocks cluster distribution, we have developed a configuration infrastructure with well-defined inheritance properties that leverages and builds on de facto standards including: XML (with standard parsers), RedHat Kickstart, HTTP transport, CGI, SQL databases, and graph constructs to easily define cluster appliances. Our approach neither resorts to replication of configuration files nor does it require building a "golden" image reference. By relying on this descriptive and programmatic infrastructure and carefully demarking configuration information from the software packages (which is a bit delivery mechanism), we can easily handle the heterogeneity of appliances, easily deal with small hardware differences among particular instances of appliances (such as IDE vs. SCSI), and support large hardware differences (like /spl times/86 vs. IA64) with the same infrastructure. Our mechanism is easily extended to other descriptive infrastructures (such as Solaris Jumpstart as a backend target) and has been proven on over a 100 clusters (with significant hardware and configuration differences among these clusters).

High performance computing, storage, visualization, and database infrastructures are increasing geometrically in complexity as scientists move towards grid-based computing. While this is natural, it has the effect of pushing computational capabilities beyond the reach of scientists because of the time needed to harness the infrastructure. Hiding the complexity of networked resources becomes essential if scientists are to utilize them effectively. In this work, we describe our efforts to integrate various computational chemistry components into a scientific computing environment. We briefly describe improvements we have made to individual components of the chemistry environment as well as future directions, followed by a more in-depth discussion of our strategy for integration into a grid workflow environment based on web services, which enables access to remote resources while shielding users from the complexities of the grid infrastructures. A preliminary schema for storing data obtained from computational chemistry calculations is also described.

High-performance computing clusters (commodity hardware with low-latency, high-bandwidth interconnects) based on Linux, are rapidly becoming the dominant computing platform for a wide range of scientific disciplines. Yet, straightforward software installation, maintenance, and health monitoring for large-scale clusters has been a consistent and nagging problem for non-cluster experts. The NPACI Rocks toolkit takes a fresh perspective on management and installation of clusters to dramatically simplify software version tracking, and cluster integration. The toolkit incorporates the latest Red Hat distribution (including security patches) with additional cluster-specific software. Using the identical software tools used to create the base distribution, users can customize and localize Rocks for their site. Strong adherence to widely-used (de facto) tools allows Rocks to move with the rapid pace of Linux development. Version 2.1 of the toolkit is available for download and installation. To date, 10 clusters spread among 5 institutions have been built using this toolkit.

We describe our vision, goals and plans for HARNESS, a distributed, reconfigurable and heterogeneous computing environment that supports dynamically adaptable parallel applications. HARNESS builds on the core concept of the personal virtual machine as an abstraction for distributed parallel programming, but fundamentally extends this idea, greatly enhancing dynamic capabilities. HARNESS is being designed to embrace dynamics at every level through a pluggable model that allows multiple distributed virtual machines (DVMs) to merge, split and interact with each other. It provides mechanisms for new and legacy applications to collaborate with each other using the HARNESS infrastructure, and defines and implements new plug-in interfaces and modules so that applications can dynamically customize their virtual environment. HARNESS fits well within the larger picture of computational grids as a dynamic mechanism to hide the heterogeneity and complexity of the nationally distributed infrastructure. HARNESS DVMs allow programmers and users to construct personal subsets of an existing computational grid and treat them as unified network computers, providing a familiar and comfortable environment that provides easy-to-understand scoping.