Scientific applications built on wide-area distributed systems such as emerging cloud based architectures and the legacy grid computing infrastructure often struggle with user adoption even though they succeed from a systems research perspective. This paper examines the coupling of user-centered design processes with modern distributed systems. Further in this paper, we describe approaches for conceptualizing a product that solves a recognized need: to develop a data gateway to serve the data management and research needs of experimentalists of electron microscopes and similar shared scientific instruments in the context of a research service laboratory. The purpose of the data gateway is to provide secure, controlled access to data generated from a wide range of scientific instruments. From the functional perspective, we focus on the basic processing of raw data that underlies the lab's "business" processes, the movement of data from the laboratory to central access and archival storage points, and the distribution of data to respective authorized users. Through the gateway interface, users will be able to share the instrument data with collaborators or copy it to remote storage servers. Basic pipelines for extracting additional metadata (through a pluggable parser framework) will be enabled. The core contribution described in this paper, building on the aforementioned distributed data management capabilities, is the adoption of user-centered design processes for developing the scientific user interface. We describe the user-centered design methodology for exploring user needs, iteratively testing the design, learning from user experiences, and adapting what we learn to improve design and capabilities. We further conclude that user-centered design is, in turn, best enabled by an adaptable distributed systems framework. A key challenge to implementing a user-centered design is to have design tools closely linked with a software system architecture that can evolve over time while providing a highly available data gateway. A key contribution of this paper is to share the insights from crafting such an evolvable design-build-evaluate-deploy architecture and plans for iterative development and deployment.

Flooding is one of the most prevalent and costliest global disasters. Disaster managers face significant challenges in managing essential information for preparedness, response, and recovery efforts. The development of an open access global flood alerting system for effective classification of potential impacts and the formulation of effective emergency response measures requires the incorporation of a wide variety of flood outputs derived from hydrologic and hydraulic models as well as from remote sensing derived data sets from multiple satellite/sensor platforms. We seek to rapidly classify flood severity using a model of models (MoM) approach that leverages products of existing flood models and incorporates Synthetic Aperture Radar (SAR) derived outputs for ground-truthing of model results and delineation of flood impact areas. The flood severity classification along with potential impacts estimated by using optical imagery will be disseminated as alerts using the Pacific Disaster Center's DisasterAWARE® decision support platform to users globally.

Quantifying Uncertainty and Kinematics of Earthquakes (QUAKES-A) provides an analytic center framework for creating a uniform crustal deformation reference model for the active plate margin of California by fusing InSAR, topographic, and GNSS geodetic imaging data. The objective is to provide tools for sampling spatial processes ranging from local to earthquake faults to broad tectonic deformation and temporal processes ranging from immediately following earthquakes to long-term tectonics. A reference model allows exploration of a uniform model and comparison to new data.

Geodetic imaging is useful for measuring topography and motions of the Earth's surface. Geodetic imaging measurements can be used, for example, to measure earthquakes, landslides, debris flows, wildfire extent, volcanos, and anthropogenic changes such as fluid withdrawal from aquifers. Different geodetic measurements sample different parts of the spatio-temporal deformation field. Combining the measurements and analysis improves understanding of a broad range of Earth surface processes. Here we describe a concept to Quantify Uncertainty and Kinematics of Earth Systems (QUAKES) that combines radar interferometry and optical imaging into one airborne platform and the analysis system required to analyze and model the data.

Summary Jetstream is a first of its kind system for the NSF — a distributed production cloud resource. We review the purpose for creating Jetstream, discuss Jetstream's key characteristics, describe our experiences from the first year of maintaining an OpenStack‐based cloud environment, and share some of the early scientific impacts achieved by Jetstream users. Jetstream offers a unique capability within the XSEDE‐supported US national cyberinfrastructure, delivering interactive virtual machines (VMs) via the Atmosphere interface. As a multi‐region deployment that operates as an integrated system, Jetstream is proving effective in supporting modes and disciplines of research traditionally underrepresented on larger XSEDE‐supported clusters and supercomputers. Already, Jetstream has been used to perform research and education in biology, biochemistry, atmospheric science, earth science, and computer science.

We describe the use of the Jetstream research-cloud, a purpose-built system with the goal of supporting "long-tail" research by providing a flexible, on-demand research infrastructure, to provide scalable back-end resources for science gateways. In addition to providing cloud-like resources for on-demand science, Jetstream offers the capability to instantiate long-running clusters which support science gateways. Science gateways are web-based systems built on computationalinfrastructure which provide commonly-used tools to a community of users. We created a persistent clusteron the Jetstream system which is connected to the SEAGrid science gateway and provides additional compute resources for a variety of quantum chemistry calculations. We discussthe further application of toolkits provided by the Extreme Science and Engineering Discovery Environment (XSEDE) tobuild general-purpose clusters on the research cloud.

The main goal of the US Science Gateways Community Institute (SGCI) is to serve science gateways to achieve sustainability and growth. Science gateways allow science and engineering communities to access shared data, software, computing services, instruments, educational materials, and other resources specific to their disciplines. Thus, science gateways are a subgroup of scientific software and the means for addressing software sustainability are also suitable for science gateways and vice versa, e.g., best practices for software engineering. Since science gateways are tailored to specific communities, understanding users' requirements is critical for sustainability. SGCI consists of five service areas that closely interact with each other. The Incubator acknowledges the value of business strategy to inform well-designed science gateways and offers two main types of services: individualized consultancy, tailored to specific challenges a gateway faces, and the Science Gateways Bootcamp. The cornerstone of the Bootcamp is a one-week onsite intensive workshop where participants create their own roadmap for a sustainable science gateway via sessions with experts, hands-on exercises, and group work. This paper offers an overview of the work of the Incubator and shares lessons learned from the inaugural session of the Bootcamp in April 2017.

eScience middleware frameworks integrating multiple virtual organizations must incorporate comprehensive user identity and access management solutions. In this paper we examine usage patterns for these systems and map the patterns to widely used security standards and approaches. We focus on science gateways, a class of distributed system cyberinfrastructure. Science gateways are end user environments that provide access to a wide range of academic and commercial computing and storage resources for virtual organizations. Successful gateways focus on specific scientific communities and domains, but they build on many reusable features that can be provided by general purpose hosted platform services that can support multiple tenants. Providing a security framework for identity and access management for such hosted service removes the burden for each gateway to handle its user identity management and control access to its critical resources. From the resource provider's point of view, it provides a basis for more uniform accounting and auditing. Challenges arise from the range of gateways (both legacy and newly created), the range of technologies used to build them, and the range of end user environments (Web, mobile, desktop, and programmatic API clients) that gateways provide. Using Apache Airavata as an implementation, we examine three common gateway types based on where the user identity information is held and how these can be treated in a unified manner using OAuth2 and OpenID-Connect. Our solutions for identity and access management are not specific to Apache Airavata but can be generally applied to any e-Science platform.

GeoGateway is a web-enabled map-based system for analysis, modeling, and response of geodetic imaging products for studying earthquakes and crustal deformation. The system provides a data product search and analysis gateway for scientific discovery, field use, and disaster response. To be effective users require data overlay and visualization, interactive analysis features, and data product download. The data products of focus in this project are NASA's UAVSAR and spaceborne interferometric radar, (InSAR), geologic earthquake faults, Global Positioning System (GPS) position time series, and seismicity.

Summary GenApp generates applications on an extensible set of target languages for scientific modules. GenApp utilizes JavaScript object notation (JSON) format for all definition files. To create an application, definition files are created for global directives, menu, and modules. Target languages have definition files detailing the steps‐mapping code fragments to output. Modules must be wrapped to accept and produce JSON as defined in the module's definition file. Execution models are not defined by GenApp; they are included in target language code fragments. Previously, GenApp included target languages of HTML5/PHP, Qt3/C++, and Qt4/C++ with execution models of direct local execution, a web server, or a web server accessible resource. A Google Summer of Code (GSoC) 2014 student demonstrated Airavata‐managed execution in GenApp's current target languages. Subsequently, Airavata's API and GenApp have evolved. Two GSoC‐2015 students updated the previous Airavata integration to support the current API and extend target languages to include Qt5/C++, Qt5/Android, and Java. GenApp was initially developed to wrap modules utilized in the small angles scattering field but is not restricted to this discipline. The GenApp philosophy is to minimize effort of the researcher to deploy modules and insure preservation in an evolving software landscape. Generated applications are in production and used by small angle scattering researchers. Copyright © 2015 John Wiley & Sons, Ltd.

Summary Science Gateways provide scientists with tools for creating, executing, and monitoring scientific experiments on multiple resource infrastructures. Apache Airavata abstracts interactions between gateways and distributed computing infrastructures like Extreme Science and Engineering Discovery Environment, international grids, and campus clusters. Airavata consists of several component services such as the API server, Orchestrator, Workflow Interpreter, Credential Store, and Application Factory. In addition, Airavata uses third party software, including RabbbitMQ for messaging, MySQL for production database management, and Apache Zookeeper for internal communications. In this paper, we discuss our initial experiences with leveraging open source technologies to manage Airavata and its dependent components to deploy, detect, and restart failed components in an auto‐scaling platform. Such capabilities will allow Airavata services to be deployed in a wide area, large Virtual Machine (VM) based cluster, and a developer's laptop. The emerging technologies in cloud computing and Big Data that address these needs are the following: Docker, Marathon, and Apache Mesos. Docker is a Linux‐based lightweight container that allows different applications to run isolated from each other but safely share the machine's resources. Docker images of applications can be published in registries and retrieved for execution in the target infrastructures. Marathon provides a cluster‐wide init and control system for services, including Docker containers. Mesos provides a cluster‐wide framework to schedule tasks based on fine‐grained resource needs. Mesosphere provides the packages, scripts, and web interface to ease the use of these technologies. We present the design, experience, and lessons learned from integrating Mesos, Docker, and Marathon with Apache Airavata. Copyright © 2015 John Wiley & Sons, Ltd.

Summary A new framework (GenApp) for rapid generation of scientific applications running on a variety of systems including science gateways has recently been developed. This framework currently builds a GUI and/or web‐based user interface for a variety of target environments on a collection of executable modules. The method for execution of modules has limited framework restrictions: primarily the requirement of wrapping the application to accept input and output formatted in JavaScript Object Notation (JSON). Initial implementation supports direct execution on a user's workstation, a web server, or a compute resource accessible from the web server. After a successful initial workshop utilizing the framework to create a web‐based user interface wrapping a scientific software suite, it was discovered that long‐running jobs would sometimes fail, because of the loss of a Transmission Control Protocol (TCP) connection. This precipitated an improvement to the execution method with the bonus of easily allowing multiple web clients to attach to the running job. To support a diversity of queue managed compute resources, a Google ‘Summer of Code’ project was completed to integrate the Apache Airavata middleware as an additional execution model within the GenApp framework. New features of file management, job management with progress, and message box support are described. Concurrency and Computation: Practice and Experience, 2015.© 2015 Wiley Periodicals, Inc.

This deposit includes (1) the invitation and reminder messages sent to the survey sample and (2) the survey questions, including instructions for branching.

Summary This paper provides an overview and roadmap of the Apache Airavata software system for science gateways. Gateways use Airavata to manage application and workflow executions on a range of backend resources (grids, computing clouds, and local clusters). Airavata's design goal is to provide component abstractions for major tasks required to provide gateway application management. Components are not directly accessed but are instead exposed through component programming interfaces. This design allows gateway developers to take full advantage of Airavata's capabilities and Airavata developers (including those interested in middleware research) to modify Airavata's implementations and behavior. This is particularly important as Airavata evolves to become a scalable, elastic ‘platform as a service’ for science gateways. We illustrate the capabilities of Airavata through the discussion of usage vignettes. As an Apache Software Foundation project, Airavata's open community governance model is as important as its software base. We discuss how this works within Airavata and how it may be applicable to other distributed computing infrastructure and cyberinfrastructure efforts. Copyright © 2015 John Wiley & Sons, Ltd.

Science Gateways bridge multiple computational grids and clouds, acting as overlay cyber infrastructure. Gateways have three logical tiers: a user interfacing tier, a resource tier and a bridging middleware tier. Different groups may operate these tiers. This introduces three security challenges. First, the gateway middleware must manage multiple types of credentials associated with different resource providers. Second, the separation of the user interface and middleware layers means that security credentials must be securely delegated from the user interface to the middleware. Third, the same middleware may serve multiple gateways, so the middleware must correctly isolate user credentials associated with different gateways. We examine each of these three scenarios, concentrating on the requirements and implementation of the middleware layer. We propose and investigate the use of a Credential Store to solve the three security challenges.

Science gateways provide user-centric views to cyberinfrastructure resources, simplifying usage and enabling a richer user experience. To enable the goals of a science gateway and the communities of scientists it supports, gateway developers need to be able to spend more time on designing and developing the user experience and less time on wrestling with the underlying technology (such as HTML5, CSS, and JavaScript). In this poster, we describe our experiences using Twitter Bootstrap and AngularJS frameworks to address this balance between design and implementation, empowering developers to create better styled and easily maintainable websites.

Over the last fifteen years, science gateways have proven to be fertile ground for cyberinfrastructure research, while at the same time dramatically increasing the usage and accessibility of cyberinfrastructure to scientists and educators. Gateway developers, however, still face many challenges in building and operating these complex infrastructures that address the dynamic and emerging challenges of computational infrastructure. Through the XSEDE Science Gateway Program, we propose to solicit information on the knowledge, experience, and software from widely used science gateways. We will compile the information into a “Science Gateway Cookbook” which will democratize the experiences in developing, operating, and sustaining science gateways. We hope this will be a good literature reference and also avoid some repetition in the development process.

MapReduce is increasingly considered as a useful parallel programming model for large-scale data processing. It exploits parallelism among execution of primitive map and reduce operations. Hadoop is an open source implementation of MapReduce that has been used in both academic research and industry production. However, its implementation strategy that one map task processes one data block limits the degree of concurrency and degrades performance because of inability to fully utilize available resources. In addition, its assumption that task execution time in each phase does not vary much does not always hold, which makes speculative execution useless. In this paper, we present mechanisms to dynamically split and consolidate tasks to cope with load balancing and break through the concurrency limit resulting from fixed task granularity. For single-job systems, two algorithms are proposed for circumstances where prior knowledge is known and unknown. For multi-job cases, we propose a modified shortest-job-first strategy, which minimizes job turnaround time theoretically when combined with task splitting. We compared the effectiveness of our approach to the default task scheduling strategy using both synthesized and trace-based workloads. Simulation results show that our approach improves performance significantly.

Many science applications require more and more computing power as the amount of input data keeps increasing. To simplify using large-scale clusters and complicated application codes, and to facilitate cross-disciplinary collaboration, there has been substantial research and development work on Web-based science gateways. With numerous gateways needing to be developed for many different scientific domains, there has also been a long-standing need for reusable codes and extensible component models. During the previous decade, the component model for many gateways was the Java portlet. To overcome some of the port let model's limitations, new gateways take a different approach that utilizes modern Web technologies. In this paper, we examine the use of new standards such as Open Social, Gadgets, and W3C Widgets to build science gateway user interfaces. These standards overcome many shortcomings of the older port let development model. As general-purpose Web standards, however, they lack support for specialized science gateway requirements and so must be extended. We propose a generic architecture in which Open Social is integrated with backend services and grid infrastructures. We implement and evaluate these concepts in the Gadget Container, software developed by the authors.

The development of cloud infrastructures has stimulated interest in virtualized block storage systems, exemplified by Amazon Elastic Block Store (EBS), Eucalyptus’ EBS implementation, and the Virtual Block Store (VBS) system. Compared with other solutions, VBS is designed for flexibility, and can be extended to support various Virtual Machine Managers and Cloud platforms. However, due to its single-volume-server architecture, VBS has the problem of single point of failure and low scalability. This paper presents our latest improvements to VBS for solving these problems, including a new distributed architecture based on the Lustre file system, new workflows, better reliability and scalability, and read-only volume sharing. We call this improved implementation VBS-Lustre. Preliminary tests show that VBS-Lustre can provide both better throughput and higher scalability in multiple attachment scenarios than VBS. VBS-Lustre could potentially be applied to solve some challenges for current cluster file systems, such as metadata management and small file access.

Abstract We introduce a distributed high‐performance Information Service Architecture, which forms a metadata replica hosting system to manage both highly dynamic small‐scale metadata and relatively large static metadata associated with Grid/Web Services. We present an empirical evaluation of the proposed architecture and investigate its practical usefulness. The results demonstrate that the proposed system achieves high performance and fault tolerance with negligible processing overheads. The results also indicate that efficient decentralized hybrid Information Service Architectures can be built by utilizing publish‐subscribe‐based messaging schemes. Copyright © 2010 John Wiley & Sons, Ltd.

Abstract Traditional techniques in building science portals and gateways are being challenged by new techniques such as Web 2.0 and Cloud Computing. This paper discusses some of our efforts to evaluate these techniques as we evolve the QuakeSim architecture. We believe that architecturally both traditional and newer approaches for Gateways are very similar; thus, giving us a path for moving to hybrid approaches. In this paper, we specifically evaluate techniques for building interactive user interfaces that rely on remote services; architectural approaches for managing massive job submissions that can include both parallel and serial jobs; and an architectural prototype for building component‐based containers compatible with emerging standards. Copyright © 2009 John Wiley & Sons, Ltd.

Abstract This paper describes Indiana University's implementation, performance testing, and use of a large high performance computing system. IU's Big Red, a 20.48 TFLOPS IBM e1350 BladeCenter cluster, appeared in the 27th Top500 list as the 23rd fastest supercomputer in the world in June 2006. In spring 2007, this computer was upgraded to 30.72 TFLOPS. The e1350 BladeCenter architecture, including two internal networks accessible to users and user applications and two networks used exclusively for system management, has enabled the system to provide good scalability on many important applications while being well manageable. Implementing a system based on the JS21 Blade and PowerPC 970MP processor within the US TeraGrid presented certain challenges, given that Intel‐compatible processors dominate the TeraGrid. However, the particular characteristics of the PowerPC have enabled it to be highly popular among certain application communities, particularly users of molecular dynamics and weather forecasting codes. A critical aspect of Big Red's implementation has been a focus on Science Gateways, which provide graphical interfaces to systems supporting end‐to‐end scientific workflows. Several Science Gateways have been implemented that access Big Red as a computational resource—some via the TeraGrid, some not affiliated with the TeraGrid. In summary, Big Red has been successfully integrated with the TeraGrid, and is used by many researchers locally at IU via grids and Science Gateways. It has been a success in terms of enabling scientific discoveries at IU and, via the TeraGrid, across the US. Copyright © 2009 John Wiley & Sons, Ltd.

The fast development of cloud computing systems stimulates the needs for a standalone block storage system to provide persistent block storage services to virtual machines maintained by clouds. This paper presents the Virtual Block Store (VBS) System, a standalone block storage system built on the basis of LVM, iSCSI, and Xen hypervisor, which can provide basic block storage services such as volume creation and attachment. The concept and functional interface of VBS are based on Amazon Elastic Block Store (EBS) service; moreover, VBS works independently with an existing LVM volume server and Xen nodes, and thus can be easily extended to support other types of volume servers and virtual machine managers, or integrated with various cloud computing systems. Preliminary I/O benchmark results are presented and analyzed, indicating that a VBS volume can provide throughput that is similar to an ATA over Ethernet virtual device.

Abstract We discuss the application of Web 2.0 to support scientific research (e‐Science) and related ‘e‐more or less anything’ applications. Web 2.0 offers interesting technical approaches (protocols, message formats, and programming tools) to build core e‐infrastructure (cyberinfrastructure) as well as many interesting services (Facebook, YouTube, Amazon S3/EC2, and Google maps) that can add value to e‐infrastructure projects. We discuss why some of the original Grid goals of linking the world's computer systems may not be so relevant today and that interoperability is needed at the data and not always at the infrastructure level. Web 2.0 may also support Parallel Programming 2.0—a better parallel computing software environment motivated by the need to run commodity applications on multicore chips. A ‘Grid on the chip’ will be a common use of future chips with tens or hundreds of cores. Copyright © 2008 John Wiley & Sons, Ltd.

Abstract Web 2.0 approaches are revolutionizing the Internet, blurring lines between developers and users and enabling collaboration and social networks that scale into the millions of users. As discussed in our previous work, the core technologies of Web 2.0 effectively define a comprehensive distributed computing environment that parallels many of the more complicated service‐oriented systems such as Web service and Grid service architectures. In this paper we build upon this previous work to discuss the applications of Web 2.0 approaches to four different scenarios: client‐side JavaScript libraries for building and composing Grid services; integrating server‐side portlets with ‘rich client’ AJAX tools and Web services for analyzing Global Positioning System data; building and analyzing folksonomies of scientific user communities through social bookmarking; and applying microformats and GeoRSS to problems in scientific metadata description and delivery. Copyright © 2009 John Wiley & Sons, Ltd.

Problem solving environments (PSEs) are increasingly important for scientific discovery. Today's most challenging problems often require multi-disciplinary teams, the ability to analyze very large amounts of data, and the need to rely on infrastructure built by others rather than reinventing solutions for each science team. The TeraGrid Science Gateways program recognizes these challenges and works with science teams to harness high-end resources that significantly extend a PSE's functionality.

Many scientific fields routinely generate huge datasets. In many cases, these datasets are not static but rapidly grow in size. Handling these types of datasets, as well as allowing sophisticated queries necessitates scalable distributed database systems, in which scientists are efficiently able to search the datasets. In this paper we present the architecture, implementation and performance analysis of a scalable, distributed database system built on software based virtualization environments. The system architecture makes use of a software partitioning of the database based on data clustering, SQMD (single query multiple database) mechanism, a Web service interface, and virtualization software technologies. The system allows uniform access to concurrently distributed databases, using the SQMD mechanism based on the publish/subscribe paradigm. We highlight the scalability of our architecture by applying it to a database of 17 million chemical structures. In addition to simple identifier based retrieval, we will present performance results for shape similarity queries, which is extremely, time intensive with traditional architectures.

E-Science users will want to construct, share, execute and monitor sequence of tasks. These tasks may execute on machines ranging from their local workstations to high-end, grid-enabled compute resources. Often, these tasks are legacy applications written in various programming and scripting languages and are designed to be run in a single user environment rather than as a Web application. These tasks often need to be tied together into composite applications that need to span multiple computing resources. The majority of the scientific experiments in E-Science involve orchestrating multiple tasks in the correct fashion to produce scientific computational experiments. Scientific workflows have proven to be a coherent and abstract framework that is capable of capturing such scientific experiments and hence have gained popularity among the scientific community. Many of the current E-Science projects have adopted different workflow systems and consequently have adopted custom workflow execution and description standards. We argue that the core of the execution semantics used by most scientific workflows can be expressed as a small common set, and we further investigate the possibility of having a higher level workflow description language that is capable of capturing and possibly bridging these different workflow systems. In this demonstration we present a set of E-Science development tools for managing these problems: a high level workflow composition monitoring and enactment tool, a toolkit that is capable of providing Web service interfaces for command line scientific applications, and a service registry that is used as a repository for sharing. This domain independent workflow suite will allow users from wide verity of environments to selectively and securely share their applications as Web services and construct workflows with these services. The suite also features on-demand service creation, workflow orchestration and monitoring. Furthermore, these components can be used individually or collectively to build a small to large-scale E-Science infrastructures.

Compute-intensive scientific applications are heavily reliant on the available quantity of computing resources. The Grid paradigm provides a large scale computing environment for scientific users. However, conventional Grid job submission tools do not provide a high-level job scheduling environment for these users across multiple institutions. For extremely large number of jobs, a more scalable job scheduling framework that can leverage highly distributed clusters and supercomputers is required. In this paper, we propose a high-level job scheduling Web service framework, Swarm. Swarm is developed for scientific applications that must submit massive number of high-throughput jobs or workflows to highly distributed computing clusters. The Swarm service itself is designed to be extensible, lightweight, and easily installable on a desktop or small server. As a Web service, derivative services based on Swarm can be straightforwardly integrated with Web portals and science gateways. This paper provides the motivation for this research, the architecture of the Swarm framework, and a performance evaluation of the system prototype.

Microarray technology is a high-throughput experimental technique that can measure expression levels of hundreds of thousands of genes simultaneously. To interpret massive data from gene-expression microarray experiments, biologists encounter computational and analytical challenges. This is especially challenging for small research labs that lack local computing and bioinformatics expertise. Here, we introduce a virtual analysis system for microarray gene expression data in computing clouds with flexible and configurable GUI workflow engine so that biologists are able to analyze the data in many angles without worrying about computational and bioinformatics issues.

We report on a unified on-the-fly, Web services-based observation/analysis/modeling environment for crustal deformation and natural hazards research, intended as a plug-in service for early warning systems, transfer of rapid information to civilian decision makers and the media, and educational purposes. We demonstrate an early warning system for a large earthquake in southern California using the components developed under several NASA-funded projects, including real-time GPS network infrastructure, Geophysical Resources Web Services, and GPS Explorer data portal with its on-the-fly earthquake modeling software. We generate an on-the-fly earthquake fault model based on simulated real-time (dynamic and coseismic) displacements computed by the GPS network, and viewable through GPS Explorer. Our objective is to demonstrate an operational service that could be transitioned to appropriate civilian agencies in southern California.

Abstract As service‐oriented architecture principles have gained importance, an emerging need has appeared for methodologies to locate desired services that provide access to their capability descriptions. These services must typically be assembled into short‐term service collections that, together with code execution services, are combined into a meta‐application to perform a particular task. To address the metadata requirements of these problems, we introduce a hybrid Information Service to manage both stateless and stateful (transient) metadata. We leverage the two widely used Web Service standards: Universal Description, Discovery and Integration (UDDI) and Web Services Context (WS‐Context), in our design. We describe our approach and experiences when designing ‘semantics’. We report the results from a prototype of the system that is applied to a mobile environment for optimizing Web Service communications. Copyright © 2007 John Wiley & Sons, Ltd.

Abstract We discuss the development and application of Web‐service‐based geographical information system (GIS) Grids. Following the WS‐I+ approach of building Grids on Web service standards, we have developed data Grid components for archival and real‐time data, map generating services that can be used to build user interfaces, information services for storing both stateless and stateful metadata, and service orchestration and management tools. Our goal is to support dynamically assembled Grid service collections that combine both GIS services with more traditional Grid capabilities such as file transfer and remote code execution. We are applying these tools to problems in earthquake modeling and forecasting, but we are attempting to build general purpose tools by using and extending appropriate standards. Copyright © 2008 John Wiley & Sons, Ltd.

Abstract Grids and Geospatial Information Systems (GIS) are both based on distributed service architectures and have complementary capabilities. GIS systems provide a comprehensive set of services for managing maps, geospatial data sets, and geospatial information that can be applied to areas ranging from access to scientific data by researchers to disaster planning and emergency management. The data and information focus of GIS are being augmented with the computational and virtual organization capabilities of Grid computing by many projects, including the ones by the contributors to this special issue. This editorial introduction serves as an overview of the issues discussed at the GIS‐Grid Workshop in the Open Grid Forum and the follow‐on papers of this special issue. Copyright © 2008 John Wiley & Sons, Ltd.

We present a method for performing mode classification of real-time streams of GPS surface position data. Our approach has two parts: an algorithm for robust, unconstrained fitting of hidden Markov models (HMMs) to continuous-valued time series, and SensorGrid technology that manages data streams through a series of filters coupled with a publish/subscribe messaging system. The SensorGrid framework enables strong connections between data sources, the HMM time series analysis software, and users. We demonstrate our approach through a Web portal environment through which users can easily access data from the SCIGN and SOPAC GPS networks in Southern California, apply the analysis method, and view results. Ongoing real-time mode classifications of streaming GPS data are displayed in a map-based visualization interface

The service-oriented architecture has come a long way in solving the problem of reusability of existing software resources. Grid applications today are composed of a large number of loosely coupled services. While this has opened up new avenues for building large, complex applications, it has made the management of the application components a nontrivial task. Use of services existing on different platforms, implemented in different languages and presence of variety of network constraints further complicates management. This paper investigates problems that emerge when there is a need to uniformly manage a set of distributed services. We present a scalable, fault-tolerant management framework. Our empirical evaluation shows that the architecture adds an acceptable number of additional resources for providing scalable, fault-tolerant management framework.

The service-oriented architecture has come a long way in solving the problem of reusability of existing software resources. Grid applications today are composed of a large number of loosely coupled services. While this has opened up new avenues for building large, complex applications, it has made the management of the application components a non-trivial task. Management is further complicated when services exist on different platforms, are written in different languages, present in varying administrative domains restricted by firewalls and are susceptible to failure. This paper investigates problems that emerge when there is a need to uniformly manage a set of distributed services. We present a scalable, fault-tolerant management framework. Our empirical evaluation shows that the architecture adds an acceptable number of additional resources making the approach feasible.

Abstract We review the efforts of the Open Grid Computing Environments collaboration. By adopting a general three‐tiered architecture based on common standards for portlets and Grid Web services, we can deliver numerous capabilities to science gateways from our diverse constituent efforts. In this paper, we discuss our support for standards‐based Grid portlets using the Velocity development environment. Our Grid portlets are based on abstraction layers provided by the Java CoG kit, which hide the differences of different Grid toolkits. Sophisticated services are decoupled from the portal container using Web service strategies. We describe advance information, semantic data, collaboration, and science application services developed by our consortium. Copyright © 2006 John Wiley & Sons, Ltd.

Abstract We discuss our message‐based approach to managing real‐time data streams and building higher level services to produce and consume them. Our messaging system acts as a substrate that can be used to provide qualities of service to various streaming applications ranging from audio–video collaboration systems to sensor Grids. The messaging substrates are composed of distributed, hierarchically arranged message broker networks. Services such as filters are deployed along the edges of the network. We discuss the role of management systems for both broker networks and filter services: broker network topologies must be created and maintained, and distributed filters must be arranged in appropriate sequences. These managed broker networks may be applied to a wide range of problems. We discuss applications to audio–video collaboration in some detail and also describe applications to streaming Global Positioning System data streams. These provide specific application filters that can transform and republish message streams to the broker system. Copyright © 2006 John Wiley & Sons, Ltd.

Abstract We present the initial architecture and implementation of VLab, a Grid and Web‐Service‐based system for enabling distributed and collaborative computational chemistry and material science applications for the study of planetary materials. The requirements of VLab include job preparation and submission, job monitoring, data storage and analysis, and distributed collaboration. These components are divided into client entry (input file creation, visualization of data, task requests) and back‐end services (storage, analysis, computation). Clients and services communicate through NaradaBrokering, a publish/subscribe Grid middleware system that identifies specific hardware information with topics rather than IP addresses. We describe three aspects of VLab in this paper: (1) managing user interfaces and input data with JavaBeans and Java Server Faces; (2) integrating Java Server Faces with the Java CoG Kit; and (3) designing a middleware framework that supports collaboration. To prototype our collaboration and visualization infrastructure, we have developed a service that transforms a scalar data set into its wavelet representation. General adaptors are placed between the endpoints and NaradaBrokering, which serve to isolate the clients/services from the middleware. This permits client and service development independently of potential changes to the middleware. Copyright © 2007 John Wiley & Sons, Ltd.

Recent specifications such as WS-management and WS-distributed management have stressed the importance of management of resources and services and propose methods towards querying Web services to gather the meta-data associated with these services. Management often entails system setup, querying system metadata, manipulation of system parameters at runtime and taking actions based on the system parameters to tune system performance. Real-time applications require rapid deployment of application components and demand results in real time. In this paper we present the HPSearch system which enables dynamic management of the system including both streams and Web services, and rapid deployment of applications via a scripting interface. We illustrate the functioning of the system by modeling a data streaming application and rapidly deploying the system and application components.

We study the current architecture of the grid and Web services and that of the global information grid (GiG) with the Network Centric Operations and Warfare (NCOW) from the Department of Defense. We compare the GiG core enterprise services with those being developed for Grids (the open grid services architecture) and Web Services (so called WS-* specifications), identifying both similarities and differences. We discuss both modeling and simulation with HLA (high level architecture) and broad defense NCOW applications. We illustrate this analysis with an open geospatial community (OGC) compatible set of geographical information system grid services. We illustrate the use of grids to efficiently support realtime simulation by an application of grids to audio-video conferencing.

We designed eSports - a collaborative and synchronous video annotation platform, which is to be used in Internet scale cross-platform grid computing environment to facilitate computer supported cooperative work (CSCW) in education settings such as distance sport coaching, distance classroom etc. Different from traditional multimedia annotation systems, eSports provides the capabilities to collaboratively and synchronously play and archive real time live video, to take snapshots, to annotate video snapshots using whiteboard and to play back the video annotations synchronized with original video streams. eSports is designed based on the grid based collaboration paradigm $the shared event model using NaradaBrokering, which is a publish/subscribe based distributed message passing and event notification system. In addition to elaborate the design and implementation of eSports, we analyze the potential use cases of eSports under different education settings. We believed that eSports is very useful to improve the online collaborative coaching and education.

This work describes an approach to building Grid applications based on the premise that users who wish to access and run these applications prefer to do so without becoming experts on Grid technology. We describe an application architecture based on wrapping user applications and application workflows as Web services and Web service resources. These services are visible to the users and to resource providers through a family of Grid portal components that can be used to configure, launch, and monitor complex applications in the scientific language of the end user. The applications in this model are instantiated by an application factory service. The layered design of the architecture makes it possible for an expert to configure an application factory service with a custom user interface client that may be dynamically loaded into the portal.

Computational portal developers often reimplement functionality found in existing portals because there are no common discovery and access mechanisms in place to enable sharing of portal functions. The Web services architecture provides an implementation independent, protocol based mechanism for entities to find, share, and invoke remote services. We developed complementary web services at SDSC and IU that generate batch scripts for different batch queuing schedulers.

Computational web portals are designed to simplify access to diverse sets of high performance computing resources, typically through an interface to computational Grid tools. An important shortcoming of these portals is their lack of interoperable and reusable services. This paper presents an overview of research efforts undertaken by our group to build interoperating portal services around a Web Services model. We present a comprehensive view of an interoperable portal architecture, beginning with core portal services that can be used to build Application Web Services, which in turn may be aggregated and managed through portlet containers.

Abstract We describe an information system architecture for the ACES (Asia–Pacific Cooperation for Earthquake Simulation) community. It addresses several key features of the field—simulations at multiple scales that need to be coupled together; real‐time and archival observational data, which needs to be analyzed for patterns and linked to the simulations; a variety of important algorithms including partial differential equation solvers, particle dynamics, signal processing and data analysis; a natural three‐dimensional space (plus time) setting for both visualization and observations; the linkage of field to real‐time events both as an aid to crisis management and to scientific discovery. We also address the need to support education and research for a field whose computational sophistication is rapidly increasing and spans a broad range. The information system assumes that all significant data is defined by an XML layer which could be virtual, but whose existence ensures that all data is object‐based and can be accessed and searched in this form. The various capabilities needed by ACES are defined as grid services, which are conformant with emerging standards and implemented with different levels of fidelity and performance appropriate to the application. Grid Services can be composed in a hierarchical fashion to address complex problems. The real‐time needs of the field are addressed by high‐performance implementation of data transfer and simulation services. Further, the environment is linked to real‐time collaboration to support interactions between scientists in geographically distant locations. Copyright © 2002 John Wiley & Sons, Ltd.

Abstract In this paper we describe the basic services and architecture of Gateway, a commodity‐based Web portal that provides secure remote access to unclassified Department of Defense computational resources. The portal consists of a dynamically generated, browser‐based user interface supplemented by client applications and a distributed middle tier, WebFlow. WebFlow provides a coarse‐grained approach to accessing both stand‐alone and Grid‐enabled back‐end computing resources. We describe in detail the implementation of basic portal features such as job submission, file transfer, and job monitoring and discuss how the portal addresses security requirements of the deployment centers. Finally, we outline future plans, including integration of Gateway with Department of Defense testbed Grids. Copyright © 2002 John Wiley & Sons, Ltd.