Abstract The National Science Foundation's Extensible Terascale Facility (ETF), or TeraGrid ( http://www.teragrid.org/ ), is entering its operational phase. An example of an ETF science gateway effort is the Neutron Science TeraGrid Gateway (NSTG). The Oak Ridge National Laboratory (ORNL) resource provider effort (ORNL‐RP) now in operation is bridging the gap between a large‐scale experimental community and the TeraGrid as a large‐scale national cyberinfrastructure. Of particular importance here is the collaboration with the Spallation Neutron Source (SNS) at ORNL. The U.S. Department of Energy's SNS ( http://www.sns.gov/ ) at ORNL will be commissioned in the spring of 2006 as the world's brightest source of neutrons. Neutron science users can run experiments, generate datasets, perform data reduction, analysis, visualize results, collaborate with remotes users, and archive long‐term data in repositories with curation services. The ORNL‐RP and the SNS data analysis group have spent 18 months developing and exploring user requirements, including the creation of prototypical services such as a facility portal, data, and application execution services. We describe results from these efforts and discuss implications for science gateway creation. Finally, we show incorporation into implementation planning for the NSTG and SNS architectures. The plan is for a primarily portal‐based user interaction supported by a service‐oriented architecture for functional implementation. Published in 2006 by John Wiley & Sons, Ltd.

We have architected and evaluated a new kind of data resource, one that is composed of a logical collection of ephemeral data streams that could be viewed as a collection of publish-subscribe "channels" over which rich data-access and semantic operations can be performed. This paper contributes new insight to stream processing under the highly asynchronous stream workloads often found in data-driven scientific applications, and presents insights gained through porting a distributed stream processing system to a grid services framework. Experimental results reveal limits on stream processing rates that are directly tied to differences in stream rates.

Abstract — Mesoscale meteorology forecasting as a data driven application is capable of reacting to events in real-time. We explore a framework for dynamic filtering and mining of data products to generate timely triggers for invoking forecasting applications. In this paper, we present our framework, which couples the Calder stream processing system developed at Indiana University for filter processing and trigger generation, and data mining algorithms developed as part of the ADaM data mining tool kit developed at ITSC, UAH, which detect events for trigger generation.

Provenance management has become an integral part of many large-scale distributed computing systems. Tracking the history of data and its usage has led to better understanding of system requirements as well as user needs. Still, the need for an intelligent service that matches the system requirements with user needs is not satisfied. We propose a meta-provenance service that infers context from the provenance information of distributed entities and uses this contextual information to satisfy user needs. We describe our meta-provenance framework by way of describing its implementation in the Calder system. The Calder streaming system enables dynamic invocation of forecast models in LEAD by using a distributed mesh of data mining agents. The meta-provenance service enables sophisticated mapping of user queries from the LEAD portal down to the set of few data mining agents that execute them. Also our meta-provenance service can work at multiple levels of contextual granularity.

The contribution of this work has two folds. First, we extend the current query planners' cost metric space by introducing network bandwidth cost, query deployment cost and query re-using cost; second, we develop a suite of algorithms for re-using existing query fragments under different scenarios. One of the most important reusable queries is called structure-sharable query.

This paper presents an experimental evaluation of a grid-based continuous query solution to access data streams. The results presented in this study are mixed, however. We are migrating to a netCDF-based streaming model to measure system behavior in a setting that more accurately reflects a real use scenario. We are also working on enabling approximate query processing support to Calder to deal with sudden drop offs and changes in stream rates.