Robust scientific methods require the exploration of the parameter space of a system (some of which can be run in parallel on distributed resources), and may involve complete state space exploration, experimental design, or numerical optimization techniques. Many-Task Computing (MTC) provides a framework for performing robust design, because it supports the execution of a large number of otherwise independent processes. Further, scientific workflow engines facilitate the specification and execution of complex software pipelines, such as those found in real science and engineering design problems. However, most existing workflow engines do not support a wide range of experimentation techniques, nor do they support a large number of independent tasks. In this paper, we discuss Nimrod/K—a set of add in components and a new run time machine for a general workflow engine, Kepler. Nimrod/K provides an execution architecture based on the tagged dataflow concepts, developed in 1980s for highly parallel machines. This is embodied in a new Kepler "Director” that supports many-task computing by orchestrating execution of tasks on on clusters, Grids, and Clouds. Further, Nimrod/K provides a set of "Actors” that facilitate the various modes of parameter exploration discussed above. We demonstrate the power of Nimrod/K to solve real problems in cardiac science.

Modern chemistry involves a mix of experimentation and computer-supported theory. Historically, these skills have been provided by different groups, and range from traditional “wet” laboratory science to advanced numerical simulation. This paper discusses the application of advanced e-Science software tools to electrochemistry research and involves collaboration between laboratories at Monash and Oxford Universities. In particular, we show how the Nimrod/OK tool can be used to automate the estimation of electrochemical parameters in Fourier transformed voltammetry. Replacing an ad-hoc manual process with e-Science tools both accelerates the process and produces more accurate solutions. In this work much attention was given to the role of the scientist user. During the research, Nimrod/K (on which Nimrod/OK is built) was extended to shield that user from technical details of the grid infrastructure, a new system of file management and grid abstraction has been incorporated.

Computational grids have been at the forefront of e-science supercomputing for several years now. Many issues have been settled over the years, but some may only appear to be so. One of these issues is firewall traversal. Many solutions have been proposed and developed. We have developed two such solutions ourselves: Remus and Romulus, but like all other solutions, they are limited in application. Others are working on proposed standards or solutions based on existing Internet standards and RFCs. However, we still have production-level grids that instead operate their grid resources on an open firewall policy. Some propose moving grids on top of peer-topeer networks and/or overlay networks, or rebuilding grids on top of clouds instead. Existing grid infrastructures have not rushed to follow either path, however, as the required changes will take considerable effort and cost for currently running systems. This paper investigates the problem and offers a different proposal: a minor revision to the grid architecture. In order to support what we propose, we will look at several proposed solutions and identify their limitations. We also classify them into two distinct approaches, and discuss how each one is not by itself sufficient for all situations. Then we shall show that a slight improvement to the grid protocol architecture provides a multi-pronged architectural solution.

Most commercial microscopes are stand-alone instruments, controlled by dedicated computer systems. These provide limited storage and processing capabilities. Virtual microscopes, on the other hand, link the image capturing hardware and data analysis software into a wide area network of high performance computers, large storage devices and software systems. In this paper we discuss extensions to Grid workflow engines that allow them to execute scientific experiments on virtual microscopes. We demonstrate the utility of such a system in a biomedical case study concerning the imaging of cancer and antibody based therapeutics.

Due to its ability to provide high-performance computing environment, the grid has become an important infrastructure to support eScience. To utilise the grid for parameter sweep experiments, workflow technology combined with tools such as Nimrod/K are used to orchestrate and automate scientific services provided on the grid. As parameter sweeping over a workflow needs to be executed numerous times, it is more efficient to execute multiple instances of the workflow in parallel. However, this parallel execution can be delayed as every workflow instance requires the same set of resources leading to resource competition problem. Although many algorithms exist for scheduling grid workflows, there is little effort in considering multiple workflow instances and resource competition in the scheduling process. In this paper, we proposed a scheduling algorithm for parameter sweep workflow based on resource competition. The proposed algorithm aims to support multiple workflow instances and avoid allocating resources with high resource competition to minimise delay due to the blocking of tasks. The result is evaluated using simulation to compare with an existing scheduling algorithm.

Computational grids are now mainstream facilities for e-research worldwide. While enterprise grids exist within organizations, national grids have become common, usually consisting of government as well as academic facilities. Such facilities are not uncommonly lenient with blanket policies to allow inbound and outbound grid traffic. This is far from ideal, from a security perspective, but given the dynamic nature of grid use, it is impractical to keep restrictive firewalls and manually keep up with on-demand firewall reconfiguration. Other solutions are necessary, where security is not sacrificed. Apart from first generation solutions that were mostly not sufficiently generic, standardization work is now ongoing, but exclusively aimed at firewall virtualization. We argue for an architectural solution that encompasses firewall virtualization as well as other methods that can be more appropriate in many environments. This paper describes our notion of the missing layer between grid and fabric, which we refer to as the virtual connectivity layer. We have developed two implementations within this layer and discuss how they fit into a complete and well-defined architectural solution.

Determining the X-ray crystallographic structures of proteins using the technique of molecular replacement (MR) can be a time and labor-intensive trial-and-error process, involving evaluating tens to hundreds of possible solutions to this complex 3D jigsaw puzzle. For challenging cases indicators of success often do not appear until the later stages of structure refinement, meaning that weeks or even months could be wasted evaluating MR solutions that resist refinement and do not lead to a final structure. In order to improve the chances of success as well as decrease this timeframe, we have developed a novel grid computing approach that performs many MR calculations in parallel, speeding up the process of structure determination from weeks to hours. This high-throughput approach also allows parameter sweeps to be performed in parallel, improving the chances of MR success.

Abstract Applications that span multiple virtual organizations (VOs) are of great interest to the e‐science community. However, our recent attempts to execute large‐scale parameter sweep applications (PSAs) for real‐world climate studies with the Nimrod/G tool have exposed problems in the areas of fault tolerance, data storage and trust management. In response, we have implemented a task‐splitting approach that facilitates breaking up large PSAs into a sequence of dependent subtasks, improving fault tolerance; provides a garbage collection technique that deletes unnecessary data; and employs a trust delegation technique that facilitates flexible third party data transfers across different VOs. Copyright © 2008 John Wiley & Sons, Ltd.

Abstract Many production Grid and e‐Science infrastructures have begun to offer services to end‐users during the past several years with an increasing number of scientific applications that require access to a wide variety of resources and services in multiple Grids. Therefore, the Grid Interoperation Now—Community Group of the Open Grid Forum—organizes and manages interoperation efforts among those production Grid infrastructures to reach the goal of a world‐wide Grid vision on a technical level in the near future. This contribution highlights fundamental approaches of the group and discusses open standards in the context of production e‐Science infrastructures. Copyright © 2009 John Wiley & Sons, Ltd.

In recent years there has been a great deal of interest in "scientific workflows". These allow scientists to specify large computational experiments involving a range of different activities, such as data integration, modelling and analysis, and visualization, to name a few. Activities can be composed, often using a graphical programming environment, so that the output of one stage can be passed as input to the next, forming a pipeline of arbitrary complexity. Scientific workflows have been used to great effect in a number of different disciplines including computational chemistry, ecology and bioinformatics.

A challenge for Grid computing is the difficulty in developing software that is parallel, distributed and highly dynamic. Whilst there have been many general purpose mechanisms developed over the years, Grid programming still remains a low level, error prone task. Scientific workflow engines can double as programming environments, and allow a user to compose dasiavirtualpsila Grid applications from pre-existing components. Whilst existing workflow engines can specify arbitrary parallel programs, (where components use message passing) they are typically not effective with large and variable parallelism. Here we discuss dynamic dataflow, originally developed for parallel tagged dataflow architectures (TDAs), and show that these can be used for implementing Grid workflows. TDAs spawn parallel threads dynamically without additional programming. We have added TDAs to Kepler, and show that the system can orchestrate workflows that have large amounts of variable parallelism. We demonstrate the system using case studies in chemistry and in cardiac modelling.

Applications that span multiple virtual organizations (VOs) are of great interest to the eScience community. However, recent attempts to execute large-scale parameter sweep applications (PSAs) with the Nimrod/G tool have exposed problems in the areas of fault tolerance, data storage and trust management. In response, we have implemented a task-splitting approach, which breaks up large PSAs into a sequence of dependent subtasks, improving fault tolerance; provides a garbage collection technique, which deletes unnecessary data; and employs a trust delegation technique that facilitates flexible third party data transfers across different VOs.

Structural biology research places significant demands upon computing and informatics infrastructure. Protein production, crystallization and X-ray data collection require solutions to data management, annotation, target tracking and remote experiment monitoring. Structure elucidation is computationally demanding and requires user-friendly interfaces to high-performance computing resources. Here we discuss how these challenges are being met at the Protein Crystallography Unit at Monash University. Specifically, we have developed informatics solutions for each stage in the structural biology pipeline, from DNA cloning through to protein structure determination. This infrastructure will be pivotal for accelerating the process of structural discovery and will be of significant interest to other laboratories worldwide.