Accurate estimation of the amount of memory required is an important step in discovering and selecting configurations for computational jobs in a grid environment. In order to mitigate the adverse impact of over and under estimation of memory requirements, job submitters need to carry out benchmarking in order to understand the memory usage behaviour of specific applications. An automated mechanism for learning the memory usage pattern can allow them to focus more on the results of the jobs. This paper presents mechanisms that automate the process of understanding the memory usage behaviour of jobs. The mechanism uses the provenance data from previously completed jobs. The proposed mechanism enables populating an information model describing the memory usage patterns of applications. The paper also presents techniques for making predictions on the memory usage while the development of the knowledge base is still being carried out. The utility of the mechanism is demonstrated by showing its increasing ability to predict the memory usage for incoming jobs. Also highlighted are the factors that effect the pace in which the system acquires knowledge about the memory usage behaviour.

Job submission in high performance computing workloads exhibits a diurnal pattern similar to electrical prices. While high-priority jobs may need immediate access to resources, by altering the cluster scheduler to delay the execution of lower-priority jobs when power prices are high, significant cost savings can be achieved. Reduction of power demands by consumers such as data centres when energy availability is low, as signaled by high prices, can also help to simplify challenges faced in reducing the carbon footprint of the electrical grid. In this paper we discuss patterns in electrical pricing and also look at some challenges in integrating more volatile, but environmentally friendly renewable energy sources into the electrical grid. Simulation results are also presented showing that high-priority jobs can still receive rapid service while achieving 25-50% electricity cost savings for lower priority jobs.

Managing the execution of scientific applications in a heterogeneous grid computing environment can be a daunting task, particularly for long running jobs. Increasing fault tolerance by checkpointing and migrating jobs between resources requires expertise and time of the scientist. Automation of such tasks can allow the scientist to focus more on the scientific results and less on the technical details. In this paper a generic framework for managing and automating the execution of jobs is presented. It uses of a variety of information models describing systems, policies, and application details/requirements to make suitable decisions on where and how to run, checkpoint, migrate and reconfigure jobs as needed. To demonstrate the utility of the framework, it is used as part of a simulation study to assess the impact availability of application memory usage information has on meeting the QoS objectives of job submitters and on overall utilization of resources. The study shows that with greater availability of memory usage information, the execution management framework is able to better meet user objectives and improve utilization of resources, particularly when the objective is to make more efficient use of resources.

Accessing, running and sharing applications and data presents researchers with many challenges. Cloud computing and social networking technologies have the potential to simplify or eliminate many of these challenges. Cloud computing technologies can provide scientists with transparent and on-demand access to applications served over the Internet in a dynamic and scalable manner. Social networking technologies provide a means for easily sharing applications and data. In this paper we present an on-line/on-demand interactive application service. The service is built on a cloud computing infrastructure that dynamically provisions virtualized application servers based on user demand. An open source social networking platform is leveraged to establish a portal front end that enables applications and results to be easily shared between researchers. Furthermore, the service works with existing/legacy applications without requiring any modifications.

Automating the execution of applications in grid computing environments is a complicated task due to the heterogeneity of computing resources, resource usage policies, and application requirements. Applications differ in memory usage, performance, scalability and storage usage. Having knowledge of this information can aid in matching jobs to resources and in selecting appropriate configuration parameters such as the number of processors to run on and memory requirements for those resources. This paper presents an application memory usage model that can be used to aid in selecting appropriate job configurations for different resources. The model can be used to represent how memory scales with the number of processors, the memory usage of different types of processes, and changes in memory usage during execution. It builds on a previously developed information model used for describing resources, resource usage policies and limited information on applications. An analysis of the memory usage model illustrating its use towards automating job execution in grid computing environments is also presented.

ldquoWeb 2.0rdquo and ldquocloud computingrdquo are revolutionizing the way IT infrastructure is accessed and managed. Web 2.0 technologies such as blogs, wikis and social networking platforms provide Internet users with easier mechanisms to produce Web content and to interact with each other. Cloud computing technologies are aimed at running applications as services over the Internet on a scalable infrastructure. In this paper we explore the advantages of using Web 2.0 and cloud computing technologies in an enterprise setting to provide employees with a comprehensive and transparent environment for utilizing applications. To demonstrate the effectiveness of this approach we have developed an environment that uses a social networking platform to provide access to a legacy application. The application is hosted on an internal cloud computing infrastructure that adapts dynamically to user demands. Initial feedback suggests this approach provides an improved user experience while simplifying management and increasing effective utilization of the underlying IT resources.

Understanding how resources are allocated in a cluster or grid environment can be a daunting task. The number of schedulable processors has increased dramatically, resulting in a greater need for resource allocation analysis tools. This increase comes from hardware developments as well as inter-domain resource sharing. This paper describes a toolset for monitoring the allocation of resources in a high performance computing cluster or grid environment. Specifically, we are interested in monitoring and analyzing the allocation of resources in a batch scheduled universe. The aim is to categorize the data so that it is useful for understanding particular questions about resource allocation.

Grid computing environments typically federate heterogeneous resource clusters belonging to several organizations. To fully realize the promise of a grid environment, it is necessary to support tools that help obtain insights into the behaviour of individual clusters. This paper describes a cluster service called Replay that simplifies the development and maintenance of such tools. The service provides a common model-based interface for obtaining current and historical information about a cluster. Replay can manage multiple views which allows tools to obtain information about an existing cluster as well as information that shows how a cluster might have behaved under alternate configurations and workloads. The model-based interface allows tools to be ported to different clusters with little effort. Furthermore, Replay uses different mechanisms to manage information that typically changes infrequently and information that can change in a more dynamic, continuous manner allowing it to handle information more efficiently than existing services that provide similar functionality. The paper presents a job analysis tool to illustrate the utility of the service

Many grid computing projects involve sharing and transferring large volumes of data. It is often important to maximise data throughput. Grid computing environments can be highly heterogeneous in nature and could involve large round trip times between collaborating sites. The actual throughput achieved could be much lower than the available bandwidth due to TCP dynamics. One technique that helps improve TCP performance in heterogeneous network environments is split TCP connections, where the connection is split at one or more points in the network. This paper examines the use of this technique for GridFTP file transfers. Results are promising showing up to four times improvement over GridFTP file transfers involving just a single end-to-end TCP connection

Summary form only given. This paper describes the use of the "Plan 9 from Bell Labs " distributed operating system as a grid computing infrastructure. In particular it compares solutions using the de facto standard middleware toolkit for grids, Globus, to an environment constructed using computers running the Plan 9 operating system. These environments are compared based on the features they offer in the context of grid computing: authentication, security, data management, and resource discovery.

This paper discusses the use of a parallel discrete-event network emulator called the Internet Protocol Traffic and Network Emulator (IP-TNE) for Web server benchmarking. The experiments in this paper demonstrate the feasibility of high-performance WAN emulation using parallel discrete-event simulation techniques on shared-memory multiprocessors. Our experiments with the Apache Web server achieve 3400 HTTP transactions per second for simple Web workloads, and 1000 HTTP transactions per second for realistic Web workloads, for static document retrieval across emulated WAN topologies of up to 4096 concurrent Web/TCP clients. The results show that WAN characteristics, including round-trip delays, link speeds, packet losses, packet sizes, and bandwidth asymmetry, all have significant impacts on Web server performance. WAN emulation enables stress testing and benchmarking of Web server performance in ways that may not be possible in simple LAN test scenarios.

This paper studies the performance of the transmission control protocol (TCP) over asynchronous transfer mode (ATM) when asymmetric digital subscriber line (ADSL) technology is used in the local loop. TCP can experience performance degradation in this network architecture because of protocol conversion overhead and data losses due to transmission errors. A simulation model is developed to simulate ADSL network components and noisy local loops. The simulation experiments are designed to study the impact of channel errors on TCP performance for unidirectional bulk data transfers, using both an independent error model and a burst error model. The primary performance metrics are cell loss ratio, packet loss ratio, and TCP effective throughput. The simulation results illustrate how TCP is affected by channel errors, as well as the impacts of the TCP maximum segment size (MSS), switch buffer size, bandwidth asymmetry, and the percentage of noisy lines.

Testing distributed applications over the Internet is fraught with problems: due to the inability to control a wide area network consistent, reproducible performance experiments are not possible. A system is described that uses a parallel discrete event simulator that can act as a real-time network emulator. Real Internet Protocol (IP) traffic generated by application programs running on user workstations can interact with modelled traffic in the emulator; thus providing a controlled test environment for distributed applications. Parallel execution enables the emulator to simulate large virtual networks and to model traffic interactions that could not be done in real-time sequentially. This paper gives an overview of the emulator and explores the various external data routing methods that the emulator supports. These routing methods allow the emulator to be operated in shared environments with certain constraints, as well as in dedicated test environments. Preliminary performance results are included.