Abstract Load balancing increases the efficient use of existing resources for parallel and distributed applications. At a coarse level of granularity, advances in runtime systems for parallel programs have been proposed in order to control available resources as efficiently as possible by utilizing idle resources and using task migration. Simultaneously, at a finer granularity level, advances in algorithmic strategies for dynamically balancing computational loads by data redistribution have been proposed in order to respond to variations in processor performance during the execution of a given parallel application. Combining strategies from each level of granularity can result in a system which delivers advantages of both. The resulting integration is systemic in nature and transfers the responsibility of efficient resource utilization from the application programmer to the runtime system. This paper presents the design and implementation of a system that combines an algorithmic fine‐grained data parallel load balancing strategy with a systemic coarse‐grained task‐parallel load balancing strategy, and reports on recent experimental results of running a computationally intensive scientific application under this integrated system. The experimental results indicate that a distributed runtime environment which combines both task and data migration can provide performance advantages with little overhead. It also presents proposals for performance enhancements of the implementation, as well as future explorations for effective resource management. Copyright © 2001 John Wiley & Sons, Ltd.

Networked workstations represent an increasingly popular distributed platform for running large parallel programs. They can present a low-cost alternative to purchasing supercomputer time or additional usable computational capability. Several capabilities are desirable in order to harness workstations, including support for a widely accepted parallel programming environment, task migration, intelligent resource allocation, fault tolerance, and totally transparent support of these features. The Hector system is designed to provide these capabilities to MPI programs. The structure of the system and experiences using the system on loaded workstations to run scientific codes are described. Copyright © 1999 John Wiley & Sons, Ltd.

General-purpose programmers have come to expect a high degree of portability among widely varying architectures. Advances in run-time systems for parallel programs have been proposed in order to harness available resources as efficiently as possible. Simultaneously, advances in algorithmic methods of dynamically balancing computational load have been proposed in order to respond to variations in actual performance and therefore in run-time. The primary mechanism for harnessing idle resources effectively, task migration, can be used alongside the primary mechanism for dynamic load balancing, data redistribution. Besides the fact that the two methods can be used simultaneously to spur further increases in performance, the run-time information-gathering infrastructure necessary to detect and use idle resources can also benefit dynamically load-balanced applications. This paper describes an architecture for and preliminary implementation of a system that combines data-parallel load balancing with task-parallel load balancing. Performance test results are included as well.

Harnessing the computational capabilities of a network of workstations promises to off-load work from overloaded supercomputers onto largely idle resources overnight. Several capabilities are needed to do this, including support for an architecture-independent parallel programming environment, task migration, automatic resource allocation, and fault tolerance. The Hector distributed run-time environment is designed to present these capabilities transparently to programmers. MPI programs can be run under this environment on homogeneous clusters with no modifications to their source code needed. The design of Hector, its internal structure, and several benchmarks and tests are presented.

In order to use networks of workstations in parallel processing applications, several schemes have been devised to allow processes on different, possibly heterogeneous, platforms to communicate with one another. The Message-Passing Interface (MPI) is one such scheme that allows for message-passing across different architectures. The MPI specification does not make provisions for the migration of a process between machines. This paper describes the work required to modify, an MPI implementation to allow for task migration. It also describes "Hector", our heterogeneous computing task allocator that is used to migrate tasks automatically and improve the overall performance of a parallel program.