In this paper we present the results of analyzing different approaches to implementing dynamic allocation on a large space-shared Linux cluster using an actual historical workload. We take the approach that dynamic allocation of nodes to processes can be initiated by the application as opposed to the more commonly studied approach in which the operating system requests an application to change the number of nodes it is allocated. We take the stance that jobs requesting dynamic allocation should have a minimized impact on the rest of the system. We conclude that this occurs when dynamic allocation is implemented in a non-blocking, non-priority fashion. We give a proposal for an implementation of dynamic allocation under MPI-2 which is non-priority and gives the user the choice of blocking and non-blocking while waiting for additional nodes.

This paper describes the components of a runtime system for launching parallel applications and presents performance results for starting a job on more than a thousand nodes of a workstation cluster. This runtime system was developed at Sandia National Laboratories as part of the Computational Plant (Cplant™) project, which is deploying large-scale parallel computing clusters using commodity hardware and the Linux operating system. We have designed and implemented a flexible runtime system that allows for launching parallel jobs on thousands of nodes in a matter of seconds. The interactions of the components are described, and the key issues that address the scalability and performance of the runtime system are discussed. We also present performance results of launching executables of varying sizes on more than a thousand nodes.

Large MPP systems can neither solve grand-challenge scientific problems nor enable large scale industrial and governmental simulations if they rely on extensions to workstation system software. At Sandia National Laboratories we have developed, with our vendors, a new system architecture for high-end computing. Highest performance is achieved by providing applications with a light-weight interface to a collection of processing nodes. Usability is provided by creating node partitions specialized for user access, networking, and I/O. The entire system is glued together by a data movement interface which we call portals. Portals allow data to flow between processing nodes with minimal system overhead while maintaining a suitable degree of protection and reconfigurability.