An important advance in cluster computing is the evolution from single processor clusters to multi-processor SMP clusters. Due to the increased complexity in the memory model on SMP clusters, new approaches are needed for applications that make use of distributed-memory paradigms. This paper presents new communications software developments that are designed to take advantage of SMP cluster hardware. Although the specific focus is on the central field of computational chemistry and materials science, as embodied in the popular electronic structure package GAMESS (General Atomic and Molecular Electronic Structure System), the impact of these new developments will be far broader in scope. Following a summary of the essential features of the distributed data interface (DDI) in the current implementation of GAMESS, the new developments for SMP clusters are described. The advantages of these new features are illustrated using timing benchmarks on several hardware platforms, using a typical computational chemistry application.

This paper describes a novel distributed data parallel self consistent field (SCF) algorithm and the distributed data coupled perturbed Hartree-Fock (CPHF) step of an analytic Hessian algorithm. The distinguishing features of these algorithms are: (a) columns of density and Fock matrices are distributed among processors, (b) pairwise dynamic load balancing and an efficient static load balancer were developed to achieve a good workload, and (c) network communication time is minimized via careful analysis of data flow in the SCF and CPHF algorithms. By using a shared memory model, novel work load balancers, and improved analytic Hessian steps, we have developed codes that achieve superb performance. The performance of the CPHF code is demonstrated on a large biological system.