We discuss parallel performance of structured adaptive mesh refinement calculations using the SAMRAI library. We focus on fundamental aspects of adaptive gridding and dynamic computation of changing data dependencies. Previous analysis of performance of large-scale parallel adaptive calculations revealed poor scaling in these operations. Specifically, we found that these operations are inexpensive for small problems, but that their costs can become unacceptable for problems run on large numbers of processors. This paper describes subsequent developments involving graph- and tree-based algorithms that reduce runtime complexity and substantially increase scalability. We characterize performance on realistic adaptive problems using up to 512 processors of an IBM SP system and up to 1024 processors of a Linux cluster.

In distributed simulations, the delivered performance of networks of heterogeneous computers degrades severely if the computations are not load balanced. In this work we consider the distributed simulation of TURNS (Transonic Unsteady Rotor Navier Stokes), a 3-D space CFD code. We propose a load balancing heuristic for simulations on networks of heterogeneous workstations. Our algorithm takes into account the CPU speed and memory capacity of the workstations. Test run comparisons with the equal task allocation algorithm demonstrated significant efficiency gains.

This paper discusses the design and performance of the parallel data communication infrastructure in SAMRAI, a software framework for structured adaptive mesh refinement (SAMR) multi-physics applications. We describe requirements of such applications and how SAMRAI abstractions manage complex data communication operations found in them. Parallel performance is characterized for two adaptive problems solving hyperbolic conservation laws on up to 512 processors of the IBM ASCI Blue Pacific system. Results reveal good scaling for numerical and data communication operations but poorer scaling in adaptive meshing and communication schedule construction phases of the calculations. We analyze the costs of these different operations, addressing key concerns for scaling SAMR computations to large numbers of processors, and discuss potential changes to improve our current implementation.

This paper explores the parallel performance of structured overset CFD computations for multicomponent bodies in which there is relative motion between component parts. The two processes that dominate the cost of such problems are the flow solution on each component and the inter-grid connectivity solution. A two-part static-dynamic load balancing scheme is proposed in which the static part balances the load for the flow solution and the dynamic part re-balances, if necessary, the load for the connectivity solution. This scheme is coupled with existing parallel implementations of the OVERFLOW flow solver and DCF3D connectivity routine and used for unsteady calculations about aerodynamic bodies on the IBM SP2 and IBM SP multi-processors. This paper also describes the parallel implementation of a new solution-adaption scheme based on structured Cartesian overset grids.