Asynchronous iterative methods tolerate straggling processors by allowing workers to proceed with stale data, but at a cost: the iterates become inconsistent, potentially degrading convergence. We investigate whether convergence accelerators such as Anderson acceleration compensate for this degradation. We experimentally study three fixed-point iterations: the Jacobi method for sparse linear systems, value iteration for the Bellman equation, and the Hartree–Fock self-consistent field (SCF) iteration. The experiments are conducted using a high-performance execution framework, Ray, which abstracts the complexity of distributed systems and enables code parallelization and fault injection with minimal changes.

Heterogeneous Graph Neural Networks (HGNNs) excel in modeling complex, multi-typed relationships across diverse domains, yet their vulnerability to backdoor attacks remains unexplored. To address this gap, we conduct the first investigation into the susceptibility of HGNNs to existing graph backdoor attacks, revealing three critical issues: (1) high attack budget required for effective backdoor injection, (2) inefficient and unreliable backdoor activation, and (3) inaccurate attack effectiveness evaluation. To tackle these issues, we propose the Heterogeneous Graph Backdoor Attack (HGBA), the first backdoor attack specifically designed for HGNNs, introducing a novel relation-based trigger mechanism that establishes specific connections between a strategically selected trigger node and poisoned nodes via the backdoor metapath. HGBA achieves efficient and stealthy backdoor injection with minimal structural modifications and supports easy backdoor activation through two flexible strategies: Self-Node Attack and Indiscriminate Attack. Additionally, we improve the ASR measurement protocol, enabling a more accurate assessment of attack effectiveness. Extensive experiments demonstrate that HGBA far surpasses multiple state-of-the-art graph backdoor attacks in black-box settings, efficiently attacking HGNNs with low attack budgets. Ablation studies show that the strength of HBGA benefits from our trigger node selection method and backdoor metapath selection strategy. In addition, HGBA shows superior robustness against node feature perturbations and multiple types of existing graph backdoor defense mechanisms. Finally, extension experiments demonstrate that the relation-based trigger mechanism can effectively extend to tasks in homogeneous graph scenarios, thereby posing severe threats to broader security-critical domains.

Summary Computational chemistry is at the forefront of solving urgent societal problems, such as polymer upcycling and carbon capture. The complexity of modeling these processes at appropriate length and time scales is mainly manifested in the number and types of chemical species involved in the reactions and may require models of several thousand atoms and large basis sets to accurately capture the chemical complexity and heterogeneity in the physical and chemical processes. The quantum chemistry package General Atomic and Molecular Electronic Structure System (GAMESS) has a wide array of methods that can efficiently and accurately treat complex chemical systems. In this work, we have used the GAMESS Effective Fragment Molecule Orbital (EFMO) method for electronic structure calculation of a challenging mesoporous silica nanoparticle (MSN) model surrounded by about 4700 water molecules to investigate the strong scaling and GPU offloading on hybrid CPU‐GPU nodes. Experiments were performed on the Perlmutter platform at the National Energy Research Scientific Computing Center. Good strong scaling and load balancing have been observed on up to 88 hybrid nodes for different settings of the execution parameters for the calculation considered here. When GPUs are oversubscribed by offloading work from multiple CPU processes, using the NVIDIA multi‐process service (MPS) has consistently reduced time to solution and energy consumed. Additionally, for some configuration parameter settings, oversubscription with MPS improved performance by up to 5.8% over the case without oversubscription.

Summary The Golub‐Kahan bidiagonalization is widely used in the singular value decomposition of rectangular matrices and has been generalized to an iterative solver for symmetric indefinite linear systems with a two‐by‐two block structure. In this work, we present a scalability study of this generalized solver as implemented in a recent release of the parallel numerical library PETSc (Portable, Extensible Toolkit for Scientific Computation). We present an improved solver performance for the two‐dimensional (2D) Stokes equations as compared to previous work. Furthermore, we investigate the performance of different parallel inner solvers in the outer Golub‐Kahan iteration for a three‐dimensional Stokes problem. The study includes parallel sparse direct solvers and multigrid methods. When increasing the number of cores for a fixed total problem size, the solver exhibits good speedups of up to 50% at the 1024 core count. For the tests in which the total problem size grows while the workload in each core stays constant, the parallel performance of the solver scales almost linearly with the increase in the core counts. In particular, the computation time increases only by about 15% when the number of cores increases from 80 to 1024 for a 2D test case.

Summary To improve power consumption of applications at the runtime, modern processors provide frequency scaling capabilities, which along with workload optimization, are also available on GPU accelerators. In this work, a runtime strategy is proposed to distribute a given power allocation among the host components and the GPU according to the current application performance and power usage, such that GPU execution is prioritized over CPU for power allocation to maximize application performance. Next, the strategy is tailored to an application, a quantum‐chemistry package GAMESS for ab initio electronic structure calculations. Specifically, GAMESS hybrid CPU–GPU implementation as provided in the Libcchem library is considered. Experiments, performed on a 28‐core node with a Kepler GPU, resulted in performance gains of up to 50% under the proposed strategy and the largest power allocation considered here as compared with the scenario when this allocation was equally distributed among the computing‐platform components.

This study seeks to understand the soft error vulnerability of asynchronous iterative methods, with a focus on stationary iterative solvers such as Jacobi. The implementations make use of hybrid parallelism where the computational work is distributed over multiple nodes using MPI and parallelized on each node using openMP. A series of experiments is conducted to measure the impact of an undetected soft fault on an asynchronous iterative method, and to compare and contrast several techniques for simulating the occurrence of a fault and then recovering from the effects of the faults. The data shows that the two numerical soft-fault models tested here more consistently than a “bit-flip” model produce bad enough behavior to test a variety of recovery strategies, such as those based on partial checkpointing.

For modern parallel applications, modeling their general execution characteristics, such as power and time, is difficult due to a great many factors affecting software-hardware interactions, which is also exacerbated by the dearth of measuring and monitoring tools for novel architectures, such as Intel Xeon Phi processors. To address this modeling challenge, the present work proposes to employ the Empirical Mode Decomposition (EMD) method to describe an execution as a series of modes culminating in a single residual trend, for which, in its turn, a model equation is obtained as a non-linear fit. As outcome, an overall energy consumption may be predicted using this model. A real-world quantum-chemistry application GAMESS and a molecular-dynamics proxy application CoMD were considered in the experiments. The results demonstrate that the energy modeled ranges within 10-30% of the measured energy, depending on the length of execution.

We propose to use a randomization technique based on Random Butterfly Transformations (RBT) in the Algebraic Recursive Multilevel Solver (ARMS) to improve the preconditioning phase in the iterative solution of sparse linear systems.We integrated the RBT technique into the parallel version of ARMS (pARMS).The preliminary experimental results on some matrices from the Davis' collection show an improvement of the convergence and accuracy of the results when compared with existing implementations of the pARMS preconditioner.

Summary Sparse matrix vector multiply (SpMVM) is an important kernel that frequently arises in high performance computing applications. Due to its low arithmetic intensity, several approaches have been proposed in literature to improve its scalability and efficiency in large scale computations. In this paper, our target systems are high end multi‐core architectures and we use messaging passing interface + open multiprocessing hybrid programming model for parallelism. We analyze the performance of recently proposed implementation of the distributed symmetric SpMVM, originally developed for large sparse symmetric matrices arising inab initionuclear structure calculations. We study important features of this implementation and compare with previously reported implementations that do not exploit underlying symmetry. Our SpMVM implementations leverage the hybrid paradigm to efficiently overlap expensive communications with computations. Our main comparison criterion is the ‘CPU core hours’ metric, which is the main measure of resource usage on supercomputers. We analyze the effects of topology‐aware mapping heuristic using simplified network load model. We have tested the different SpMVM implementations on two large clusters with 3D Torus and Dragonfly topology. Our results show that the distributed SpMVM implementation that exploits matrix symmetry and hides communication yields the best value for the ‘CPU core hours’ metric and significantly reduces data movement overheads. Copyright © 2015 John Wiley & Sons, Ltd.

The Fortran95 code VTDIRECT95, based on the original MPI, has been modified to use MPI-2. An option for VTDIRECT95 is to divide the feasible box into subdomains, and concurrently apply the global direct search algorithm DIRECT within each subdomain. When the number of subdomains is greater than one, a bug causes VTDIRECT95 to occasionally sample outside the given feasible box, which is serious if the objective function is not defined outside the given box. This bug has been fixed, and the sample output files have been updated to reflect the correction. For completeness, the package VTDIRECT95 now contains both the MPI-1 (with the multiple subdomain bug fixed) and the MPI-2 versions of the code.

SUMMARY Energy consumption has become a major design constraint in modern computing systems. With the advent of petaflops architectures, power‐efficient software stacks have become imperative for scalability. Techniques such as dynamic voltage and frequency scaling (called DVFS) and CPU clock modulation (called throttling) are often used to reduce the power consumption of the compute nodes. To avoid significant performance losses, these techniques should be used judiciously during parallel application execution. For example, its communication phases may be good candidates to apply the DVFS and CPU throttling without incurring a considerable performance loss. They are often considered as indivisible operations although little attention is being devoted to the energy saving potential of their algorithmic steps. In this work, two important collective communication operations, all‐to‐all and allgather, are investigated as to their augmentation with energy saving strategies on theper‐callbasis. The experiments prove the viability of such a fine‐grain approach. They also validate a theoretical power consumption estimate for multicore nodes proposed here. While keeping the performance loss low, the obtained energy savings were always significantly higher than those achieved when DVFS or throttling were switched on across the entire application run. Copyright © 2012 John Wiley & Sons, Ltd.

Although high-performance computing has always been about efficient application execution, both energy and power consumption have become critical concerns owing to their effect on operating costs and failure rates of large-scale computing platforms. Modern microprocessors are equipped with the capabilities to reduce their power consumption using techniques such as dynamic voltage and frequency scaling (DVFS) and CPU clock modulation (called throttling). Without careful application, however, DVFS and throttling may cause a significant performance loss due to system overhead. This work presents design considerations for a runtime procedure that dynamically analyzes blocking point-to-point communications, groups them according to the proposed criteria, and applies frequency scaling by analyzing both communication and architectural parameters without penalizing the performance much. Experiments, performed on NAS parallel benchmarks verify the proposed design by exhibiting energy savings of as much as 11% with a performance loss as low as 2%.

VTDIRECT95 is a Fortran 95 implementation of D. R. Jones' deterministic global optimization algorithm called DIRECT , which is widely used in multidisciplinary engineering design, biological science, and physical science applications. The package includes both a serial code and a data-distributed massively parallel code for different problem scales and optimization (exploration vs. exploitation) goals. Dynamic data structures are used to organize local data, handle unpredictable memory requirements, reduce the memory usage, and share the data across multiple processors. The parallel code employs a multilevel functional and data parallelism to boost concurrency and mitigate the data dependency, thus improving the load balancing and scalability. In addition, checkpointing features are integrated into both versions to provide fault tolerance and hot restarts. Important algorithm modifications and design considerations are discussed regarding data structures, parallel schemes, error handling, and portability. Using several benchmark functions and real-world applications, the software is evaluated on different systems in terms of optimization effectiveness, data structure efficiency, parallel performance, and checkpointing overhead. The package organization and usage are also described in detail.

One of the emerging computational approaches in nuclear physics is the configuration interaction (CI) method for solving the many-body nuclear Hamiltonian in a sufficiently large single-particle basis space to obtain exact answers - either directly or by extrapolation. The lowest eigenvalues and corresponding eigenvectors for very large, sparse and unstructured nuclear Hamiltonian matrices are obtained and used to evaluate additional experimental quantities. These matrices pose a significant challenge to the design and implementation of efficient and scalable algorithms for obtaining solutions on massively parallel computer systems. In this paper, we describe the computational strategies employed in a state-of-the-art CI code MFDn (Many Fermion Dynamics - nuclear) as well as techniques we recently developed to enhance the computational efficiency of MFDn. We will demonstrate the current capability of MFDn and report the latest performance improvement we have achieved. We will also outline our future research directions.

Abstract Sparse matrix computations are ubiquitous in high‐performance computing applications and often are their most computationally intensive part. In particular, efficient solution of large‐scale linear systems may drastically improve the overall application performance. Thus, the choice and implementation of the linear system solver are of paramount importance. It is difficult, however, to navigate through a multitude of available solver packages and to tune their performance to the problem at hand, mainly because of the plethora of interfaces, each requiring application adaptations to match the specifics of solver packages. For example, different ways of setting parameters and a variety of sparse matrix formats hinder smooth interactions of sparse matrix computations with user applications. In this paper, interfaces designed for components that encapsulate sparse matrix computations are discussed in the light of their matching with application usability requirements. Consequently, we distinguish three levels of interfaces, high, medium, and low, corresponding to the degree of user involvement in the linear system solution process and in sparse matrix manipulations. We demonstrate when each interface design choice is applicable and how it may be used to further users' scientific goals. Component computational overheads caused by various design choices are also examined, ranging from low level, for matrix manipulation components, to high level, in which a single component contains the entire linear system solver. Published in 2007 by John Wiley & Sons, Ltd.

Modern performance tools provide methods for easy integration into an application for performance evaluation. For a large-scale scientific software package that has been under development for decades and with developers around the world, several obstacles must be overcome in order to utilize modern performance tools and explore performance bottlenecks. In this paper, we present our experience in integrating performance tools with one popular computational chemistry package. We discuss the difficulties we encountered and the mechanisms developed to integrate performance tools into this code. With performance tools integrated, we show one of the initial performance evaluation results, and discuss what other challenges we are facing to conduct performance evaluation for large-scale scientific packages.

Abstract Iterative methods play an important role in solving large‐scale systems of linear equations that arise in real‐world applications. Due to numerous linear system properties that may affect the solution, it is rather difficult for a user to develop a good sparse linear system solver from scratch. Thus, various collections of solution methods are made available to the user. One such software package is SPARSKIT, which is well known in the scientific community. Written in FORTRAN77 and provided with a cumbersome interface, it is considered, however, a legacy code. Our objective is to enable its wider usage in modern applications and to facilitate further SPARSKIT enhancements. Applying a ‘peer‐component’ design, we have created a set of SPARSKIT components that: (a) incorporate both original and new iterative methods; (b) are readily extensible with more methods; (c) may be connected to applications in a component framework; and (d) provide access from a variety of programming languages. Tools available from the Common Component Architecture (CCA) Forum enabled our component design of SPARSKIT. Copyright © 2006 John Wiley & Sons, Ltd.

We describe a system for aggregating heterogeneous resources from distinct administrative domains into an enterprise-wide compute grid, such that the aggregated resource provides the services of reliable and flexible queuing, scheduling, execution, and monitoring of batch applications. The system provides scheduling across multiple cluster grids, user account mapping across domains, and file staging, thereby enabling the consolidation of organization-wide distributed resources into a virtual resource, while preserving local control of resources. The concept of abstract queue, as the unit of aggregating heterogeneous resources, is introduced and instantiated for distributed resource scheduling. The proposed system is an open source, standards-based alternative to similar commercial systems

Globally convergent, probability-one homotopy methods have proven to be very effective for finding all the isolated solutions to polynomial systems of equations. After many years of development, homotopy path trackers based on probability-one homotopy methods are reliable and fast. Now, theoretical advances reducing the number of homotopy paths that must be tracked and handling singular solutions have made probability-one homotopy methods even more practical. POLSYS_GLP consists of Fortran 95 modules for finding all isolated solutions of a complex coefficient polynomial system of equations. The package is intended to be used on a distributed memory multiprocessor in conjunction with HOMPACK90 (Algorithm 777), and makes extensive use of Fortran 95-derived data types and MPI to support a general linear product (GLP) polynomial system structure. GLP structure is intermediate between the partitioned linear product structure used by POLSYS_PLP (Algorithm 801) and the BKK-based structure used by PHCPACK. The code requires a GLP structure as input, and although finding the optimal GLP structure is a difficult combinatorial problem, generally physical or engineering intuition about a problem yields a very good GLP structure. POLSYS_GLP employs a sophisticated power series end game for handling singular solutions, and provides support for problem definition both at a high level and via hand-crafted code. Different GLP structures and their corresponding Bezout numbers can be systematically explored before committing to root finding.

The general atomic and molecular electronic structure system (GAMESS) is a program for ab initio molecular quantum chemistry calculations. This work presents a modification to the integration model of network information conveyer and application notification (NICAN) into GAMESS for the concurrent execution of sequential GAMESS jobs. In the work presented here, NICAN acts as an application-level co-scheduler in distributed SMP environments. The primary goal of such a co-scheduler is to increase throughput of parallel GAMESS calculations

Summary form only given. We present a sophisticated and efficient parallel scheme for the DIRECT global optimization algorithm of Jones et al. (1993). Although several sequential implementations for this algorithm have been successfully applied to large scale MDO problems, few parallel versions of the DIRECT algorithm have addressed well algorithm characteristics such as a single starting point, an unpredictable workload, and a strong data dependency. These challenges engender many interesting design issues including domain decomposition, data access and management, and workload balancing. A hierarchical parallel scheme has been developed to address these challenges at three levels. Each level is supported by parallel and distributed data structures to access shared data sets, distribute workload, or exchange messages. Parameter estimation problems in systems biology provide an ideal application context for the present work. Global nonlinear parameter estimation results obtained on a 200 node Linux cluster are given for a cell cycle model for frog eggs.

Summary form only given. High-performance applications place great demands on the computation and communication resources of a distributed computing platform. If the availability of the resources changes dynamically, the application performance may suffer. This is especially true for cluster environments, which are often heterogeneous and require tedious tuning for high-performance applications. We describe a packet probing technique to detect contention on the cluster nodes to which application is mapped. The technique is light-weight and may be a priori tuned to a given network type, so that it is used at an application's run-time. We also show an easy integration of packet probing as a module of a recently developed communication middleware, which provides an application with a run-time access to the dynamic computing system information and which invokes application adaptations.

Applications involving large sparse nonsymmetric linear systems encourage parallel implementations of robust iterative solution methods, such as GMRES(k). One variation of GMRES(k) is to adapt the restart value k for any given problem and use Householder reflections in the orthogonalization phase to achieve high accuracy. The Householder transformations can be performed without global communications and modified to use an arbitrary row distribution of the coefficient matrix. The effect of this modification on the GMRES(k) performance is discussed here. This paper compares the abilities of various parallel GMRES(k) implementations to maintain fixed efficiency with increase in problem size and number of processors.

HOMPACK90 is a Fortran 90 version of the Fortran 77 package HOMPACK (Algorithm 652), a collection of codes for finding zeros or fixed points of nonlinear systems using globally convergent probability-one homotopy algorithms.Three qualitatively different algorithmsordinary differential equation based, normal flow, quasi-Newton augmented Jacobian matrix-are provided for tracking homotopy zero curves, as well as separate routines for dense and sparse Jacobian matrices.A high level driver for the special case of polynomial systems is also provided.Changes to HOMPACK include numerous minor improvements, simpler and more elegant interfaces, use of modules, new end games, support for several sparse matrix data structures, and new iterative algorithms for large sparse Jacobian matrices.

Homotopy algorithms to solve a nonlinear system of equations f(x) = 0 involve tracking the zero curve of a homotopy map p(a, λ, x) from λ = 0 until λ = 1. When the algorithm nears or crosses the hyperplane λ = 1, an “end game” phase is begun to compute the solution x¯ satisfying p(a, λ, x¯) = f(x¯) = 0. This note compares several end game strategies, including the one implemented in the normal flow code FIXPNF in the homotopy software package HOMPACK.