A brief overview of performance evaluation and benchmarking is presented. It is shown that traditional performance measurements recorded in these activities are really a direct measurement of the parallelism in software and hardware. A framework called the 'Path to Performance' that identifies the agents and activities that change the parallelism as it moves from problem to solution is developed. It is shown where the various current application benchmarks apply probes on the Path and which agents are being measured. The traditional time-based performance measurements are recast into parallelism-based performance measurements to show that understanding performance implies understanding the parallelism. A simulation tool and methodology are described for measuring and comparing the executed parallelism on a single CRAY Y-MP CPU. Application of the methodology to several of the Perfect Benchmarks suggests that the benchmarking utility of these programs on the Y-MP may be questioned because they exercise the machine in the same way.>

The workload of the NSF supercomputer center at the University of Illinois is decomposed into its major constituent programs, and performance characteristics of those programs are described in detail.The resulting data is compared with similar data obtained from standard benchmarks showing that applications level programs are clearly distinct from kernel and algorithm benchmarks.The workload decomposition shows that overidl utilization is quite high but is heavily influenced by a small number of sophisticated users with large allocations who run highly efficient programs.Some high level profifing data leads to an analysis of the benefits of inliniug subroutines.A number of other preliminary results are also described.This effort, called the Supercomputer Applications Database (SAD) Project, is the first detailed workload analysis of a US national supercomputer center.

A tracing library for the Cray X-MP and Cray 2 supercomputers has been developed that supports the low-overhead capture of execution events for sequential and multitasked programs. This library has been extended to use the automatic instrumentation facilities on these machines, allowing trace data from routine entry and exit, and other program segments to be captured. To assess the utility of the trace-based tools, three of the Perfect Benchmark codes have been tested in scalar and vector models with the tracing instrumentation. In addition to computing summary execution statistics from the traces, interesting execution dynamics appear when studying the trace histories. It is also possible to compare codes across the two architectures by correlating the event traces. It is concluded that adding tracing support in Cray supercomputers can have significant returns in improved performance characterization and evaluation.>

This package performs an automatic round-off error analysis of numerical algorithms, which is useful in determining the numerical properties (stability) of such algorithms.The approach taken with this software differs from that of Miller [4, 5] by dealing with relative as opposed to absolute error analysis.This is the natural approach taken for dealing with floating-point arithmetic.In addition, this software gives an accurate analysis of an algorithm for some fixed data, rather than searches for data for which an approximate analysis indicates that the algorithm performs poorly.It uses one of two methods, either an exact method [1] or a heuristic algorithm [1-3].T h e following paragraphs summarize the previous work in [2], and it is assumed that the reader is familiar with the concepts and techniques presented there.To perform the error analysis, the algorithm being analyzed is represented as a directed graph with each numeric quantity being a node in this graph.Directed arcs lead from operands to their results.Each node's value is subject to error, called local relative error.This error affects those subsequent nodes that are computed using the contaminated node as an operand.All of the local relative errors together are used to produce a total relative error for each node.F r o m the

EffectsA study of relative error propagation In numerical algorithms is given.Roundoff errors and their effects are Identified, and a method IS presented which efficiently calculates these effects Present methods in use are O(l 2) The time and storage required by the method presented here is O(l), where l Is the number of operations in the straight-line algorithm being analyzed These savings will enable the analysis of larger and more complex algorithms Key Words and Phrases automatic roundoff analysis, numerical stability, numerical linear algebra CR Categories: 5.10, 5 11, 5 14