A brief overview of performance evaluation and benchmarking is presented. It is shown that traditional performance measurements recorded in these activities are really a direct measurement of the parallelism in software and hardware. A framework called the 'Path to Performance' that identifies the agents and activities that change the parallelism as it moves from problem to solution is developed. It is shown where the various current application benchmarks apply probes on the Path and which agents are being measured. The traditional time-based performance measurements are recast into parallelism-based performance measurements to show that understanding performance implies understanding the parallelism. A simulation tool and methodology are described for measuring and comparing the executed parallelism on a single CRAY Y-MP CPU. Application of the methodology to several of the Perfect Benchmarks suggests that the benchmarking utility of these programs on the Y-MP may be questioned because they exercise the machine in the same way.>

The workload of the NSF supercomputer center at the University of Illinois is decomposed into its major constituent programs, and performance characteristics of those programs are described in detail.The resulting data is compared with similar data obtained from standard benchmarks showing that applications level programs are clearly distinct from kernel and algorithm benchmarks.The workload decomposition shows that overidl utilization is quite high but is heavily influenced by a small number of sophisticated users with large allocations who run highly efficient programs.Some high level profifing data leads to an analysis of the benefits of inliniug subroutines.A number of other preliminary results are also described.This effort, called the Supercomputer Applications Database (SAD) Project, is the first detailed workload analysis of a US national supercomputer center.