Irregular loop nests in which the loop bounds are determined dynamically by indexed arrays are difficult to compile into expressive parallel constructs, such as segmented scans and reductions. In this paper, we describe a suite of transformations to automatically parallelize such irregular loop nests, even in the presence of recurrences. We describe a simple, general loop flattening transformation, along with new optimizations which make it a viable compiler transformation. A robust recurrence parallelization technique is coupled to the loop flattening transformation, allowing parallelization of segmented reductions, scans, and combining-sends over arbitrary associative operators. We discuss the implementation and performance results of the transformations in a parallelizing Fortran 77 compiler for the Cray C90 supercomputer. In particular, we focus on important sparse matrix-vector multiplication kernels, for one of which we are able to automatically derive an algorithm used by one of the fastest library routines available.

State space exploration is of prime importance in the study of finite state sequential systems, with several efforts aimed at compact representation of the state space in order to tackle the state explosion problem. In the work presented on formal verification of inductively-defined hardware, we have identified a useful class of Boolean functions called linearly inductive functions (LIFs). We explore the relationship between our LIF representation and the classic DFA (deterministic finite state automaton) representation of a sequential function, and examine the associated tradeoffs. We show that our LIF representation corresponds to a minimal reverse DFA, i.e. a minimal DFA which accepts the language consisting of the reverse input strings, where our implicit method for obtaining an LIF representation does not require explicit construction of a forward DFA. Its practical usefulness arises from our demonstration that reverse DFAs for several datapath circuits are exponentially more compact than the classic forward DFAs. Furthermore, in comparison to the traditional representation of DFAs as state transition diagrams, our LIF representations allow for state-sharing while maintaining decomposition, resulting in memory savings in practice.>

We present a method for automatically extracting parallel prefix programs from sequential loops, even in the presence of complicated conditional statements. Rather than searching for associative operators in the loop body directly, the method rests on the observation that functional composition itself is associative. Accordingly, we model the loop body as a multivalued function of multiple parameters, and look for a closed-form representation of arbitrary compositions of loop body instances. Careful analysis of conditionals allows this search to succeed in cases where existing automatic methods fail. The method has been implemented and used to generate code for the iWarp parallel computer.

We consider a class of practically useful Boolean functions, called linearly inductive functions (LIFs), and present a canonical representation as well as algorithms for their automatic symbolic manipulation. LIFs can be used to capture structural induction in parameterized circuit descriptions, whereby our LIF representation provides a fixed-sized representation for all size instances of a circuit. Furthermore, since LIFs can naturally capture the temporal induction inherent in sequential system descriptions, our representation also provides a canonical form for sequential functions. This allows for a wide range of applications of symbolic LIF manipulation in the verification and synthesis of digital systems. We also present practical results from a preliminary implementation of a general purpose LIF package.

Performance is a central issue in parallel computing. In this paper, we describe our approach to teaching advanced undergraduates and graduate students about the fundamentals of measuring and analyzing the performance of programs running on a variety of parallel machines. This approach can be applied to virtually any type of parallel machine, as well as to parallel program simulators. Although performance analysis can serve many purposes, we focus on the needs of the parallel programmer: understanding the behavior of algorithms and programs, and making informed choices among them.

Abstract: "Data-parallel programming languages have many desirable features, such as single-thread semantics and the ability to express fine-grained parallelism. However, it is challenging to implement such languages efficiently on conventional MIMD multiprocessors, because these machines incur a high overhead for small grain sizes. This paper presents compile-time analysis techniques for data-parallel program graphs that reduce these overheads in two ways: by stepping up the grain size, and by relaxing the synchronous nature of the computation without altering the program semantics.The algorithms partition the program graph into clusters of nodes such that all nodes in a cluster have the same loop structure, and futher refine these clusters into epochs based on generation and consumption patterns of data vectors. This converts the fine-grain parallelism in the original program to medium-grain loop parallelism, which is better suited to MIMD machines. A compiler has been implemented based on these ideas. We present performance results for data-parallel kernels analyzed by the compiler and converted to single-program multiple-data (SPMD) code running on an Encore Multimax."

Parallel computers are becoming increasingly important for scientists and engineers in a wide range of disciplines.However, most undergraduate students rarely have the opportunity to program a parallel computer.Furthermore, the field of parallel computers is rapidly

A method is presented for automated verification of synchronous pipelined circuits, based on symbolic simulation and the well-known program verification concept of representation functions. The use of representation functions to allow straightforward formulation of readable and intuitive specifications is demonstrated, along with the use of a symbolic switch-level simulator to automatically prove that a circuit meets its specification. As an example, a systolic stack with more than 5000 transistors can be formally verified in a few minutes on a VAX 8800.>

A parallel algorithm for a line-finding Hough transform that runs on a linearly connected, SIMD (single-instruction, multiple-data-stream) vector of processors is described. The authors show that a high-precision transform, usually considered to be an expensive global operation, can be performed efficiently, in two to three times real time, with only local, communication on a long vector. The algorithm also illustrates a decomposition principle that has wide application in algorithm design for large linear arrays. A review of straight-line Hough transform implementations is also presented.>

This paper describes the scan line array processor (SLAP), a new architecture designed for high-performance yet low-cost image computation. A SLAP is a SIMD linear array of processors, and hence is easy to build and scales well with VLSI technology; yet appropriate special features and programming techniques make it efficient for a surprisingly wide variety of low and medium level computer vision tasks. We describe the basic SLAP concept and some of its variants, discuss a particular planned implementation, and indicate its performance on computer vision and other applications.

Highly parallel VLSI computing structures consist of many processing elements operating simultaneously. In order for such processing elements to communicate among themselves, some provision must be made for synchronization of data transfer. The simplest means of synchronization is the use of a global clock. Unfortunately, large clocked systems can be difficult to implement because of the inevitable problem of clock skews and delays, which can be especially acute in VLSI systems as feature sizes shrink. For the near term, good engineering and technology improvements can be expected to maintain the feasibility of clocking in such systems; however, clock distribution problems crop up in any technology as systems grow. An alternative means of enforcing necessary synchronization is the use of self-timed asynchronous schemes, at the cost of increased design complexity and hardware cost. Realizing that different circumstances call for different synchronization methods, this paper provides a spectrum of synchronization models; based on the assumptions made for each model, theoretical lower bounds on clock skew are derived, and appropriate or best possible synchronization schemes for large processor arrays are proposed.

A number of tree-structured multiprocessor designs have been proposed for performing a group of dictionary operations (INSERT, DELETE, EXTRACTMIN, NEAR, etc.) on a set of keys. These designs typically use one processor for each key stored and operate with constant throughput, assuming unit time to communicate and compare keys. This assumption breaks down in applications with long keys. This paper describes a machine which uses a number of processors proportional to the maximum length of a key to achieve constant throughput, regardless of key length. This design has important practical advantages over the family of tree-structured machines, and demonstrates that processor-intensive VLSI structures are not always the best route to a high-performance system.

Highly parallel VLSI computing structures consist of many processing elements operating simultaneously. In order for such processing elements to communicate among themselves, some provision must be made for synchronization of data transfer. The simplest means of synchronization is the use of a global clock. Unfortunately, large clocked systems can be difficult to implement because of the inevitable problem of clock skews and delays, which can be especially acute in VLSI systems as feature sizes shrink. For the near term, good engineering and technology improvements can be expected to maintain the feasibility of clocking in such systems; however, clock distribution problems crop up in any technology as systems grow. An alternative means of enforcing necessary synchronization is the use of self-timed, asynchronous schemes, at the cost of increased design complexity and hardware cost. Realizing that different circumstances call for different synchronization methods, this paper provides a spectrum of synchronization models; based on the assumptions made for each model, theoretical lower bounds on clock skew are derived, and appropriate or best-possible synchronization schemes for large processor arrays are proposed. One set of models is based on assumptions that allow the use of a pipelined clocking scheme, where more than one clock event is propagated at a time. In this case, it is shown that even assuming that physical variations along clock lines can produce skews between wires of the same length, any one-dimensional processor array can be correctly synchronized by a global pipelined clock while enjoying desirable properties such as modularity, expandability and robustness. This result cannot be extended to two-dimensional arrays, however—the paper shows that under this assumption, it is impossible to run a clock such that the maximum clock skew between two communicating cells will be bounded by a constant as systems grow. For such cases or where pipelined clocking is unworkable, a synchronization scheme incorporating both clocked and “asynchronous” elements is proposed.

In recent years, many systolic algorithms have been proposed as solutions to computationally demanding problems in signal and image processing and other areas. Such algorithms exploit the regularity and parallelism of problems to achieve high performance and low I/O requirements. Since systolic algorithms generally consist of a few types of simple processors, or systolic cells, connected in a regular pattern, they are less expensive to design and implement than more general machines.