The use of software implemented distributed shared memory (SDSM) to provide shared memory programming environments on networks of workstations and message-passing parallel computers has become quite popular. However, the memory reference patterns of many shared memory programs lead to poor performance on such systems. The authors propose hardware assist to improve the performance of SDSM systems faced with problematic reference patterns. An example of such a system is described. Operating system software in Mach is used to provide internode sharing in the example system, but is assisted through hardware support for maintaining update-based coherence of replicated pages. Simulations driven by hardware-collected parallel reference traces are used to provide an indication of the expected performance of the system.>

The class of NUMA (nonuniform memory access time) shared memory architectures is becoming increasingly important with the desire for larger scale multiprocessors. In such machines, the placement and movement of code and data are crucial to performance. The operating system can play a role in managing placement through the policies and mechanisms of the virtual memory subsystem. In this paper, we develop an analytic model of memory system performance of a Local/Remote NUMA architecture based on approximate mean-value analysis techniques. The model assumes that a simple workload model based on a few parameters can often provide insight into the general behavior of real applications. The model is validated against experimental data obtained with the DUnX operating system kernel for the BBN GP1000 while running a synthetic workload. The results of this validation show that in general, model predictions are quite good, though in some cases the model fails to include the effect of unexpected behaviors in the implementation. Experiments investigate the effectiveness of dynamic multiple-copy page placement. We investigate the cost of incorrect policy decisions by introducing different percentages of policy error and measuring their effect on performance.

Dynamic page placement policies for NUMA (nonuniform memory access time) shared-memory architectures are explored using two approaches that complement each other in important ways. The authors measure the performance of parallel programs running on the experimental DUnX operating system kernel for the BBN GP1000, which supports a highly parameterized dynamic page placement policy. They also develop and apply an analytic model of memory system performance of a local/remote NUMA architecture based on approximate mean-value analysis techniques. The model is validated against experimental data obtained with DUnX while running a synthetic workload. The results of this validation show that, in general, model predictions are quite good. Experiments investigating the effectiveness of dynamic page-placement and, in particular, dynamic multiple-copy page placement the cost of replication/coherency fault errors, and the cost of errors in deciding whether a page should move or be remotely referenced are described.>

Shared memory multiprocessors are attractive because they are programmed in a manner similar to uniprocessors. The UMA class of shared memory multiprocessors is the most attractive, from the programmer's point of view, since the programmer need not be concerned with the placement of code and data in the physical memory hierarchy. Scalable shared memory multiprocessors, on the other hand, tend to present at least some degree of non-uniformity of memory access to the programmer, making the NUMA class an important one to consider. In this paper, we investigate the role that DUnX, an operating system supporting dynamic page placement on a BBN GP1000, might play in simplifying the memory model presented to the applications programmer. We consider a case study of psolu, a real scientific application originally targeted for a NUMA architecture. We find that dynamic page placement can dramatically improve the performance of a simpler implementation of psolu targeted for an UMA memory architec...

The study of operating systems level memory management policies for nonuniform memory access time (NUMA) shared memory multiprocessors is an area of active research. Previous results have suggested that the best policy choice often depends on the application under consideration, while others have reported that the best policy depends on the particular architecture. Since both observations have merit, we explore the concept of policy tuning on an application/architecture basis.We introduce a highly tunable dynamic page placement policy for NUMA multiprocessors, and address issues related to the tuning of that policy to different architectures and applications. Experimental data acquired from our DUnX operating system running on two different NUMA multiprocessors are used to evaluate the usefulness, importance, and ease of policy tuning.Our results indicate that while varying some of the parameters can have dramatic effects on performance, it is easy to select a set of default parameter settings that result in good performance for each of our test applications on both architectures. This apparent robustness of our parameterized policy raises the possibility of machine-independent memory management for NUMA-class machines.

Nonuniformity of memory accessis an almost inevitable feature of the memory architecture in shared memory multiprocessor designs that can scale to large numbers of processors.One implication of NUMA architectures is that the placement and movement of code and data are crucial to performance.As memory architectures become more complex and the nonuniformity becomes less well hidden, system software must assume a larger role in providing memory management support for the programmer.This paper investigates the role of the operating system.We take an experimental approach to evaluating a wide-range of memory management policies.The target NUMA environment is BBN's GP-1OOOmultiprocessor.Extensive local modifications have been made to the memory management subsystem of BBN's nX operating system to support multiple policy implementations.Policy comparisons are based on the measured performance of real parallel applications.Our results show that there are memory management policies implemented in our system that can improve the performance of programs written using the simpler uniform memory access (UMA) programming model.While achieving the level of performance of a highly tuned NUMA program is still a difficult problem, some examples come close.There appears to be no single policy that can be considered the best over our set of test applications.Investigations into the contributions made by individual policy features toward overall behavior of the workload provide some insight into the design of a set of effective policies.