ABSTRACT As high‐performance computing (HPC) becomes a tool used in many different workflows, quality of service (QoS) becomes increasingly important. In many cases, this includes the reliable execution of an HPC job and the generation of the results by a certain deadline. The resource and job management system (RJMS) or simply RMS is responsible for receiving the job requests and executing the jobs with a deadline‐oriented policy to support the workflows. In this article, we evaluate how well static resource management policies cope with deadline‐constrained HPC jobs and explore two variations of a dynamic policy in this context. As the Hilbert curve‐based approach used by the SLURM workload manager represents the state‐of‐the‐art in production environments, it was selected as one of the static allocation strategies. The Manhattan median approach as a second allocation strategy was introduced as a research work that aims to minimize the communication overhead of the parallel programs by providing compact partitions more than the Hilbert curve approach. In contrast to the static partitions provided by the Hilbert curve approach and the Manhattan median approach, the leak approach focuses on supporting dynamic runtime behavior of the jobs and assigning nodes of the HPC system on demand at runtime. Since the contiguous leak version also relies on a compact set of nodes, the noncontiguous leak can provide additional nodes at a greater distance from the nodes already used by the job. Our preliminary results clearly show that a dynamic policy is needed to meet the requirements of a modern deadline‐oriented RMS scenario.

Reducing the cost of memory accesses, both in terms of performance and energy consumption, is a major challenge in shared-memory architectures. Modern systems have deep and complex memory hierarchies with multiple cache levels and memory controllers, leading to a Non-Uniform Memory Access (NUMA) behavior. In such systems, there are two ways to improve the memory affinity: First, by mapping threads that share data to cores with a shared cache, cache usage and communication performance are optimized. Second, by mapping memory pages to memory controllers that perform the most accesses to them and are not overloaded, the average cost of accesses is reduced. We call these two techniques thread mapping and data mapping, respectively. Thread and data mapping should be performed in an integrated way to achieve a compounding effect that results in higher improvements overall. Previous work in this area requires expensive tracing operations to perform the mapping, or require changes to the hardware or to the parallel application. In this paper, we propose kMAF, a mechanism that performs integrated thread and data mapping in the kernel. kMAF uses the page faults of parallel applications to characterize their memory access behavior and performs the mapping during the execution of the application based on the detected behavior. In an evaluation with a large set of parallel benchmarks executing on three NUMA architectures, kMAF achieved substantial performance and energy efficiency improvements, close to an Oracle-based mechanism and significantly higher than previous proposals.

One of the main challenges for parallel architectures is the increasing complexity of the memory hierarchy, which consists of several levels of private and shared caches, as well as interconnections between separate memories in NUMA machines. To make full use of this hierarchy, it is necessary to improve the locality of memory accesses by reducing accesses to remote caches and memories, and using local ones instead. Two techniques can be used to increase the memory access locality: executing threads and processes that access shared data close to each other in the memory hierarchy (thread affinity), and placing the memory pages they access on the NUMA node they are executing on (data affinity). Most related work in this area focuses on either thread or data affinity, but not both, which limits the improvements. Other mechanisms require expensive operations, such as memory access traces or binary analysis, require changes to hardware or work only on specific parallel APIs.

Forecast precisions of climatological models are limited by computing power and time available for the executions. As more and faster processors are used in the computation, the resolution of the mesh adopted to represent the Earth's atmosphere can be increased, and consequently the numerical forecast is more accurate and shows local phenomena. However, a finer mesh resolution, able to include local phenomena in a global atmosphere integration, is still not possible. To overcome this situation, different mesh refinement levels can be used at the same time for different areas. In this context, this paper evaluates how mesh refinement at run time can improve performance for climatological models. In order to contribute with this analysis, an online dynamic mesh refinement was developed. It increases mesh resolution in parts of a parallel distributed model, when special atmosphere conditions are registered during the execution. The results show that the parallel execution of this improvement provides better resolution for the meshes, without a significant increase of execution time.

Process placement is a technique widely used on parallel machines with heterogeneous interconnects to reduce the overall communication time. For instance, two processes which communicate frequently are mapped close to each other. Finding the optimal mapping between threads and cores in a shared-memory environment (for example, OpenMP and Pthreads) is an even more complex task due to implicit communication. In this work, we examine data sharing patterns between threads in different workloads and use those patterns in a similar way as messages are used to map processes in cluster computers. We evaluated our technique on a state-of-the-art multicore processor and achieved moderate improvements in the common case and considerable improvements in some cases, reducing execution time by up to 45%.

In this paper we will describe a model for BSP (Bulk Synchronous Parallel) process rescheduling called MigBSP. Considering the scope of BSP applications, its differential approach is the combination of three metrics - Memory, Computation and Communication - in order to measure the Potential of Migration of each BSP process. In this context, this paper addresses both the efficiency and the adaptivity perspectives of this model over our multi-cluster machine.

While OpenMP conceptually allows to vary the degree of parallelism from one parallel region to the next in order to adapt to the system load, this might still be too coarse-grained in certain scenarios. Especially applications designed for parallelism may stay within one parallel region for a long time. This may lead either to an oversubscribed system where individual applications are not restricted in their degree of parallelism, or to an underutilized system, because individual applications are restricted to a too small degree of parallelism. In this paper, we tackle both problems by dynamically restricting the number of active threads within a parallel region without violating the OpenMP specification.

We have developed a model called MigBSP that controls processes rescheduling in BSP (bulk synchronous parallel)applications. A BSP application is composed by one or more supersteps, each one containing both computation and communication phases followed by a synchronization barrier. Since the barrier waits for the slowest process, MigBSPpsilas final idea is to adjust the processes location in order to reduce the superstepspsila times. Considering the scope of the BSP model, the novel ideas of MigBSPare: (i) combination of three metrics - memory, computation and communication - to measure the potential of migration of each BSP process; (ii) use of both computation and communication patterns to control processespsila regularity;(iii) adaptation regarding the periodicity to launch the processes rescheduling. This paper describes MigBSP and presents some experimental results and related work.

The performance of Web servers and application servers is a crucial factor for the success of the underlying business activity. Current commercial servers (such as Apache and Microsoftpsilas IIS) usually employ a thread-based concurrency policy for executing request simultaneously. However, with this policy the throughput tends to collapse when massive overload occurs. Event-based concurrency has been proposed as an alternative, but - similar to the thread-based approach - it suffers in its pure form from a potential memory access bottleneck. This is avoided by a server based on a staged event-driven architecture (SEDA) which combines the two approaches to take advantage of locality in data and code within each state. In this paper, we present a three layer observer/controller architecture for dynamic performance control of SEDA-based application servers. Control is achieved by a strategy which adapts the resources assigned to each stage based on the observed throughput and queue length of each stage using a feedback loop. To validate our strategy and to compare with other strategies, we implemented a simulation environment. The results gained demonstrate that our approach successfully self-optimizes the CPU time allocation and achieves a competitive system throughput while avoiding performance degradation under overload and dynamic changes in the system.

Grid applications can be modeled as a composition of rather independent tasks. There are two approaches to define such a workflow either by combining multiple applications to build a more complex functionality or by splitting up an existing application. In this paper we analyze the latter process. We present a compute intensive application for climatology simulation and the options available to split it up. Using the simulation mode of our grid broker, we were able to compare the different workflow specifications before actually executing the workflows. This case study showed, using finer grained workflows-which usually need more adjustments to the software-allows better performance in the grid.

For resource management in grid environments, advance reservations turned out to be very useful and hence are supported by a variety of grid toolkits. However, failure recovery for such systems has not yet received the attention it deserves. In this paper, we address the problem of remapping reservations to other resources, when the originally selected resource fails. Instead of dealing with jobs already running, which usually means checkpointing and migration, our focus is on jobs that are scheduled on the failed resource for a specific future period of time but not started yet. The most critical factor when solving this problem is the estimation of the downtime. We avoid the drawbacks of under- or overestimating the downtime by a dynamic load-based approach that is evaluated by extensive simulations in a grid environment and shows superior performance compared to estimation-based approaches.

In general, two types of resource reservations in computer networks can be distinguished: immediate reservations which are made in a just-in-time manner and advance reservations which allow to reserve resources a long time before they are actually used. Advance reservations are especially useful for grid computing but also for a variety of other applications that require network quality-of-service, such as content distribution networks or even mobile clients, which need advance reservation to support handovers for streaming video. With the emerged MPLS standard, explicit routing can be implemented also in IP networks, thus overcoming the unpredictable routing behavior which so far prevented the implementation of advance reservation services. The impact of such advance reservation mechanisms on the performance of the network with respect to the amount of admitted requests and the allocated bandwidth has so far not been examined in detail. We show that advance reservations can lead to a reduced performance of the network with respect to both metrics. The analysis of the reasons shows a fragmentation of the network resources. In advance reservation environments, additional new services can be defined such as malleable reservations and can lead to an increased performance of the network. Four strategies for scheduling malleable reservations are presented and compared. The results of the comparisons show that some strategies increase the resource fragmentation and are therefore unsuitable in the considered environment while others lead to a significantly better performance of the network. Besides discussing the performance issue, the software architecture of a management system for advance reservations is presented.

In this paper, we present an integration of SCI (Scalable Coherent Interface) into the TCP/IP protocol stack of Linux for high bandwidth low latency communication within two or more computers connected with SCI. We describe a network driver which uses SCI at the physical layer. We explain some further optimization techniques and the resulting performance improvement. In this context we shown a technique to eliminate a copy operation on the receiver side of the Linux-TCP/IP stack.

Current processor allocation techniques for highly parallel systems are based on centralized front-end based algorithms. As a result, the applied strategies are restricted to static allocation, low parallelism and weak fault tolerance. To lift these restrictions, we are investigating a distributed approach to the processor allocation problem in large distributed memory machines. A contiguous and a noncontiguous version of a distributed dynamic processor allocation strategy are proposed and studied. Simulations compare the performance of the proposed strategies with that of well-known centralized algorithms. We also present the results of experiments on a Simens hpcline Primergy Server with 96 nodes that show distributed allocation is feasible with current technologies.

To execute a parallel program on a multicomputer system, the tasks of the program have to be mapped to the particular processors of the parallel machine. The aim of the mapping is twofold: (i) to achieve a balanced load on the processors (partitioning problem) and (ii) to keep communication delays low by placing communicating tasks closely together (mapping). Since both the communication structure of the program and the interconnection structure of the parallel machine can be represented as graphs, the mapping problem can be regarded as a graph embedding problem to minimize communication costs. As a new heuristic approach to this NP-hard problem we apply Kohonen's self-organizing maps to establish a topology-preserving embedding. Experimental results are presented and compared to other approaches to this problem. The most attractive feature of our new method is that it can be extremely well parallelized.

A relational database workload analyzer (REDWAR) is developed to characterize the workload in a DB2 environment. This is applied to study a production DB2 system where a structured query language (SQL) trace for a two-hour interval and an image copy of the database catalog were obtained. The results of the workload study are summarized. The structure and complexity of SQL statements, the makeup and run-time behavior of transactions/queries, and the composition of relations and views are discussed. The results obtained provide the important information needed to build a benchmark workload to evaluate the alternative design tradeoffs of database systems.>

An examination is made of a technique to derive the time-stamp or interval of time-stamps dynamically by using limited time-stamp history information to re-order transaction at commit time and to derive a back-shifted time-stamp for certification to reduce read-write conflicts. An analytic model to quantify the performance improvement by comparison with the basic time-stamp certification scheme is developed. Since the committed conflicting transaction may itself be back-shifted, the analytic model needs to estimate the distribution of the back-shift of the certification time-stamp in order to determine the probability of successfully back-shifting a transaction requesting commit. It is shown how this relatively complex protocol can be analyzed using a few simple approximations, validate the analysis through simulations, and the range of parameter values for which this approach is most beneficial is determined.>