Nowadays, performance optimization is more or less an established procedure in high-performance computing (HPC) centers. To sustainably increase compute efficiency of such systems, we need to increase the awareness of efficiency on both the operator's and the users' side. Therefore, we propose an infrastructure for continuous monitoring and analysis, which automatically characterizes HPC jobs and provides a systematic approach to identify underperforming compute jobs with optimization potential. The recorded metadata and time-series data can be visualized live at runtime or post-mortem and are eventually stored for long-term analysis. The monitoring has a negligible overhead on the compute nodes and neither influences nor limits the user applications.

Nowadays, the daily work of many research communities is characterized by an increasing amount and complexity of data. This makes it increasingly difficult to manage, access and utilize the data to ultimately gain scientific insights based on it. At the same time, domain scientists want to focus on their science instead of IT. The solution is research data management to store data in a structured way enabling easy discovery for future reference and usage. An integral part is the use of metadata. With it, data becomes accessible by its content and context instead of its name and location only. The use of metadata shall be as automatic and seamless as possible in order to foster a high usability. Here, we present the architecture and developments of the Metadata Management for Applied Sciences (MASi) project that is currently building a comprehensive research data management service. MASi extends the existing KIT Data Manager framework by a generic metadata programming interface and a generic graphical web interface. Furthermore, MASi is OAI compliant and supports the OAI-PMH protocol while providing support for provenance information using ProvONE, a well-established and accepted provenance model. To illustrate the practical applicability of the MASi service, we present the adoption of initial use cases within geography, chemistry and digital humanities. The MASi research data management service is currently being prepared to go into production to satisfy the complex and varying requirements in an efficient, useable and sustainable way.

With the explosion of the number of compute nodes, the bottleneck of future computing systems lies in the network architecture connecting the nodes. Addressing the bottleneck requires replacing current backplane-based network topologies. We propose to revolutionize computing electronics by realizing embedded optical waveguides for onboard networking and wireless chip-to-chip links at 200-GHz carrier frequency connecting neighboring boards in a rack. The control of novel rate-adaptive optical and mm-wave transceivers needs tight interlinking with the system software for runtime resource management.

Research data is increasingly important in order to gain insights from scientific data. To optimally foster this, the management of research data is required to be usable, customizable and fast. We enable this by building up the MASi research data management repository service, based on the KIT DM framework. The aim is on utilizing a single repository instance to serve multiple arbitrary community use cases. Due to their diverse data characteristics the performance of the MASi service has to be fitting across the different cases. We evaluate the performance along three initial heterogeneous use cases. Various aspects are investigated; First, the object insertion and query performance of the database along the object fill level. Second and third, the ingest and download performance of digital objects using real-life data sets. Highly favorable performance characteristics are shown.

Modern processors incorporate several performance monitoring units, which can be used to count events that occur within different components of the processor. They provide access to information on hardware resource usage and can therefore be used to detect performance bottlenecks. Thus, many performance measurement tools are able to record them complementary to information about the application behavior. However, the exact meaning of the supported hardware events is often incomprehensible due to the system complexity and partially lacking or even inaccurate documentation. For most events it is also not documented whether a certain rate indicates a saturated resource usage. Therefore, it is usually difficult to draw conclusions on the performance impact from the observed event rates. In this paper, we evaluate whether hardware performance counters can be used to measure the capacity utilization within the memory hierarchy and estimate the impact of memory accesses on the achieved performance. The presented approach is based on a small selection of micro-benchmarks that constantly stress individual components in the memory subsystem, ranging from caches to main memory. These workloads are used to identify hardware performance counters that provide good estimates for the utilization of individual components in the memory hierarchy. However, since access latencies can be interleaved with computing instructions, a high utilization of the memory hierarchy does not necessarily result in low performance. We therefore also investigate which stall counters provide good estimates for the number of cycles that are actually spent waiting for the memory hierarchy.

In this paper we study the correlation of node failures in time and space. Our study is based on measurements of a production high performance computer over an 8-month time period. We draw possible types of correlations between node failures and show that, in many cases, there are direct correlations between observed node failures. The significance of such a study is twofold: achieving a clearer understanding of correlations between node failures and enabling failure detection as early as possible. The results of this study are aimed at helping the system administrators minimize (or even prevent) the destructive effects of correlated node failures.

The Message Passing Interface (MPI) includes nonblocking collective operations that support additional overlap between computation and communication. These new operations enable complex data movement between large numbers of processes. However, their asynchronous behavior hides and complicates the detection of defects in their use. We highlight a lack of correctness tool support for these operations and extend the MUST runtime MPI correctness tool to alleviate this complexity. We introduce a classification to summarize the types of correctness analyses that are applicable to MPI's nonblocking collectives. We identify complex wait-for dependencies in deadlock situations and incorrect use of communication buffers as the most challenging types of usage errors. We devise, demonstrate, and evaluate the applicability of correctness analyses for these errors. A scalable analysis mechanism allows our runtime approach to scale with the application. Benchmark measurements highlight the scalability and applicability of our approach at up to 4,096 application processes and with low overhead.

Dynamic load balancing with diffusive methods is known to provide minimal load transfer and requires communication between neighbor nodes only. These are very attractive properties for highly parallel systems. We compare diffusive methods with state-of-the-art geometrical and graph-based partitioning methods on thousands of nodes. When load balancing overheads, i.e. repartitioning computation time and migration, have to be minimized, diffusive methods provide substantial benefits.

Distributed data life cycles consist of data sources, data and computing components as well as data sinks and user facing elements. The complexity of the underlying systems is ever rising with the increasing heterogeneity and distribution of components and environments. Researchers would like to focus on their specific research topic without the need to learn these systems in detail. Accessible data life cycles enable scientists to do better science more efficiently and obtain results, which would not have been possible without these advanced technologies. For this objective, abstraction to hide complexity and automation to avoid manual tasks are a necessity. These are embodied in the three conceptual data life cycle challenges, namely data, computing and utilization. Concepts and technologies to manage these challenges are explored and exemplified on the basis of the general data life cycle and MoSGrid (Molecular Simulation Grid) science gateway. In this context, we especially focus on teaching in drug design and quantum chemistry research use cases. Further cases are presented elucidating various challenges in adapting the concepts and technologies to wind energy data analysis and the XSEDE research infrastructure.

A major challenge in the design of contemporary microprocessors is the increasing number of cores in conjunction with the persevering need for cache coherence. To achieve this, the memory subsystem steadily gains complexity that has evolved to levels beyond comprehension of most application performance analysts. The Intel Has well-EP architecture is such an example. It includes considerable advancements regarding memory hierarchy, on-chip communication, and cache coherence mechanisms compared to the previous generation. We have developed sophisticated benchmarks that allow us to perform in-depth investigations with full memory location and coherence state control. Using these benchmarks we investigate performance data and architectural properties of the Has well-EP micro-architecture, including important memory latency and bandwidth characteristics as well as the cost of core-to-core transfers. This allows us to further the understanding of such complex designs by documenting implementation details the are either not publicly available at all, or only indirectly documented through patents.

Scientific data life cycles are becoming more and more demanding. Data amounts are seen to be ever increasing while at the same time advanced IT infrastructures become more common resulting in a much broader user group demanding a high usability. These challenges are met with a distributed science gateway infrastructure, which in turn is complex in nature. In this situation a main part is missing, a generic metadata management concept. The approach and architecture of such a concept, the current status, an evaluation, and an outlook will be described in this publication. The main goals are to enable scientists to easily manage and use large amounts of data and provide a generic way to enable the fast transfers to a multitude of scientific use cases.

SUMMARY State‐of‐the‐art research in a variety of natural sciences depends heavily on methods of computational chemistry, for example, the calculation of the properties of materials, proteins, catalysts, and drugs. Applications providing such methods require a lot of expertise to handle their complexity and the usage of high‐performance computing. The MoSGrid (molecular simulation grid) infrastructure relieves this burden from scientists by providing a science gateway, which eases access to and usage of computational chemistry applications. One of its cornerstones is the molecular simulations markup language (MSML), an extension of the chemical markup language. MSML abstracts all chemical as well as computational aspects of simulations. An application and its results can be described with common semantics. Using such application, independent descriptions users can easily switch between different applications or compare them. This paper introduces MSML, its integration into a science gateway, and its usage for molecular dynamics, quantum chemistry, and protein docking. Copyright © 2013 John Wiley & Sons, Ltd.

While various HPC tools use Tree-Based Overlay Networks (TBONs) to increase their scalability, some use cases do not map well to a tree-based hierarchy. We provide the concept of intralayer communication to improve this situation, where nodes in a specific hierarchy layer may exchange messages directly with each other. This concept targets data preprocessing that allows tool developers to avoid load imbalances in higher hierarchy levels. We implement intralayer communication within the Generic Tools Infrastructure (GTI) that provides TBON services, as well as a high-level abstraction to ease the creation of scalable runtime tools. An extension of GTI's abstractions allows simple and efficient use of intralayer communication. We demonstrate this capability with a runtime message matching tool for MPI's point-to-point communication, which we evaluate in an application study with up to 16,384 processes. Low overheads for two benchmark suites show the applicability of our approach, while a stress test demonstrates close to constant overheads across scales. The stress test measurements demonstrate that intralayer communication reduces application slowdown by two orders of magnitude at 2,048 processes, compared to a previous TBON-based implementation.

Heterogeneous systems consisting of general-purpose processors and different types of hardware accelerators are becoming more and more common in HPC systems. Especially Field Programmable Gate Arrays (FPGAs) provide an energy-efficient way to achieve high performance. Numerous application areas, including bio- and neuroinformatics, require enormous processing capability and employ simple computation cores, elementary data structures and algorithms highly suitable for FPGAs. To allow an efficient work with distributed FPGAs, it is necessary to provide a simple and scalable integration of these FPGAs in a common cluster architecture and to permit an easy access to these resources. Our approach enables a system-wide dynamic partitioning, a batch-based administration and the monitoring of FPGA resources. The system can easily be reconfigured to user-specific requirements and provides a high degree of flexibility and performance.

One of the most urgent challenges in event based performance analysis is the enormous amount of collected data. A real-time event reduction is crucial to enable a complete in-memory event tracing workflow, which circumvents the limitations of current parallel file systems to support event tracing on large scale systems. However, a traditional single flat memory buffer fails to support real-time event reduction. To address this issue, we present a hierarchical memory buffer, which is capable to support event reduction. We show that this hierarchical memory buffer does not introduce additional overhead, regardless of the grade of reduction. In addition, we evaluate its main parameter: the size of the internal memory bins. The hierarchical memory buffer is based on the Open Trace Format 2, a state-of-the-art Open Source event trace library used by the performance analysis tools VAMPIR, SCALASCA, and TAU.

Energy efficiency is of steadily growing importance in virtually all areas from mobile to high performance computing. Therefore, lots of research projects focus on this topic and strongly rely on power measurements from their test platforms. The need for finer grained measurement data-both in terms of temporal and spatial resolution (component breakdown)-often collides with very rudimentary measurement setups that rely e.g., on non-professional power meters, IMPI based platform data or model-based interfaces such as RAPL or APM. This paper presents an in-depth study of several different AC and DC measurement methodologies as well as model approaches on test systems with the latest processor generations from both Intel and AMD. We analyze most important aspects such as signal quality, time resolution, accuracy, and measurement overhead and use a calibrated, professional power analyzer as our reference.

Runtime error detection tools detect many classes of MPI usage errors, including errors in collective communication calls. However, they often face scalability challenges. We present runtime checks for MPI collective operations that use a Tree-Based Overlay Network (TBON) for scalability and that provide full datatype matching. While we can use transitive correctness properties for most checks, some collective operations impose non-transitive correctness properties, e.g., MPI_Alltoallv, where we use an intralayer communication within the TBON to distribute datatype matching information. An overhead study with stress tests and two benchmark suites demonstrates applicability and scalability at 4,096, 2,048 and 16,384 processes respectively.

Communication analysis of parallel applications based on event traces depends on correct matching of associated MPI send and receive events. Selective monitoring techniques, however, may result in incomplete MPI event traces and, in that case, current matching strategies fail. In this paper we introduce an additional unique identifier for each message to make MPI events distinguishable from others. Therefore, it is possible to identify missing MPI events and match all remaining MPI events correctly. An overhead study with a real-life application and a benchmark suite demonstrates the applicability and benefits of this approach.

We present a particle-in-cell simulation of the relativistic Kelvin-Helmholtz Instability (KHI) that for the first time delivers angularly resolved radiation spectra of the particle dynamics during the formation of the KHI. This enables studying the formation of the KHI with unprecedented spatial, angular and spectral resolution. Our results are of great importance for understanding astrophysical jet formation and comparable plasma phenomena by relating the particle motion observed in the KHI to its radiation signature.

The widely used Message Passing Interface (MPI) with its multitude of communication functions is prone to usage errors. Runtime error detection tools aid in the removal of these errors. We develop MUST as one such tool that provides a wide variety of automatic correctness checks. Its correctness checks can be run in a distributed mode, except for its deadlock detection. This limitation applies to a wide range of tools that either use centralized detection algorithms or a timeout approach. In order to provide scalable and distributed deadlock detection with detailed insight into deadlock situations, we propose a model for MPI blocking conditions that we use to formulate a distributed algorithm. This algorithm implements scalable MPI deadlock detection in MUST. Stress tests at up to 4,096 processes demonstrate the scalability of our approach. Finally, overhead results for a complex benchmark suite demonstrate an average runtime increase of 34% at 2,048 processes.

Digital instruments and simulations are creating an ever-increasing amount of data. The need for institutions to acquire these data and transfer them for analysis, visualization, and archiving is growing as well. In parallel, networking technology is evolving, but at a much slower rate than our ability to create and store data. Single fiber 100 Gbps networking solutions have recently been deployed as national infrastructure. This article describes our experiences with data movement and video conferencing across a networking testbed, using the first commercially available single fiber 100 Gbps technology. The testbed is unique in its ability to be configured for a total length of 60, 200, or 400 km, allowing for tests with varying network latency. We performed low-level TCP tests and were able to use more than 99.9% of the theoretical available bandwidth with minimal tuning efforts. We used the Lustre file system to simulate how end users would interact with a remote file system over such a high performance link. We were able to use 94.4% of the theoretical available bandwidth with a standard file system benchmark, essentially saturating the wide area network. Finally, we performed tests with H.323 video conferencing hardware and quality of service (QoS) settings, showing that the link can reliably carry a full high-definition stream. Overall, we demonstrated the practicality of 100 Gbps networking and Lustre as excellent tools for data management.

The Special Session on “Pathways to Servers of the Future” outlines a new research program set up at Technische Universität Dresden addressing the increasing energy demand of global internet usage and the resulting ecological impact of it. The program pursues a novel holistic approach that considers hardware as well as software adaptivity to significantly increase energy efficiency, while suitably addressing application demands. The session presents the research challenges and industry perspective.

Event tracing is an important tool for understanding the performance of parallel applications. As concurrency increases in leadership-class computing systems, the quantity of performance log data can overload the parallel file system, perturbing the application being observed. In this work we present a solution for event tracing at leadership scales. We enhance the I/O forwarding system software to aggregate and reorganize log data prior to writing to the storage system, significantly reducing the burden on the underlying file system for this type of traffic. Furthermore, we augment the I/O forwarding system with a write buffering capability to limit the impact of artificial perturbations from log data accesses on traced applications. To validate the approach, we modify the Vampir tracing toolset to take advantage of this new capability and show that the approach increases the maximum traced application size by a factor of 5x to more than 200,000 processes.

Runtime detection of semantic errors in MPI applications supports efficient and correct large-scale application development. However, current approaches scale to at most one thousand processes and design limitations prevent increased scalability. The need for global knowledge for analyses such as type matching, and deadlock detection presents a major challenge. We present a scalable tool infrastructure - the Generic Tool Infrastructure (GTI) - that we will use to implement MPI runtime error detection tools and that applies to other use cases. GTI supports simple offloading of tool processing onto extra processes or threads and provides a tree based overlay network (TBON) for creating scalable tools that analyze global knowledge. We present its abstractions and code generation facilities that ease many hurdles in tool development, including wrapper generation, tool communication, trace reductions, and filters. GTI ultimately allows tool developers to focus on implementing tool functionality instead of the surrounding infrastructure. Further, we demonstrate that GTI supports scalable tool development through a lost message detector and a phase profiler. The former provides a more scalable implementation of important base functionality for MPI correctness checking, while the latter tool demonstrates that GTI can serve as the basis of further types of tools. Experiments with up to 2048 cores show that GTI's scalability features apply to both tools.

Efficient deadlock-free routing strategies are crucial to the performance of large-scale computing systems. There are many methods but it remains a challenge to achieve lowest latency and highest bandwidth for irregular or unstructured high-performance networks. % We investigate a novel routing strategy based on the single-source-shortest-path routing algorithm and extend it to use virtual channels to guarantee deadlock-freedom. We show that this algorithm achieves minimal latency and high bandwidth with only a low number of virtual channels and can be implemented in practice. % We demonstrate that the problem of finding the minimal number of virtual channels needed to route a general network deadlock-free is NP-complete and we propose different heuristics to solve the problem. We implement all proposed algorithms in the Open Subnet Manager of InfiniBand and compare the number of needed virtual channels and the bandwidths of multiple real and artificial network topologies which are established in practice. % Our approach allows to use the existing virtual channels more effectively to guarantee deadlock-freedom and increase the effective bandwidth of up to a factor of two. Application benchmarks show an improvement of up to 95\%. Our routing scheme is not limited to InfiniBand but can be deployed on existing InfiniBand installations to increase network performance transparently without modifications to the user applications.

The performance of computationally intensive scientific applications on the underlying system can be maximized by providing application-level load balancing of loop iterates via the use of dynamic loop scheduling (DLS) algorithms. These DLS methods are based on probabilistic analyses, and therefore account for unpredictable variations of algorithmic, systemic and application level characteristics. A considerable number of DLS algorithms has been proposed in the last decade, and some of them have been effectively integrated into scientific and engineering applications, yielding significant performance improvements. However, scheduling scientific applications in large-scale distributed systems where, the chances of failure, such as processor or link failure, are high, makes the problem of achieving a load balanced execution even more challenging. Although real experiments are necessary to verify the benefits of using DLS, they prove to be very time consuming when every level of detail is required for the assessment of the execution of complex, data parallel and irregular scientific applications using DLS on a wide range of architectural platforms and computational environments. Thus, we propose the use of simulators which can give results that are not always experimentally measurable with the current technology. Simulations also help in studying the problem at various levels of abstraction and provide practical feedback. In this paper, we discuss the implementation of DLS techniques in Alea, a Grid Sim based scheduling simulator. Based on the simulation results, we further compare the load balancing characteristics of these methods in a simulated parallel and distributed computing environment.

We are pleased to present a special issue of the best papers presented at the Scientific Day of the International Supercomputing Conference (ISC), during June 2007 in Dresden, Germany.ISC has always been an event where the academia and the industry meet to exchange ideas and shape the future of HPC.Started in Mannheim, Germany, in 1986 by Prof. Hans Werner Meuer, this conference has traditionally featured a diverse set of invited presentations.2007 marked an important change in the ISC-the Scientific Day-which offered supercomputing experts for the first time an opportunity to submit papers to be presented at the event through a peer review process.A total of 45 papers were submitted for consideration and reviewed by leading experts in supercomputing and high-performance computing.A total of 16 presentations were given at the inaugural Scientific Day.Six papers were accepted for publication in this special issue of Concurrency and Computation: Practice and Experience.As is typical of the ISC itself, the paper contributions cover a wide variety of important aspects of high-performance computers.The papers from ISC07 Scientific Day presented here may be grouped into three broad categories: papers on making effective and efficient use of supercomputers; papers regarding the measurement of performance of supercomputers; and papers describing particular aspects of specialized supercomputer hardware.Three papers dealt with the topic of making effective and efficient use of supercomputers.The subject of power consumption generally is very often in the news today, and particularly so within the high-performance computing community.The paper by DiRienzo et al. describes the development of a power aware job scheduler.The work by Stewart et al. presents the experiences of the integration of a large system into a grid environment.In addition to the operational and performance issues it describes the presentation of user-friendly working environments using science gateways.The impact of migrating from a single core to a multicore CPU is analyzed by Brightwell et al.This paper presents specific features of the Cray XT3 performance measured with microbenchmarks and analyzed with respect to the observed application performance to clarify the impact of the migration of a large system to multicore CPUs.This paper was selected as an ISC award winning paper.Two papers discussed performance measurement.The paper by Mueller et al. presents an important new application benchmark suite for MPI parallel applications.This suite was developed by the SPEC High-Performance Group and is a significant contribution to the general area of supercomputer performance measurement.The features of this benchmark suite are presented in detail together with the first performance results on a number of parallel systems.The contribution of Mucci and Mohan presents the creation of a vendor-independent open source performance suite for Linux clusters.It also gives a brief overview of the current efforts in the field of tools for performance analysis for parallel programs.

We are pleased to introduce the special issue of the best papers presented at the Scientific Day of the International Supercomputing Conference (ISC), during June 2008 in Dresden, Germany. ISC always has been an important platform for academia and industry to exchange ideas and to shape the future of HPC. Started in 1986 by Prof. Hans Werner Meuer in Mannheim, Germany, this conference has traditionally featured a diverse set of invited presentations. In 2008, the Scientific Day again offered supercomputing experts the opportunity to submit papers to be presented at the conference through a peer review process. A total of 40 papers were submitted for consideration and reviewed by leading experts in supercomputing and high-performance computing. A total of 18 presentations were given at the Scientific Day in Dresden. Six papers were accepted for publication in this special issue of Concurrency and Computation: Practice and Experience. As is typical of the ISC itself, the paper contributions cover a wide variety of important aspects of high-performance computing. The paper by Bouteiller, Bosilica, and Dongarra addresses the issue of fault tolerance by introducing a novel approach to message logging and implementing it in Open MPI. With this implementation, the authors are able to demonstrate significant performance improvements compared with other solutions. Performance measurement and analysis tools on large-scale systems are treated by Mohr, Wylie, and Wolf. The paper includes a comparison of tools scaling beyond 8000 processes and measurements using their own tool with up to 65 536 processes. This paper was selected as one of the two ISC'08 award winning papers. Performance portability is the topic of two papers. The first by Gabriel et al. describes an abstract data and communication library that provides runtime optimization of application-level communication patterns frequently found in scientific applications. The second by Turek et al. received the PRACE award and is focused on a finite element-based solver toolkit that executes efficiently on commodity clusters, vector supercomputers, and GPGPUs. Improvements for I/O are discussed by Balaji et al. Here, an environment for distributed I/O is presented and applied to gene sequence analysis with mpiBLAST. The analysis was distributed over nine sites and involved more than a petabyte of data. The paper was selected for the second ISC'08 award. The simulation of the strength and stiffness of human bones is covered by Bekas et al. The computation requires to scale up to thousands of processors; the paper contains a detailed performance and scaling study. We would like to thank the authors for contributing papers on their research in high-performance computing to this special publication, and all the reviewers for providing constructive reviews and helping to shape the contributions Finally, we would like to thank the co-editors of this journal—Professors Geoffrey Fox (Pervasive Technology Institute and School of Informatics, Indiana University) and Luc Moreau (School of Electronics and Computer Science, University of Southampton) for providing us the opportunity to create this special issue. With this special issue, we publish some of the best papers from the ISC Scientific Day, and we hope to contribute to the field of high-performance computing and aid the continued success of this aspect of the conference.

Across a broad range of applications, multicore technology is the most important factor that drives today's microprocessor performance improvements. Closely coupled is a growing complexity of the memory subsystems with several cache levels that need to be exploited efficiently to gain optimal application performance. Many important implementation details of these memory subsystems are undocumented. We therefore present a set of sophisticated benchmarks for latency and bandwidth measurements to arbitrary locations in the memory subsystem. We consider the coherency state of cache lines to analyze the cache coherency protocols and their performance impact. The potential of our approach is demonstrated with an in-depth comparison of ccNUMA multiprocessor systems with AMD (Shanghai) and Intel (Nehalem-EP) quad-core x86-64 processors that both feature integrated memory controllers and coherent point-to-point interconnects. Using our benchmarks we present fundamental memory performance data and architectural properties of both processors. Our comparison reveals in detail how the microarchitectural differences tremendously affect the performance of the memory subsystem.

In the past several years, many scientific applications from various domains have taken advantage of e-science infrastructures that share storage or computational resources such as supercomputers, clusters or PC server farms across multiple organizations. Especially within e-science infrastructures driven by high-performance computing (HPC) such as DEISA, online visualization and computational steering (COVS) has become an important technique to save compute time on shared resources by dynamically steering the parameters of a parallel simulation. This paper argues that future supercomputers in the Petaflop/s performance range with up to 1 million CPUs will create an even stronger demand for seamless computational steering technologies. We discuss upcoming challenges for the development of scalable HPC applications and limits of future storage/IO technologies in the context of next generation e- science infrastructures and outline potential solutions.

Today's HPC environments are increasingly complex in order to achieve highest performance. Hardware platforms introduce features like out-of-order execution, multi-level caches, multi-cores, non-uniform memory access etc. Application software combines OpenMP, MPI, optimized libraries and various types of compiler optimization to exploit potential performance.To reach a reasonable percentage of the theoretical peak performance, three fundamental steps need to be accomplished. First, correctness must be guaranteed especially during the course of optimization. Second, the actual performance achieved needs to be determined. In particular the contributions/limitations of all sub-systems involved (CPU, memory, network, I/O) have to be identified. Third, actual optimization can only be successful with the previously obtained knowledge.Those steps are by no means trivial. There are sophisticated tools beyond simple profiling to support the HPC user. The tutorial introduces a variety of such tools: it shows how they play together and how they scale with long-running massively parallel cases.

The Open Trace Format (OTF) is a DOE-sponsored initiative to help deliver open, scalable performance tracing tools for HPC systems. OTF is an open specification of trace information to provide a target for trace generation and to allow trace analysis and visualization tools to operate efficiently at large scale. The Technical University of Dresden and ParaTools, Inc. developed the first version OTF with support from Lawrence Livermore National Laboratory.The BOF has two goals. First, we will report on the current status of OTF. This will include a review of the OTF specification and a description of the OTF reader/writer library to be released into open source at SC06. We will also report on recent ports of the library to HPC platforms.The second goal is to invite community involvement in the OTF initiative and to establish a working group chartered with evolving the OTF specification in the future.

Compressed complete call graphs (cCCGs) are a newly developed memory data structure for event based program traces. The most important advantage over linear lists or arrays traditionally used is the ability to apply lossy or lossless data compression. The compression scheme is completely transparent with respect to read access decompression is not required. This approach is a new way to cope with todays challenges when analyzing enormous amounts of trace data. The article focuses on CCG construction and compression, querying and evaluation are briefly covered.

The use of a cluster for distributed performance analysis of parallel trace data is discussed. We propose an analysis architecture that uses multiple cluster nodes as a server to execute analysis operations in parallel and communicate to remote clients where performance visualization and user interactions occur. The client-server system developed, VNG, is highly configurable and is shown to perform well for traces of large size, when compared to leading trace visualization systems.

Article Free Access Share on Parallel programs and background load: efficiency studies with the PAR-Bench system Authors: Wolfgang E. Nagel Zentralinstitut für Angewandte Mathematik, Forschungszentrum Jülich GmbH (KFA), Postfach 1913, 5170 Jülich, Fed. Rep. Germany Zentralinstitut für Angewandte Mathematik, Forschungszentrum Jülich GmbH (KFA), Postfach 1913, 5170 Jülich, Fed. Rep. GermanyView Profile , Markus A. Linn Zentralinstitut für Angewandte Mathematik, Forschungszentrum Jülich GmbH (KFA), Postfach 1913, 5170 Jülich, Fed. Rep. Germany Zentralinstitut für Angewandte Mathematik, Forschungszentrum Jülich GmbH (KFA), Postfach 1913, 5170 Jülich, Fed. Rep. GermanyView Profile Authors Info & Claims ICS '91: Proceedings of the 5th international conference on SupercomputingJune 1991 Pages 365–375https://doi.org/10.1145/109025.109111Online:01 June 1991Publication History 0citation217DownloadsMetricsTotal Citations0Total Downloads217Last 12 Months5Last 6 weeks4 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

Modern supercomputers like CRAY X-MP and IBM 3090 VF achieve their high computing speed by using both vector and parallel hardware. The available multitasking concepts supporting concurrent execution of tasks within a single application have been designed for different purposes: owing to the small dispatching overhead, fine-grain parallelism allows parallelization of small units of computation, usually chunks of a DO loop. Larger units of computation, such as arithmetic intensive subroutines, may be processed independently using coarse-grain parallelism.

Today, numerical simulation is used more and more to get detailed information about crystal growth processes. In the Czochralski crystal growth from electrically conductive melts, the application of an external magnetic field has become a very useful technique for improving crystal quality. The integration of the magnetic field into the simulation program leads to a system of coupled partial differential equations, i.e. incompressible Navier-Stokes equations and convective heat equations in combination with an external magnetic field, which have to be solved. Because of the regular access to data this problem is suitable for vector computers. The CRAY X-MP provides vectorization and multiprocessing capabilities. With multitasking it is possible to run different parts of one program in parallel. Both multitasking concepts, macrotasking and microtasking, were examined to evaluate their potential to speed-up the Czochralski crystal growth simulation program implemented on a CRAY X-MP.