One of the earliest technical application areas for discrete simulation has been communication systems and networks. This area continues to be a focus of simulation activity as communication systems continue to evolve. Specific uses run the gamut from chip-level networks, evolving and novel RF waveforms and hardware, high performance networks in clusters and data centers, up through Internet scale protocols and applications. At each level evolving and new requirements and possible solutions are being developed, and often first in simulation in advance of hardware/software availability. In many cases simulation has fully entered the engineering design cycle, leading to pre-manufacture exploration of performance tradeoffs and edge cases. In this paper we review some of these developments and point to the novel challenges they pose for the simulators themselves.

Productivity from day one on supercomputers that leverage new technologies requires significant preparation. An institution that procures a novel system architecture often lacks sufficient institutional knowledge and skills to prepare for it. Thus, the "Center of Excellence" (CoE) concept has emerged to prepare for systems such as Summit and Sierra, currently the top two systems in the Top 500. This paper documents CoE experiences that prepared a workload of diverse applications and math libraries for a heterogeneous system. We describe our approach to this preparation, including our management and execution strategies, and detail our experiences with and reasons for using different programming approaches. Our early science and performance results show that the project enabled significant early seismic science with up to a l4X throughput increase over Cori. In addition to our successes, we discuss our challenges and failures so others may benefit from our experience.

The capture of data about the events executed by a discrete event simulation can easily lead to very large trace data files. While disk space is relatively inexpensive and mostly capable of storing these large trace files, the manipulation and analysis of these large trace files can prove difficult. Furthermore, some types of analysis must be performed in-core and they cannot be performed with the trace data exceeds the size of the physical RAM where the analysis is performed. Because of these limits, it is often necessary to strictly limit the simulation run time to satisfy the analysis time memory limits. Experience with the DESMetrics tool suite (a collection of tools to analyze event trace files), demonstrates that our in-memory analysis tools are limited to trace files on the order of 10GB (on a machine with 24GB of RAM). Furthermore, even when it is possible to analyze large trace files, the run time costs of performing this analysis can take several days to complete. While high performance analysis of traces data is not strictly necessary, the results should be available within some reasonably bounded time frame. This paper explores techniques to overcome the limits of analyzing very large event trace files. While explorations for out-out-core analysis have been examined as part of this work, the run time costs for out-of-core processing can increase processing time 10-fold. As a result, the work reported here will focus on an approach to capture and analyze small samples from the event trace file. The work reported in this paper will examine how closely the analysis from sampling matches the analysis from a full trace file. Two techniques for comparison are presented. First a visual comparison of analysis results between the full trace and a trace sample are presented. Second, numerical quantification of the different analysis results (between the full trace and trace sample) will be reported using the Wasserstein, Directed Hausdorff, and Kolmogorov-Smirnov distance metrics. Finally, the ability to process trace samples from a very large trace file of 80GB is demonstrated.

Discrete event simulations (DES) are central to exploration of "what-if" scenarios in many domains including networks, storage devices, and chip design. Accurate simulation of dynamically varying behavior of large components in these domains requires the DES engines to be scalable and adaptive in order to complete simulations in a reasonable time. This paper takes a step towards development of such a simulation engine by redesigning ROSS, a parallel DES engine in MPI, in Charm++, a parallel programming framework based on the concept of message-driven migratable objects managed by an adaptive runtime system. In this paper, we first show that the programming model of Charm++ is highly suitable for implementing a PDES engine such as ROSS. Next, the design and implementation of the Charm++ version of ROSS is described and its benefits are discussed. Finally, we demonstrate the performance benefits of the Charm++ version of ROSS over its MPI counterpart on IBM's Blue Gene/Q supercomputers. We obtain up to 40% higher event rate for the PHOLD benchmark on two million processes, and improve the strong-scaling of the dragonfly network model to 524, 288 processes with up to 5x speed up at lower process counts.

The fully automatic generation of code that establishes the reversibility of arbitrary C/C++ code has been a target of research and engineering for more than a decade as reverse computation has become a central notion in large scale parallel discrete event simulation (PDES). The simulation models that are implemented for PDES are of increasing complexity and size and require various language features to support abstraction, encapsulation, and composition when building a simulation model. In this paper we focus on parallel simulation models that are written in C++ and present an approach and an evaluation for a fully automatically generated reversible code for a kinetic Monte-Carlo application implemented in C++. Although a significant runtime overhead is introduced with our technique, the assurance that the reverse code is generated automatically and correctly, is an enormous win that allows simulation model developers to write forward event code using the entire C++ language, and have that code automatically transformed into reversible code to enable parallel execution with the Rensselaer's Optimistic Simulation System (ROSS).

Time Warp is an optimistic synchronization protocol for parallel discrete event simulation that coordinates the available parallelism through its rollback and antimessage mechanisms. In this paper we present the results of a strong scaling study of the ROSS simulator running Time Warp with reverse computation and executing the well-known PHOLD benchmark on Lawrence Livermore National Laboratory's Sequoia Blue Gene/Q supercomputer. The benchmark has 251 million PHOLD logical processes and was executed in several configurations up to a peak of 7.86 million MPI tasks running on 1,966,080 cores. At the largest scale it processed 33 trillion events in 65 seconds, yielding a sustained speed of 504 billion events/second using 120 racks of Sequoia. This is by far the highest event rate reported by any parallel discrete event simulation to date, whether running PHOLD or any other benchmark. Additionally, we believe it is likely to be the largest number of MPI tasks ever used in any computation of any kind to date.