This paper introduces Block Data Representations (BDR), a framework for exploring and evaluating a wide spectrum of narrow-precision formats for deep learning. It enables comparison of popular quantization standards, and through BDR, new formats based on shared microexponents (MX) are identified, which outperform other state-of-the-art quantization approaches, including narrow-precision floating-point and block floating-point. MX utilizes multiple levels of quantization scaling with ultra-fine scaling factors based on shared microexponents in the hardware. The effectiveness of MX is demonstrated on real-world models including large-scale generative pretraining and inferencing, and production-scale recommendation systems.

Many applications running on high-performance computing systems share limited resources such as the last-level cache, often resulting in lower performance. Intel recently introduced a new control mechanism, called cache allocation technology (CAT), which controls the cache size used by each application. To intelligently utilize this technology for automated management, it is essential to accurately identify application performance behavior for different cache allocation scenarios. In this work, we show a novel approach which automatically builds a prediction model for application performance changes with CAT. We profile the workload characteristics based on Intel Top-down Microarchitecture Analysis Method (TMAM), and train the model using machine learning. The model predicts instructions per cycle (IPC) across available cache sizes allocated for the applications. We also design a dynamic cache management technique which utilizes the prediction model and intelligently partitions the cache resource to improve application throughput. We implemented and evaluated the proposed framework in Intel PMU profiling tool running on Xeon Platinum 8186 Skylake processor. In our evaluation, we show that the proposed model accurately predicts the IPC changes of applications with 4.7% error on average for different cache allocation scenarios. Our predictive online cache managements achieves improvements on application performance of up to 25% as compared to a prediction-agnostic policy.

Trace-driven simulation of chip multiprocessor (CMP) systems offers many advantages over execution-driven simulation, such as reducing simulation time and complexity, allowing portability, and scalability. However, trace-based simulation approaches have difficulty capturing and accurately replaying multithreaded traces due to the inherent nondeterminism in the execution of multithreaded programs. In this work, we present SynchroTrace, a scalable, flexible, and accurate trace-based multithreaded simulation methodology. By recording synchronization events relevant to modern threading libraries (e.g., Pthreads and OpenMP) and dependencies in the traces, independent of the host architecture, the methodology is able to accurately model the nondeterminism of multithreaded programs for different hardware platforms and threading paradigms. Through capturing high-level instruction categories, the SynchroTrace average CPI trace Replay timing model offers fast and accurate simulation of many-core in-order CMPs. We perform two case studies to validate the SynchroTrace simulation flow against the gem5 full-system simulator: (1) a constraint-based design space exploration with traditional CMP benchmarks and (2) a thread-scalability study with HPC-representative applications. The results from these case studies show that (1) our trace-based approach with trace filtering has a peak speedup of up to 18.7× over simulation in gem5 full-system with an average of 9.6× speedup, (2) SynchroTrace maintains the thread-scaling accuracy of gem5 and can efficiently scale up to 64 threads, and (3) SynchroTrace can trace in one platform and model any platform in early stages of design.

The vertical interlayer communication channel plays a critical role in defining the performance of a 3-D networkon-chip (NoC). In this paper, an arbitration-free design for the shared vertical channels is proposed. The proposed vertical arbitration-free 3-D NoC is compared with other 3-D NoC architectures using traditional synthetic traffic patterns and Rentian traffic emulating applications for chip multiprocessors. The results of the analysis show comparable performance in throughput, energy, and latency compared to a symmetric 3-D NoC with savings up to ≈ 20% in area. The proposed NoC is superior in performance to a 3-D NoC utilizing vertical arbitration with a similar area footprint.

The emergence of Internet of Things increases the complexity and the heterogeneity of computing platforms. Migrating workload between various platforms is one way to improve both energy efficiency and performance. Effective migration decisions require accurate estimates of its costs and benefits. To date, these estimates were done by either instrumenting the source code/binaries, thus causing high overhead, or by using power estimates from hardware performance counters, which work well for individual machines, but until now have not been accurate for predicting across different architectures. In this paper, we propose P4, a new Phase-based Power and Performance Prediction framework which identifies cross-platform application power and performance at runtime for heterogeneous computing systems. P4analyzes and detects machine-independent application phases by characterizing computing platforms offline with a set of benchmarks, and then builds neural network-based models to automatically identify and generalize the complex cross-platform relationships for each benchmark phase. It then leverages these models along with performance counter measurements collected at runtime to estimate performance and power consumption if it were running on a completely different computing platform, including a different CPU architecture, without ever having to run it on there. We evaluate the proposed framework on four commercial heterogeneous platforms, ranging from X86 servers to mobile ARM-based architecture, with 129 industry-standard benchmarks. Our experimental results show that P4can predict the power and performance changes with only 6.8% and 5.6% error, respectively, even for completely different architectures from the ones applications ran on.

The emergence of Network-on-Chip (NoC) as a communication paradigm for Multi-Processor System-on-Chips (MPSoCs) significantly exacerbates the need to provide a methodology that optimizes the energy consumption of the overall system. This is especially important when factoring in current Network-on-Chip advances which have multiple communication media such as on-chip wireless or nano-photonics links, hybrid with traditional wired links. All of these media have different energy profiles, and if not taken into consideration the system will incur a higher power consumption throughout the runtime of the application. In this work, the case for EDP (energy-delay product) optimization between different levels of a multi-level Network-on-Chip is presented. Using a dynamic, energy aware algorithm, the EDP improvement is compared to a multi-level Network-on-Chip using a statically optimized routing. The proposed routing algorithm handles the different types of energy-delay profiles of multiple links. The end product is a methodology that lowers the overall energy consumption by optimizing the energy profile of the Network-on-Chip while also minimizing the network delay.

Trace-driven simulation of chip multiprocessor (CMP) systems offers many advantages over execution-driven simulation, such as reducing simulation time and complexity, and allowing portability, and scalability. However, trace-based simulation approaches have encountered difficulty capturing and accurately replaying multi-threaded traces due to the inherent non-determinism in the execution of multi-threaded programs. In this work, we present SynchroTrace, a scalable, flexible, and accurate trace-based multi-threaded simulation methodology. The methodology captures synchronization- and dependency-aware, architectureagnostic, multi-threaded traces and uses a replay mechanism that plays back these traces correctly. By recording synchronization events and dependencies in the traces, independent of the host architecture, the methodology is able to accurately model the non-determinism of multi-threaded programs for different platforms. We validate the SynchroTrace simulation flow by successfully achieving the equivalent results of a constraint-based design space exploration with the Gem5 Full-System simulator. The results from simulating benchmarks from PARSEC 2.1 and Splash-2 show that our trace-based approach with trace filtering has a peak speedup of up to 18.4ξ over simulation in Gem5 Full-System with an average of about 7.5ξ speedup. We are also able to compress traces up to 74% of their original size with almost no impact on accuracy.

Hierarchical and multi-network networks-on-chip (NoCs) have been proposed in the literature to improve the energy- and performance-efficient scalability of the traditional flat-mesh NoC architecture. Theoretically, based on a small-world network-based analysis, traditional hierarchical NoCs are expected to provide good scalability. However, the traditional theoretical analysis (e.g. for small-worldness) does not take into account the congestion phenomenon experienced in such networks. Counterintuitively, as shown in this work, breaking the hierarchy in traditional hierarchical NoCs and utilizing the proposed locality-aware network utilization (NU) balancing technique performs better. This improvement in performance is observed through experimental analysis, which is contrasted with the theoretical analysis that does not account for congestion. In addition to the novelties for hierarchical networks, the application of the proposed locality-aware NU balancing scheme is extended to multi-network NoC topologies (with already separated networks). Results of the analysis show the superiority of applying the locality-aware NU balancing technique for a throughput and energy-efficient scaling of the multi-network NoC architectures, much like those of the hierarchical NoCs. For instance, for a NoC with 1024 nodes, the proposed NU balancing technique provides up to 95% higher throughput efficiency and consumes up to 29% less energy per flit compared to the best NoC topology without the NU balancing technique. The analysis also helps to render the choice of a NoC topology for traffic patterns varying in locality and nonlocality on exascale computing CMPs.

A novel methodology is proposed for thread mapping on a chip-multiprocessor (CMP) system with a network-on-chip (NoC). This novel mapping leverages multi-threaded traces produced by a binary instrumentation tool, which classifies the communication and computation events for each thread of a multi-threaded program application. Processing these binary instrumentation traces after profiling, a static thread mapping is computed to improve the NoC performance.

Hybrid wireless NoCs are proposed to improve the communication throughput and energy compared to flat mesh-based NoCs. However, the multi-faceted design of a hybrid wireless NoC presents a conundrum amongst the different design paradigms. In this work, a methodology with minimal design complexity is presented for the design of hybrid wireless NoCs encompassing the different paradigms. To illustrate one use of the methodology, path loss models for the wireless channel implemented with various operating frequencies and substrate types are presented which are to be used in the design space exploration of the NoC.

The feasibility of the dynamic reconfigurability of the network layer of a hybrid wireless network-on-chip (NoC) that uses on-chip antennas for the wireless network layer and metal interconnects for the wired network layer is studied. The reconfigurability of the NoC is analyzed using a circuit co-simulation technique with a 3D finite element method (FEM) based full-wave electro-magnetic analysis of the antennas. The die and the circuits are modeled according to a typical complementary metal oxide semiconductor (CMOS) technology. It is shown that, it is possible to have 1) at least two different frequency domains for the signal sources and 2) the dynamic switching of the signal sinks between the two frequency domains, with minimal design and area overhead. When implemented, the proposed reconfigurable hybrid network architecture can reduce the latency and increase the network throughput.

The electromagnetic interaction of on-chip antennas and metal interconnects modeled in a 250 nm complementary metal-oxide semiconductor (CMOS) technology is investigated. A finite element method (FEM) based 3-D full-wave solver is used to perform the electromagnetic field analysis. It is shown that there can be significant signal coupling between the on-chip transmitting antenna and the metal interconnects on a die (-12.09 dB for a 1.6 mm long, 2 um wide interconnect at a distance of 1 um from the antenna). Design considerations for metal interconnects in the presence of on-chip antennas are presented in order to limit the undesirable electromagnetic coupling.