Heterogeneous computing has become increasingly prevalent, driven by the huge computational demands of highperformance computing (HPC) and artificial intelligence (AI) workloads. Yet, evaluating these workloads on modern systems poses significant challenges. Existing tools for the instrumentation and analysis of CPU and GPU applications run separately, introducing timing differences and limiting the ability to capture their runtime interactions. To address this problem, we introduce XPU-Pin, a framework that enables simultaneous CPU and GPU binary instrumentation in a single execution. XPU-Pin integrates the CPU instrumentation framework Pin with GPU instrumentation frameworks such as NVBit (for NVIDIA GPUs) and GTPin (for Intel GPUs). This approach allows for the holistic analysis of heterogeneous workloads irrespective of the platform it executes. Leveraging the co-analysis capabilities of XPU-Pin, we present a novel methodology called XPU-Point to select simulatoragnostic representative samples of heterogeneous CPU-GPU workloads. The XPU-Point methodology employs tools developed with the XPU-Pin framework to: (a) capture the execution signature of heterogeneous programs and (b) evaluate the accuracy of selected samples on silicon, which were not possible before. We evaluate XPU-Point on diverse hardware platforms (x86 CPU with Intel/NVIDIA GPUs) using workloads such as SPECaccel 2023, SPEChpc 2021, GROMACS, AutoDock, and PyTorch. We demonstrate that XPU-Point predicts overall application performance with sampling errors typically less than $\mathbf{5} \boldsymbol{\%}$ as measured on native hardware.

High-performance, multi-core processors are the key to accelerating workloads in several application domains. To continue to scale performance at the limit of Moore’s Law and Dennard scaling, software and hardware designers have turned to dynamic solutions that adapt to the needs of applications in a transparent, automatic way. For example, modern hardware improves its performance and power efficiency by changing the hardware configuration, like the frequency and voltage of cores, according to a number of parameters, such as the technology used or the workload running at the time. With this level of dynamism, it is essential to simulate next-generation multi-core processors in a way that can both respond to system changes and accurately determine system performance metrics. Currently, no sampled simulation platform can achieve these goals of dynamic, fast, and accurate simulation of multi-threaded workloads. In this work, we propose a solution that allows for fast, accurate simulation in the presence of both hardware and software dynamism. To accomplish this goal, we present Pac-Sim, a novel sampled simulation methodology for fast, accurate sampled simulation that requires no upfront analysis of the workload. With our proposed methodology, it is now possible to simulate long-running dynamically scheduled multi-threaded programs with significant simulation speedups, even in the presence of dynamic hardware events. We evaluate Pac-Sim using the SPEC CPU2017, NPB, and PARSEC multi-threaded benchmarks with both static and dynamic thread scheduling. The experimental results show that Pac-Sim achieves a very low sampling error of 1.63% and 3.81% on average for statically and dynamically scheduled benchmarks, respectively. Pac-Sim also demonstrates significant simulation speedups as high as 523.5× (210.3× on average) for the training input set of SPEC CPU2017 running eight threads.

GPUs, due to their massively-parallel computing architectures, provide high performance for data-parallel applications. However, existing GPU simulators are too slow to enable architects to quickly evaluate their hardware designs and software analysis studies. Sampled simulation methodologies are one common way to speed up CPU simulation. However, GPUs apply drastically different execution models that challenge the sampled simulation methods designed for CPU simulations. Recent GPU sampled simulation methodologies do not fully take advantage of the GPU’s special architecture features, such as limited types of basic blocks or warps. Moreover, these methods depend on up-front analysis via profiling tools or functional simulation, making them difficult to use.

Modern optimizing compilers can exploit memory access and computation patterns to generate vectorized codes. However, such patterns in irregular programs such as SpMV are unknown until runtime due to the input dependence. Thus, either compiler’s static optimization or profile-guided optimization cannot represent the patterns for any common input, which leads to suboptimal vectorization. To address the above drawback, we propose DynVec, a framework to automatically exploit regular patterns buried deeply inside SpMV programs and apply corresponding optimizations for better vectorization. Due to the ability to represent instruction features and identify regular patterns with effective feature extraction and data re-arranging methods, DynVec can generate highly efficient vectorized codes by replacing gather/scatter/reduction operations with optimized operation groups. We evaluate DynVec on optimizing SpMV with representative sparse matrix datasets. The experiment results show that DynVec achieves significant speedup compared to the state-of-the-art SpMV implementations across a range of platforms.

Sparse Matrix-Vector Multiplication (SpMV) is an essential computation kernel for many data-analytic workloads running in both supercomputers and data centers. The intrinsic irregularity in SpMV is challenging to achieve high performance, especially when porting to new architectures. In this paper, we present our work on designing and implementing efficient SpMV algorithms on Sunway, a novel architecture with many unique features. To fully exploit the Sunway architecture, we have designed a dual-side multi-level partition mechanism on both sparse matrices and hardware resources to improve locality and parallelism. On one hand, we partition sparse matrices into blocks, tiles, and slices for different granularities. On the other hand, we partition cores in a Sunway processor into fleets, and further dedicate part of cores in a fleet as computation and I/O cores. Moreover, we have optimized the communication between partitions to further improve the performance. Our scheme is generally applicable to different SpMV formats and implementations. For evaluation, we have applied our techniques atop a popular SpMV format, CSR. Experimental results on 18 datasets show that our optimization yields up to 15.5x (12.3x on average) speedups.