As GPUs are increasingly used for high-performance and data-intensive applications, optimizing workloads and memory behavior remains a significant challenge due to complex memory hierarchies, high parallelism, and limited runtime observability. Binary instrumentation has emerged as a key technique for analyzing and profiling GPU workloads under these constraints. This paper introduces new technology within GTPin - the binary instrumentation framework for Intel GPUs - adding support for high-level instrumentation alongside the previous low-level ISA insertion approach. The new mechanism enables developers to write analysis routines in a high-level programming language (OpenCL C) and dynamically invoke them at any instruction during kernel execution. This capability enables the implementation of sophisticated profiling tools for conducting advanced on-the-fly analyses and studies on code running on Intel GPUs, such as cache modeling, memory pattern inspection, race detection, and more, while maintaining reasonably low overhead on real hardware.In this work, we describe the design of the technology, present usage examples, provide a unique overhead breakdown analysis quantifying the performance impact of both high-level and lowlevel instrumentation, and compare it with other existing GPU instrumentation frameworks. This comparison shows that even with high-level instrumentation, on average GTPin has lower overhead. These capabilities bring GTPin to parity with leading GPU binary instrumentation frameworks, thus allowing the community to investigate, research, and analyze all three major GPU vendors in a uniform way.

In the process of developing software and hardware, profiling workloads is critical. Binary Instrumentation Technology plays a key role in this task for both x86 architecture and Intel Graphics Processing Units. The GTPin framework is the first tool that allows the profiling of graphics and compute kernels running on Intel GPUs. However, GTPin capabilities are less flexible than x86 profiling tools. In this paper, we introduce the concept of “gLIT” – Long Instruction Trace for Intel GPUs. Generated on real hardware, gLIT can be replayed on a simulator or an emulator running on the CPU device, and thus, can be easily profiled and analyzed “on the fly” with analysis tools of any complexity. Since the graphics devices are extremely parallel, the gLIT trace is, by definition, a multi-threaded trace, reflecting a kernel concurrently running hundreds of hardware threads. The ability to thoroughly profile and analyze workloads is critical for improving hardware and software readiness and creates new possibilities for academic research on Intel graphics devices.

Functional and performance profiling of workloads is critical in developing software and hardware. Binary Instrumentation Technology has played a key role in this task for many years in the world of x86 architecture. However, such capabilities have not been available until recently for graphics devices, especially in the Intel Graphics Processing Unit world. The GTPin framework is the only tool that supports profiling graphics and GP-GPU kernels running on extremely parallel Intel GPU devices. GTPin supports a wide range of capabilities for software and hardware developers. With GTPin, you can profile real-world graphics and compute applications at a level of performance close to real hardware. Such an ability is critical in accelerating hardware and software readiness.

The problem of profiling a compute kernel running on the CPU is mostly solved with the help of technologies that explore a code behavior in detail. But with a last decade trend when computation spreads to other devices, more power and performance efficient, we face a high need for fine-grain code profiling on such devices. Traditional methods are not always sufficient: program counter sampling requires special hardware support, and performance simulation works slowly. In this paper, we introduce a novel method for instruction hotspots detection based on the binary instrumentation approach and demonstrate it on Intel® Graphics. This method relies on three key principles: measurement with instruction block granularity, conscious placement of probes, and combination of static and runtime information. We demonstrate its ability to highlight the hottest instructions and lines of code, and its relatively low runtime overhead comparable to a native run. The method is applicable to GPUs and accelerators with in-order architecture, and could be used in a rapidly growing segment of accelerator solutions for computer vision and artificial intelligence.