Generative artificial intelligence is widely used for various tasks such as language translation and art creation. Generative inference employs the key and value tensors to encode relational information among the tokens in the input and output sequences. Most of the existing generative inference frameworks use KV caching (KVC), which caches the key and value tensors (i.e., the KV cache) to avoid recomputing the key and value tensors for each of the processed tokens. Despite the widespread use of KVC, in-depth characterization of KVC with tensor offloading, which enables generative inference with a single GPU, remains yet to be explored.

Recent CPUs have begun to adopt non-inclusive cache hierarchies for more effective cache utilization. Non-inclusive cache hierarchies have an additional advantage in that they eliminate the vulnerability to cache-based side-channel attacks. In addition, precise timers are often disabled or added with noise to defeat timer-based side-channel attacks. With the combination of such countermeasures, existing cache- and directory-based side-channel attacks can robustly be defeated on commodity systems.In this work, we discover the vulnerability caused by the undocumented interactions between the coherence directories and Intel TSX transactions in latest Intel CPUs with non-inclusive cache hierarchies. Guided by the observation, we propose a high-precision and timer-free directory attack called DPrime+DAbort in non-inclusive cache hierarchies using Intel TSX, which nullifies the aforementioned countermeasures. Our quantitative evaluation conducted on real systems equipped with latest Intel CPUs in three different generations demonstrates the practicality of the DPrime+DAbort attack in that it can be used to attack cryptographic and genomesequencing applications. We also discuss potential countermeasures and evaluate the feasibility of an Intel TSX-based countermeasure against the DPrime+DAbort attack.