This paper presents Genie Cache that enables non-blocking miss-handling and replacement in a page-table-based DRAM cache (DC). Various DC designs have been proposed to meet the growing bandwidth demand of emerging memory-bound applications. The related literature can be categorized into hardware-based (HW-based) and page-table-based (PT-based) schemes based on their tag storage methods. HW-based designs store DC metadata (e.g., tags) in on-package DRAM for scalability but use extra bandwidth and energy for metadata access. PT-based schemes store DC tags in page table entries (PTEs), enabling virtual-to-cache address translations using the existing memory management units (MMUs) without the DC bandwidth overhead. However, their miss-handling and eviction mechanisms relying on operating systems (OS) incur nontrivial latency overhead. To minimize the OS intervention, Genie Cache implements non-blocking miss handling and replacement using a hardware unit called DRAM cache management unit (DCMU) and a novel pre-write back mechanism. In Genie Cache, DC misses detected by MMUs are forwarded to the DCMU, which handles the misses by allocating page frames and updating PTEs without calling OS routines. When the PT-based DRAM cache runs low on free pages, an eviction routine is called to flush TLBs and evict a batch of cached pages to avoid frequent TLB shootdowns. Since writing back many dirty pages in a blocking manner causes substantial application stall cycles, Genie Cache proactively writes dirty pages back to off-package memory, allowing the eviction routine to simply evict cleaned pages. Experimental results show that Genie Cache achieves 51.3% speedup over the state-of-the-art PT-based design via non-blocking miss handling and replacement.

This paper introduces a DRAM cache architecture that provides near-ideal access time and non-blocking miss handling. Previous DRAM cache (DC) designs are classified into two categories, HW-based and OS-managed schemes. Hardware-based designs implement non-blocking caches that can handle multiple DC misses using MSHRs, but they have drawbacks in metadata management since storing tags in on-package DRAM significantly increases the effective cycle time of DC accesses. In contrast, OS-managed schemes utilize PTEs for storing tags and caching them in TLBs, which can achieve ideal DC access time. However, they implement blocking caches that stall application threads on misses until cache fills are completed. To overcome the limitations of both HW-based and OS-managed schemes, this paper introduces a DRAM cache architecture named Non-blocking OS-managed DRAM cache (NOMAD). Unlike conventional caches that guarantee the presence of data on tag hits, NOMAD decouples tag and data management to enable non-blocking miss handling in an OS-managed DRAM cache. The front-end OS routines of NOMAD manage DC tags using PTEs and TLBs, and its back-end hardware handles data management in the DRAM cache. On a DC miss, the OS updates a tag, offloads a cache-fill command to the back-end, and immediately resumes an application thread without waiting for the cache fill to complete. Instead, the back-end hardware handles the cache fill without blocking the application thread. By decoupling tag and data management in NOMAD, a tag hit does not necessarily guarantee the presence of data in the DRAM cache. The back-end traces which DC lines are still in transfers and checks if the demanded part of a cache line has been transferred yet for every DC access. Notably, this back-end procedure does not require an OS intervention, thereby implementing a non-blocking DRAM cache. Experiment results show that NOMAD reduces application stall cycles by 76.1% and improves IPC by 16.7% over a state-of-the-art OS-managed scheme.