As we enter the dark silicon era, architects should not envision designs in which every transistor remains turned on permanently but rather ones in which portions of the chip are judiciously turned on/off depending on the characteristics of a workload. At the same time, due to the increasing cost per transistor, architects should also consider new ways to re-purpose transistors to increase their architectural value. In this work, we consider the design of directory-based cache coherence in light of the dark silicon era and the need to re-purpose transistors. We point out that directories are not needed all of the time, and we argue that directories (and coherence) should be off unless it is actually needed for correctness. In our design, directories will be disabled and powered off for workloads with no sharing. Then only when parallel workloads need cache coherence will directories be enabled in proportion to the sharing that is present. At the same time, we exploit the structural similarities of directories and cache. If a directory is idle, then we reconfigure it to be used as extra capacity in the last-level cache. Since our novel approach can keep most directories off, we are free to select sparse overprovisioned directory designs that are reconfigurable to large amounts of cache that can significantly boost performance when the directory is idle. We call these combined features Reconfigured Dark Directories , since directories are usually dark (off) and can be reconfigured. Our results for Reconfigurable Dark Directories running SPEC 2006 applications show a performance benefit, on average, of 17% for an 8× overprovisioned fully mapped directory on a 64-tile system under low system concurrency (10% under heavy concurrency), or a 29% average speedup for a 2× overprovisioned directory on 256-tile system (10% under heavy concurrency) to systems with a conventional sparse directory design using the same overprovisioning factor.

Extracting high memory-level parallelism (MLP) is essential for speeding up single-threaded applications which are memory bound. At the same time, the projected amount of dark silicon (the fraction of the chip powered off) on a chip is growing. Hence, Asymmetric Multicore Processors (AMP) offer a unique opportunity to integrate many types of cores, each powered at different times, in order to optimize for different regions of execution. In this work, we quantify the potential for exploiting core customization to speedup programs during regions of high MLP. Based on a careful design space exploration, we discover that an AMP that includes a narrow and fast specialized core has the potential to efficiently exploit MLP. Using the results of our analysis, we design an AMP with both an MLP and ILP specialized core, and we propose a hardware-level, application steering mechanism called Symbiotic Core Execution (SCE). SCE detects MLP phases by monitoring the L2 miss rate of the application, and it uses that information to steer the application to the best core. Interestingly, we show that L2 miss rates are important for deciding when an MLP region begins and when it ends. As a program runs, its execution migrates to a core customized for MLP during regions of high MLP; when the region ends, it is re-scheduled on the core that fits the application characteristics. Compared to a monolithic core optimized for both modes of operation, our AMP design provides a harmonic mean performance improvement of 5.3% and 6.6% for SPEC2000 and SPEC2006, respectively, with a maximum speedup of 14.5%. For the same study, it achieves a 18.3% and 21.1% energy delay 2 reduction for SPEC2000 and SPEC2006, respectively. Our findings yield an important message for designing AMPs with specialized cores: core customization enables efficient exploitation of MLP, and application steering mechanisms for MLP are simple to implement and effective.