Due to the memory hierarchy's large contribution to a microprocessor's total power, cache tuning is an ideal method for optimizing overall power consumption in embedded systems. Since most embedded systems are power and area constrained, the hardware and/or software that orchestrate cache tuning - the cache tuner - must not impose significant power and area overhead. Furthermore, as embedded systems increasingly trend towards multicore, inter-core data sharing, communication, and synchronization impose additional cache tuner design complexity, necessitating cross-core cache tuning coordination. In order to minimize cache tuner overhead, cache tuner design must consider these overheads and scalability. Whereas prior work proposes low-overhead cache tuners, scalability to multicore systems requires additional considerations. In this work, we present a low-overhead, scalable cache tuner and extensively evaluate various cache tuner design tradeoffs with respect to power and area for constrained multicore embedded systems. Based on our analysis, we formulate valuable insights and designer-assisted guidelines for modeling scalable and efficient cache tuners that best achieve optimization goals while maintaining power and area constraints.

Since multicore architectures are becoming more popular, recent multicore optimizations focus on energy consumption. In this paper, we focus on reducing the energy consumption in the data and instruction cache hierarchies in a multicore system. First, we present a level one data cache tuning heuristic for a heterogeneous multicore system, which classifies applications based on data sharing and cache behavior and uses this classification to guide cache tuning and reduce the number of cores that need to be tuned. Results reveal average energy savings of 25 percent for 2, 4, 8, and 16-core systems while searching only 1 percent of the design space. Next, we present a level one instruction cache tuning heuristic that reduces energy consumption in the instruction cache hierarchy by an average of 53 percent for 2, 4, 8, and 16-core systems, while searching less than 1 percent of the design space. Finally, we develop a custom, global hardware cache tuner for a dual-core system and show that our cache tuner has low area, energy, and power overheads.

Loop caches provide an effective method for decreasing memory hierarchy energy consumption by storing frequently executed code (critical regions) in a more energy efficient structure than the level one cache. However, due to code structure restrictions or costly design time pre-analysis efforts, previous loop cache designs are not suitable for all applications and system scenarios. We present an adaptive loop cache that is amenable to a wider range of system scenarios, which can provide an additional 20% average instruction cache energy savings (with individual benchmark energy savings as high as 69%) compared to the next best loop cache, the preloaded loop cache.

Since multi-core architectures are becoming more popular, recent multi-core optimizations focus on energy consumption. We present a level one data cache tuning heuristic for a heterogeneous multi-core system, which classifies applications based on data sharing and cache behavior, and uses this classification to guide cache tuning and reduce the number of cores that need to be tuned. Results reveal average energy savings of 25% for 2-, 4-, 8-, and 16-core systems while searching only 1% of the design space.

Even though much previous work explores varying instruction cache optimization techniques individually, little work explores the combined effects of these techniques (i.e., do they complement or obviate each other). In this paper we explore the interaction of three optimizations: loop caching, cache tuning, and code compression. Results show that loop caching increases energy savings by as much as 26% compared to cache tuning alone and reduces decompression energy by as much as 73%.

Cache tuning reveals substantial energy savings for single-core architectures, but has yet to be explored for multi-core architectures. In this paper we explore level one (L1) data cache tuning in a heterogeneous dual-core system where each data cache can have a different configuration. We show that L1 data cache tuning in a dual-core system achieves 25% average energy savings, which is comparable to single-core data cache tuning. We present the dual-core tuning heuristic CPACT, which finds cache configurations within 1% of the optimal configuration while searching only 1% of the design space. Finally, we provide valuable insights on core-interactions and data coherence revealed when tuning the multithreaded SPLASH-2 benchmarks.

Loop caches provide an effective method for decreasing memory hierarchy energy consumption by storing frequently executed code in a more energy efficient structure than the level one cache. However, due to code structure restrictions and/or costly design time pre-analysis efforts, previous loop cache designs are not suitable for all applications and system scenarios. In this paper, we present an adaptive loop cache that is amenable to a wide range of system scenarios, providing an additional 20% average instruction memory hierarchy energy savings (with individual benchmark energy savings as high as 69%) compared to the best previous loop cache design.