As the frequency gap between main memory and modern microprocessor grows, the implementation and efficiency of on-chip caches become more important. The growing latency to memory is motivating new research into load instruction behavior and selective data caching. This work investigates the classification of load instruction behavior. A new load classification method is proposed that classifies loads into those vital to performance and those not vital to performance. A limit study is presented to characterize different types of non-vital loads and to quantify the percentage of loads that are non-vital. Finally, a realistic implementation of the non-vital load classification method is presented and a new cache structure called the Vital Cache is proposed to take advantage of non-vital loads. The Vital Cache caches data for vital loads only, deferring non-vital loads to slower caches. Results: The limit study shows 75% of all loads are non-vital with only 35% of the accessed data space being vital for caching. The Vital Cache improves the efficiency of the cache hierarchy and the hit rate for vital loads. The Vital Cache increases performance by 17%.

A low-latency and high-bandwidth level-1 data cache is crucial for achieving high performance in future superscalar microprocessors. The parallel cachelets (PC) proposed in this paper provide bandwidth close to that of a multi-ported cache with implementation efficiency close to that of a multi-banked cache. In the PC scheme, the traditional level-1 data cache is replaced by a set of parallel single-ported independent caches, or cachelets. Similar to the interleaved multi-banked design, the cachelets can be concurrently accessed to provide bandwidth. However, instead of mapping data elements to the banks in an interleaved fashion based on address bits, they are dynamically assigned to cachelets based on the pattern of concurrent accesses, thus many bank conflicts can be eliminated. Furthermore, the PC scheme exhibits the attribute of implicit set associativity that allows it to outperform a direct-mapped multi-ported cache for some benchmarks. The PC scheme outperforms the multi-banked scheme by an average of 6% (IPC) across a set of SPEC95 benchmarks, and comes very close to matching the performance of the multi ported scheme. When cache access latency is taken into account, the PC scheme even outperforms the multi ported scheme by 6.4%.