•Recap Accelerator-in-Memory (AiM) & AiMX •System Extensions of AiMX Card for Datacenter •AiM & AiMX for On-device AI •Design Choices for Future AiM/AiMX •Conclusion

Hybrid cache architectures (HCAs) containing both SRAM and non-volatile memory (NVM) have been proposed to overcome the disadvantages of NVM-based cache architecture. Most previous works have concentrated on managing write-intensive blocks by storing these blocks to SRAM to reduce the number of the write operations to NVM. However, they have not focused on reducing linefill operations which also occupy a large portion of overall NVM write counts in chip-multiprocessor (CMP) environments. This paper proposes an NVM way allocation scheme, taking into account the NVM linefill counts as well as cache miss rate and the NVM write hit counts during victim selection. Three metrics are introduced to estimate the effectiveness of NVM way allocation: Miss counts change (ΔM), write counts change (ΔW), and NVM write counts change (ΔNVMW). An algorithm to minimize the write counts of NVM based on these metrics is proposed as well. Our experimental results show that dynamic energy consumption is reduced by 37.5 percent on average.

Existing hybrid cache architecture (HCA) studies have concentrated on managing write-intensive blocks to relieve the write pressure on Non-volatile memory (NVM). However, they have overlooked that the capacity of NVM is also one of important factors affecting the number of write accesses to NVM. This research investigates the relationship between the size of NVM and the write counts of NVM to efficiently design a new algorithm to reduce the write counts of NVM in HCA. Based on the analysis, it is revealed that adjusting the capacity of NVM can decrease write requests to NVM if the miss rate does not grow. This discovery leads to the necessity of NVM capacity management policy such that the size of NVM is dynamically adjusted according to the demand of applications. To implement the policy, we propose a dynamic way adjusting algorithm which dynamically monitors the optimal number of NVM ways using the stack property and disabling the unnecessary NVM ways. The experiments are conducted for two HCAs with STT-RAM and PCM. The results show that proposed management policy reduces the write accesses to STT-RAM by 77.6% and PCM by 79.6%.

This research aims to design a new phase-change RAM (PRAM)-based main memory structure, supporting the advantages of PRAM while providing performance similar to that of conventional DRAM main memory. To replace conventional DRAMs with non-volatile PRAMs as the main memory components, comparable memory access latency, overall cost, power dissipation, and memory cell endurance should be supported. For these goals, we propose a new main memory system consisting of a DRAM converter and an array of single-level cell (SLC)/multi-level cell (MLC) PRAMs. The DRAM converter consists of an aggressive fetching superblock buffer to assure better use of spatial locality and a selective filtering buffer for better use of temporal locality. The array of the SLC/MLC hybrid PRAM structure includes a combination of SLC and MLC PRAMs to enhance the lifetime of the MLC PRAM main memory and hide asymmetric read/write access latency. The proposed structure is evaluated by a trace-driven simulator using SPEC CPU 2006 and SPLASH-2 traces. Experimental results show that the proposed DRAM converter can reduce the miss rate by ∼37% and write count by ∼55% in comparison with the uniform buffer case. Also, the SLC/MLC PRAM with DRAM converter shows performance in terms of access latency and power consumption close to that of the conventional memory architecture. Thus, our proposed memory architecture can be used to replace the current DRAM-based main memory system.

This paper proposes a power-aware cache block allocation algorithm for the way-selective set-associative cache on embedded systems to reduce energy consumption without additional delay or performance degradation. For this goal, way selection logic and specialized replacement policy are designed to enable only one way of set-associative cache as in the direct-mapped cache. Overall cache access time becomes almost the same as that of a conventional set associative cache with accessing additional way selection logic. Because data array can be accessed without waiting for tag comparison, multiplexer delay can be removed totally. The simulation result shows that the proposed architecture can reduce a per access power consumption by 59% over conventional set-associative caches with average 0.06% of negligible performance loss.

We present a selective filter-bank translation lookaside buffer (TLB) system with low power consumption for embedded processors. The proposed TLB is constructed as multiple banks with a small two-bank buffer, called as a filter-bank buffer, located above its associated bank. Either a filter-bank buffer or a main bank TLB can be selectively accessed based on two bits in the filter-bank buffer. Energy savings are achieved by reducing the number of entries accessed at a time, by using filtering and bank mechanism. The overhead of the proposed TLB turns out to be negligible compared with other hierarchical structures. Simulation results show that the Energy*Delay product can be reduced by about 88% compared with a fully associative TLB, 75% with respect to a filter-TLB, and 51% relative to a banked-filter TLB.

This research is to design a new two-level TLB (translation look-aside buffer) architecture that integrates a 2-way banked filter TLB with a 2-way banked main TLB. One of the main objectives is to reduce power consumption in embedded processors by distributing the accesses to the TLB entries across several banks in a balanced manner. Thus, an advanced filtering technique is devised to reduce power dissipation by adopting a sub-bank structure at the filter TLB. And also a bank-associative structure is applied to each level of the TLB hierarchy. Simulation result shows that the miss ratio and Energy*Delay product can be improved by 59.26% and 24.9%, respectively, compared with a micro TLB with 4-32 entries, and 40.81% and 12.18%, compared with a micro TLB with 16-32 entries.