As applications demand more bandwidth, the the “memory wall” problem becomes increasingly severe. Therefore, the processing-in-memory (PIM) architecture has attracted significant research interest due to its ability to execute instructions offloaded by the processor. Existing works on PIM architectures are classified into two categories: regional offloading, where all instructions within a programmer-specified code region are offloaded, and selective offloading, where only instructions of interest are offloaded via hardware support. However, PIM architectures pose the amplified in-PIM traffic overhead challenge that endangers the performance of PIM and degrades the performance of the entire system. To address the challenge, we propose a PIM architecture, called fast PIM (fPIM), which integrates the PIM cache within each Channel Controller to optimize the data flow within the PIM. This design cooperates with theProcessing Unit Load-balancerandBehavior-based Offloaderto achieve high execution efficiency. To evaluatefPIM, we perform extensive experiments, and the results show thatfPIMreduces the workload finish time by up to$88.6\%$,$87.5\%$, and$79.6\%$(with an average of$68.7\%$,$66.2\%$, and$59.8\%$), compared to three state-of-the-art PIM designs, PEI, Fafnir, and SpaceA, respectively.

With the increasing problem complexity, more irregular applications are deployed on high-performance clusters due to the parallel working paradigm, and yield irregular memory access behaviors across nodes. However, the irregularity of memory access behaviors is not comprehensively studied, which results in low utilization of the integrated hybrid memory system compositing of stacked DRAM and off-chip DRAM. To address this problem, we devise a novel method calledSimilarity-Managed Hybrid Memory System(SM-HMS) to improve the hybrid memory system performance by leveraging the memory access similarity among nodes in a cluster. WithinSM-HMS, two techniques are proposed,Memory Access Similarity MeasuringandSimilarity-based Memory Access Behavior Sharing. To quantify the memory access similarity, memory access behaviors of each node are vectorized, and the distance between two vectors is used as the memory access similarity. The calculated memory access similarity is used to share memory access behaviors precisely across nodes. With the shared memory access behaviors,SM-HMSdivides the stacked DRAM into two sections, thesliding window sectionand theoutlier section. The shared memory access behaviors guide the replacement of thesliding window sectionwhile theoutlier sectionis managed in the LRU manner. Our evaluation results with a set of irregular applications on various clusters consisting of up to 256 nodes have shown thatSM-HMSoutperforms the state-of-the-art approaches,Cameo,Chameleon, andHyrbid2, on job finish time reduction by up to$58.6\%$,$56.7\%$, and$31.3\%$, with$46.1\%$,$41.6\%$, and$19.3\%$on average, respectively.SM-HMScan also achieve up to$98.6\%$($91.9\%$on average) of the ideal hybrid memory system performance.

Clusters have been a prevalent and successful computing framework for processing large amount of data due to their distributed and parallelized working paradigm. A task submitted to a cluster is typically divided into a number of subtasks which are designated to different work nodes running the same code but dealing with different equal portion of the dataset to be processed. Due to the existence of heterogeneity, it could easily result in stragglers unfairly slowing down the entire processing, because work nodes finish their subtasks at different rates. In this study, we aim to speed up straggling work nodes to quicken the overall processing by leveraging exhibited performance variation. More specifically, we propose StragglerHelper which conveys the memory access characteristics experienced by the forerunner to the stragglers such that stragglers can be sped up due to the accurately informed memory prefetching. A Progress Monitor is deployed to supervise the respective progresses of the work nodes and inform the memory access patterns of forerunner to straggling nodes. Our evaluation results with the SPEC MPI 2007 and BigDataBench on a cluster of 64 work nodes have shown that StragglerHelper is able to improve the execution time of stragglers by up to 99.5% with an average of 61.4%, contributing to an overall improvement of the entire cohort of the cluster by up to 46.7% with an average of 9.9% compared to the baseline cluster.

Photo service providers are facing critical challenges of dealing with the huge amount of photo storage, typically in a magnitude of billions of photos, while ensuring national-wide or world-wide satisfactory user experiences. Distributed photo caching architecture is widely deployed to meet high performance expectations, where efficient still mysterious caching policies play essential roles. In this work, we present a comprehensive study on internet-scale photo caching algorithms in the case of QQPhoto from Tencent Inc., the largest social network service company in China. We unveil that even advanced cache algorithms can only perform at a similar level as simple baseline algorithms and there still exists a large performance gap between these cache algorithms and the theoretically optimal algorithm due to the complicated access behaviors in such a large multi-tenant environment. We then expound the reasons behind this phenomenon via extensively investigating the characteristics of QQPhoto workloads. Finally, in order to realistically further improve QQPhoto cache efficiency, we propose to incorporate a prefetcher in the cache stack based on the observed immediacy feature that is unique to the QQPhoto workload. The prefetcher proactively prefetches selected photos into cache before they are requested for the first time to eliminate compulsory misses and promote hit ratios. Our extensive evaluation results show that with appropriate prefetching we improve the cache hit ratio by up to 7.4 percent, while reducing the average access latency by 6.9 percent at a marginal cost of 4.14 percent backend network traffic compared to the original system that performs no prefetching.

Photo service providers are facing critical challenges of dealing with the huge amount of photo storage, typically in a magnitude of billions of photos, while ensuring national-wide or world-wide satisfactory user experiences. Distributed photo caching architecture is widely deployed to meet high performance expectations, where efficient still mysterious caching policies play essential roles. In this work, we present a comprehensive study on internet-scale photo caching algorithms in the case of QQPhoto from Tencent Inc., the largest social network service company in China. We unveil that even advanced cache algorithms can only perform at a similar level as simple baseline algorithms and there still exists a large performance gap between these cache algorithms and the theoretically optimal algorithm due to the complicated access behaviors in such a large multi-tenant environment. We then expound the behind reasons for that phenomenon via extensively investigating the characteristics of QQPhoto workloads. Finally, in order to realistically further improve QQPhoto cache efficiency, we propose to incorporate a prefetcher in the cache stack based on the observed immediacy feature that is unique to the QQPhoto workload. Evaluation results show that with appropriate prefetching we improve the cache hit ratio by up to 7.4%, while reducing the average access latency by 6.9% at a marginal cost of 4.14% backend network traffic compared to the original system that performs no prefetching.

DRAM memory suffers from increasingly aggravating refresh penalty, which causes significant performance degradation and power consumption. As memory capacity increases, refresh penalty has become increasingly worse as more rows have to be refreshed. In this work, we propose an effective refresh approach called Compression-Aware Refresh (CAR) to efficiently mitigate refresh overheads. We apply a data compression technique to store data in a compressed format so that the resultant sparse banks need only be partially refreshed. Because of compression, data blocks which are originally distributed across all the constituent chips of a rank only need to be stored in a subset of those chips, leaving banks in the remaining chips not fully occupied. As a result, the memory controller can safely skip refreshing memory rows which contain no useful data without compromising data integrity. Such a compression-aware refresh scheme significantly reduces the refresh and thus improves overall memory performance and energy efficiency. To further take advantage of data compression, we adopt a rank subsetting technique to enable accesses to only those occupied chips for memory requests accessing compressed data blocks. Evaluations using benchmarks from SPEC CPU 2006 and the PARSEC 3.0 on recent DDR4 memory systems have shown that CAR achieves up to 1.66× performance improvement (11.7 percent on average) and up to 45.9 percent energy reduction (27.3 percent on average), and reduce memory traffic by up to 99.9 percent for zero cache lines intensive workloads with an average of 66.3 percent across all benchmarks.

DRAM memory performs periodic refreshes to prevent data loss due to charge leakage, while memory refreshes cause performance degradation and energy consumption, referred to as refresh overheads. In this paper, we propose Refresh-Oriented Prefetching (ROP) to alleviate memory refresh overheads. Before a refresh starts, ROP prefetches cache lines from the tobe-refreshed rank into an added SRAM buffer. In doing so, when a rank is undergoing refresh, memory requests can still be serviced rather than being blocked. At the core of ROP is a probabilistic prefetch model determining which cache lines are prefetched for a refresh based on the access patterns appearing in an observational window ahead of the refresh. A Pattern Profiler collects statistics about memory traffic occurring before and after the starting time of each refresh operation in a period of training time and it outputs two conditional probabilities which are used to control subsequent prefetch decisions. A Prefetcher maintains a prediction table which helps to ascertain access patterns appearing around refresh operations. The prediction table is updated every time an access occurs to the to-be-nextrefreshed ran during the observational window and is consulted to decide which cache lines are prefetched. Extensive evaluation results with benchmarks from SPEC CPU2006 on a DDR4 memory have demonstrated that with ROP memory performance can be improved by up to 9.2% (3.3% on average) for singlecore simulations, while reducing the overall memory energy by up to 6.7% (3.6% on average), relative to an auto-refresh baseline memory. Moreover, it increases the Weighted Speedup by up to 2.22X (1.32X on average) for 4-core multiprogram simulations, while reducing energy by up to 48.8% (24.4% on average).

This paper introduces a new memory scheduling policy called LAMS, which is inspired by a recently proposed memory architecture and targets for future high capacity memory systems. As memory capacity increases, the bit-lines connected to memory row buffers become much longer, dramatically lengthening memory access latency, due to increased parasitic capacitance. Recent study has proposed to partition long bit-lines into near and far (relative to the row buffer) segments via inserting isolation transistors such that access to near segment can be accomplished much faster, while access to far segment remains nearly the same. However, how to effectively leverage the new memory architecture still remains unexplored. We suggest to take advantage of this new memory architecture via performing latency-aware memory scheduling for pending requests to explore their performance potentials. In this scheduling policy, each memory request is classified to one of the following three categories, row-buffer hit, near-buffer, and far-buffer. Based on the classification, it issues requests in the order of row-buffer hit → near-buffer → far-buffer. In doing so, it avoids long-latency requests blocking short-latency memory requests, reducing total memory queuing time in the memory controller and improving overall memory performance. Our evaluation results on a simulated memory system show that comparing with the commonly used FR-FCFS scheduler, our LAMS improves performance and energy efficiency by up to 20.6% and 34%, respectively. Even comparing with the four competitive schedulers chosen from memory scheduling champion (MSC), LAMS still improves performance and energy efficiency by up to 6.1% and 23.4%, respectively.

The chunk fragmentation problem inherently associated with deduplication systems significantly slows down the restore performance, as it causes the restore process to assemble chunks which are distributed in a large number of containers as a result of storage indirection. Existing solutions attempting to address the fragmentation problem either sacrifice deduplication efficiency or require additional memory resources. In this work, we propose a new restore cache scheme, which accelerates the restore process using the same amount of cache space as that of the traditional LRU restore cache. We leverage the recipe knowledge to recognize the containers which will soon be accessed for restoring a backup version and classify those containers into bursty containers which are differentiated from other regular containers. Bursty and regular containers are then put in two separate caches, respectively. Bursty containers, containing many chunks that will be needed for restore within a short period of time, are put in a smaller cache managed at the container granularity. On the contrary, regular containers are put in the other bigger cache managed at the chunk granularity, with chunks which will not be used dropped off at the time when the containers are brought in. In doing so, bursty containers have better chances to be quickly evicted from the restore cache, avoiding their unnecessarily occupying cache space for too long. Our evaluation results have demonstrated that our proposed cache scheme can improve restore speed factor by up to 3.05X and reduce the number of container reads by 67.3% on average, relative to a conventional LRU restore cache.

DRAM memory is suffering increasingly aggravating refresh penalty, which no longer causes trivial performance degradation and power consumption. As memory capacity increases, refresh penalty has become increasingly worse as more rows have to be refreshed. In this work, we propose a simple, practical, and effective refresh approach called CAR (Compression-Aware Refresh) to efficiently mitigate refresh overheads. We apply data compression technique to store data in compressed format so that data blocks which are originally distributed across all the constituent chips of a rank only need to be stored in a subset of those chips, leaving banks in the remaining chips not fully occupied. As a result, the memory controller can safely skip refreshing memory rows which contain no useful data without compromising data integrity. Such a compression-aware refresh scheme can result in significant refresh savings and thus improve overall memory performance and energy efficiency. Moreover, to further take advantage of data compression, we adopt the rank subsetting technique to enable accesses to only those occupied chips for memory requests accessing compressed data blocks. Evaluations using benchmarks from SPEC CPU 2006 and the PARSEC 3.0 on the recent DDR4 memory systems have shown that CAR can achieve up to 1.66x performance improvement (11.7% on average).