Processing-in-Memory architecture presents a promising solution to alleviate the data movement bottleneck that arises from transferring data between memory and compute units in traditional processor-centric systems, particularly for DNN applications. However, this architecture introduces two inherent overheads: PIM code offloading and data transferring between CPU and memory. To address these issues, we propose two register-based addressing modes, indexed and base-offset addressing, for DMA descriptor-based in-DRAM PIM ISAs. Our full-system performance evaluation demonstrates that the approach significantly reduces the overheads, resulting in up to 1.94x speedup compared to the baseline PIM, additionally only with $4.65 \%$ area and $8.61 \%$ power consumption.

As the parameter size of deep neural networks (DNNs) increases, high bandwidth memory (HBM) is widely adopted to satisfy the growing demand for memory bandwidth. However, due to the shorter retention time caused by higher on-chip temperature, HBM requires more frequent refresh operations, resulting in significant refresh energy and performance overhead. In this paper, we propose SHIFT ECC, a lightweight and robust ECC scheme for INT8 quantized DNNs on HBM, to reduce refresh operations while maintaining inference accuracy. SHIFT ECC enhances DNN reliability by converting negative weights into positive weights, eventually mitigating the primary cause of retention errors (mostly 1→0 bit errors). Additionally, SHIFT ECC applies stronger ECC to the upper bits (more important bits) of DNN weights while protecting the lower bits (less important bits) with weaker ECC, which further enhances the robustness of DNNs with the same number of parity bits. Our evaluation results show that when the proportion of 1→0 bit errors is 100% and 99%, SHIFT ECC reduces average refresh energy by 32.6% and 35.0%, respectively, reducing average memory read latency by 21.7% compared to the state-of-the-art refresh reduction technique.

Recently Processing-in-Memory (PIM) has become a promising solution to achieve energy-efficient computation in data-intensive applications by placing computation near or inside the memory. In most Deep Learning (DL) frameworks, a user manually partitions a model’s computational graph (CG) onto the computing devices by considering the devices’ capability and the data transfer. The Deep Neural Network (DNN) models become increasingly complex for improving accuracy; thus, it is exceptionally challenging to partition the execution to achieve the best performance, especially on a PIM-based platform requiring frequent offloading of large amounts of data. This article proposes two novel algorithms for DL inference to resolve the challenge: low-overhead profiling and optimal model partitioning. First, we reconstruct CG by considering the devices’ capability to represent all the possible scheduling paths. Second, we develop a profiling algorithm to find the required minimum profiling paths to measure all the node and edge costs of the reconstructed CG. Finally, we devise the model partitioning algorithm to get the optimal minimum execution time using the dynamic programming technique with the profiled data. We evaluated our work by executing the BERT, RoBERTa, and GPT-2 models on the ARM multicores with the PIM-modeled FPGA platform with various sequence lengths. For three computing devices in the platform, i.e., CPU serial/parallel and PIM executions, we could find all the costs only in four profile runs, three for node costs and one for edge costs. Also, our model partitioning algorithm achieved the highest performance in all the experiments over the execution with manually assigned device priority and the state-of-the-art greedy approach.

The Deep Neural Network (DNN), Recurrent Neural Network (RNN) applications, rapidly becoming attractive to the market, process a large amount of low-locality data; thus, the memory bandwidth limits their peak performance. Therefore, many data centers actively adapt high-bandwidth memory like HBM2/HBM2E to resolve the problem. However, this approach would not provide a complete solution since it still transfers the data from the memory to the computing unit. Thus, processing-in-memory (PIM), which performs the computation inside memory, has attracted attention. However, most previous methods require the modification or the extension of core pipelines and memory system components like memory controllers, making the practical implementation of PIM very challenging and expensive in development. In this article, we propose a Silent-PIM that performs the PIM computation with standard DRAM memory requests; thus, requiring no hardware modifications and allowing the PIM memory device to perform the computation while servicing non-PIM applications' memory requests. We can achieve our design goal by preserving the standard memory request behaviors and satisfying the DRAM standard timing requirements. In addition, using standard memory requests makes it possible to use DMA as a PIM's offloading engine, resulting in processing the PIM memory requests fast and making a core perform other tasks. We compared the performance of three Long Short-Term Memory models (LSTM) kernels on real platforms, such as the Silent-PIM modeled on the FPGA, GPU, and CPU. For (p ×512) ×(512 ×2048) matrix multiplication with a batch size p varying from 1 to 128, the Silent-PIM performed up to 16.9x and 24.6x faster than GPU and CPU, respectively, p=1, which was the case without having any data reuse. At p=128, the highest data reuse case, the GPU performance was the highest, but the PIM performance was still higher than the CPU execution. Similarly, at (p ×2048) element-wise multiplication and addition, where there was no data reuse, the Silent-PIM always achieved higher than both CPU and GPU. It also showed that when the PIM's EDP performance was superior to the others in all the cases having no data reuse.

Dynamic Random Access Memory (DRAM) standards have evolved for higher bandwidth, larger capacity, and lower power consumption, so their specifications have become complicated to satisfy the design goals. These complex implementations have significantly increased the test time overhead for design verification; thus, a tremendous amount of command sequences are used. However, since the sequences generated by real machines or memory simulators are the results of scheduling for high performance, they result in low test coverage with repetitive patterns. Eventually, various workloads should be applied to increase the coverage, but this approach incurs significant test time overhead. A few preliminary studies have been proposed to generate predefined or random sequences to cover various test cases or increase test coverage. However, they have limitations in representing various memory behaviors of real workloads. In this article, we define a performance metric for estimating the test coverage when using command sequences. Then, our experiment shows that the coverage of a real machine and a simulator is low and similar. Also, the coverage patterns are almost the same in all tested benchmarks. To alleviate the problem, we propose a test-oriented command scheduling algorithm that increases the test coverage while preserving the memory behaviors of workloads and reducing the test time overhead by extracting representative sequences based on the similarity between command sequences. For the sequence extraction and the coverage estimation, our test sequences are embedded into vectors using bag-of-Ngrams. Compared to the simulator, our algorithm achieves 2.94x higher coverage while reducing the test overhead to 7.57%.

The warp scheduler plays an important role in the GPU for efficient utilization of hardware resources. However, the efficiency of the warp scheduler is often limited by the L1 cache (especially, L1 data cache) capacity; providing large capacity for an L1 cache is challenging due to the increased latency. In this paper, we adopt Monolithic 3D (M3D) technology to design a large capacity L1 cache for GPU performance enhancement, not deteriorating the latency. Our evaluation results show that the M3D L1 cache improves GPU performance by 2.18~2.24× on average, compared to the 2D conventional L1 cache.

As the device capacity of Dynamic Random Access Memory (DRAM) increases, refresh operation becomes a significant contributory factor toward total power consumption and memory throughput of the device. To reduce the problems associated with the refresh operation, a multi-rate refresh technique that changes the refresh period based on the retention time of DRAM cells has been proposed. Unfortunately, the multi-rate refresh technique has a scalability issue, because the additional storage and logic overhead on a memory controller increases as the device capacity increases. In this article, we propose a novel redundancy repair technique to increase the refresh period of DRAM by using a universal hashing technique. Our redundancy repair technique efficiently repairs both hard faults, which occur during the manufacturing process, and weak cells that have short retention time using the remaining spare elements after the process. Also, our technique solves the Variable Retention Time problem by repairing weak cells at boot time by exploiting the Built-in self-repair (BISR) technique and Error Correction Code. Our technique outperforms a conventional BISR redundancy repair with very little hardware overhead, and ensure reliability with more extended refresh period in the entire system. In particular, our experimental results show that our BISR technique achieves 100% repair rate at a 384ms refresh period in 1.0e-6 hard fault BER configuration, and reduces the refresh energy consumption by 83.9% compared to the 64ms refresh and 12% compared to the conventional multi-rate refresh technique for the state-of-the-art 4Gb device.

As dynamic random access memory (DRAM) cells continue to be scaled down for higher density and capacity, they have more faults. Thus, DRAM reliability becomes a major concern in computer systems. Previous studies have proposed many techniques preserving the reliability in various system components, such as DRAM internal, memory controller, caches, and operating systems. By reviewing the techniques, we identified the following two considerations: First, it is possible to recover faults with reasonable overhead at high fault rate only if the recovery unit is fine-grained. Second, since hardware modification requires additional cost in the employment of a technique, a pure software-based recovery technique is preferable. However, in the existing software-based recovery technique, the recovery unit is too coarse-grained to tolerate the high fault rate. In this article, we propose a pure software-based recovery technique with fine-granularity. Our key idea is based on heap segments being managed by the system library with variable-sized chunks to handle dynamic allocation in user applications. In our technique, faulty blocks in pages are offlined by marking them as allocated chunks. Thus, not only fault-free pages but also the remaining clean blocks in faulty pages are allowed to be usable space. Our technique is implemented by modifying the operating system and the system library. Since hardware assistance is unnecessary in the implementation, we evaluated our method on a real machine. Our evaluation results show that our technique has negligible performance overhead at high bit error rate (BER) 5.12e-5, which a hardware-based recovery technique could not tolerate without unacceptable area overhead. Also, at the same BER, our method provides 5.22× usable space, compared with page-offline, which is the state-of-the-art pure software-based technique.

Recently, scaling down dynamic random access memory (DRAM) has become more of a challenge, with more faults than before and a significant degradation in yield. To improve the yield in DRAM, a redundancy repair technique with intra-subarray replacement has been extensively employed to replace faulty elements (i.e., rows or columns with defective cells) with spare elements in each subarray. Unfortunately, such technique cannot efficiently handle a biased distribution of faulty cells because each subarray has a fixed number of spare elements. In this article, we propose a novel redundancy repair technique that uses a hashing method to solve this problem. Our hashing technique reorganizes replacement regions by changing the way in which their replacement information is referred, thus making faulty cells become evenly distributed to the regions. We also propose a fast repair algorithm to find the best hash function among all possible candidates. Even if our approach requires little hardware overhead, it significantly improves the yield when compared with conventional redundancy techniques. In particular, the results of our experiment show that our technique saves spare elements by about 57% and 55% for a yield of 99% at BER 1e-6 and 5e-7, respectively.

Recently, phase change memory (PCM) has been emerging as a strong replacement for DRAM owing to its many advantages such as nonvolatility, high capacity, low leakage power, and so on. However, PCM is still restricted for use as main memory because of its limited write endurance. There have been many methods introduced to resolve the problem by either reducing or spreading out bit flips. Although many previous studies have significantly contributed to reducing bit flips, they still have the drawback that lower bits are flipped more often than higher bits because the lower bits frequently change their bit values. Also, interblock wear-leveling schemes are commonly employed for spreading out bit flips by shifting input data, but they increase the number of bit flips per write. In this article, we propose a noble content-aware bit shuffling (CABS) technique that minimizes bit flips and evenly distributes them to maximize the lifetime of PCM at the bit level. We also introduce two additional optimizations, namely, addition of an inversion bit and use of an XOR key, to further reduce bit flips. Moreover, CABS is capable of recovering from stuck-at faults by restricting the change in values of stuck-at cells. Experimental results showed that CABS outperformed the existing state-of-the-art methods in the aspect of PCM lifetime extension with minimal overhead. CABS achieved up to 48.5% enhanced lifetime compared to the data comparison write (DCW) method only with a few metadata bits. Moreover, CABS obtained approximately 9.7% of improved write throughput than DCW because it significantly reduced bit flips and evenly distributed them. Also, CABS reduced about 5.4% of write dynamic energy compared to DCW. Finally, we have also confirmed that CABS is fully applicable to BCH codes as it was able to reduce the maximum number of bit flips in metadata cells by 32.1%.

With the advent of the HTML5 standard, JavaScript is increasingly processing computationally intensive, data-parallel workloads. Thus, the enhancement of JavaScript performance has been emphasized because the performance gap between JavaScript and native applications is still substantial. Despite this urgency, conventional JavaScript compilers do not exploit much of parallelism even from data-parallel JavaScript applications, despite contemporary mobile devices being equipped with expensive parallel hardware platforms, such as multicore processors and GPGPUs. In this article, we propose an automatically parallelizing JavaScript compiler that targets emerging, data-parallel HTML5 applications by leveraging the mature affine loop analysis of conventional static compilers. We identify that the most critical issues when parallelizing JavaScript with a conventional static analysis are ensuring correct parallelization, minimizing compilation overhead, and conducting low-cost recovery when there is a speculation failure during parallel execution. We propose a mechanism for safely handling the failure at a low cost, based on compiler techniques and the property of idempotence. Our experiment shows that the proposed JavaScript parallelizing compiler detects most affine parallel loops. Also, we achieved a maximum speedup of 3.22 times on a quad-core system, while incurring negligible compilation and recovery overheads with various sets of data-parallel HTML5 applications.

Voltage scaling is known to be an efficient way of saving power and energy within a system, and large caches such as LLCs are good candidates for voltage scaling considering their constantly increasing size. However, the V CCMIN problem, in which the lower bound of scalable voltage is limited by process variation, has made it difficult to exploit the benefits of voltage scaling. Lowering V CCMIN incurs multibit faults, which cannot be efficiently resolved by current technologies due to their high complexity and power consumption. We overcame the limitation by exploiting the data redundancy of memory hierarchy. For example, cache coherence states and several layers of cache organization naturally expose the existence of redundancy within cache blocks. If blocks have redundant copies, their V CCMIN can be lowered; although more faults can occur in the blocks, they can be efficiently detected by simple error detection codes and recovered by reloading the redundant copies. Our scheme requires only minor modifications to the existing cache design. We verified our proposal on a cycle accurate simulator with SPLASH-2 and PARSEC benchmark suites and found that the V CCMIN of a 2MB L2 cache can be further lowered by 0.1V in 32nm technology with negligible degradation in performance. As a result, we could achieve 15.6% of reduction in dynamic power and 15.4% of reduction in static power compared to the previous minimum power.

This paper describes a low-jitter delay-locked loop (DLL)-based clock generator for dynamic frequency scaling in the extendable instruction set computing (EISC) processor. The DLL-based clock generator provides the system clock with frequencies of 0.5× to 8× of the reference clock, according to the workload of the EISC processor. The proposed analog self-calibration method and a phase detector with an auxiliary charge pump can effectively reduce the delay mismatch between delay cells in the voltage-controlled delay line and the static phase offset due to the current mismatch in the charge pump, respectively. The self-calibrated output waveform exhibits 9.7 ps of RMS jitter and 73.7 ps of peak-to-peak jitter at 120 MHz. The prototype clock generator implemented in a 0.18-μm CMOS process occupies an active area of 0.27 mm2and consumes 15.56 mA.

The multi-band orthogonal frequency-division multiplexing modem needs to process large amount of computations in short time for support of high data rates, i.e., up to 480 Mbps. In order to satisfy the performance requirement while reducing power consumption, a multi-way parallel architecture has been proposed. But the use of the high degree parallel architecture would increase chip resource significantly, thus a resource efficient design is essential. In this paper, we introduce several novel optimization techniques for resource efficient implementation of the baseband modem which has highly, i.e., 8-way, parallel architecture, such as new processing structures for a (de)interleaver and a packet synchronizer and algorithm reconstruction for a carrier frequency offset compensator. Also, we describe how to efficiently design several other components. The detailed analysis shows that our optimization technique could reduce the gate count by 27.6% on average, while none of techniques degraded the overall system performance. With 0.18-μm CMOS process, the gate count and power consumption of the entire baseband modem were about 785 kgates and less than 381 mW at 66 MHz clock rate, respectively.

This paper proposes a new runtime parallelization technique, based on a dynamic optimization framework, to automatically parallelize single-threaded legacy programs. It heavily leverages the optimistic concurrency of transactional memory. This work addresses a number of challenges posed by this type of parallelization and quantifies the trade-offs of some of the design decisions, such as how to select good loops for parallelization, how to partition the iteration space among parallel threads, how to handle loop-carried dependencies, and how to transition from serial to parallel execution and back. The simulated implementation of runtime parallelization shows a potential speedup of 1.36 for the NAS benchmarks and a 1.34 speedup for the SPEC 2000 CPU floating point benchmarks when using two cores for parallel execution.

This paper presents an architectural approach to the design of low power reconfigurable finite impulse response (FIR) filter. The approach is well suited when the filter order is fixed and not changed for particular applications, and efficient trade-off between power savings and filter performance can be made using the proposed architecture. Generally, FIR filter has large amplitude variations in input data and coefficients. Considering the amplitude of both the filter coefficients and inputs, the proposed FIR filter dynamically changes the filter order. Mathematical analysis on power savings and filter performance degradation and its experimental results show that the proposed approach achieves significant power savings without seriously compromising the filter performance. The power savings is up to 41.9% with minor performance degradation, and the area overhead of the proposed scheme is less than 5.3% compared to the conventional approach.

In this paper, we propose a timing controller design to use a hierarchical structure of control signals for plasma display panels (PDPs). Also, we used a double buffering and a repeatable FIFO in order to reduce the workload of memory accesses for control data, and provided a graphical user interface program for easy control data management. Our prototype system runs at 83 MHz on Spartan-3A DSP FPGA, and the new design achieves the reduction of 73 % in resource usage from the previous implementation.

In this paper, we present a novel architecture of a block interleaver in MB-OFDM systems based on Mixed Radix System (MRS). We prove mathematically that the proposed architecture can support bit permutations in the interleaving process. The hierarchical property of our proposed MRS-based design methodology allows the proposed architecture to support all the required data rates in the MB-OFDM systems with simple modular design. Furthermore, the same design to be used for the interleaver can also be used for the operation of de-interleaving, which reduces the implementation complexity significantly. The latency of our architecture is as low as 6 MB-OFDM symbols. In addition, when comparing our proposed architecture with the conventional approach, we are able to reduce the implementation complexity by 85.5%, 69.4%, and 40.3% for 80, 200, and 480 Mb/s data rates, respectively, while improving our operating maximum clock frequency by more than 3.3 times over the conventional design. We also show that the power consumption is reduced by 87.4%, 73.6%, and 39.8% for 80, 200, and 480 Mb/s, respectively.

One of major design concerns about wireless headphone is power consumption. In this paper, we propose a novel design for extreme low power headphone implementation by extending the EPC Class-1 Generation 2 RFID protocol for delivering stream data. We prototyped a reader as a stream generator, and a passive tag as an audio receiver to consume less power than any other protocols for wireless headphones.

This paper presents an H.264/AVC baseline profile decoder based on a SoC platform design methodology. The overall decoding throughput is increased by optimized software and a dedicated hardware accelerator. We minimize the number of bus accesses and use macroblock (MB) level pipeline processing techniques to achieve a real time operation. We implemented and verified a prototype on a SoC platform with a 32-bit RISC CPU core and FPGA module. Our design can process up to 20 frames/sec with QCIF (176/spl times/144). The proposed architecture can be easily applied to many mobile video application areas such as a digital camera and a DMB (digital multimedia broadcasting) phone.

In this paper, we present our compiler infrastructure, called Jaguar for Java reconfigurable computing. The Jaguar compiler translates compiled Java methods, i.e. sequence of bytecodes into Verilog synthesizable code modules with exploiting the maximum operational parallelism in applications. Our compiler infrastructure consists of two major components: One is a compiler to generate synthesizable Verilog codes from Java applications, which performs full compilation passes, such as bytecode parsing, intermediate representation (IR) construction, program analysis, optimization, and code emission. The other component is the Java Virtual Machine (JVM), which provides Java execution environment to compiler-generated hardware. The JVM runs on a host processor and the generated hardware does on FPGA. Differently from previous work, our compiler infrastructure is a complete and solid solution for Java reconfigurable computing. We present how to design our compiler framework. Our infrastructure improves the performance by 66% on average and by up to 174% in measured benchmarks. Also we discuss the performance issues in detail, especially focusing on overhead of interactions between JVM and Jaguar hardware.

This paper presents an H.264/AVC baseline profile decoder based on an SOC platform design methodology. The overall decoding throughput is increased by optimized software and a dedicated hardware accelerator. We minimize the number of bus accesses and use macroblock level pipeline processing techniques to achieve a real time operation. We implemented and verified a prototype on an SOC platform with a 32-bit RISC CPU core and FPGA module. Our design can process up to 30 frames/sec with CIF_(352times288). The proposed architecture can be easily applied to many mobile video application areas such as a digital camera and a DMB (digital multimedia broadcasting) phone

Recent proposals for multithreaded architectures employ speculative execution to allow threads with unknown dependences to execute speculatively in parallel. The architectures use hardware speculative storage to buffer speculative data, track data dependences and correct incorrect executions through roll-backs. Because all memory references access the speculative storage, current proposals implement speculative storage using small memory structures to achieve fast access. The limited capacity of the speculative storage causes considerable performance loss due to speculative storage overflow whenever a thread's speculative state exceeds the speculative storage capacity. Larger threads exacerbate the overflow problem but are preferable to smaller threads, as larger threads uncover more parallelism.In this article, we discover a new program property called memory reference idempotency . Idempotent references are guaranteed to be eventually corrected, though the references may be temporarily incorrect in the process of speculation. Therefore, idempotent references, even from nonparallelizable program sections, need not be tracked in the speculative storage, and instead can directly access nonspeculative storage (i.e., conventional memory hierarchy). Thus, we reduce the demand for speculative storage space in large threads. We define a formal framework for reference idempotency and present a novel compiler-assisted speculative execution model. We prove the necessary and sufficient conditions for reference idempotency using our model. We present a compiler algorithm to label idempotent memory references for the hardware. Experimental results show that for our benchmarks, over 60% of the references in nonparallelizable program sections are idempotent.

Recent proposals for Chip Multiprocessors (CMPs) advocate speculative, or implicit, threading in which the hardware employs prediction to peel off instruction sequences (i.e., implicit threads) from the sequential execution stream and speculatively executes them in parallel on multiple processor cores. These proposals augment a conventional multiprocessor, which employs explicit threading, with the ability to handle implicit threads. Current proposals focus on only implicitly-threaded code sections. This paper identifies, for the first time, the issues in combining explicit and implicit threading. We present the Multiplex architecture to combine the two threading models. Multiplex exploits the similarities between implicit and explicit threading, and provides a unified support for the two threading models without additional hardware. Multiplex groups a subset of protocol states in an implicitly-threaded CMP to provide a write-invalidate protocol for explicit threads.

Recent proposals for multithreaded architectures allow threads with unknown dependences to execute speculatively in parallel. These architectures use hardware speculative storage to buffer uncertain data, track data dependences and roll back incorrect executions. Because all memory references access the speculative storage, current proposals implement this storage using small memory structures for fast access. The limited capacity of the speculative storage causes considerable performance loss due to speculative storage overflow whenever a thread's speculative state exceeds the storage capacity. Larger threads exacerbate the over-flow problem but are preferable to smaller threads, as larger threads uncover more parallelism. In this paper, we discover a new program property called memory reference idempotency. Idempotent references need not be tracked in the speculative storage, and instead can directly access non-speculative storage (i.e., the conventional memory hierarchy). Thus, we reduce the demand fo r speculative storage space. We define a formal framework for reference idempotency and present a novel compiler-assisted speculative execution model. We prove the necessary and sufficient conditions for reference idempotency using our model. We present a compiler algorithm to label idempotent memory references for the hardware. Experimental results show that for our benchmarks, over 60% of the references in non-parallelizable program sections are idempotent.

We present a new software scheme, called compiler-assisted I-cache prediction (CIP) for energy reduction in instruction caches. With the help of compiler-supplied information, the processor is able to turn off substantial portions of the I-cache. The necessary cache sets are only turned on during the execution of individual code sections. The CIP scheme is based on the processor's ability to predict code sections that are about to execute and on the compiler's ability to precisely inform the hardware about the size of these code sections. Our techniques grew out of work with optimizing compilers for speculative parallel microarchitectures. The use of this target machine class is further motivated by the fact that speculative processors have the potential to overcome limitations in the compiler parallelization of many applications, especially non-numerical programs. Speculative microarchitectures are also among the most promising emerging architectures that can take advantage of the ever-increasing levels of chip integration. We will show that our new techniques can lead up to 90% I-cache energy savings in general-purpose applications without significant execution overhead. We believe that this is a substantial step towards the goal of making such chips integral parts of mobile computing devices, such as laptops, palm tops, and cellular phones.