Huge embedding tables in modern deep learning recommender models (DLRM) require prohibitively large memory during training and inference. This paper proposes FIITED, a system to automatically reduce the memory footprint via FIne-grained In-Training Embedding Dimension pruning. By leveraging the key insight that embedding vectors are not equally important, FIITED adaptively adjusts the dimension of each individual embedding vector during model training, assigning larger dimensions to more important embeddings while adapting to dynamic changes in data. We prioritize embedding dimensions with higher frequencies and gradients as more important. To enable efficient pruning of embeddings and their dimensions during model training, we propose an embedding storage system based on virtually-hashed physically-indexed hash tables. Experiments on two industry models and months of realistic datasets show that FIITED can reduce DLRM embedding size by more than 65’ while preserving model quality, outperforming state-of-the-art in-training embedding pruning methods and increasing the reduction ratio by 1.3× to 1.67×. On public datasets, FIITED can reduce the size of embedding tables by 2.2× to 800× with negligible accuracy drop, achieving 1.05× to 12.5× improvement in reduction ratio compared to baselines while improving model throughput.

Modern processors often need to switch among different power states based on usage scenarios, energy availability, and thermal conditions. Dynamic voltage and frequency scaling (DVFS) is a commonly used power management strategy to trade off performance and energy. As transistor scaling is reaching its limit, the viable supply voltage range where DVFS can operate is shrinking, which limits its effectiveness. To extend the performance-energy trade-off capabilities in modern processors, this article proposes dynamic core scaling (DCS) that does not rely on voltage scaling. DCS dynamically adjusts the active superscalar datapath resources so that programs run at a given percentage of their maximum speed while minimizing energy consumption at the same time. Since DCS does not need voltage scaling, it can be combined with DVFS to achieve greater energy savings. To effectively manage performance-energy trade-offs using a combination of DCS and DVFS, this article proposes an oracle controller that demonstrates the optimal control strategy, and two practical controllers that are applicable in real implementations. Evaluations using an 8-way superscalar processor implemented in a 45-nm circuit show that DCS is more effective in performance-energy trade-offs than DVFS at the high performance end for a number of SPEC CPU2000 benchmarks. When used together with DVFS, DCS saves an additional 20 percent of a full-size core’s energy on average. At the minimum operating voltage, DVFS hits its limit, while DCS is still able to achieve an average of 46 percent further energy reduction.

Dynamic voltage and frequency scaling (DVFS) is commonly employed on modern superscalar processors to reduce energy when peak performance is not needed or allowed. As technology scales, the effectiveness of DVFS is limited by the shrinking viable supply voltage range. This work proposes dynamic core scaling (DCS) to extend performance-energy tradeoff capabilities in superscalar processors. DCS ensures that programs run at a given percentage of their maximum speed and, at the same time, minimizes energy consumption by dynamically adjusting the active superscalar datapath resources. Evaluations using an 8-way superscalar processor implemented on 45nm circuit infrastructure show that DCS is more effective in performance-energy tradeoffs than DVFS at the high performance end. When used together with DVFS, DCS saves an additional 20% of a full-size core's energy on average. At the minimum operating voltage, DVFS stops reducing energy, while DCS is still able to achieve an average of 46% further energy reduction.

To minimize the access latency of set-associative caches, the data in all ways are read out in parallel with the tag lookup. However, this is energy inefficient, as only the data from the matching way is used and the others are discarded. This paper proposes an early tag lookup (ETL) technique for L1 instruction caches that determines the matching way one cycle earlier than the cache access, so that only the matching data way need be accessed. ETL incurs no performance penalty and insignificant hardware overhead. Evaluation on a 4-way set-associative L1 instruction cache in 45nm technology shows that ETL reduces the read energy by 68% on average.

Floating point multiplication is one of the most frequently used arithmetic operations in a wide variety of applications, but the high power consumption of the IEEE-754 standard floating point multiplier prohibits its implementation in many low power systems, such as wireless sensors and other battery-powered embedded systems, and limits performance scaling in high performance systems, such as CPUs and GPGPUs for scientific computation. This paper presents a low-power accuracy-configurable floating point multiplier based on Mitchell's Algorithm. Post-layout SPICE simulations in a 45nm process show same-delay power reductions up to 26X for single precision and 49X for double precision compared to their IEEE-754 counterparts. Functional simulations on six CPU and GPU benchmarks show significantly better power reduction vs. quality degradation trade-offs than existing bit truncation schemes.

To achieve high performance, conventional superscalar processors maintain maximum front-end instruction delivery bandwidth, which is often suboptimal when program behavior and priority metrics change. This paper proposes an adaptive front-end throttling technique that dynamically adjusts the front-end instruction delivery bandwidth as program behavior changes to optimize a target metric, being performance, energy, or an arbitrary trade-off between them. Circuit-level synthesis (45nm FreePDK) and simulation show that adaptive front-end throttling incurs negligible overhead but achieves average improvements of 7%, 28%, 28%, and 32% for performance, energy, energy-delay product, and energy-delay-squared product, respectively, over all benchmarks on an 8-way superscalar processor.

Voltage noise not only detracts from reliability and performance, but has been used to attack system security. Most systems are completely unaware of fluctuations occurring on nanosecond time scales. This paper quantifies the threat to FPGA-based systems and presents a solution approach. Novel measurements of transients on 28nm FPGAs show that extreme activity in the fabric can cause enormous undershoot and overshoot, more than 10× larger than what is allowed by the specification. An existing voltage sensor is evaluated and shown to be insufficient. Lastly, a sensor design using reconfigurable logic is presented; its time-to-digital converter enables sample rates 500× faster than the 28nm Xilinx ADC. This enables quick characterization of transients that would normally go undetected, thereby providing potentially useful data for system optimization and helping to defend against supply voltage attacks.

We present a parallelized and pipelined architecture for a generalized Laguerre-Volterra MIMO system to identify the time-varying neural dynamics underlying spike activities. The proposed architecture consists of a first stage containing a vector convolution and MAC (Multiply and Accumulation) component, a second stage containing a pre-threshold potential updating unit with an error approximation function component, and a third stage consisting of a gradient calculation unit. A flexible and efficient architecture that can accommodate different design speed requirements are generated. Simulation results are rigorously analyzed. A hardware IP library for versatile models and applications is proposed. The design runs on a Xilinx Virtex-6 FPGA and the processing core produces data samples at a maximum clock rate of 357MHz, which is 4.37 × 105times faster than the corresponding software model running on an AMD Pheono 9750 Quad Core Processor. It occupies 216,766 LUTs, maximum 12 block-RAMs, and 2016 DSP-blocks.

We present a full-parallelized and pipelined architecture for a generalized Laguerre-Volterra MIMO system to identify the time-varying neural dynamics underlying spike activities. The proposed architecture consists of a first stage containing a vector convolution and MAC (Multiply and Accumulation) component, a second stage containing a pre-threshold potential updating unit with an error approximation function component, and a third stage consisting of a gradient calculation unit. A flexible and efficient architecture that can accommodate different design speed requirements is generated. The design runs on a Xilinx Virtex-6 FPGA and the processing core produces data samples at a speed of 1.33×106data frames/sec, which is 3.1×103times faster than the corresponding C model running on an Intel i7-860 Quad Core Processor.