Huge embedding tables in modern deep learning recommender models (DLRM) require prohibitively large memory during training and inference. This paper proposes FIITED, a system to automatically reduce the memory footprint via FIne-grained In-Training Embedding Dimension pruning. By leveraging the key insight that embedding vectors are not equally important, FIITED adaptively adjusts the dimension of each individual embedding vector during model training, assigning larger dimensions to more important embeddings while adapting to dynamic changes in data. We prioritize embedding dimensions with higher frequencies and gradients as more important. To enable efficient pruning of embeddings and their dimensions during model training, we propose an embedding storage system based on virtually-hashed physically-indexed hash tables. Experiments on two industry models and months of realistic datasets show that FIITED can reduce DLRM embedding size by more than 65’ while preserving model quality, outperforming state-of-the-art in-training embedding pruning methods and increasing the reduction ratio by 1.3× to 1.67×. On public datasets, FIITED can reduce the size of embedding tables by 2.2× to 800× with negligible accuracy drop, achieving 1.05× to 12.5× improvement in reduction ratio compared to baselines while improving model throughput.

As we enter an era witnessing the closer end of Dennard scaling, where further reduction in power supply-voltage to reduce power consumption becomes more challenging in conventional systems, a goal of developing a system capable of performing large computations with minimal area and power overheads needs more optimization aspects. A rigorous exploration of alternate computing techniques, which can mitigate the limitations of Complementary Metal-Oxide Semiconductor (CMOS) technology scaling and conventional Boolean systems, is imperative. Reflecting on these lines of thought, in this article we explore the potential of non-Boolean computing employing nano-oscillators for performing varied functions. We use a two coupled nano-oscillator as our basic computational model and propose an architecture for a non-Boolean coupled oscillator based co-processor capable of executing certain functions that are commonly used across a variety of approximate application domains. The proposed architecture includes an accuracy tunable knob, which can be tuned by the programmer at runtime. The functionality of the proposed co-processor is verified using a soft coupled oscillator model based on Kuramoto oscillators. The article also demonstrates how real-world applications such as Vector Quantization, Digit Recognition, Structural Health Monitoring, and the like, can be deployed on the proposed model. The proposed co-processor architecture is generic in nature and can be implemented using any of the existing modern day nano-oscillator technologies such as Resonant Body Transistors (RBTs), Spin-Torque Nano-Oscillators (STNOs), and Metal-Insulator Transition (MITs) . In this article, we perform a validation of the proposed architecture using the HyperField Effect Transistor (FET) technology-based coupled oscillators, which provide improvements of up to 3.5× increase in clock speed and up to 10.75× and 14.12× reduction in area and power consumption, respectively, as compared to a conventional Boolean CMOS accelerator executing the same functions.

There are an increasing number of neuromorphic hardware platforms designed to efficiently support neural network inference tasks. However, many applications contain structured processing in addition to classification. Being able to map both neural network classification and structured computation onto the same platform is appealing from a system design perspective. In this paper, we perform a case study on mapping the feature extraction stage of pedestrian detection using Histogram of Oriented Gradients (HoG) onto a neuromophic platform. We consider three implementations: one that approximates HoG using neuromorphic intrinsics, one that emulates HoG outputs using a trained network, and one that allows feature extraction to be absorbed into classification. The proposed feature extraction methods are implemented and evaluated on neuromorphic hardware (IBM Neurosynaptic System). Our study shows that both a designed approximation and a "parroted" emulation can achieve similar accuracy, and that the latter appears to better capitalize on limited training and resource budgets, compared to the absorbed approach, while also being more power efficient than the programmed approach by a factor of 6.5x-208x.

We report on the implementation of a path planning algorithm on the TrueNorth neurosynaptic system. Our implementation exploits processing in the temporal domain within the architectural constraints of the TrueNorth chip to deduce the optimal path. The optimal path is computed on the TrueNorth chip for grid maps with dimensions as large as 173 × 168 nodes consuming 70.0mW at an operating voltage of 0.8V.

Deep neural networks (DNN) have been shown to be very effective at solving challenging problems in several areas of computing, including vision, speech, and natural language processing. However, traditional platforms for implementing these DNNs are often very power hungry, which has lead to significant efforts in the development of configurable platforms capable of implementing these DNNs efficiently. One of these platforms, the IBM TrueNorth processor, has demonstrated very low operating power in performing visual computing and neural network classification tasks in real-time. The neuron computation, synaptic memory, and communication fabrics are all configurable, so that a wide range of network types and topologies can be mapped to TrueNorth. This reconfigurability translates into the capability to support a wide range of low-power functions in addition to feed-forward DNN classifiers, including for example, the audio processing functions presented here.In this work, we propose an end-to-end audio processing pipeline that is implemented entirely on a TrueNorth processor and designed to specifically leverage the highly-parallel, low-precision computing primitives TrueNorth offers. As part of this pipeline, we develop an audio feature extractor (LATTE) designed for implementation on TrueNorth, and explore the tradeoffs among several design variants in terms of accuracy, power, and performance. We customize the energy-efficient deep neuromorphic networks structures that our design utilizes as the classifier and show how classifier parameters can trade between power and accuracy. In addition to enabling a wide range of diverse functions, the reconfigurability of TrueNorth enables re-training and re-programming the system to satisfy varying energy, speed, area, and accuracy requirements. The resulting system's end-to-end power consumption can be as low as$14.43\text{mW}$, which would give up to 100 hours of continuous usage with button cell batteries (CR3023$1.5\; \text{Whr}$) or 450 hours with cellphone batteries (iPhone 6s$6.55\; \text{Whr}$).

With recent advances in silicon technology, previously intractable Deep Neural Network (DNN) solutions to complex visual, auditory, and other sensory perception problems are now practical for real-time, energy constrained systems. One such advancement is IBM's TrueNorth neurosynaptic processor, containing 1 million neurons and 256 million synapses, consuming 65mW of power, and capable of operating in real-time for a variety of applications. In this work, we explore how auditory features can be extracted on the TrueNorth processor using low numerical precision while maintaining algorithmic fidelity for DNN based spoken digit recognition on isolated words from the TIDIGITS dataset. Further, we show that our Low-power Audio Transform with TrueNorth Ecosystem (LATTE) is capable of achieving a 24× reduction in energy for feature extraction over a baseline FPGA implementation using standard MFCC audio features, while only incurring a 3 - 6% accuracy penalty.

Current mode logic (CML) circuits have been widely used in high-speed data transceivers. The lower-voltage-swing makes the switching speed of CML much higher than the static logic can achieve, so it is worthy to adopt the CML circuits at the cost of higher power consumption in the high-speed applications. In order to obtain a better power efficiency (Frequency/power) in CML, it is critical to reduce the power consumption while maintaining the high operating frequency. This paper proposes an alternative approach by building the CML circuits with tunneling-field-effect-transistor (Tunnel FETs or TFETs) to achieve a high-throughput, low-voltage interface circuit design. By taking advantage of its steep subthreshold slope (less than 60 mV/dec), TFET exhibits the same on/off current ratio at the input voltage swing interval much lower than that of the MOSFETs, which enables the supply voltage scaling in CML circuits. For a design target data-rate (20 Gbps for multiplexer and 50 Gbps for buffer), our simulations show that the proposed TFET CML circuits are able to reduce the supply voltage from 0.6 V in conventional Si FinFET CML circuits to as low as 0.3 V while using the same constant tail current. As a result, a power consumption reduction of approximately 50% is achieved by the proposed TFET CML circuits, making the TFET CML approach a promising candidate for future low-power, high-performance applications.

This paper proposes Multiplexer-Flip-Flops (MUX-FFs) to be a high-throughput and low-cost solution for serial link transmitters. We also propose Multiplexer-Latches (MUX-Latches) that possess the logic function of combinational circuits and storing capacity of sequential circuits. Adopting the pipeline with MUX-FFs, which are composed of cascaded latches and MUX-Latches, many latch gates for sequencing can be removed. Analysis shows that an 8-to-1 serializer in the pipeline topology with MUX-FFs reduces 52% gate-count compared to the traditional pipeline topology. To verify the function of the proposed design, a chips is implemented with the proposed 8-to-1 serializer with MUX-FFs in 90 nm CMOS technology. The measured results show that the proposed serializer with MUX-FFs are bit-error-free (with BER-12), operating at up to 12 Gbit/s.

In this paper, a novel multiplexer-flip-flop (MUX-FF) topology using the current mode logic (CML) is presented. A CML multiplexer-latch (MUX-latch) is proposed by combining a multiplexer and the loopback storage part of a latch into a single module so that the buffer part of a latch can be removed. A MUX-FF is implemented by cascading two stages of MUX-latches. The output of a MUX-FF is edge-triggered, so it is insensitive to input noise. All the paths from inputs to the output are symmetric. Power and area can be reduced due to the removal of DFFs. Simulation results show that a MUX-FF can achieve a similar frequency as a conventional tree-type MUX by saving 56% of area and 72% of power consumption.