This paper presents a tensor-based algorithm that leverages a hardware accelerator for inferencing decision-tree-based machine learning models. The algorithm has been integrated in a public software library and is demonstrated on an IBM z16 server, using the Telum processor with the Integrated Accelerator for AI. We describe the architecture and implementation of the algorithm and present experimental results that demonstrate its superior runtime performance compared with popular CPU-based machine learning inference implementations.

We introduce a highly heterogeneous and programmable compute-in-memory (CIM) accelerator architecture for deep neural network (DNN) inference. This architecture combines spatially distributed CIM memory array “tiles” for weight-stationary, energy-efficient multiply–accumulate (MAC) operations, together with heterogeneous special-function compute cores for auxiliary digital computation. Massively parallel vectors of neuron activation data are exchanged over short distances using a dense and efficient circuit-switched 2-D mesh, offering full end-to-end support for a wide range of DNN workloads, including CNNs, long-short-term-memory (LSTM), and transformers. We discuss the design of the “analog fabric”—the 2-D grid of tiles and compute cores interconnected by the 2-D mesh—and address the efficiency in both mapping of DNNs onto the hardware and in pipelining of various DNN workloads across a range of batch sizes. We show, for the first time, system-level assessments using projected component parameters for a realistic “analog AI” system, based on dense crossbar arrays of low-power nonvolatile analog memory elements, while incorporating a single common analog fabric design that can scale to large networks by introducing data transport between multiple analog AI chips. Our performance estimates for several networks, including large LSTM and bidirectional encoder representations from transformers (BERT), show highly competitive throughput while offering$40\times $–$140\times $higher energy efficiency than NVIDIA A100—thus illustrating the strong promise of analog AI and the proposed architecture for DNN inference applications.

New coding schemes based on generalized concatenated codes are proposed for emerging nonvolative memory technologies. Key requirements are high code rate, low latency, high throughput and the ability to correct chipkill failures while sustaining high data reliability despite raw bit error rates up to 10-3. New concatenated codes based on Reed-Solomon codes have been designed for payload sizes of 512B, 1kB, and 2kB; they have high rates above 0.8 and a high data reliability with a decoder target BER of 10-15. An FPGA-based implementation of the decoder validates the low latency and high throughput: for an operating clock frequency of 250MHz, the decoding latency is 236ns and a 9.3GB/s throughput is achieved.

Next-generation memory (NGM) technologies present a major opportunity but also a significant challenge, due to their intricate reliability issues. In particular, multilevel-cell (MLC) storage is highly desirable for increasing storage capacity and lowering total cost-per-bit. In phase-change memory (PCM), MLC storage is hampered by sensitivity to temperature variations and resistance drift. A novel drift-invariant detection (DID) scheme that estimates variable read thresholds based on ordered statistics and clustering of the soft read-back signals from a small block of 32 cells has been developed and implemented in hardware to improve reliability and prolong data retention. A low-complexity implementation of the DID on a FPGA platform comprises 20'000 LUTs and 6'000 flip-flops and has a latency of 90ns. We present results from an extensive performance verification that ascertains highly reliable data retrieval up to 13 orders of magnitude in time after programming. Such elevated reliability is necessary for the most anticipated application of NGM, namely persistent far-memory, where the NGM is used as a large memory pool, possibly together with DRAM.

In this paper a novel approach to optimize digital integrated circuits yield with regards to speed and area/power for aggressive scaling technologies is presented. The technique is intended to reduce the effects of intra-die variations using redundancy applied only on critical parts of the circuit. The inherent property of the technique is that the improvement in the maximum frequency the circuit can run is higher for the larger variations. The work shows that the technique can be already applied for 65nm CMOS technology process where a beneficial delay vs. area/power tradeoff can be made. However, a significant benefit is expected for future nanoscale CMOS technologies such as 45nm and 32nm nodes and in low-voltage applications.

The precise evaluation of the reliability of logic circuits has a significant importance in highly-defective and future nanotechnologies. It allows efficient comparison of fault-tolerance techniques, and enables designs improvement with respect to their reliability figure. This paper presents a novel, accurate and scalable method for modeling the output probability density functions (PDFs) of logic circuits. Our method combines probability theory with concepts from logic synthesis and testing. The PDFs are modeled using the acquired circuit output probability of failure and PDFs of gates in the last two layers of the output cone. Unlike the existing output PDF modeling techniques, the proposed method is directly applicable to standard CMOS design. Simulation results of benchmark circuits demonstrate the accuracy of the method. Several potential applications of the proposed technique include the analysis of averaging (analog) fault-tolerant techniques, fine-grained redundancy insertion, and reliability-driven design optimization.

This paper presents a new approach for assessing the reliability of nanometer-scale devices prior to fabrication and a practical reliability architecture realization. A four-layer architecture exhibiting a large immunity to permanent as well as random failures is used. Characteristics of the averaging/thresholding layer are emphasized. A complete tool based on Monte Carlo simulation for a-priori functional fault tolerance analysis was used for analysis of distinctive cases and topologies. A full chip CMOS integrated design of the 128-bit AES cryptography algorithm with multiple cores that incorporate reliability architectures is shown.

An assessment of the fault-tolerance properties of single-ended and differential signaling is shown in the context of a high defect density environment, using a robust error-absorbing circuit architecture. A software tool based on Monte-Carlo simulations is used for the reliability analysis of the examined logic families. A benefit of the differential circuit over standard single-ended is shown in case of complex systems. Moreover, analysis of reliability of different circuits and discussion on the optimal granularity of redundant blocks was made.