Deep neural network (DNN) training workloads are increasingly susceptible to hardware failures in datacenters. For example, Google experienced "mysterious, difficult to identify problems" in their TPU training systems due to hardware failures [7]. Although these particular problems were subsequently corrected through significant efforts, they have raised the urgency of addressing the growing challenges emerging from hardware failures impacting many DNN training workloads.

In response to innovations in machine learning (ML) models, production workloads changed radically and rapidly. TPU v4 is the fifth Google domain specific architecture (DSA) and its third supercomputer for such ML models. Optical circuit switches (OCSes) dynamically reconfigure its interconnect topology to improve scale, availability, utilization, modularity, deployment, security, power, and performance; users can pick a twisted 3D torus topology if desired. Much cheaper, lower power, and faster than Infiniband, OCSes and underlying optical components are <5% of system cost and <3% of system power. Each TPU v4 includes SparseCores, dataflow processors that accelerate models that rely on embeddings by 5x--7x yet use only 5% of die area and power. Deployed since 2020, TPU v4 outperforms TPU v3 by 2.1x and improves performance/Watt by 2.7x. The TPU v4 supercomputer is 4x larger at 4096 chips and thus nearly 10x faster overall, which along with OCS flexibility and availability allows a large language model to train at an average of ~60% of peak FLOPS/second. For similar sized systems, it is ~4.3x--4.5x faster than the Graphcore IPU Bow and is 1.2x--1.7x faster and uses 1.3x--1.9x less power than the Nvidia A100. TPU v4s inside the energy-optimized warehouse scale computers of Google Cloud use ~2--6x less energy and produce ~20x less CO2e than contemporary DSAs in typical on-premise data centers.

Google deployed several TPU generations since 2015, teaching us lessons that changed our views: semi-conductor technology advances unequally; compiler compatibility trumps binary compatibility, especially for VLIW domain-specific architectures (DSA); target total cost of ownership vs initial cost; support multi-tenancy; deep neural networks (DNN) grow 1.5X annually; DNN advances evolve workloads; some inference tasks require floating point; inference DSAs need air-cooling; apps limit latency, not batch size; and backwards ML compatibility helps deploy DNNs quickly. These lessons molded TPUv4i, an inference DSA deployed since 2020.

This article consists only of a collection of slides from the author's conference presentation.

Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC---called a Tensor Processing Unit (TPU) --- deployed in datacenters since 2015 that accelerates the inference phase of neural networks (NN). The heart of the TPU is a 65,536 8-bit MAC matrix multiply unit that offers a peak throughput of 92 TeraOps/second (TOPS) and a large (28 MiB) software-managed on-chip memory. The TPU's deterministic execution model is a better match to the 99th-percentile response-time requirement of our NN applications than are the time-varying optimizations of CPUs and GPUs that help average throughput more than guaranteed latency. The lack of such features helps explain why, despite having myriad MACs and a big memory, the TPU is relatively small and low power. We compare the TPU to a server-class Intel Haswell CPU and an Nvidia K80 GPU, which are contemporaries deployed in the same datacenters. Our workload, written in the high-level TensorFlow framework, uses production NN applications (MLPs, CNNs, and LSTMs) that represent 95% of our datacenters' NN inference demand. Despite low utilization for some applications, the TPU is on average about 15X -- 30X faster than its contemporary GPU or CPU, with TOPS/Watt about 30X -- 80X higher. Moreover, using the CPU's GDDR5 memory in the TPU would triple achieved TOPS and raise TOPS/Watt to nearly 70X the GPU and 200X the CPU.

Carbon nanotube field-effect transistors (CNFETs) are excellent candidates for building highly energy-efficient electronic systems of the future. Fundamental limitations inherent to carbon nanotubes (CNTs) pose major obstacles to the realization of robust CNFET digital very large-scale integration (VLSI): 1) it is nearly impossible to guarantee perfect alignment and positioning of all CNTs despite near-perfect CNT alignment achieved in recent years; 2) CNTs can be metallic or semiconducting depending on chirality; and 3) CNFET circuits can suffer from large performance variations, reduced yield, and increased susceptibility to noise. Today's CNT process improvements alone are inadequate to overcome these challenges. This paper presents an overview of: 1) imperfections and variations inherent to CNTs; 2) design and processing techniques, together with a probabilistic analysis framework, for robust CNFET digital VLSI circuits immune to inherent CNT imperfections and variations; and 3) recent experimental demonstration of CNFET digital circuits that are immune to CNT imperfections. Significant advances in design tools can enable robust and scalable CNFET circuits that overcome the challenges of the CNFET technology while retaining its energy-efficiency benefits.

Carbon Nanotube Field-Effect Transistors (CNFETs) are excellent candidates for designing highly energy-efficient future digital systems. However, carbon nanotubes (CNTs) are inherently highly subject to imperfections that pose major obstacles to robust CNFET digital VLSI. This paper summarizes commonly raised questions and concerns about CNFET technology through a series of frequently asked questions. The specific questions addressed in this paper are motivated by recent advances in the field since the publication of our earlier paper on frequently asked questions in the Proceedings of the 2009 Design Automation Conference.

Variations in the spatial density of carbon nanotubes (CNTs), resulting from the lack of precise control over CNT positioning during chemical synthesis, is a major hurdle to the scalability of carbon nanotube field effect transistor (CNFET) circuits. Such CNT density variations can lead to non-functional CNFET circuits. This paper presents a probabilistic framework for modeling the CNT count distribution contained in a CNFET of given width, and establishes the accuracy of the model using experimental data obtained from CNT growth. Using this model, we estimate the impact of CNT density variations on the yield of CNFET very large-scale integrated circuits. Our estimation results demonstrate that CNT density variations can significantly degrade the yield of CNFETs, and can be a major concern for scaled CNFET circuits. Finally, we analyze the impact of CNT correlation (i.e., correlation of CNT count between CNFETs) that exists in CNT growth, and demonstrate how the yield of a CNFET storage circuit (primarily limited by its noise immunity) can be significantly improved by taking advantage of such correlation.

Carbon Nanotubes (CNTs) are grown using chemical synthesis, and the exact positioning and chirality of CNTs are very difficult to control. As a result, "small-width" Carbon Nanotube Field-Effect Transistors (CNFETs) can have a high probability of containing no semiconducting CNTs, resulting in CNFET failures. Upsizing these vulnerable small-width CNFETs is an expensive design choice since it can result in substantial area/power penalties. This paper introduces a processing/design co-optimization approach to reduce probability of CNFET failures at the chip-level. Large degree of spatial correlation observed in directional CNT growth presents a unique opportunity for such optimization. Maximum benefits from such correlation can be realized by enforcing the active regions of CNFETs to be aligned with each other. This approach relaxes the device-level failure probability requirement by 350X at the 45nm technology node, leading to significantly reduced costs associated with upsizing the small-width CNFETs.

Carbon Nanotube Field-Effect Transistors (CNFETs) can potentially provide significant energy-delay-product benefits compared to silicon CMOS. However, CNFET circuits are subject to several sources of imperfections. These imperfections lead to incorrect logic functionality and substantial circuit performance variations. Processing techniques alone are inadequate to overcome the challenges resulting from these imperfections. An imperfection-immune design methodology is required. We present an overview of imperfection-immune design techniques to overcome two major sources of CNFET imperfections: metallic Carbon Nanotubes (CNTs) and CNT density variations.

Nanoelectromechanical (NEM) relays are promising candidates for programmable routing in Field-Programmable-Gate Arrays (FPGAs). This is due to their zero leakage and potentially low on-resistance. Moreover, NEM relays can be fabricated using a low-temperature process and, hence, may be monolithically integrated on top of CMOS circuits. Hysteresis characteristics of NEM relays can be utilized for designing programmable routing switches in FPGAs without requiring corresponding routing SRAM cells. Our simulation results demonstrate that the use of NEM relays for programmable routing in FPGAs can simultaneously provide 43.6% footprint area reduction, 37% leakage power reduction, and up to 28% critical path delay reduction compared to traditional SRAM-based CMOS FPGAs at the 22nm technology node.

Carbon Nanotube Field-Effect Transistors (CNFETs) show promise as extensions to silicon-CMOS. Ideal CNFET circuits can potentially provide 20X Energy-Delay-Product benefits over silicon-CMOS at the 16 nm technology node. However, several challenges must be overcome before such performance benefits can be experimentally realized. In this paper, we present a brief overview of CNFET technology, and address commonly raised concerns through a series of Frequently Asked Questions (FAQs). We also provide a CNFET technology outlook which includes a survey of challenges as well as existing and potential solutions to these challenges.

Carbon Nanotubes (CNTs) are grown using chemical synthesis. As a result, it is extremely difficult to ensure exact positioning and uniform density of CNTs. Density variations in CNT growth can compromise reliability of Carbon Nanotube Field Effect Transistor (CNFET) circuits, and result in increased delay variations. A parameterized model for CNT density variations is presented based on experimental data extracted from aligned CNT growth. This model is used to quantify the impact of such variations on design metrics such as noise margin and delay variations of CNFET circuits. Finally, we analyze correlation that exists in aligned CNT growth, and demonstrate how the reliability of CNFET circuits can be significantly improved by taking advantage of such correlation.

Carbon Nanotube Field-Effect Transistors (CNFETs) show big promise as extensions to silicon-CMOS because: 1) Ideal CNFETs can provide significant energy and performance benefits over silicon-CMOS, and 2) CNFET processing is compatible with existing silicon-CMOS processing. However, future gigascale systems cannot rely solely on existing chemical synthesis for guaranteed ideal devices. VLSI-scale logic circuits using CNFETs must overcome major challenges posed by: 1) Misaligned and mis-positioned Carbon Nanotubes (CNTs); 2) Metallic CNTs; and, 3) CNT density variations. This paper performs detailed analysis of the impact of these challenges on CNFET circuit performance. A combination of design and processing techniques, presented this paper, can enable VLSI-scale CNFET logic circuits that are immune to high rates of inherent imperfections. These techniques are inexpensive compared to traditional defect- and fault-tolerance, do not impose major changes in VLSI design flows, and are compatible with VLSI processing because they do not require special customization on chip-by-chip basis.

Carbon Nanotube Field Effect Transistors (CNFETs), consisting of semiconducting single-walled Carbon Nanotubes (CNTs), show great promise as extensions to silicon CMOS and in large-area electronics. While there has been significant progress at a single-device level, a major gap exists between such results and their transformation into VLSI CNFET technologies. Major CNFET technology challenges include mis-positioned CNTs, metallic CNTs, and wafer-scale processing. We present design and processing techniques to overcome these challenges. Experimental results demonstrate the effectiveness of the presented techniques. Mis-positioned CNTs can result in incorrect logic functionality of CNFET circuits. A new layout design technique produces CNFET circuits for arbitrary logic functions that are immune to a large number of mis-positioned CNTs. This technique is significantly more efficient compared to traditional defect- and fault-tolerance. Furthermore, it is VLSI-compatible and does not require changes to existing VLSI design and manufacturing flows. A CNT can be semiconducting or metallic depending upon the arrangement of carbon atoms. Typical CNT synthesis techniques yield ~33% metallic CNTs. Metallic CNTs create source-drain shorts in CNFETs resulting in excessive leakage (Ion/Ioffon/Ioffin the range of 103-105, and overcome the limitations of existing metallic-CNT removal techniques. The above techniques are demonstrated for complex logic structures using wafer-scale growth of (99.5%) aligned CNTs on single-crystal quartz and wafer-scale CNT transfer from quartz to silicon. Such an integrated approach enables experimental demonstration of cascaded CNFET logic circuits.

Metallic carbon nanotubes (CNTs) pose a major barrier to the design of digital logic circuits using CNT field-effect transistors (CNFETs). Metallic CNTs create source to drain shorts in CNFETs, resulting in undesirable effects such as excessive leakage and degraded noise margins. No known CNT growth technique guarantees 0% metallic CNTs. Therefore, special processing techniques are required for removing metallic CNTs after CNT growth. This paper presents a probabilistic model which incorporates processing and design parameters and enables quantitative analysis of the impact of metallic CNTs on leakage, noise margin, and delay variations of CNFET-based digital logic circuits. With practical constraints on these key circuit performance metrics, the model provides design and processing guidelines that are required for very large scale integration (VLSI)-scale metallic-CNT-tolerant digital circuits.

Metallic carbon nanotubes (CNTs) create source-drain shorts in carbon nanotube field effect transistors (CNFETs), causing excessive leakage, degraded noise margin and delay variation. There is no known CNT growth technique that guarantees 0% metallic CNTs. Therefore, metallic CNT removal techniques are necessary. Unfortunately, such removal techniques alone are imperfect and insufficient. This paper demonstrates the necessity for co-optimization of processing techniques for metallic CNT removal together with CNFET-based circuit design. We present a probabilistic CNFET circuit model which forms the basis for such co-optimization, and use the model to derive design and processing guidelines that enable design of CNFET-based digital circuits with practical constraints on leakage, noise margin and delay variations. These guidelines are essential for designing robust metallic- carbon-nanotube-tolerant digital circuits.

Carbon-nanotube (CNT) field-effect transistors (CNFETs) are promising extensions to silicon CMOS. Simulations show that CNFET inverters fabricated with a perfect CNFET technology have 13 times better energy delay product compared with 32-nm silicon CMOS inverters. The following two fundamental challenges prevent the fabrication of CNFET circuits with the aforementioned advantages: 1) misaligned and mispositioned CNTs and 2) metallic CNTs. Misaligned and mispositioned CNTs can cause incorrect functionality. This paper presents a technique for designing arbitrary logic functions using CNFET circuits that are guaranteed to implement correct functions even in the presence of a large number of misaligned and mispositioned CNTs. Experimental demonstration of misaligned and mispositioned CNT-immune logic structures is also presented.

Carbon Nanotube Field-Effect Transistors (CNFETs) are promising candidates as extensions to Silicon CMOS due to excellent CV/I device performance. An ideal CNFET inverter fabricated using a perfect CNFET technology can have 5.1 times faster F04 delay and 2.6 times lower energy per cycle compared to a 32nm Silicon CMOS inverter. Two fundamental challenges prevent us from creating CNFET-based logic designs with the advantages quoted above: 1. Misaligned Carbon Nanotubes (CNTs), and 2. Metallic CNTs. Misaligned CNTs can result in incorrect logic function implementations. This paper presents a technique for designing CNFET-based arbitrary logic functions that are guaranteed to be correct even in the presence of a large number of misaligned CNTs.

Signature analyzers are designed to enable identification of failing test response bits directly from failing signatures, without any special diagnosis mode. This ability is useful for yield learning from the large volume of data available from failing chips during production test. The signature analyzers described also tolerate unknown logic values (X's) and are useful for built-in-self-test and test compression with yield analysis support. Actual defective chip data demonstrates the effectiveness of the presented techniques. Depending on the desired accuracy of failing response bit identification and the number of X's, test response data is reduced by up to two orders of magnitude