The semiconductor industry is steadily shifting toward disaggregated system design to overcome the scalability (yield and cost) limitations of monolithic integration. 2.5D integration offers a compelling pathway for realizing such systems, enabling the assembly of heterogeneous chiplets—compute, memory, analog & I/O, with dense interconnects and high bandwidth. Our previous work showcased experimental results from a multi-foundry chiplet design over a large passive silicon base. There were 20 chip slots (CS) on an interposer that can be configured with compute die (CD) or memory die (MD) chiplets. Building on this foundation, this paper introduces a methodology for system technology co-optimization (STCO) across several vectors. These include varying the number of chip slots, configuring slots with different numbers of MDs and CDs, sweeping the inter-die bandwidth, choosing between different technology nodes for the performance limiting MDs. This work demonstrates how the design and technology choices affect system performance, power, and cost across different workloads, empowering designers to select optimal configurations for their specific needs.

This article presents 3DNN-Xplorer, the first machine learning (ML)-based framework for predicting the performance of heterogeneous 3-D deep neural network (DNN) accelerators. Our ML framework facilitates the design space exploration (DSE) of heterogeneous 3-D accelerators with a two-tier compute-on-memory (CoM) configuration, considering 3-D physical design factors. Our design space encompasses four distinct heterogeneous 3-D integration styles, combining 28- and 16-nm technology nodes for both compute and memory tiers. Using extrapolation techniques with ML models trained on 10-to-256 processing element (PE) accelerator configurations, we estimate the performance of systems featuring 75–16384 PEs, achieving a maximum absolute error of 13.9% (the number of PEs is not continuous and varies based on the accelerator architecture). To ensure balanced tier areas in the design, our framework assumes the same number of PEs or on-chip memory capacity across the four integration styles, accounting for area imbalance resulting from different technology nodes. Our analysis reveals that the heterogeneous 3-D style with 28-nm compute and 16-nm memory is energy-efficient and offers notable energy savings of up to 50% and an 8.8% reduction in runtime compared to other 3-D integration styles with the same number of PEs. Similarly, the heterogeneous 3-D style with 16-nm compute and 28-nm memory is area-efficient and shows up to 8.3% runtime reduction compared to other 3-D styles with the same on-chip memory capacity.

Heterogeneous 3D integration, blending multiple technology nodes, emerges as a promising strategy for enhancing performance and maintaining low power consumption in next-generation computing systems. This paper presents Hetero-3D, an RTL-to-GDS design flow tailored specifically for heterogeneous 3D ICs. Hetero-3D integrates an area-unbalanced 3D floorplanner with an ML-based power delivery and signal router, working in tandem for rigorous PPA (Power, Performance, and Area) optimization. Using two CPU benchmarks, we showcase a remarkable 15% increase in maximum frequency and a substantial 50% decrease in voltage drop compared to homogeneous 3D baselines. Moreover, Hetero-3D effectively addresses voltage drop issues in the power delivery network while delivering an additional 5% frequency boost. This study emphasizes the EDA solutions that unlock the potential of mixed-node stacking as a crucial enabler for performance scaling in future ICs.

This paper presents 3DNN-Xplorer, the first machine learning (ML)-based framework for predicting the performance of heterogeneous 3D DNN accelerators. Our ML frame-work facilitates the design space exploration of heterogeneous 3D accelerators with a 2-tier compute-on-memory configuration, considering 3D physical design factors. Our design space encompasses four distinct heterogeneous 3D integration styles, combining 28nm and 16nm technology nodes for both compute and memory tiers. Using extrapolation techniques with ML models trained on 16, 32, and 64 PE accelerator configurations, we estimate the performance of systems featuring 128, 256, 512, 1024, and 2048 PEs, achieving a maximum absolute error of 12.7%. To ensure balanced tier areas in the design, our framework assumes the same number of PEs or on-chip memory capacity across the four integration styles, accounting for area imbalance resulting from different technology nodes. Our analysis reveals that the heterogeneous 3D style with 28nm compute and 16nm memory is energy-efficient and offers notable energy savings of up to 50% and an 8.8% reduction in runtime compared to other 3D integration styles with the same number of PEs. Similarly, the heterogeneous 3D style with 16nm compute and 28nm memory is area-efficient and shows up to 8.3% runtime reduction compared to other 3D styles with the same on-chip memory capacity.

D Place and Route (P&R) flows either involve true-3D placement algorithms or use commercial 2D tools to transform a 2D design into a 3D design. Irrespective of the nature of the placers, several placement parameters in these tools affect the quality of the final 3D designs. Different parameter settings work well with different circuits, and it is impossible to manually tune them for a particular circuit. Automated approaches involving reinforcement learning have been shown to adapt and learn the parameter settings and create trained models. However, their effectiveness depends on the input dataset quality. Using a set of 10 netlists and 10–21 handpicked placement parameters in P&R flows involving pseudo-3D or true-3D placement, the dataset quality is analyzed. The datasets are the design metrics obtained through different P&R stages, such as placement optimization, clock tree synthesis, or 3D partitioning and global routing. The training runtime and the quality of the final design metrics are compared. On a pseudo-3D flow, the training takes around 126–290 hours, whereas, on a true-3D placer-based flow, it takes around 305–410 hours. It is observed that the datasets obtained from different stages lead to drastically different final design results. With the RL-based training processes, the quality of results in 3D designs improves by up to 23.7% compared to their corresponding untrained P&R flows.

This work identifies the architectural and design scaling limits of 2-D flexible interconnect deep neural network (DNN) accelerators and addresses them with 3-D ICs. We demonstrate how scaling up a baseline 2-D accelerator in the$X/Y$dimension fails and how vertical stacking effectively overcomes the failure. We designed multitier accelerators that are$1.67\times $faster than the 2-D design. Using our 3-D architecture and circuit codesign methodology, we improve throughput, energy efficiency, and area efficiency by up to$5\times $,$1.2\times $, and$3.9\times $, respectively, over 2-D counterparts. The IR-drop in our 3-D designs is within 10.7% of VDD, and the temperature variation is within 12 °C.

In this paper, we show that true 3D placement approaches, enhanced with reinforcement learning, can offer further PPA improvements over pseudo-3D approaches. To accomplish this goal, we integrate an academic true 3D placement engine into a commercial-grade 3D physical design flow, creating ART-3D flow (Analytical 3D Placement with Reinforced Parameter Tuning-based 3D flow). We use a reinforcement learning (RL) framework to find optimized placement parameter settings of the true 3D placement engine for a given netlist and perform high-quality 3D placement. We then use an efficient 3D optimization and routing engine based on a commercial place and route (P&R) tool to maintain or improve the benefits reaped from true 3D placement till design signoff. We evaluate our 3D flow by designing several gate-only and processor benchmarks on a commercial 28nm technology node. Our proposed 3D flow involving true 3D placement offers the best PPA results compared to existing 3D P&R flows and reduces power consumption by up to 31%, improves effective frequency by up to 25%, and therefore reduces power-delay product by up to 43% compared with commercial 2D IC design flow. These improvements predominantly come from RL-based parameter tuning, as it improves the performance of the 3D placer by up to 12%.

Monolithic 3D (M3D) is an emerging heterogeneous integration technology that overcomes the limitations of the conventional through-silicon-via (TSV) and provides significant performance uplift and power reduction. However, the ultra-dense 3D interconnects impose significant challenges during physical design on how to best utilize them. Besides, the unique low-temperature fabrication process of M3D requires dedicated design-for-test mechanisms to verify the reliability of the chip. In this article, we provide an in-depth analysis on these design and test challenges in M3D. We also provide a comprehensive survey of the state-of-the-art solutions presented in the literature. This article encompasses all key steps on M3D physical design, including partitioning, placement, clock routing, and thermal analysis and optimization. In addition, we provide an in-depth analysis of various fault mechanisms, including M3D manufacturing defects, delay faults, and MIV (monolithic inter-tier via) faults. Our design-for-test solutions include test pattern generation for pre/post-bond testing, built-in-self-test, and test access architectures targeting M3D.

Tier partitioning is one of the most critical stages in monolithic 3-D (M3D) integrated circuits (ICs) implementation flows. It transforms 2-D netlists into 3-D by performing tier assignment for each design instance, which directly impacts the power, performance, and area (PPA) metrics of final 3-D full-chip designs. However, the current state-of-the-art tier partitioning approach named bin-based min-cut algorithm has fundamental flaws that lead to severe drawbacks, such as timing degradation, 3-D routing overhead, and redundant monolithic intertier vias (MIVs) insertion. To overcome these issues, in this article, we propose TP-GNN, an unsupervised graph learning-based tier partitioning framework that utilizes graph neural networks (GNNs) and advanced machine learning (ML) techniques to perform tier partitioning. The proposed framework comprehends design- and technology-related parameters properly so that it is generalizable to various netlists and technologies. In addition, it can be integrated with any style of M3D design flows that require tier assignments of standard cells. In the experiments, we validate the proposed framework on seven industrial designs with two different fashions of M3D implementation flows: 1) partitioning-first (Snap3D) and 2) partitioning-last (Shrunk2D and Compact2D) flows. We demonstrate that our framework, TP-GNN, significantly improves the 3-D quality of results (QoR) across most testing designs in a large margin compared with the bin-based min-cut tier partitioning algorithm. Specifically, in OpenPiton, an RISC-V-based multicore system, we observe 27.4%, 7.7%, and 20.3% improvements in performance, wirelength, and energy-per-cycle, respectively. Finally, we perform a case study by applying the proposed framework to a heterogeneous M3D design flow, Pin3D, on a commercial CPU design and observe that TP-GNN reaches better partitioning solutions than the existing partitioning approaches for heterogeneous 3-D ICs.

One of the advantages of 3D IC technology is its ability to integrate different devices such as CMOS, SRAM, and RRAM, or multiple technology nodes of single or different devices onto a single chip due to the presence of multiple tiers. This ability to create heterogeneous 3D ICs finds a wide range of applications, from improving processor performance by integrating better memory technologies to building compute-in-memory ICs to support advanced machine learning algorithms. This paper discusses the current trends and future directions for the physical design of heterogeneous 3D ICs. We summarize various physical design and optimization flows, integration techniques, and existing academic works on heterogeneous 3D ICs.

Carbon nanotube FETs (CNFETs) are emerging as an alternative to silicon devices for next-generation computing systems. However, imperfect carbon nanotube deposition during CNFET fabrication can lead to the formation of difficult-to-etch CNT aggregates in the active layer. These CNT aggregates can form parasitic CNFETs (para-FETs) that are modulated by adjoining gate contacts or back-end-of-line metal layers, thereby forming conditional shorts and stuck-at faults. We show that even weak (parametric) para-FETs can lead to a degraded static noise margin in CNFET-based design. We propose ParaMitE, a layout optimization method that horizontally flips selected standard cells in situ to minimize the number of para-FETs that can arise due to unetched CNTs. As we modify only the cell orientation (and not the cell placement), the impact on the power, timing, and wire length of the CNFET-based design is negligible. Simulation results for several benchmarks show that the proposed method can mitigate up to 60% of the possible para-FET locations (90% of the most critical locations) with only a 3% increase in the total wire length. ParaMitE can enable yield ramp-up at the foundry by providing guidance on which para-FETs can be avoided by design, and conversely, which CNT aggregates must be removed through processing steps.

The 2-D CMOS process technology scaling may have reached its pinnacle, yet it is not feasible to manufacture all computing elements at lower technological nodes. This has opened a new branch of chip designing that allows chiplets on different technological nodes to be integrated into a single package using interposers, the passive interconnection mediums. However, establishing a high-frequency communication over an entirely passive layer is one of the significant design challenges of 2.5-D systems. In this article, we present a robust clocking architecture for a 2.5-D system consisting of 64 processor cores. This clocking scheme consists of two major components, namely, interposer clocking and on-chiplet clocking. The interposer clocking consists of clocks used to achieve global synchronicity and clocks for interchiplet communication established using the AIB protocol. We synthesized these clocking components using commercial EDA tools and analyzed them using standard tools, on-chip, and package models. We also compare these results against a 2-D design of the same benchmark and another 2.5-D clocking architecture. Our experiments show that the absolute clock power is up to 16% less, and the ratio of clock power to system power is up to 4% less in the 2.5-D design than its 2-D counterpart.

Resistive random access memory (RRAM)-based compute-in-memory architecture helps overcome the bottleneck caused by large memory transactions in the convolutional neural network (CNN) accelerators. However, their deployment using 2-D IC technology faces challenges, as today's RRAM cells remain at legacy nodes above 20 nm due to high programming voltages. Besides, power-hungry analog-to-digital converter (ADC) units limit the throughput of RRAM accelerators. In this article, we present the first-ever heterogeneous (multiple technology nodes) mixed-signal monolithic 3-D IC designs of the RRAM CNN accelerator. Our RRAM remains at legacy 40-nm nodes in one tier, but CMOS periphery scales toward advanced 28/16 nm in another tier. Our 3-D designs overcome the bottleneck caused by ADCs and offer up to 4.9× improvement in energy efficiency in TOPS/W and up to 50% reduction in footprint area over 40-nm 2-D IC designs. Compared with existing 2-D works, our 3-D architecture offers up to 28.6× improvement in energy efficiency.

In this paper, we propose RTL-to-GDS design flow for monolithic 3D ICs (M3D) built with carbon nanotube field-effect transistors and resistive memory. Our tool flow is based on commercial 2D tools and smart ways to extend them to conduct M3D design and simulation. We provide a post-route optimization flow, which exploits the full potential of the underlying M3D process design kit (PDK) for power, performance and area (PPA) optimization. We also conduct IR-drop and thermal analysis on M3D designs to improve the reliability. To enhance the testability of our M3D designs, we develop design-for-test (DFT) methodologies and integrate a low-overhead built-in self-test module into our design for testing inter-layer vias (ILVs) as well as logic circuitries in the individual tiers. Our benchmark design is RISC-V Rocketcore, which is an open source processor. Our experiments show 8.1% of power, 19.6% of wirelength and 55.7% of area savings with M3D designs at iso-performance compared to its 2D counterpart. In addition, our IR-drop and thermal analyses indicate acceptable power and thermal integrity in our M3D design.

Compute in-memory (CIM) is a promising technique that minimizes data transport, the primary performance bottleneck and energy cost of most data intensive applications. This has found wide-spread adoption in accelerating neural networks for machine learning applications. Utilizing a crossbar architecture with emerging non-volatile memories (eNVM) such as dense resistive random access memory (RRAM) or phase change random access memory (PCRAM), various forms of neural networks can be implemented to greatly reduce power and increase on chip memory capacity. However, compute in-memory faces its own limitations at both the circuit and the device levels. Although compute in-memory using the crossbar architecture can greatly reduce data transport, the rigid nature of these large fixed weight matrices forfeits the flexibility of traditional CMOS and SRAM based designs. In this work, we explore the different synchronization barriers that occur from the CIM constraints. Furthermore, we propose a new allocation algorithm and data flow based on input data distributions to maximize utilization and performance for compute-in memory based designs. We demonstrate a$\boldsymbol{7.47}\times$performance improvement over a naive allocation method for CIM accelerators on ResNet18.

A new trend in system-on-chip (SoC) design is chiplet-based IP reuse using 2.5-D integration. Complete electronic systems can be created through the integration of chiplets on an interposer, rather than through a monolithic flow. This approach expands access to a large catalog of off-the-shelf intellectual properties (IPs), allows reuse of them, and enables heterogeneous integration of blocks in different technologies. In this article, we present a highly integrated design flow that encompasses architecture, circuit, and package to build and simulate heterogeneous 2.5-D designs. Our target design is 64core architecture based on Reduced Instruction Set Computer (RISC)-V processor. We first chipletize each IP by adding logical protocol translators and physical interface modules. We convert a given register transfer level (RTL) for 64-core processor into chiplets, which are enhanced with our centralized network-onchip. Next, we use our tool to obtain physical layouts, which is subsequently used to synthesize chip-to-chip I/O drivers and these chiplets are placed/routed on a silicon interposer. Our package models are used to calculate power, performance, and area (PPA) and reliability of 2.5-D design. Our design space exploration (DSE) study shows that 2.5-D integration incurs 1.29× power and 2.19× area overheads compared with 2-D counterpart. Moreover, we perform DSE studies for power delivery scheme and interposer technology to investigate the tradeoffs in 2.5-D integrated chip (IC) designs.

A new trend in complex SoC design is chiplet-based IP reuse using 2.5D integration. In this paper we present a highly-integrated design flow that encompasses architecture, circuit, and package to build and simulate heterogeneous 2.5D designs. We chipletize each IP by adding logical protocol translators and physical interface modules. These chiplets are placed/routed on a silicon interposer next. Our package models are then used to calculate PPA and signal/power integrity of the overall system. Our design space exploration study using our tool flow shows that 2.5D integration incurs 2.1x PPA overhead compared with 2D SoC counterpart.