The long-term deployment of data-driven AI technology using artificial neural networks (ANNs) should be scalable and maintainable when new data becomes available. To insure smooth adaptation, the learning must be cumulative so that the network consumes new data without compromising its inference performance based on past data. Such incremental accumulation of learning experience is known as progressive learning. In this paper, we address the open problem of tuning the hyperparameters of neural networks during progressive learning. A hyper-parameter optimization framework is proposed that selects the best hyper-parameter values on a task-by-task basis. The neural network model adapts to each progressive learning task by adjusting the hyper-parameters under which the neural architecture is incrementally grown. Several hyper-parameter search strategies are explored and compared in support of progressive learning. In contrast to the predominant practice of using imaging datasets in machine learning, we have used cybersecurity datasets to illustrate the advantages of the proposed hyper-parameter tuning algorithms.

Synthesis and place & route tools are highly leveraged for modern digital design. But, despite continuous improvement in CAD tool performance, products in competitive markets often set PPA (performance, power, area) targets beyond what the tools can natively deliver. These aggressive targets lead to circuit designers attempting to tune a vast number of design flow parameters in search of near-optimal design specific flow recipes. Compounding the complex design flow parameter tuning problem is that many digital design tools exhibit nondeterminism, i.e., run-to-run variation. While CAD tool nondeterminism is typically considered an undesirable behavior, this paper proposes design flow tuning methodologies that take advantage of nondeterminism. We propose techniques that employ a combination of running targeted scenarios multiple times to exploit positive deviations nondeterminism can produce and leverage the best observed runs as seeds for multi-phase tuning. We introduce three seed variants for multi-phase tuning that have a spectrum of characteristics, trading off PPA improvement and reduce run-to-run variation. Our experimental analysis using high-performance industrial designs show that the proposed novel techniques outperform an existing state-of-the-art industrial design flow tuning program across all PPA metrics. Furthermore, our proposed approaches reduce run-to-run variation of the best scenarios, leading to a more predictable design flow.

Modern logic and physical synthesis tools provide numerous options and parameters that can drastically affect design quality; however, the large number of options leads to a complex design space difficult for human designers to navigate. Fortunately, machine learning approaches and cloud computing environments are well suited for tackling complex parameter tuning problems like those seen in VLSI design flows. This paper proposes a holistic approach where online and offline machine learning approaches work together for tuning industrial design flows. We describe a system called SynTunSys (STS) that has been used to optimize multiple industrial high-performance processors. STS consists of an online system that optimizes designs and generates data for a recommender system that performs offline training and recommendation. Experimental results show the collaboration between STS online and offline machine learning systems as well as insight from human designers provide best-of-breed results. Finally, we discuss potential new directions for research on design flow tuning.

The growing prevalence and computational demands of Artificial Intelligence (AI) workloads has led to widespread use of hardware accelerators in their execution. Scaling the performance of AI accelerators across generations is pivotal to their success in commercial deployments. The intrinsic error-resilient nature of AI workloads present a unique opportunity for performance/energy improvement through precision scaling. Motivated by the recent algorithmic advances in precision scaling for inference and training, we designed RaPiD1, a 4-core AI accelerator chip supporting a spectrum of precisions, namely, 16 and 8-bit floating-point and 4 and 2-bit fixed-point. The 36mm2RaPiD chip fabricated in 7nm EUV technology delivers a peak 3.5 TFLOPS/W in HFP8 mode and 16.5 TOPS/W in INT4 mode at nominal voltage. Using a performance model calibrated to within 1% of the measurement results, we evaluated DNN inference using 4-bit fixed-point representation for a 4-core 1 RaPiD chip system and DNN training using 8-bit floating point representation for a 768 TFLOPs AI system comprising 4 32-core RaPiD chips. Our results show INT4 inference for batch size of 1 achieves 3 - 13.5 (average 7) TOPS/W and FP8 training for a mini-batch of 512 achieves a sustained 102 - 588 (average 203) TFLOPS across a wide range of applications.

Advances in deep neural networks (DNNs) and the availability of massive real-world data have enabled superhuman levels of accuracy on many AI tasks and ushered the explosive growth of AI workloads across the spectrum of computing devices. However, their superior accuracy comes at a high computational cost, which necessitates approaches beyond traditional computing paradigms to improve their operational efficiency. Leveraging the application-level insight of error resilience, we demonstrate how approximate computing (AxC) can significantly boost the efficiency of AI platforms and play a pivotal role in the broader adoption of AI-based applications and services. To this end, we present RaPiD, a multi-tera operations per second (TOPS) AI hardware accelerator core (fabricated at 14-nm technology) that we built from the ground-up using AxC techniques across the stack including algorithms, architecture, programmability, and hardware. We highlight the workload-guided systematic explorations of AxC techniques for AI, including custom number representations, quantization/pruning methodologies, mixed-precision architecture design, instruction sets, and compiler technologies with quality programmability, employed in the RaPiD accelerator.

Logic synthesis and physical design (LSPD) tools automate complex design tasks previously performed by human designers. One time-consuming task that remains manual is configuring the LSPD flow parameters, which significantly impacts design results. To reduce the parameter-tuning effort, we propose an LSPD parameter recommender system that involves learning a collaborative prediction model through tensor decomposition and regression. Using a model trained with archived data from multiple state-of-the-art 14nm processors, we reduce the exploration cost while achieving comparable design quality. Furthermore, we demonstrate the transfer-learning properties of our approach by showing that this model can be successfully applied for 7nm designs.

Low voltage architecture and design are key enablers of high throughput per watt in heterogeneous, accelerator-rich many-core designs. However, such low voltage operation poses significant challenges because of difficulties in achieving reliable functionality of on-chip memories, particularly SRAMs at these design points. In this paper, we present a technique of low-voltage neural network acceleration, where the embedded SRAM architecture is equipped with a novel application-aware supply voltage boosting capability. This technique mitigates low-voltage induced failures, while enabling Very low voltage (VLV)1operation during most of the application run, resulting in substantial improvement in net energy efficiency. We present a framework to evaluate the impact of low-voltage SRAM errors on machine learning applications and characterize trade-offs between output inference accuracy and energy efficiency in our application-programmable supply boosted SRAM architecture. Using the proposed technique we push the limits on the minimum operable voltage (Vmin) for the desired output quality. As a proof of concept, we demonstrate these techniques on Dante, a Deep Neural Network (DNN) accelerator chip taped out in state-of-the art 14nm technology.

The combination of growth in compute capabilities and availability of large datasets has led to a re-birth of deep learning. Deep Neural Networks (DNNs) have become state-of-the-art in a variety of machine learning tasks spanning domains across vision, speech, and machine translation. Deep Learning (DL) achieves high accuracy in these tasks at the expense of 100s of ExaOps of computation; posing significant challenges to efficient large-scale deployment in both resource-constrained environments and data centers.

This paper is a tutorial-style introduction to a special session on: Effective Voltage Scaling in the Late CMOS Era. It covers the fundamental challenges and associated solution strategies in pursuing very low voltage (VLV) designs. We discuss the performance and system reliability constraints that are key impediments to VLV. The associated trade-offs across power, performance and reliability are helpful in inferring the optimal operational voltage-frequency point. This work was performed under the auspices of an ongoing DARPA program (named PERFECT) that is focused on maximizing system-level energy efficiency.

Modern logic and physical synthesis tools provide numerous options and parameters that can drastically impact design quality; however, the large number of options leads to a complex design space difficult for human designers to navigate. By employing intelligent search strategies and parallel computing we can tackle this parameter tuning problem, thus automating one of the key design tasks conventionally performed by a human designer. In this paper we present a novel learning-based algorithm for synthesis parameter optimization. This new algorithm has been integrated into our existing autonomous parameter-tuning system, which was used to design multiple 22nm industrial chips and is currently being used for 14nm chips. These techniques show, on average, over 40% reduction in total negative slack and over 10% power reduction across hundreds of 14nm industrial processor macros while reducing overall human design effort. We also present a new higher-level system that manages parameter tuning of multiple designs in a scalable way. This new system addresses the needs of large design teams by prioritizing the tuning effort to maximize returns given the available compute resources.

We propose an efficient physics-based mixed-mode statistical simulation methodology for nanoscale devices and circuits. Here, 3-D Technology Computer Aided Design models pose a barrier for efficient simulation of variability as they generally involve millions of nodes in their mesh representations. The proposed methodology, which has been implemented for FinFET/tri-gate static random access memory (SRAM) design, overcomes this barrier by leveraging advanced physics-based 2-D (P2-D) devices with optimized meshes that are derived from 3-D FinFET models with tuned device parasitics. This enables physics-based simulation as well as physics-based variability input parameters. To improve accuracy, an embedded automated flow enables extraction of all external nodal parasitics, directly from a 3-D FinFET circuit layout representation. The circuits consisting of advanced P2-D devices are then back annotated with the nodal parasitics to enable fast and accurate SRAM dynamic margin mixed-mode simulations. Results demonstrate up to 200× speedup compared with traditional 3-D device simulations, and around five orders of magnitude wall clock time improvement on account of fast statistical methodologies, which are superior in comparison with traditional Monte Carlo analysis. This makes it feasible to supplant often inaccurate compact model-based simulations by true mixed-mode device simulations in statistical engines. The proposed physics-based methodology is also shown to corroborate well with hardware measurements.

The design complexity of modern high performance processors calls for innovative design techniques and methodologies for achieving time-to-market goals. New design techniques are also needed to curtail power increases that inherently arise from ever increasing performance targets. This paper describes new processor design and optimization approaches that bridge the gap between high performance and low power. These techniques are flexible as they rely on automated synthesis-centric optimizations to enable power reduction without sacrificing performance. These methodology innovations contributed to the industry leading performance of the POWER8 processor.

In this paper, we present a novel algorithm for latch placement, LatchPlanner which enables a placement engine to deliver high quality placement for datapath-oriented design. Datapath-oriented VLSI designs are in general hand-crafted by human at high cost, as understanding and capturing datapath structure is critical for the performance. The conventional placement algorithms by itself cannot exploit the underlying datapath due to lack of logic structure recognition and inaccurate/approximated wirelength estimation. LatchPlanner addresses such drawbacks by placing and fixing latches in the datapath context, a key element in datapath structure. By taking placed/fixed latches as constraints, a placer can find a more datapath-friendly placement effectively, which results in higher-quality hardware. LatchPlanner begins latch clustering/sizing/ordering to prepare the following steps, a) global latch placement based on linear programming to place latch clusters, and b) local latch placement based on network flow optimization to place latches within each cluster. Experimental results on eighteen industrial benchmarks show that LatchPlanner improves total wirelength by 32%, total negative slack by 25%, and area by 3% without CPU overhead over a commercial placement engine, and delivers near semi-custom-quality solutions.

An increasing focus on design productivity is shifting microprocessor design to more synthesis-centric design methodologies. Low power design is also rising in importance, even for higher performance server chips. This paper proposes a design methodology capitalizing on the relatively low cost of parallel synthesis job submission for design space exploration. By tailoring the design flow for parallel and iterative design space exploration we attempt to maximize the return on investment (ROI) of design effort. The methodology was applied to the IBM POWER7+™ microprocessor to save power during the second release of the chip. This paper provides an overview of the methodology as well as chip hardware measurements showing power savings.

In deep sub-micro designs, more functions are integrated into one chip, and datapath has become a critical part of the design. Typical datapath consists an array of bit slices. The inherent high degree regularity of datapaths is especially attractive to the placement and routing to achieve regular layout with high density and high performance. However, the current design methodology may generate inferior datapath designs because the datapath regularity cannot be well understood by the traditional design tools. In previous works, several techniques are proposed to preserve/re-identify datapath structures. However, they either restrict the datapath optimization or have little tolerance on bit slice difference.

The time-to-market pressures combined with the immense power reduction design space of VLSI design call for an evaluation of power savings opportunities prior to the investment in design effort. This paper presents an estimation methodology for predicting the power savings of circuit tuning for an industrial chip design project. A comparison between the estimated and actual power savings realized through tuning over 100 macros on the chip validates the accuracy of this estimation methodology.

Regularly structured design techniques can combat complexity on a variety of fronts. We present the Structured and Tuned Array Generation (STAG) design methodology, which provides a complete design solution from logic to layout for regularly structured circuits. The STAG circuit tuning constraints are a key component of the methodology. The tuning contraints first guide a SPICE-level tuner to a violation free region in the design space. Secondly, the tuning methodology provides flexibility for targeting a variety of design contraints and objectives. Design examples illustrate STAG's ability for fast turnaround time as well as for high performance and timing critical random logic.

We consider sparsity, fanout, and radix as three dimensions in the design space of regular parallel prefix adders and present a unified formalism to describe such structures.

As the nanoelectronics field reaches the maturity needed for circuit-level integration, modeling approaches are needed that can capture nonclassical behaviors in a compact manner. This paper proposes a universal device model (UDM) for two-terminal devices that addresses the challenge of correctly balancing accuracy, complexity, and flexibility. The UDM qualitatively captures fundamental classical and quantum phenomena and enables nanoelectronic circuit design and simulation. We discuss the motivation behind this modeling approach as well as the underlying details of the model. Furthermore, we present a circuit example of the model in action.

As the dominating CMOS technology is fast approaching a "brick wall," new opportunities arise for competing solutions. Nanoelectronics has achieved several breakthroughs lately and promises to overcome many of the limitations intrinsic to current semiconductor approaches. Most of the results in this area reported until now focus on devices and interconnect; this work goes several steps further and presents issues related to circuits and architecture. Based on proposed nanoscale interconnect and device structures, we explore the design space available to the nanoelectronic circuit designer and system architect.

The challenge of extending Moore's Law past the physical and economic barriers of present semiconductor technologies calls for novel nanoelectronic solutions. Circuits composed of mixed silicon semiconductors and nanoelectronics can provide a means for gradually switching technology paradigms. We suggest a design methodology to accompany this concept. Furthermore, we explore design tradeoffs for a nanoscale crossbar technology that supports CMOS/nano co-design.