The transformative power of Large Language Models (LLMs) is reshaping the role of AI in post-silicon test engineering. This paper summarizes our experience in leveraging LLMs to develop AI agents specifically tailored for this domain. Central to our approach is a two-stage process: first, we utilize the reasoning capabilities of LLMs to systematically interpret user queries; second, we invoke a grounding process to execute tasks as directed by these queries. This grounding ensures seamless integration of the LLM with existing test engineering infrastructure, enabling the AI agent to autonomously perform tasks within an established framework. Using the Intelligent Engineering Assistant (IEA) as a case study, we demonstrate how domain-specific, LLM-powered AI agents can automate critical aspects of test engineering, and highlight the potential of LLMs to revolutionize post-silicon test engineering through intelligent, context-aware automation.

This paper introduces IEA-plugin, a novel AI agent-based reasoning module developed as a new front-end for the Intelligent Engineering Assistant (IEA). The primary objective of IEA-plugin is to utilize the advanced reasoning and coding capabilities of Large Language Models (LLMs) to effectively address two critical practical challenges: capturing diverse engineering requirements and improving system scalability. Built on the LangGraph agentic programming platform, IEA-plugin is specifically tailored for industrial deployment and integration with backend test data analytics tools. Compared to the previously developed IEA-Plot (introduced two years ago), IEA-plugin represents a significant advancement, capitalizing on recent breakthroughs in LLMs to deliver capabilities that were previously unattainable.

This paper introduces WM-Graph, a novel approach designed for flexible analytics of wafermaps. The key concept behind WM-Graph is the construction of a wafermap graph, where individual wafermaps are connected if they exhibit semi-equivalence. This graph-based structure allows a wide range of analytics to be performed using established graph algorithms. Unlike traditional multi-class classification methods, WM-Graph enables more versatile analyses, making it possible to answer complex, practical questions that would otherwise be difficult to address. We explain the technical innovations that underpin the WM-Graph approach and demonstrate how to perform certain analytical tasks with simple graph operations. The effectiveness of the WM-Graph approach is validated through experiments using the public WM-811K dataset and a proprietary dataset from a recent production line.

This paper presents key ideas behind IEA-Plot, a software framework designed to conduct test data analytics through chat. We use wafer-based data analytics as an application example to discuss the ideas. IEA-plot interacts with a user through a dialog and produces plots according to user instructions. At the core of IEA-Plot is a knowledge graph connecting a frontend natural language parser to a backend API. This knowledge graph captures our analytics knowledge in the specific context. Usage examples are presented based on test data collected from a recent production line.

This volume contains the papers presented at the 2023 International Test Conference, held from October 10 - 12 at the Disneyland Hotel, Anaheim California. ITC is the world's premier conference dedicated to electronic test. This year's ITC continued with its mission to play a unique role as an information sharing forum, where the wide range of its offerings allows ITC participants to learn, network and conduct business. This year's program included a top-notch technical program, vibrant exhibitors, informationpacked tutorials, interactive technical panels, three focused workshops, as well as the all-important networking that these events provide. The technical program was designed to optimize personal interactions on all levels.

We present a novel approach where wafer map pattern analytics are driven by natural language queries. At the core is a semantic parser that translates a user query into a meaning representation comprising instructions to generate a summary plot. The allowable plot types are pre-defined which serve as an interface that communicates user intents to the analytics software backend. Application results on wafer maps from a recent production line are presented to explain the capabilities and benefits of the proposed approach.

In the context of analyzing wafer maps, we present a novel approach to enable analytics to be driven by user queries. The analytic context includes two aspects: (1) grouping wafer maps based on their failure patterns and (2) for a failure pattern found at wafer probe, checking to see whether there is a correlation to the result from the final test (feedforward) and to the result from the E-test (feedback). We introduce language driven analytics and show how a formal language model in the backend can enable natural language queries in the frontend. The approach is applied to analyze test data from a recent product line, with interesting findings highlighted to explain the approach and its use.

We present a novel approach called MINiature Interactive Offset Networks (or MINIONs). We use wafer map classification as an application example. A Minion is trained with a specially-designed one-shot learning scheme. A collection of Minions can be used to patch a master model. Experiment results are provided to explain the potential areas Minions can help and their unique benefits.

In this work, we consider learning a wafer plot recognizer where only one training sample is available. We introduce an approach called Manifestation Learning to enable the learning. The underlying technology utilizes the Variational AutoEncoder (VAE) approach to construct a so-called Manifestation Space. The training sample is projected into this space and the recognition is achieved through a pre-trained model in the space. Using wafer probe test data from an automotive product line, this paper explains the learning approach, its feasibility and limitation.

Robustness is a key requirement for deploying a machine learning (ML) based solution. When a solution involves a ML model whose robustness is not guaranteed, ensuring robustness of the solution might rely on continuous checking of the ML model for its validity after the solution is deployed in production. Using wafer image classification as an example, this paper introduces tensor-based methods that help improve robustness of a neural-network-based classification approach and facilitate its deployment. Experiment results based on data from a commercial product line are presented to explain the key ideas behind the tensor-based methods.

This paper presents an automated flow to classify wafer plots obtained based on production test data. The wafer plots are based on pass/fail locations. The classification is achieved through wafer pattern recognition models built with two sets of techniques, Generative Adversarial Networks and Tensor analysis. The primary focus is on developing the automatic flow. Experiment results based on production test data from a microcontroller product line will be presented to demonstrate the usefulness of the proposed classification flow.

A machine learning (ML) solution can be non-robust and when it is deployed, can make mistakes on the future unseen data. Consequently, deployment of a ML solution might demand continuous service from its ML developer. Using wafer image classification as an example, this paper presents the design of a ML solution where its deployment is facilitated by the continuous service from its ML expert.

In production test data analytics, it is often that an analysis involves the recognition of a conceptual pattern on a wafer map. A wafer pattern may hint a particular issue in the production by itself or guide the analysis into a certain direction. In this work, we introduce a novel approach to recognize patterns on a wafer map of pass/fail locations. Our approach utilizes Tucker decomposition to find projection matrices that are able to project a wafer pattern represented by a small set of training samples into a nearly-diagonal matrix. Properties of such a matrix are utilized to recognize wafers with a similar pattern. Also included in our approach is a novel method to select the wafer samples that are more suitable to be used together to represent a conceptual pattern in view of the proposed approach.

Applying machine learning in Electronic Design Automation (EDA) has received growing interests in recent years. One approach to analyze data in EDA applications can be called feature-based analytics. In this context, the paper explains the inadequacy of adopting a traditional machine learning problem formulation view. Then, an alternative machine learning view is suggested where learning from data is treated as an iterative search process. The theoretical and practical considerations for implementing such a search process are discussed in the context of various applications.

An analytic process is iterative between two agents, an analyst and an analytic toolbox. Each iteration comprises three main steps: preparing a dataset, invoking an analytic tool, and evaluating the result, where dataset preparation and result evaluation, conducted by the analyst, are largely domain-knowledge driven. In this work, the focus is on automating the result evaluation step. In particular, we consider the problem to recognize plots that are deemed interesting by an analyst. We propose a methodology to learn such analyst's intent based on Generative Adversarial Networks (GANs) and demonstrate its applications in the context of production yield optimization using data collected from several product lines.

Applying machine learning in design and test has been a growing field of interest in recent years. Many potential applications have been demonstrated and tried. This tutorial paper provides a review of author's experience in this field, in particular how the perspective for applying machine learning evolved from one stage to another where the latest perspective is based on an autonomous system view to apply machine learning. The theoretical and practical barriers leading to this view are highlighted with selected applications.

The special session focuses on using Machine Learning (ML) techniques on different applications in test and diagnosis. The first contribution discusses how to close the gap between working silicon and a working system by using ML. The second presentation then talks an alternative ML view and its various applications such as functional verification, Fmax prediction, and production yield optimization. The last presentation discusses using supervised ML on volume diagnosis to further improve the accuracy of identifying root causes.

Feature selection is essential to rule learning in the context of functional verification. In practice today, features are selected manually and the selection requires domain knowledge. In contrast, this work proposes using automatic feature extraction from design documents as a viable approach to support rule learning. To demonstrate its effectiveness, document-extracted features are employed to learn the rules for covering a set of assertions based on a commercial SoC. Experiments show that 100%-accurate rules can be obtained for more than 70% of the assertions.

Security verification relies on using direct tests manually prepared. Test preparation often requires intensive efforts from experts with in-depth domain knowledge. This work presents an approach to learn from direct tests written by an expert. After the learning, the learned model acts as a surrogate for the expert to produce new tests. The learning software comprises a database for accumulating and sharing security verification knowledge. The learning approach uses process discovery to build an upper-bound model and continuously adds constraints to refine it. We demonstrate the feasibility and effectiveness of the learning approach in a commercial SoC verification environment.

This work studies a data-driven methodology for detecting systematic defects using layout-aware scan diagnosis data. As part of volume diagnosis, this methodology focuses on ranking the most systematic defective signatures, while possible random defects are also present in the wafer. The main analysis components utilize χ2Independence Tests to establish systematic relationships between reported defective signatures and defects, while data clustering and net repeating signals are used to amplify these systematic defective signals. Based on consensus results from the analysis components, the methodology provides physical candidates to facilitate the discovery of potential yield limiters. Finally, methodology effectiveness and evaluations are presented through application on production wafers from three 14nm products.

With wafer fabs running at near full capacity, it is a constant challenge to maintain high yields. Many different products are fabricated by the same equipment. So the sudden change in product yield, a yield excursion, can have a significant impact to many different products. Therefore, it is critical to detect an excursion as early as possible and fix the cause in order to minimize the impact. This paper introduces a two-part methodology for excursion detection and quality improvement. This methodology is based on a novel method called Kernel Based Clustering. First, a screening method will be described for removing die in close proximity to the cluster of failing dies. Second, a cluster commonality methodology will be described for detecting common clusters in terms of shape, region on the wafer and failure mode. This methodology was evaluated with 40k wafers from a 30-week production period. These wafers came from 15 different products developed on the same technology. During this period, a process excursion occurred that impacted many of these products. It will be shown that the kernel based clustering algorithm effectively identifies and removes high-risk dies around the failure cluster. It will also be shown that common clusters can be identified across multiple products and with this capability, the time to detection can be reduced.

Outlier screening is a popular approach employed for automotive product lines. There have been many outlier methods proposed. In practice, it is desirable to choose the “best” outlier method. This work develops a notion of applicability associated with an outlier method on a given set of wafers. A measure for applicability is proposed and experiment results are presented to illustrate its effects for finding outliers and for analyzing customer returns based on data collected from several automotive product lines.

An analytics process is subjective to the perspective of the analyst. This paper presents a learning approach that models the process of how an analyst conducts analytics. The approach is applied in the context of correlation analysis for production yield optimization. The benefit is demonstrated by showing that learning from resolving a yield issue for one automotive product line can help resolve a yield issue for another automotive product line.

Applying modern data mining in electronic design automation and test has become an area of growing interest in recent years. This paper reviews some of the recent developments in the area. It begins by introducing several key concepts in machine learning and data mining, followed by a review of different learning approaches. Then, the experience of developing a practical data mining application is described, including promises demonstrated through positive results based on industrial settings and challenges explained in the respective application contexts. Future research directions are summarized at the end.

Summary form only given. Modern IC design and manufacturing have progressed in leaps and bounds, resulting in unimaginable integration, and power-performance advancements. This progress has been accompanied by adverse design-layout-process interactions and increased defect sensitivity. Controlling these complex interactions has exacted a steep price in terms of delaying yield ramp, extending silicon validation to characterize and fix marginal effects, and test screening being overwhelmed in time and volume to be able to ensure outgoing customer quality. There is, however, a bright side. Each step of the manufacturing, validation and test process generates information. This information, if effectively organized and analyzed, has the potential to result in efficiency and quality improvements that parallel in scale to the manufacturing process itself. Recent developments in data analytics methods have enabled harnessing of this information towards some benefits, while promising much more. This panel will explore major problems that can potentially be solved with advanced analytics, and also current solutions in the market targeted at some of these problems. Experts from integrated and fabless design houses will present their perspective on problems they encounter, while vendors of EDA solutions on data analytics will shed light on the nature of problems solved by current methods and those that will be addressed by solutions to come.

Outlier screening is a popular approach for testing automotive products. In practice, developing an outlier model can be subjective, making justification of the model challenging. In this paper we propose a new concept called Consistency which provides a data-driven objective way to assess an outlier model. We study the development of outlier models in view of this new model consistency concept and report experimental findings on an automotive product line.

This paper reviews the data mining methodologies proposed for functional test content optimization where tests are sequences of instructions or transactions. Basic machine learning concepts and the key ideas of these methodologies are explained. Challenges for implementing these methodologies in practice are illustrated. Promises are demonstrated through experimental results based on industrial verification settings.

This paper describes two separate learning flows for improving the efficiency of simulation-based design analysis. Machine learning concepts and methods are explained in the context of realizing the two learning flows. Experimental results are presented to demonstrate their feasibility. Generality of the proposed learning flows is illustrated using the kernel-based learning concept.

In this work, we study the generalization of an outlier model from two perspectives, temporal and spatial. We show that model generalization with existing distribution-based outlier analysis methods can vary significantly. We then propose a “big data” outlier analysis approach together with a probability-based outlier evaluation for improving model generalization. Experiments are conducted based on two automotive product lines to explain the concepts and demonstrate the effectiveness of the proposed approach.

This paper discusses the basic principles of applying data mining in Electronic Design Automation. It begins by introducing several important concepts in statistical learning and summarizes different types of learning algorithms. Then, the experience of developing a practical data mining application is described, including promises that are demonstrated through positive results based on industrial settings and constraints explained in their respective application contexts.

This paper investigates how data mining can be applied in functional debug, which is formulated as the problem of explaining a functional simulation error based on human-understandable machine states. We present a rule discovery methodology comprising two steps. The first step selects relevant state variables for constructing the mining dataset. The second step applies rule learning to extract rules that differentiates the tests that excite error behavior from those that do not. We explain the dependency of the second step on the first step and considerations for implementing the methodology in practice. Application of the proposed methodology is illustrated through experiments conducted on a recent commercial SoC design.

Univariate outlier analysis has become a popular approach for improving quality. When a customer return occurs, multivariate outlier analysis extends the univariate analysis to develop a test model for preventing similar returns from happening. In this context, this work investigates the following question: How simple multivariate outlier modeling can be? The interest for answering this question are twofold: (1) to facilitate implementation of a test model in test application and (2) to ensure robustness of the methodology. In this work, we explain that based on a Gaussian assumption, a simpler covariance-based outlier analysis approach can be sufficient over a more complex density-based approach such as one-class SVM. We show that correlation among tests can be a good metric to rank potential outlier models. Based on these observations a simple outlier analysis methodology is developed and applied to effectively analyze customer returns from two automotive product lines.

This work presents a novel yield optimization methodology based on establishing a strong correlation between a group of fails and an adjustable process parameter. The core of the methodology comprises three advanced statistical correlation methods. The first method performs multivariate correlation analysis to uncover linear correlation relationships between groups of fails and measurements of a process parameter. The second method partitions a dataset into multiple subsets and tries to maximize the average of the correlations each calculated based on one subset. The third method performs statistical independence test to evaluate the risk of adjusting a process parameter. The methodology was applied to an automotive product line to improve yield. Five process parameter changes were discovered which led to significant improvement of the yield and consequently significant reduction of the yield fluctuation.

This work proposes a methodology of knowledge extraction from constrained-random simulation data. Feature-based analysis is employed to extract rules describing the unique properties of novel assembly programs hitting special conditions. The knowledge learned can be reused to guide constrained-random test generation towards uncovered corners. The experiments are conducted based on the verification environment of a commercial processor design, in parallel with the on-going verification efforts. The experimental results show that by leveraging the knowledge extracted from constrained-random simulation, we can improve the test templates to activate the assertions that otherwise are difficult to activate by extensive simulation.

This talk discusses several application examples to illustrate the basic principles of applying data mining in design and test. Two types of data mining are seen in most of the applications: novelty detection and feature-based rule learning. The experience of developing a practical data mining flow is summarized. Promises are demonstrated with positive experimental results based on industrial settings.

This work presents three pattern mining methodologies for inter-wafer abnormality analysis. Given a large population of wafers, the first methodology identifies wafers with abnormal patterns based on a test or a group of tests. Given a wafer of interest, the second methodology searches for a test perspective that reveals the abnormality of the wafer. Given a particular pattern of interest, the third methodology implements a monitor to detect wafers containing similar patterns. This paper discusses key elements for implementing each of the methodologies and demonstrates their usefulness based on experiments applied to a high-quality SoC product line.

Novel test detection is an approach to improve simulation efficiency by selecting novel tests before their application [1]. Techniques have been proposed to apply the approach in the context of processor verification [2]. This work reports our experience in applying the approach to verifying a commercial processor. Our objectives are threefold: to implement the approach in a practical setting, to assess its effectiveness and to understand its challenges in practical application. The experiments are conducted based on a simulation environment for verifying a commercial dual-thread low-power processor core. By focusing on the complex fixed-point unit, the results show up to 96% saving in simulation time. The main limitation of the implementation is discussed based on the load-store unit with initial promising results to show how to overcome the limitation.

One of the challenges of functional test content optimization, in the context of performance validation, is to predict from a high level model an event of interest observed in either a detailed simulation or in silicon testing. This work uses peak power validation as an example to study the potential of using learning algorithms to uncover the correlations between the different levels of abstraction. Using the OpenSPARC T2 microprocessor as the driving example, we have studied the use of three learning algorithms for building models to explain the events of interest in the output of a power simulation. These models are built based on features extracted from a high-level view of the design. We show that the learned models can be used to select assembly programs that are likely to produce similar interesting events, and also can be used to produce constrained random assembly programs capable of exposing the events of our interest.

This work studies the potential of capturing customer returns with models constructed based on multivariate analysis of parametric wafer sort test measurements. In such an analysis, subsets of tests are selected to build models for making pass/fail decisions. Two approaches are considered. A preemptive approach selects correlated tests to construct multivariate test models to screen out outliers. This approach does not rely on known customer returns. In contrast, a reactive approach selects tests relevant to a given customer return and builds an outlier model specific to the return. This model is applied to capture future parts similar to the return. The study is based on test data collected over roughly 16 months of production for a high-quality SoC sold to the automotive market. The data consists of 62 customer returns belonging to 52 lots. The study shows that each approach can capture returns not captured by the other. With both approaches, the study shows that multivariate test analysis can have a significant impact on reducing customer return rates especially during the later period of the production.

Burn-in is a common test approach to screen out unreliable parts. The cost of burn-in can be significant due to long burn-in periods and expensive equipment. This work studies the potential of using parametric test data to reduce the time of burn-in. The experiment focuses on developing parametric test models based on test data collected after 10 hours of burn-in to predict parts likely-to-fail after 24 and 48 hours of burn-in. Our study shows that 24-hour and 48-hour burn-in failures behave abnormally in multivariate parametric test spaces after 10 hours of burn-in. Hence, it is possible to develop multivariate test models to identify these likely-to-fail parts early in a burn-in cycle. This study is carried out on 8 lots of test data from a burn-in experiment based on a 3-axis accelerometer design. The study shows that after 10 hours of burn-in, it is possible to identify a large portion of all parts that do not require longer burn-in time, potentially providing significant cost saving.

introduction Share on Introduction to special section on verification challenges in the concurrent world Authors: Sandip Ray University of Texas at Austin, TX University of Texas at Austin, TXView Profile , Jayanta Bhadra Freescale Semiconductor Inc., Austin, TX Freescale Semiconductor Inc., Austin, TXView Profile , Magdy S. Abadir Freescale Semiconductor Inc., Austin, TX Freescale Semiconductor Inc., Austin, TXView Profile , Li-C. Wang University of California at Santa Barbara, CA University of California at Santa Barbara, CAView Profile , Aarti Gupta NEC Laboratories America, Inc., Princeton, NJ NEC Laboratories America, Inc., Princeton, NJView Profile Authors Info & Claims ACM Transactions on Design Automation of Electronic SystemsVolume 17Issue 3June 2012 Article No.: 19pp 1–3https://doi.org/10.1145/2209291.2209292Published:05 July 2012Publication History 0citation172DownloadsMetricsTotal Citations0Total Downloads172Last 12 Months1Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

This work proposes a wafer probe parametric test set optimization method for predicting dies which are likely to fail in the field based on known in-field or final test fails. Large volumes of wafer probe data across 5 lots and hundreds of parametric measurements are optimized to find test sets that help predict actually observed test escapes and final test failures. Simple rules are generated to explain how test limits can be tightened in wafer probe to prevent test escapes and final test fails with minimal overkill. The proposed method is evaluated on wafer probe data from a current automotive IC with near zero DPPM requirements resulting in improved test quality and reduced test cost.

This paper studies the potential of using wafer probe tests to predict the outcome of future tests. The study is carried out using test data based on an SoC design for the automotive market. Given a set of known failing parts, there are two possible approaches to learn. First a single binary classification model can be learned to model all failing parts. We show that this approach can be effective if the failing parts are compatible in learning. Second, an individual outlier model can be learned for each failing part. We show that this approach is suitable for learning failing parts such as customer returns, where each may have a unique failing behavior. We also show that with Principal Component Analysis (PCA), a learning model can be visualized in two or three dimensional PC space, which facilitates an engineer to manually select or adjust the model.

Customer returns are defective parts that pass all functional and parametric tests, but fail in the field. To prevent customer returns, this paper analyzes wafer probe test data and tries to understand what it takes to screen them out during testing. Because these parts pass all tests, analyzing their signatures based on the original test perspective does not make sense. In this work, we search for a novel test perspective where the test signatures from parametric measurements can be used to separate the returned parts from the rest of population. Our study shows that in order to effectively screen customer returns during wafer test, a multivariate screening methodology is desired. This study is based on analyzing over 1000 parametric wafer probe tests and dies from seven lots, each lot containing one returned part. We demonstrate that analyzing customer returns from a multivariate test perspective leads to robust and conservative results.

This paper studies the problem of automatic assertion extraction at the input boundary of a given unit embedded in a system. This paper proposes a data mining approach that analyzes simulation traces to extract the assertions. We borrow two key concepts from the sequential data mining and develop an effective assertion extraction approach specific to our problem. These two concepts are (1) the slide-window-based episode definition that decides the space of all potential assertions and (2) the Support-Confidence framework that evaluates the meaningfulness of potential assertions using a given simulation trace. We implement the approach in a system simulation environment built on the AMBA 2.0 standard. Experimental results demonstrate the feasibility of the proposed approach and validity of extracted assertions are verified by comparing to the transactions defined in the specification.

System test has been the standard measurement to evaluate performance variability of high-performance microprocessors. The question of whether or not many of the lower-cost alternative tests can be used to reduce system test has been studied for many years. This paper utilizes a data-learning approach for correlating three test datasets, structural test, ring oscillator test, and scan flush test, with system test. With the data-learning approach, higher correlation can be found without altering test measurements or test conditions. Rather, the approach utilizes new optimization algorithms to extract more useful information in the three test datasets, with particular success using the structural test data. To further minimize test cost, process monitoring measurements (ring oscillator and scan flush tests) are used to reduce the need for high-frequency structural test. We demonstrate our methodology on a recent high-performance microprocessor design.

Traditional diagnosis of defects is based on an assumed fault model. A failing chip is diagnosed to find the subset of faults that can best explain the failure. This paper illustrates a link between this traditional perspective of diagnosis and a new perspective where diagnosis is seen as a form of data learning. We explain that both defect diagnosis and data learning are solving so-called ill-posed problems and the technique for solving such a problem is called regularization. We illustrate a diagnosis framework that employs various data learning techniques to implement two diagnosis approaches: feature ranking and rule extraction. This diagnosis framework is designed to uncover design-related issues that cause systematic uncertainties or any unexpected behavior in silicon. We review the work that has been accomplished for implementing this framework and further discuss issues with its practical application.

Due to the magnitude and complexity of design and manufacturing processes, it is unrealistic to expect that models and simulations can predict all aspects of silicon behavior accurately. When unexpected behavior is observed in the post-silicon stage, one desires to identify the causes and consequently identify the fixes. This paper studies one formulation of the design-silicon mismatch problem. To analyze unexpected behavior, silicon behavior is partitioned into two classes, one class containing instances of unexpected behavior and the other with rest of the population. Classification rule learning is applied to extract rules to explain why certain class of behavior occurs. We present a rule learning algorithm that analyzes test measurement data in terms of design features to generate rules, and conduct controlled experiments to demonstrate the effectiveness of the proposed approach. Results show that the proposed learning approach can effectively uncover rules responsible for the designsilicon mismatch even when significant noises are associated with both the measurement data and the class partitioning results for capturing the unexpected behavior.

This paper proposes an online functional test selection approach based on novelty detection. Unlike other test selection methods, the idea of this paper is selecting novel functional tests to improve coverage from a large pool of available test programs before simulation. A graph based encoding scheme is developed to measure the similarity between test programs and map them into a set of feature vectors. We employ one-class SVM as the learning algorithm to detect novel tests to be simulated. While leaving the general test selection framework unchanged, the developed test program similarity measure can easily be tailored to specific applications and coverage targets based on existing simulation results. Experiments on a public domain MIPS processor design are presented to demonstrate the effectiveness of the approach.

In this paper we demonstrate how statistical learning support vector machine (SVM) algorithms can be applied to modeling analog circuits. The success of these types of techniques has been traditionally achieved by using large sets of training data. However, analog data is expensive in terms of simulation time and hardware testing; therefore, achieving high modeling accuracy with limited datasets has become a challenge. The proposed sampling method dynamically forms datasets based on its selection of dominant support vectors, requiring less data while maintaining the same level of model accuracy. The rest of the modeling flow, including the learning and regression methods, is also discussed. We present two industry designs to validate this approach throughout the paper.

This paper proposes a kernel-based functional test program generation approach for microprocessor test and verification. The fundamental idea in this approach is to select high quality test programs before the simulation from a large number of biased random test programs. Unlike a direct test program generation approach, a selection approach demands much less domain knowledge and intervention from its user for achieving a similar coverage goal, making it more applicable for scenarios targeting on different coverage objectives. We will demonstrate the effectiveness and efficiency of such an approach through performing experiments on a MIPS processor design.

Speed-limiting paths are critical paths that limit the performance of one or more silicon chips. This paper present a data mining methodology for analyzing speed-limiting paths extracted from AC delay test measurements. Based on data collected on 15 packaged silicon units of a four-core microprocessor design, we show that the proposed methodology can efficiently discovered actionable, design-related knowledge that would be difficult to find otherwise.

The use of low-cost structural Fmax measurement as a replacement for in-system Fmax measurement for speed binning has been aided by the use of a data-learning approach that can be used to build a reliable system Fmax predictor given structural Fmax. This paper uses industry test measurements to demonstrate why a data-learning approach for correlation is better than simple correlation approaches, how to select the most relevant structural Fmax, and how the proposed methodology works on multiple lots.

Multiple input switching (MIS) on off-path inputs is known to increase the delay through a gate. However, due to the complexity of incorporating MIS in timing analysis, design flows typically ignore the effect of MIS. Test tools also do not attempt to maximize the off-path switching to maximize delays through a path. In this paper we study the impact of not maximizing the switching on the off-path inputs in different process corners. We present quantitative data to estimate the test escape or mis-binning that could result if MIS is not considered as part of our path-delay test generation process.

Success of simulation-based functional verification depends on the quality and diversity of the verification tests that are simulated. The objective of test generation methods is to generate tests that exercise as much different functionality of the hardware designs as possible. In this paper, we propose a novel methodology that generates a model of the verification tests in a given test set using unsupervised support vector analysis. One potential application is to use this model to select tests that are likely to exercise functionality that has not been tested so far. Since this selection can be done before simulation, it can be used to filter redundant tests and reduce required simulation cycles. Our methodology can be combined with a test generation method like constrained-random test generation to increase its effectiveness without making fundamental changes to the verification flow. Experimental results based on application of the proposed methodology to the OpenSparc T1 processor are reported to demonstrate the practicality of our approach.

This paper presents a novel path selection methodology to select paths for monitoring unexpected systematic timing effects. The methodology consists of three components: path filtering, path encoding, and path clustering. Given a large set of critical paths, in path filtering, the goal is to filter out paths that cannot be functionally sensitized. To explore the space of unexpected timing effects, a set of features are defined to encode paths into path vectors. Each feature is a source of concern that may potentially contribute to the cause of an unexpected timing effect. Finally, a kernel-based clustering algorithm is employed to group similar path vectors into clusters from which the best representative paths are selected for post-silicon monitoring. The effectiveness of our proposed methodology is demonstrated through experiments on an industrial ASIC design.

In optimizing high-performance designs, speed limiting paths (speed-paths) impact the performance and power trade-off. Timing tools attempt to model and capture all such paths on a chip. Due to the high performance nature of these designs, critical paths predicted by the timing tools often do not match the actual speedpaths found on silicon chips. Early silicon data therefore is used to identify the speedpaths, and further performance optimization is carried out by pushing the delays on these paths. In this context, the paper presents a novel data mining approach that analyzes a small number of identified speedpaths against a large number of non-speedpaths. The result of this analysis for each speedpath is a set of hypotheses explaining why the path is special. These hypotheses can be used in guiding the search for the root causes, or in predicting additional paths as potential speedpaths. We demonstrate the feasibility of this approach and summarize our findings based on analysis of silicon speedpaths collected from a 65nm microprocessor.

As lithography process nodes shrink to sub-wavelength levels generating acceptable layout patterns becomes a challenging problem. Traditionally, complex convolution based lithography simulations are used to estimate areas of high variability. These methods are slow and infeasible for large scale full chip analysis. This work proposes a solution to this problem by using machine learning techniques to identify layout areas that are more prone to variability. A novel target layout representation is proposed, and the latest support vector machine (SVM) algorithms are used to detect variability within standard cells and between cells in a simulated full chip layout.

In test and diagnosis, one often runs into the situation that after analyzing a set of samples, a few of these samples are identified as being ¿special¿. Then, in a large population of samples one desires to identify all samples that are ¿similar¿ to the special samples. The process is called a similarity search. This paper presents a feature based similarity search approach and discusses three potential methods to implement this approach. These methods are (1) building a model to capture the characteristics of the non-special samples, (2) building a model to capture the characteristics of the special samples, and (3) searching for the hypotheses to explain individually why each sample is special. We apply similarity search to the speedpath analysis problem where special samples are special paths that limit the performance of silicon chips. The goal is to identify more paths in the design with similar characteristics to the speedpaths. The effectiveness of the three methods are analyzed based on speedpath data collected from a high-performance microprocessor.

The question of whether or not structural test measurements can be used to predict functional or system Fmax, has been studied for many years. This paper presents a data learning approach to study the question. Given Fmax values and structural delay measurements on a set of sample chips, we propose a method called conformity check whose goal is to select a subset of conformal samples such that a more reliable predictor can be built on. Our predictor consists of two models, a conformal model that decides on a given chip if its Fmax is predictable or not, and a prediction model that outputs the predicted Fmax based on results obtained from structural test measurements. We explain the data learning methodology and study various data learning techniques using frequency data collected on a high-performance microprocessor design.

This work proposes a methodology to minimize the application cost of outlier analysis when applied to delay testing in the presence of systematic variability. Support vector machine (SVM) outlier analysis algorithms and traditional entropy measures are used to detect delay defects by choosing a minimum number of suitable test clocks. Monte Carlo simulations generate realistic test data while information content measurements guide test clock selection. Exhaustive simulation found trade-offs between reducing the number of clocks, patterns, and chip samples. Substantial cost reduction was obtained with proper clock selection, while minimizing both test patterns and circuit samples required for effective outlier analysis.

Explaining the mismatch between predicted timing behavior from modeling and simulation, and the observed timing behavior measured on silicon chips can be very challenging. Given a list of potential sources, the mismatch can be the aggregate result caused by some of them both individually and collectively, resulting in a very large search space. Furthermore, observed data are always corrupted by some unknown statistical random noises. In this paper, we examine how trying to explain the mismatch observed on silicon can be classified as an ill-posed problem, where ill posed means that the solution may not be unique or stable. Thus, a small change in the observed response can have a large change in the predicted solution. To solve ill-posed problems, a statistical learning theory uses a principle called regularization. This paper proposes using a statistical learning method called support vector (SV) analysis to statistically analyze all known sources of uncertainty with the objective to rank which sources contribute the most to the observed mismatch. Experimental results are presented under different error assumption models to compare two kinds of SV ranking approaches to four other ranking approaches, where some use the idea of regularization and others do not. This paper is concluded by showing a self cross-validation approach to validate the ranking results when there is no true ranking available, as the case with actual silicon.

We show that rising and falling delays in gates can differ considerably. Simulation data, using 40 nm and 65 nm process technology, shows an increasing trend and that the slow transition delay could be two times of the faster transition delay. This translates to an asymmetry between the rise and fall delays along a path. Based on this we propose refinements to the following delay test methodology: (i) selection of robust path delay tests for delay characterization; (ii) refinements to small delay defects coverage metric; and (iii) an alternative to the inline resistance fault (IRF) model for selecting TDF tests.

Explaining the mismatch between predicted timing behavior from modeling and simulation, and the observed timing behavior measured on silicon chips can be very challenging. Given a list of potential sources, the mismatch can be the aggregate result caused by some of them both individually and collectively, resulting in a very large search space. Furthermore, observed data are always corrupted by some unknown statistical random noises. To overcome both challenges, this paper proposes a statistical diagnosis framework that formulates the diagnosis problem as a regression learning problem. In this diagnosis framework, the objective is to rank a set of features corresponding to the list of potential sources of concern. The rank is based on measured silicon path delay data such that a feature inducing a larger unexpected timing deviation is ranked higher. Experimental results are presented to explain the learning method. Diagnosis effectiveness will be demonstrated through benchmark experiments and on an industrial design.

In high performance designs, speed-limiting logic paths (speedpaths) impact the power/performance trade-off that is becoming critical in our low power regimes. Timing tools attempt to model and predict the delay of all the paths on a chip, which may be in the millions. These delay predictions often have a significant error and when silicon is measured there is a large variation of path delays as compared to the prediction of the tools. This variation may be caused by process, environmental or other effects that are often unpredictable. It is therefore desirable to use early silicon data to better predict and model potential speedpaths for subsequent silicon steppings. In this paper, we present a novel machine learning-based approach that uses a small number of identified speedpaths to predict a larger set of potential speedpaths, thus significantly enhancing the traditional timing prediction flows post-silicon. We demonstrate the feasibility of this approach and summarize our findings based on the analysis of silicon speedpaths from a 65nm P4 microprocessor.

Extensive software-based simulation continues to be the mainstream methodology for functional verification of designs. To optimize the use of limited simulation resources, coverage metrics are essential to guide the development of effective test suites. Traditional coverage metrics are defined based on either a functional model or a structural model of the design. If our goal is to select a subset of tests from a set of tests, using these coverage metrics require simulation of the entire set before the effectiveness of tests can be compared. In this paper, we propose a novel methodology that estimates the input space covered by a set of tests. We use unsupervised support vector analysis to learn such a space, resulting in a subset of tests that represent the original set of tests. A direct application of this methodology is to select tests before simulation in order to reduce simulation cycles. Consequently, simulation effectiveness can be improved. Experimental results based on application of the proposed methodology to the OpenSparc T1 processor are reported to demonstrate the practicality of our approach.

For sub-65 nm design, there can be many timing effects not explicitly and/or accurately modeled and simulated. For design-silicon timing convergence, this paper describes a novel path-based diagnosis approach that analyzes and ranks potential design related issues causing the unexpected timing effects. We explain in detail how a path can be encoded with a set of diverse "features" based on one's knowledge of the potential issues. We explain how these features can be interpreted differently in a data learning algorithm based on adjusting a so-called kernel function. Then, we explain how kernel-based data learning can be used to rank the importance of features such that a feature contributing the most to design-silicon timing mismatch is ranked the highest. We conclude the paper by showing an application result on an industrial ASIC design.

This work provides a survey study of several outlier analysis techniques and compares their effectiveness in the context of delay testing. Three different approaches are studied, an Euclidean-distance based algorithm, random forest, and one-class support vector machine (SVM), from which more advanced methods are derived and analyzed. We conclude that one-class SVM using a polynomial kernel is most effective for detecting delay defects, while keeping overkills minimized. The best models were successfully validated and a feasible approach to delay testing using one-class SVM is proposed.

In the post-silicon stage, timing information can be extracted from two sources: (1) on-chip monitors and (2) delay testing. In the past, delay test data has been overlooked in the correlation study. In this paper, we take path delay testing as an example to illustrate how test data can be incorporated in the overall design-silicon correlation effort. We describe a path-based methodology that correlates measured path delays from the good chips, to the path delays predicted by timing analysis. We discuss how statistical data mining can be employed for extracting information and show experimental results to demonstrate the potential of the proposed methodology.

Unit-level verification is a critical step to the success of full-chip functional verification for microprocessor designs. In the unit-level verification, a unit is first embedded in a complex software that emulates the behavior of surrounding units, and then a sequence of stimuli is applied to measure the functional coverage. In order to generate such a sequence, designers need to comprehend the relationship between boundaries at the unit under verification and at the inputs to the emulation software. However, figuring out this relationship can be very difficult. Therefore, this paper proposes an incremental learning framework that incorporates an ordered-binary-decision-forest(OBDF) algorithm, to automate estimating the controllability of unit-level signals and to provide full-chip level information for designers to govern these signals. Mathematical analysis shows that the proposed OBDF algorithm has lower model complexity and lower error variance than the previous algorithms. Meanwhile, a commercial microprocessor core is also applied to demonstrate that controllability of input signals on the load/store unit in the microprocessor core can be estimated automatically and information about how to govern these signals can also be extracted successfully.

As technology scales, it is becoming increasingly difficult for simulation and timing models to accurately predict silicon timing behavior. When a collection of chips fail in timing in a similar way, diagnosis and silicon debug look to find the root-causes for the failure. However, little work has been done to develop a methodology that looks for useful design information in the good-chip data. This paper describes a path-based methodology that correlates measured path delays from the good chips, to the path delays predicted by timing analysis. We explain how to utilize this methodology for evaluating the risk of timing modeling.

Functional test-bench development is a tedious and time-consuming process that requires tremendous engineering effort. Developing proper test-benches is crucial for both functional verification and post-silicon performance validation. Constrained random test generation is a popular approach to alleviate the burden of test-bench development. This paper presents an automatic constraint extraction tool that can be easily integrated with an existing commercial constrained random test generation framework. This tool extracts constraints by analyzing test-bench simulation data. These constraints, when added into a test-bench, can provide controllability of signals that are deeply embedded in a complex design. We develop simulation data mining algorithms for constraint extraction and demonstrate the effectiveness of our approach based on OpenSparc Tl microprocessor.

In this work, we present using random forests statistical learning to analyze post-silicon delay test data. We introduce the concept of parametric delay test as a new perspective for extracting more information from delay test. First, a methodology for outlier identification is presented to aid defect characterization of initial sample chips. Second, a methodology for production test is presented, including automated pattern-set reduction analysis. Finally, a strategy for adaptive test is presented.

Statistical static timing analysis (SSTA) has been a popular research topic in recent years. A fundamental issue with applying SSTA in practice today is the lack of reliable and efficient statistical timing models (STM). Among many types of parameters required to be carefully modeled in an STM, spatial delay correlations are recognized as having significant impact on SSTA results. In this work, we assume that exact modeling of spatial delay correlations is quite difficult, and propose an experimental methodology to resolve this issue. The modeling accuracy requirement is relaxed by allowing SSTA to impose upper bounds and lower bounds on the delay correlations. These bounds can then be refined through learning the actual delay correlations from path delay testing on silicon. We utilize SSTA as the platform for learning and propose a Bayesian approach for learning spatial delay correlations. The effectiveness of the proposed methodology is illustrated through experiments on benchmark circuits.

Process variations cause different behavior of timing-dependent effects across different chips. In this work, we analyze one example of timing-dependent effects, cross-coupling capacitance, and the complex problem space created by considering coupling and process variations together. The delay of a critical path under these conditions is difficult to bound for design and test. We develop a methodology that analyzes this complex space by decomposing the problem space along three dimensions: the aggressor space, test space, and sample space. For design, we utilize an OBDD-based approach to prune the aggressor space based on logical constraints, which can be combined with a worst-case timing window simulator to prune based on both logical and timing constraints. After pruning, the reduced aggressor space can be used to derive a more accurate timing bound. Solving the problems in the test and sample spaces is postponed to the post-silicon stage, where we propose a test selection methodology for bounding the delay of every sample. This methodology is based on probability density estimation and has a tradeoff between the number of tests to apply and the tightness of the delay bound obtained. Experimental results based on benchmark examples are presented to show the effectiveness of the proposed methodology.

In simulation-based functional verification, composing and debugging testbenches can be tedious and time-consuming. A simulation data-mining approach, called TTPG (C. Wen, L-C Wang et al., 2005), was proposed as an alternative for functional test pattern generation. However, the core of simulation data-mining approach is Boolean learning, which tries to extract the simplified view of the design functionality according to the given bit-level simulation data. In this work1, an efficient data-mining engine is presented based on decision-diagram(DD)-based learning approaches. We compare the DD-based learning approaches to other known methods, such as the Nearest Neighbor method and support vector machine. We demonstrate that the proposed Boolean data miner is efficient for practical use. Finally, that the TTPG methodology incorporated with the Boolean data miner can achieve a high fault coverage (95.36%) on the OpenRISC 1200 microprocessor concludes the effectiveness of the proposed approach.

As feature sizes continue to decrease, sensitivities of design to process variations have become harder to analyze. Traditional worst-case and nominal timing analyses are not sufficient to accurately characterize these sensitivities. Timing sensitivities can be classified into timing variability, a direct result of process variations, and timing uncertainty, caused by the interaction between some timing-dependent effects and timing variability. Statistical timing analysis is an emerging approach that promises to better quantify timing variability. However, there has been little work focusing on timing uncertainty. Searching for precision tests that bound this uncertainty in design and test is inherently a statistical problem. Using cross-coupling as an example, this work describes a non-statistical framework that utilizes a Boolean satisfiability solver (SAT), ordered binary decision diagrams (OBDD), and timing window based filtering to efficiently prune the search space. Experimental results are presented to explain that such a non-statistical solution is desired and can be effective for practical use

As the timing behavior of the good and defective chips become statistical, the traditional notion that there exists a one-dimensional timing boundary to separate the good and defective behavior may no longer be true. This paper studies issues in test optimization for screening statistical delay defects. After the first silicon tapeout, test data learning based on silicon samples can be utilized to optimize the test set for mass production. This approach depends on the availability of known good and known defective samples. This paper focuses the discussion on silicon sample based test optimization. We relate this problem to binary classification and pattern selection to the feature selection problem in statistical learning. Experimental results are presented to explain the methodologies and the new concepts

Deterministic functional test pattern generation has been a long-standing open problem, which is an important problem to be solved for both design verification and manufacturing testing. One key in developing a practical functional test pattern generation approach is to avoid the exponential growth of the test generation complexity in terms of the design size. This work proposes a novel functional test generation approach where simulation results are used to guide the generation of additional tests. Our methodology avoids the complexity growth issue by converting some modules in a design into simpler and more efficient models. Then, these models are used to facilitate the actual test generation process. We develop two sets of techniques to achieve these conversions: Boolean learning for random logic and arithmetic learning for datapath modules. We demonstrate the effectiveness and discuss the. limitations of these techniques through experiments on benchmark circuits. Last, we validate the overall test generation methodology based on the OpenRISC 1200 microprocessor

A sequential SAT solver, Satori, was recently proposed (Iyer, M.K. et al., Proc. IEEE/ACM Int. Conf. on Computer-Aided Design, 2003) as an alternative to combinational SAT in verification applications. This paper describes the design of Seq-SAT, an efficient sequential SAT solver with improved search strategies over Satori. The major improvements include: (1) a new and better heuristic for minimizing the set of assignments to state variables; (2) a new priority-based search strategy and a flexible sequential search framework which integrates different search strategies; (3) a decision variable selection heuristic more suitable for solving the sequential problems. We present experimental results to demonstrate that our sequential SAT solver can achieve orders-of-magnitude speedup over Satori. We plan to release the source code of Seq-SAT.

The purpose of statistical timing simulation is to assess the impact of process variations on pattern delays. In this paper, we propose a novel hazard-aware statistical timing simulator. Our simulator characterizes timing hazards in terms of uncertainty windows whose widths are computed as random variables. Given a 2-pattern vector, the simulator estimates the transition times and uncertainty windows at each circuit output as two separate random variables. We demonstrate that this simulator achieves much higher accuracy and robustness than its predecessor. With this improved statistical timing simulator, we study its applications from three perspectives: (1) improving test effectiveness through pattern selection (2) enhancing defect detection by using multiple test frequencies (3) extending defect coverage for different voltages and temperatures. Experimental results are presented to demonstrate the benefits as well as the limitations of using the statistical timing simulator in the context of screening frequency-dependent defects.

Software-based self-test (SBST) was previously proposed as an on-chip functional test methodology. Achieving desired full-chip functional fault coverage has always been a challenge because random test program generation (RTPG) alone may not be sufficient. This work investigates the potential of using target test program generation (TTPG) to supplement the RTPG method. The proposed TTPG method utilizes simulation results to develop learned models for the surrounding modules of the block under test. Then, the learned models replace the surrounding modules around the block in the actual test generation process. Because the learned models are much simpler to handle, this method minimizes the cost of functional TPG. For developing the simulation-based learning scheme, we divide the surrounding modules into two categories: Boolean and arithmetic. We apply different techniques for each category and explain their applicability and limitations. The feasibility and effectiveness of the proposed simulation-based TTPG method in the context of supplementing RTPG for achieving high fault coverage in SBST of a RISC pipelined microprocessor design is demonstrated as well

The delay variations of a pattern set can come from two sources: (1) Different patterns sensitize different parts of the circuit and result in different delays. (2) The same pattern, applied on different chips, results in different delays because of process variations. For structural delay testing, these pattern variations may result in difficulty for finding an optimal test clock setting, which may significantly impact the defect screening effectiveness. This paper investigates the possibility of applying statistical timing analysis techniques to reduce pattern variations for structural delay testing. We develop an efficient statistical pattern-based timing simulator and devise pattern selection algorithms for reducing such variations. By constructing pattern sets with smaller variations, we show that higher screening effectiveness can be achieved. We present experimental results to demonstrate the advantages of our techniques based on benchmark circuits.

Speed path identification is an indispensable step for pushing the design timing wall and for developing the final speed binning strategy in production test. For complex high-performance designs, pre-silicon timing tools have so far not been able to deliver satisfactory results in predicting the actual speed limiting paths on the silicon. The actual speed paths are mostly uncovered through test and silicon debug, where tremendous manual effort is involved. This paper presents a novel approach as the first step for automating the speed path identification process. Our approach is silicon-based, meaning that timing information is extracted through testing of silicon sample chips. We call this step silicon learning. Based on silicon learning, we present an iterative flow for speed path identification. Experimental results are presented to explain the new methodologies and to demonstrate the effectiveness of our techniques.

Software-based self-test (SBST) was originally proposed for cost reduction in SOC test environment. Previous studies have focused on using SBST for screening logic defects. SBST is functional-based and hence, achieving a high full-chip logic defect coverage can be a challenge. This raises the question of SBST's applicability in practice. In this paper, we investigate a particular SBST methodology and study its potential applications. We conclude that the SBST methodology can be very useful for producing speed binning tests. To demonstrate the advantage of using SBST in at-speed functional testing, we develop a SBST framework and apply it to an open source microprocessor core, named OpenRISC 1200. A delay path extraction methodology is proposed in conjunction with the SBST framework. The experimental results demonstrate that our SBST can produce tests for a high percentage of extracted delay paths of which less than half of them would likely be detected through traditional functional test patterns. Moreover, the SBST tests can exercise the functional worst-case delays which could not be reached by even 1M of traditional verification test patterns. The effectiveness of our SBST and its current limitations are explained through these experimental findings.

In this article, we propose a symbolic simulation method where Boolean functions can be efficiently manipulated through a 2-domain partitioned OBDD data structure. The functional partition is applied by automatically exploring the key decision points implicitly built inside a circuit. The partition can help to significantly reduce the OBDD sizes, solving problems that could not be solved with monolithic OBDD data structure. We demonstrate the performance of the approach through the symbolic simulation of several benchmark circuits with complex control logics and datapath. The symbolic simulation based on 2-domain partitioned OBDD can be also applied in equivalence checking. It can generate the signature of functions to identify the critical partition points in the optimized gate-level netlist.

We present a novel hybrid finite-domain constraint solving engine for RTL circuits, that automatically uses data-path abstraction. We describe how DPLL search can be modified by using efficient finite-domain constraint propagation to improve communication between interacting integer and Boolean domains. This enables efficient combination of Boolean SAT and linear integer arithmetic solving techniques. We use conflict-based learning using the variables on the boundary of control and data-path for additional performance benefits. Finally, the hybrid constraint solver is experimentally analyzed using some example circuits.

This paper describes the implementation of a novel path-based learning methodology that can be applied for two purposes: (1) In a pre-silicon simulation environment, path-based learning can be used to produce a fast and approximate simulator for statistical timing simulation. (2) In post-silicon phase, path-based learning can be used as a vehicle to derive critical paths based on the pass/fail behavior observed from the test chips. Our path-based learning methodology consists of four major components: a delay test pattern set, a logic simulator, a set of selected paths as the basis for learning, and a machine learner. We explain the key concepts in this methodology and present experimental results to demonstrate its feasibility and applications.

In this paper, we provide a flexible and automatic method to partition the functional space for efficient symbolic simulation. We utilize a 2-tuple list representation as the basis for partitioning the functional space. The partitioning is carried out dynamically during the symbolic simulation based on the sizes of OBDDs. We develop heuristics for choosing the optimal partitioning points. These heuristics intend to balance the tradeoff between the time and space complexity. We demonstrate the effectiveness of our new symbolic simulation approach through experiments based on a floating point adder and a memory management unit.

Due to process variations in deep sub-micron (DSM) technologies, the effects of timing defects are difficult to capture. This paper presents a novel coverage metric for estimating the test quality with respect to timing defects under process variations. Based on the proposed metric and a dynamic timing analyzer, we develop a pattern-selection algorithm for selecting the minimal number of patterns that can achieve the maximal test quality. To shorten the run time in dynamic timing analysis, we propose an algorithm to speed up the Monte-Carlo-based simulation. Our experimental results show that, selecting a small percentage of patterns from a multiple-detection transition fault pattern set is sufficient to maintain the test quality given by the entire pattern set. We present run-time and accuracy comparisons to demonstrate the efficiency and effectiveness of our pattern selection framework.

In this paper, we propose a simple technique for estimating the standard deviation of a Gaussian random jitter component in a multi-gigahertz signal. This method may utilize existing on-chip single-shot period measurement techniques to measure the multi-gigahertz signal periods for the estimation. This method does not require an external sampling clock, or any additional measurement beyond existing techniques. Experimental results show that this extraction method can accurately estimate the random jitter variance in a multi-gigahertz signal even with the presence of a few hundred-hertz sinusoidal jitter components.

This paper presents a novel path-based learning methodology to achieve timing regression simulation. The methodology can be applied for two purposes: (1) In pre-silicon phase, regression simulation can be used to produce a fast and approximate timing simulator to avoid the high cost associated with statistical timing simulation. (2) In post-silicon phase, regression simulation can be used as a vehicle to deduce critical paths from the pass/fail behavior observed on the test chips. Our path-based learning methodology consists of four major components: a delay test pattern set, a logic simulator, a set of selected paths as the basis for learning, and a machine learner. We summarize the key concepts in our regression simulation approach and present experimental results.

This work presents a novel path-based methodology for post-silicon timing validation. In timing validation, the objective is to decide if the timing behavior observed from the silicon is consistent with that predicted by the timing model. At the core of our path-based methodology, we propose a framework to obtain the post-silicon path ranking from observing silicon timing behavior. Then, the consistency is determined by comparing the post-silicon path ranking and the pre-silicon path ranking calculated based on the timing model. Our post-silicon ranking methodology consists of two approaches: ranking optimization and path filtering. We discuss the applications of both approaches and their impacts on the path ranking results. For experiments, we utilize a statistical timing simulator that was developed in the past to derive chip samples and we demonstrate the feasibility of our methodology using benchmark circuits.

A latch-based timing analyzer is an essential tool for developing high-speed pipeline designs. As process variations increasingly influence the timing characteristics of DSM designs, a timing analyzer capable of handling process-induced timing variations for latch-based pipeline designs becomes in demand. In this work, we present a static statistical timing analyzer, STAP, for latch-based pipeline designs. Our analyzer propagates statistical worst-case delays as well as critical probabilities across the pipeline stages. We present an efficient method to handle correlations due to re-convergent fanouts. We also demonstrate the impact of not including the analysis of reconvergent fanouts in latch-based pipeline designs. Comparing to a Monte-Carlo based timing analyzer, our experiments show that STAP can accurately evaluate the critical probability that a design violates the timing constraints under a given statistical timing model. The runtime comparison further demonstrates the efficiency of our STAP.

The use of functional vectors has been an industry standard for speed binning purposes of high performance ICs. This practice can be prohibitively expensive as the ICs become faster and more complex. In comparison, structural patterns can target performance related faults in a more systematic manner. To make structural testing an effective alternative to functional testing for speed binning, structural patterns need to correlate with functional test frequencies closely. We investigate the correlation between functional test frequency and that of various types of structural patterns on MPC7455, a Motorola processor executing to the PowerPC/spl trade/ instruction set architecture.

In this paper, we propose a method for extracting the spectral information of a multi-gigahertz jittery signal. This method utilizes existing on-chip single-shot period measurement techniques to sample and measure the period of multiple cycles of the multi-gigahertz periodic signal for spectral analysis. Since measurements are made on the period of multiple cycles, but not on the period of a single cycle, a lower-speed timing measurement circuitry can be used to measure a higher-speed signal. Therefore, the proposed solution is scalable for even higher-speed signals. This method does not require an external sampling clock, nor any additional measurement beyond existing techniques. Experimental results based on simulation show that this method can accurately estimate the sinusoidal and random jitters of a multi-gigahertz signal.

In this paper, an effective algorithm is presented for multilevel circuit clustering for delay minimization, and is applicable to hierarchical field programmable gate arrays. With a novel graph contraction technique, which allows some crucial delay information of a lower-level clustering to be maintained in the contracted graph, our algorithm recursively divides the lower-level clustering into the next higher-level one in a way that each recursive clustering step is accomplished by applying a modified single-level circuit clustering algorithm based on . We test our algorithm on the two-level clustering problem and compare it with the latest algorithm in . Experimental results show that our algorithm achieves, on average, 12% more delay reduction when compared to the best results (from TLC with full node-duplication) in . In fact, our algorithm is the first one for the general multilevel circuit clustering problem with more than two levels.

Critical path selection is an indispensable step for testing of small-size delay defects. Historically, this step relies on the construction of a set of worst-case paths, where the timing lengths of the paths are calculated based upon discrete-valued timing models. The assumption of discrete-valued timing models may become invalid for modeling delay effects in the deep submicron domain, where the effects of timing defects and process variations are often statistical in nature. This paper studies the problem of critical path selection for testing small-size delay defects, assuming that circuit delays are statistical. We provide theoretical analysis to demonstrate that the new path-selection problem consists of two computationally intractable subproblems. Then, we discuss practical heuristics and their performance with respect to each subproblem. Using a statistical defect injection and timing-simulation framework, we present experimental results to support our theoretical analysis.

was shown to be effective for verifying individual array blocks. However, when applying STE to verify multiple array blocks together as a single system, the run-time OBDD sizes would often blow up. In this paper, we propose using a ”dual-rail ” symbolic simulation scheme to facilitate the application of STE proof methodology for verifying array systems. The proposed scheme implicitly partitions a given design into control domain and datapath domain, and symbolic simulation is carried out on both domains. With this scheme, the run-time OBDD sizes during the symbolic simulation for each domain can be limited. We demonstrate the effectiveness of our approach by verifying the Memory Management Unit (MMU) in Motorola high-performance microprocessors. The verification of MMU as a whole was not possible before because of the OBDD size blow-up problem when an ordinary symbolic simulator was used in the STE proof process. I.

Critical path selection is an indispensable step for AC delay test and timing validation. Traditionally, this step relies on the construction of a set of worse-case paths based upon discrete timing models. However, the assumption of discrete timing models can be invalidated by timing defects and process variation in the deep sub-micron domain, which are often continuous in nature. As a result, critical paths defined in a traditional timing analysis approach may not be truly critical in reality. In this paper, we propose using a statistical delay evaluation framework for estimating the quality of a path set. Based upon the new framework, we demonstrate how the traditional definition of a critical path set may deviate from the true critical path set in the deep sub-micron domain. To remedy the problem, we discuss improvements to the existing path selection strategies by including new objectives. We then compare statistical approaches with traditional approaches based upon experimental analysis of both defect-free and defect-injected cases.

This paper proposes a mixed-signal Built-In Self-Test (BIST) architecture based on a second-order delta-sigma modulator. This modulator, which incorporates a design-for-testability (DfT) circuitry, is capable of testing/characterizing itself using digital stimulus. This characteristic is attractive for implementing the modulator as an on-chip analog signal analyzer. When applied for mixed-signal BIST, the modulator-based analog signal analyzer is first characterized using digital stimulus. Then the analyzer is utilized to characterize the stimulus generator in the BIST application. Some critical implementation issues of the BIST architecture are also discussed.

In this paper, we propose a new methodology for diagnosis of delay defects in the deep sub micron domain. The key difference between our diagnosis framework and other traditional diagnosis methods lies in our assumptions of the statistical circuit timing and the statistical delay defect size. Due to the statistical nature of the problem, achieving 100% diagnosis resolution cannot be guaranteed. To enhance diagnosis resolution, we propose a 3-phase diagnosis methodology. In the first phase, our goal is to quickly identify a set of candidate suspect faults that are most likely to cause the failing behavior based on logic constraints. In the second phase, we obtain a much smaller suspect fault set by applying a novel diagnosis algorithm that can effectively utilize the statistical timing information based upon a single defect assumption. In the third phase, our goal is to apply additional fine-tuned patterns to successfully narrow down to more exact suspect defect locations. Using a statistical timing analysis framework, we demonstrate the effectiveness of the proposed methodology for delay defect diagnosis, and discuss experimental results based on benchmark circuits.

The developments of efficient SAT solvers have attracted tremendous research interest in recent years. The merits of these solvers are often compared in terms of their performance based upon a wide spread of benchmarks. In this paper, we extend an earlier-proposed solver design concept called (SCGL) Signal Correlation Guided Learning that is ATPG-based into a family of heuristics. Along with this SCGL family of heuristics, we classify benchmark examples according to their performance using the SCGL heuristics. With this study, we identify the class of problems that are uniquely suitable to be solved by using the SCGL approach. In particular, for solving difficult circuit-based problems at INTEL, our SCGL-based ATPG solver is able to achieve at least an order of magnitude speedup over the state-of-the-art SAT solvers. Our conclusion is that SCGL is an unique solver design concept that can complement heuristics proposed by others for solving circuit-oriented difficult problems.

This paper describes a new post-silicon validation problem for diagnosing systematic timing errors. We illustrate the differences between timing validation and the traditional logic defect diagnosis. The key difference between our validation framework and other traditional diagnosis methods lies in our assumptions of the statistical circuit timing and statistical distribution of the size of timing errors. Different algorithms are proposed and evaluated via statistical timing error injection and simulation. Due to the statistical nature of the problem, 100% diagnosis resolution often cannot be guaranteed. With a statistical timing analysis framework developed in the past, we demonstrate the new concepts in timing validation, and discuss experimental results based upon three types of systematic errors: timing correlation error, crosstalk, and single-site random-size delay perturbation.

In this paper, we study the possibility of using logic defect-level prediction models to predict the detection behavior of statistical timing defects. We compare two known logic models: the Williams-Brown (WB) model and the Mercer-Park-Grimaila-Dworak (MPGD) model. In the WB-model, the defect coverage is replaced by the n-detection transition fault coverage. We first demonstrate that both logic models may fail to predict the detection of statistical timing defects. Then, we propose an improved WB model based upon selection of the hard-to-detect transition faults. We show that, by selecting a proper subset of the hard-to-detect transition faults, the detection behavior of these faults can correlate well to the detection behavior of statistical timing defects. We explain our findings through statistical delay defect injection and simulation, and report results based upon various benchmark circuits.

In this paper, we study the problem of delay defect diagnosis based on statistical timing models. We propose a diagnosis algorithm that can effectively utilize statistical timing information based upon single defect assumption. We evaluate its performance and its applicability to single as well as multiple defect scenarios via statistical defect injection and simulation. With a statistical timing analysis framework developed in the past, we demonstrate the new concept in statistical delay defect diagnosis, and discuss experimental results using benchmark circuits.

We propose a false-path-aware statistical timing analysis framework. In our framework, cell as well as interconnect delays are assumed to be correlated random variables. Our tool can characterize statistical circuit delay distribution for the entire circuit and produce a set of true critical paths.

In conventional delay testing, the test clock is a single pre-defined parameter that is often set to be the same as the system clock. This paper discusses the potential of enhancing test efficiency by using multiple clock frequencies. The intuition behind our work is that for a given set of AC delay patterns, a carefully-selected, tighter clock would result in higher effectiveness to screen out the potential defective chips. Then, by using a smarter test clock scheme and combining with a second set of AC delay patterns, the overall quality of AC delay test can be enhanced while the cost of including the second pattern set can be minimized. We demonstrate these concepts through analysis and experiments using a statistical timing analysis framework with defect-injected simulation.

In current industrial practice, critical path selection is an indispensable step for AC delay test and timing validation. Traditionally, this step relies on the construction of a set of worse-case paths based upon discrete timing models. The assumption of discrete timing models can be invalidated by delay effects in the deep sub-micron domain, where timing defects and process variation are statistical in nature. In this paper, we study the problem of optimizing critical path selection, under both fixed delay and statistical delay assumptions. With a novel problem formulation and new theoretical results, we prove that the problem in both cases are computationally intractable. We then discuss practical heuristics and their theoretical performance bounds, and demonstrate that among all heuristics under consideration, only one is theoretically feasible. Finally, we provide consistent experimental results based upon defect-injected simulation using an efficient statistical timing analysis framework.

In conventional delay testing, two types of tests, transition tests and path delay tests, are often considered. The test clock frequency is usually set to a single pre-determined parameter equal to the system clock. This paper discusses the potential of enhancing test effectiveness by using multiple test sets with multiple clock frequencies. The two intuitions motivating our analysis are 1) multiple test sets can deliver higher test quality than a single test set, and 2) for a given set of AC delay patterns, a carefully-selected, tighter clock would result in higher effectiveness to screen out potentially defective chips. Hence, by using multiple test sets, the overall quality of AC delay test can be enhanced, and by using multiple-clock schemes the cost of adding the additional pattern sets can be minimized. In this paper, we analyze the feasibility of this new delay test methodology with respect to different combinations of pattern sets and to different circuit characteristics. We discuss the pros and cons of multiple-clock schemes through analysis and experiments using a statistical delay evaluation and delay defect-injected framework.

In the past, symbolic trajectory evaluation (STE) has been shown to be effective for verifying individual array blocks. However, when applying STE to verify multiple array blocks together as a single system, the run-time OBDD (ordered boolean decision diagrams) sizes would often blow up. In this paper, we propose the use of both an ATPG-based justification engine and symbolic simulation to facilitate the application of STE proof methodology for array systems. Our method translates a given verification problem instance into ATPG justification objectives, and partitions a given design into ATPG and symbolic simulation domains. Then, by developing a scheme that enables the ATPG justification engine to work closely with the symbolic simulator, the runtime OBDD sizes during each symbolic simulation run can be limited. We demonstrate the effectiveness of our approach by verifying the memory management units (MMU) in Motorola high-performance microprocessors. The verification of a MMU as a whole was not possible before because of the OBDD size blow-up problem when symbolic simulation is used in the STE proof process.

When testing high-performance custom designs, the implementation models for these designs can often be missing or only available late in a design cycle. To facilitate test preparation for these designs and to ensure their test quality, this paper studies ATPG approaches whose resulting test quality are less design model dependent. These ATPG approaches do not depend on the internal implementation structure of a design and utilizes multiple detection techniques to achieve the desired test quality. As a result, test quality results are transferable from one model representation to another. For high-performance custom designs, we discover that without explicit testing of the internal faults, an ATPG is able to deliver better quality performance than traditional model-dependent approaches. Experience and experimental results from a recent Motorola high-performance microprocessor is reported and discussed.

In VLSI module placement, it is very practical to consider placing some modules along the pre-specified boundaries of the chip so that the modules are easier to be connected to certain I/O pads. In this paper, we study the module placement problem where some modules have the boundary constraints, and present a simulated annealing based algorithm that represents each placement topology by a sequence-pair. The major contribution of our algorithm is that a feasible placement is always obtainable. Our algorithm has been implemented, and its effectiveness is supported by the encouraging experimental results.

Custom circuits, in contrast to those synthesized by automatic tools, are manually designed blocks of which performance is critical to the full chip operation. Testing these blocks represents a major DFT challenge and hence, a crucial time-to-market factor in microprocessor design flow. This paper compares three industry-adopted methodologies for testing custom blocks. Pros and cons are analyzed and discussed based on factors such as stability of the methodologies, resulting sizes of gate-level models, ATPG process, and testing quality in terms of non-target defect detection. Experience and results from a recent PowerPC microprocessor are reported.

Uses data collected from benchmark circuit simulations to examine the relationship between the tests which detect stuck-at faults and those which detect bridging surrogates. We show that the coefficient of correlation between these tests approaches zero as the stuck-at fault coverage approaches 100%. An enhanced version of the MPG-D model, which is based upon the number of detections of each site in a logic circuit, is shown to be superior to stuck-at fault coverage-based defective part level prediction. We then compare the accuracy of both predictors for an industrial circuit tested using two different test pattern sequences.

Predicting the final value of the defective part level after the application of a set of test vectors is not a simple problem. In order for the defective part level to decrease, both the excitation and observation of defects must occur. This research shows that the probability of exciting an as yet undetected defect does indeed decrease exponentially as the number of observations increases. In addition, a new defective part level model is proposed which accurately predicts the final defective part level (even at high fault coverages) for several benchmark circuits and which continues to provide good predictions even as changes are made an the set of test patterns applied.

If many potential defects exist at each site in an integrated circuit, then as the number of applied test patterns increases, the number of defects which remain undetected decreases monotonically. Modeling this rate of decline in defective part level is a non-trivial problem. We show that the number of times each site is observed serves as a significantly superior basis for modeling this phenomenon when contrasted with the number of faults detected. This "site observation-based" predictor not only increases the accuracy of defective part level prediction, it also provides the first quantitative method for comparing the effectiveness of various ATPG strategies to reduce the defective part level.

Custom circuits, in contrast to those synthesized by automatic tools, are manually designed blocks of which performance is critical to the full chip operation. Testing these block represents a major challenge and hence, a crucial time-to-market factor in micro-processor design flow. This paper investigates various methodologies for testing custom blocks. Issues of efficiently obtaining proper circuit model for ATPG tools as well as producing quality tests are analyzed and discussed. Tradeoffs among various methods are analyzed and compared. Experience and results based on recent PowerPC microprocessors will be reported.

For many years, non-target detection experiments have been simulated by using AND/OR bridges or gross delay faults as surrogates. For example, the defective part level can be estimated based upon surrogate detection when test patterns target stuck-at faults in the circuit. For the first time, test pattern generation techniques that attempt to maximize non-target defect detection have been used to test a real, 100% scanned, commercial chip consisting of 75 K logic gates. In this experiment, the defective part level for REDO-based patterns was 1,288 parts per million lower than that achieved by DC stuck-at based patterns generated using today's state of the art tools and techniques.

For verifying complex sequential blocks such as microprocessor embedded arrays, the formal method of symbolic trajectory evaluation (STE) has achieved great success in the past [[3], [5], [6]]. Past STE methodology for arrays requires manual creation of "assertions" to which both the RTL view and the actual design should be equivalent. In this paper, we describe a novel method to automate the assertion creation process which improves the efficiency and the quality of array verification. Encouraging results on recent PowerPC arrays will be presented.

Design validation for embedded arrays remains as a challenging problem in today's microprocessor design environment. At Somerset, validation of array designs relies on both formal verification and vector simulation. Although several methods for array design validation have been proposed and had great success, little evidence has been reported for the effectiveness of these methods with respect to the detection of design errors. In this paper, we propose a new way of measuring the effectiveness of different validation approaches based on automatic design error injection and simulation. This technique provides a systematic way for the evaluation of the quality of various validation approaches. Experimental results using different validation approaches on recent PowerPC microprocessor arrays are reported.

Recently, formal verification has become more a part of the VLSI design methodology. Formally verifying a design guarantees 100% coverage and negates the need to do simulation. Theoretically, 100% coverage is very appealing and formal verification looks to be the panacea to solve the coverage problem. However, there are many practical considerations in deploying formal verification in real design environments. These considerations if not evaluated can lead to ineffective and even erroneous formal verification methodologies. In this paper we show how to make formal verification a successful part of a design methodology by paying attention to practical considerations and knowing the limitations of formal verification. We show the errors that can result by making over generalized assumptions and how they can be avoided. We do this in the context of the design of PowerPC microprocessors. We limit ourselves to a formal verification technique commonly used in our design methodology-boolean equivalence checking.

Design validation for embedded arrays remains as a challenging problem in today's microprocessor design environment. At Somerset, validation of array designs relies on both formal verification and vector simulation. Although several methods for army design validation have been proposed and had great success, little evidence has been reported for the effectiveness of these methods with respect to the detection of design errors. In this paper, the authors propose a way of measuring the effectiveness of different validation approaches based on automatic design error injection and simulation. The technique provides a systematic way for the evaluation of the quality of various validation approaches at both logic and transistor levels. Experimental results using different validation approaches on PowerPC microprocessor arrays will be reported.

Design validation for embedded arrays remains as a challenging problem in today's microprocessor design environment. At Somerset, validation of array designs relies on both formal verification and vector simulation. Although several methods for array design validation have been proposed and had great success [Ganguly et al. 1996; Pandey et al. 1996, 1997; Wang and Abadir 1997], little evidence has been reported for the effectiveness of these methods with respect to the detection of design errors. In this paper, we measure the effectiveness of different validation approaches based on automatic design error injection and simulation. The technique provides a systematic way to evaluate various validation approaches at both logic and transistor levels. Experimental results on recent PowerPC microprocessor arrays will be discussed and reported.

Test and validation of embedded array blocks remain as a major challenge in today's processor design environment. The difficulty comes from two folds. First, the sizes of the arrays are too large to be handled by the most sophisticated sequential ATPG tools. On the other hand, the complex timing and control make it hard to model these arrays as well-defined transparent blocks which combinational ATPG tools can understand. This paper describes a novel methodology for test and validation of complex array blocks in PowerPC RISC microprocessors. Unlike traditional ATPG methods, our methodology uses formal techniques to functionally verify the arrays and then derive tests from the verification results. The superiority of these tests over the traditional ATPG tests will be discussed and shown at the transistor level through experiments on various recent PowerPC array designs.

The traditional goal of an ATPG algorithm is to achieve a high fault coverage by producing a small number of tests. However, since usually a high fault coverage does not imply a high defect coverage, such an objective can mislead us to overtrust an ATPG method which is optimal for faults but inefficient for defects. The paper presents several new principles to design an ATPG algorithm for solve this problem. The new principles direct an ATPG algorithm to improve its efficiency and reliability for detecting the non target defects through the target faults. Theory and experiments are presented to demonstrate the superiority of the new principles and the new test generation algorithm.

The traditional ATPG method relies upon faults to target all defects. Since faults do not model all possible defects, testing quality depends on the fortuitous detection of non-target defects. By analyzing different ATPG approaches, this paper intends to identify critical factors that may greatly affect the fortuitous detection. For enhancing the fortuitous detection of non-target defects through target faults, new concepts and novel ATPG methods are proposed.

How can we guarantee that a testing method will stably and efficiently achieve a very low defective part level? Traditional testing methods rely upon faults to model all defects. As technology advances, this approach becomes increasingly questionable. If only a subset of defects are modeled as faults, then as fault coverage approaches 100%, the tests mill be more and more biased in favor of fault detection. Unfortunately, this reduces the testing efficiency for defects and limits the quality level that we can achieve. In this paper, we propose models for the testing process and suggest a solution which we call "unbiased test generation". We define two types of testing bias, and these new metrics can be used to compare and evaluate test generation methods in practice.

Testing is an indispensable process to weed out the defective parts coming out of the manufacturing process. Traditionally, test generation targets on a specific fault model, usually the single stuck-at fault model, to produce tests that are expected to identify defects such as unintended shorts and opens. With this approach, the test quality relies on fortuitous detection of the non-target defects. As the quality demands and circuit sizes increase, the feasibility of test generation on a single fault model becomes questionable. In the paper, we present empirical data from experiments on ISCAS benchmark circuits to demonstrate that using traditional methods the probability of detecting nontarget defects drops rapidly as the fault coverage approaches 100%. By assuming surrogates, we explain the mechanism which produces this effect and describe a new test pattern generation approach with better testing efficiency.

Article Free Access Share on Experience in massively parallel discrete event simulation Authors: Albert G. Greenberg View Profile , Boris D. Lubachevsky View Profile , Li-C. Wang View Profile Authors Info & Claims SPAA '93: Proceedings of the fifth annual ACM symposium on Parallel Algorithms and ArchitecturesAugust 1993 Pages 193–202https://doi.org/10.1145/165231.165256Published:01 August 1993Publication History 1citation218DownloadsMetricsTotal Citations1Total Downloads218Last 12 Months7Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF