Log anomaly detection refers to the task that distinguishes the anomalous log messages from normal log messages. Transformer-based large language models (LLMs) are becoming popular for log anomaly detection because of their superb ability to understand complex and long language patterns. In this paper, we propose LogLLaMA, a novel framework that leverages LLaMA2. LogLLaMA is first finetuned on normal log messages from three large-scale datasets to learn their patterns. After finetuning, the model is capable of generating successive log messages given previous log messages. Our generative model is further trained to identify anomalous log messages using reinforcement learning (RL). The experimental results show that LogLLaMA outperforms the state-of-the-art approaches for anomaly detection on BGL, Thunderbird, and HDFS datasets.

Effectively integrating knowledge into end-to-end task-oriented dialog systems remains a challenge. It typically requires incorporation of an external knowledge base (KB) and capture of the intrinsic semantics of the dialog history. Recent research shows promising results by using Sequence-to-Sequence models, Memory Networks, and even Graph Convolutional Networks. However, current state-of-the-art models are less effective at integrating dialog history and KB into task-oriented dialog systems in the following ways: 1. The KB representation is not fully context-aware. The dynamic interaction between the dialog history and KB is seldom explored. 2. Both the sequential and structural information in the dialog history can contribute to capturing the dialog semantics, but they are not studied concurrently. In this paper, we propose a novel Graph Memory Network (GMN) based Seq2Seq model, GraphMemDialog, to effectively learn the inherent structural information hidden in dialog history, and to model the dynamic interaction between dialog history and KBs. We adopt a modified graph attention network to learn the rich structural representation of the dialog history, whereas the context-aware representation of KB entities are learnt by our novel GMN. To fully exploit this dynamic interaction, we design a learnable memory controller coupled with external KB entity memories to recurrently incorporate dialog history context into KB entities through a multi-hop reasoning mechanism. Experiments on three public datasets show that our GraphMemDialog model achieves state-of-the-art performance and outperforms strong baselines by a large margin, especially on datatests with more complicated KB information.

Spoken language understanding, usually including intent detection and slot filling, is a core component to build a spoken dialog system.Recent research shows promising results by jointly learning of those two tasks based on the fact that slot filling and intent detection are sharing semantic knowledge.Furthermore, attention mechanism boosts joint learning to achieve state-of-the-art results.However, current joint learning models ignore the following important facts: 1.Long-term slot context is not traced effectively, which is crucial for future slot filling.2. Slot tagging and intent detection could be mutually rewarding, but bidirectional interaction between slot filling and intent detection remains seldom explored.In this paper, we propose a novel approach to model long-term slot context and to fully utilize the semantic correlation between slots and intents.We adopt a key-value memory network to model slot context dynamically and to track more important slot tags decoded before, which are then fed into our decoder for slot tagging.Furthermore, gated memory information is utilized to perform intent detection, mutually improving both tasks through global optimization.Experiments on benchmark ATIS and Snips datasets show that our model achieves state-of-the-art performance and outperforms other methods, especially for the slot filling task.

We present an approach to generate assertions from natural language specifications by performing semantic analysis of sentences in the specification document. Other techniques for automatic assertion generation use information found in the design implementation, either by performing static or dynamic analysis. Our approach generates assertions directly from the specification document, so bugs in the implementation will not be reflected in the assertions. Our approach parses each sentence and examines the resulting syntactic parse trees to locate subtrees which are associated with important phrases, such as the antecedent and consequent of an implication. Formal assertions are generated using the information inside these subtrees to fill a set of assertion templates which we present. We evaluate the effectiveness of our approach using a set of statements taken from a real specification document.

We present an approach to simplify the task of extracting assertions from specifications given in natural language. Our goal is to accept and understand a broad range of linguistic variation, allowing the author of the natural language specifications to express herself freely. To enable this, we leverage the Dialogflow framework from Google. Dialogflow is usually used to build chatbots that understand and respond to conversational statements. We have trained a Dialogflow model to recognize a range of different natural language expressions of properties, and to identify key information inside the expression. The model responses to each statement with a generated SystemVerilog assertion whose semantic meaning is equivalent to that of the English statement.

We present an outline of the field of Multilevel Design Understanding by first defining and motivating the related problems, and then describing the key issues which must be addressed in future research.

As new features are introduced into the Bluetooth core specification, the ability to use Bluetooth Low Energy (BLE) technology to construct a mobile ad-hoc network (MANET) becomes a reality. Key features included in Bluetooth Specification version 4.1 are the ability for a single node to be part of multiple piconets, and the ability for a node to act dual mode, as both a piconet master and slave. These features allow the possibility of multi-hop routing spanning multiple connected piconets. Although multi-hop routing is theoretically possible with version 4.1, no multi-hop routing algorithm has never been presented which exploits this possibility. In this paper we propose an approach to scatternet formation and multi-hop routing for networks using BLE version 4.1. We define procedures for device discovery, communication between piconets and forming multi-hop scatternet. Our approach has the following properties, 1) it can be used with existing Bluetooth hardware, 2) the protocol is fully distributed, so global connectivity information is not required for formation and routing, and 3) it supports ad-hoc network formation. We have implemented our approach using real Bluetooth SoCs including the Broadcom BCM434x chipset which is used in the iPhone 6. Our experiments demonstrate the routing delay and throughput using networks containing different numbers of nodes in order to demonstrate the impact of network size on performance. As the network size increases, our protocol does not incur a large delay and achieves better resource utilization.

In order to verify natural language assertions from a specification automatically, they need to be translated into formal representations. This process is error-prone and can lead to a product that does not meet the initial intentions.We automate this process by first partitioning all assertions into subsets based on sentence similarity and then providing a translation template for each subset which must be completed by the designer. Since many assertions are described by similar sentences, the number of manual translation steps can be decreased significantly. We evaluated our approach by translating English constraint sentences from an industrial specification into SystemVerilog assertions.

There is a growing interest in software summarization and tools for automatically producing summaries. Discussions of relevant papers at recent conferences led to the observation that software summarization needs to consider migrating away from ``is this a good summary?" and towards ``is this a useful summary?" As a result, it has been suggested that to judge usefulness, one needs to view the summary through the lens of a particular task. A preliminary investigation of this suggestion was undertaken at the 2013 ICSE workshop NaturaLiSE. Initial results and lessons learned from this investigation support the notion that task plays a significant role and thus should be considered by researchers building and accessing automatic software summarization tools.

Natural language specifications are the first concrete behavioral description which is the basis for any manually generated formal behavioral model. Natural language is preferred as the initial description method mainly because it is much simpler for a designer to use than existing hardware description languages. The focus of this project is the extraction of behavioral information from a natural language specification to generate a formal behavioral description with clear and unambiguous semantics. In the initial effort presented here, we employ semantic parsing to identify key information describing bus transactions in the natural language specification. The identified information is used to generate Verilog tasks which embody bus transactions. To our knowledge, the work presented here is the first attempt to generate simulatable Verilog from natural language descriptions.

Intrusion detection approaches have been presented which detect anomalous malware behavior at runtime. Most techniques involve software-based analysis which is too slow to support the tight timing constraints often imposed on embedded systems. We propose a hardware-based intrusion detection approach which does not alter the functional performance of the system. When using a real-time operating system, the executing process changes several times each second, requiring fast adaptation on the part of the intrusion detection mechanism. We present a technique to exploit the partial runtime reconfiguration feature present on many modern field programmable gate arrays (FPGAs) to adapt intrusion detection to a new process at each context switch. The use of runtime reconfiguration enables the flexibility of software-based approaches with the performance benefits of hardware-based approaches.

We propose Specification-Based Test Generation (SBTG) which automatically generates functional tests directly from specification, rather than the HDL description of the design. The main benefit of generating tests from the specification is the ability to detect Specification-based Translation Errors (SBTEs) that occur due to a misunderstanding of the specification. Our results show that our test generation approach is more effective at detecting these errors than approaches that generate tests from the HDL code to maximize code coverage metrics.

As an industrial practice, the functional coverage models are developed based on a high-level specification of the design under verification (DUV). However, in the course of implementation a designer makes specific choices which may not be reflected well in a functional coverage model developed entirely from a high-level specification. We present a method to automatically generate implementation-aware coverage models based on the static analysis of a HDL description of the DUV. Experimental results show that the functional coverage models generated using our technique correlate well with the detection of randomly injected errors into a design

We present a coverage metric which evaluates the testing of a set of interacting concurrent processes. Existing behavioral coverage metrics focus almost exclusively on the testing of individual processes. However the vast majority of practical hardware descriptions are composed of many processes which must correctly interact to implement the system. Coverage metrics which evaluate processes separately are unlikely to model the range of design errors which manifest themselves when components are integrated to build a system. A metric which models component interactions is essential to enable validation techniques to scale with growing design complexity. We describe the effectiveness of our metric and provide results to demonstrate that coverage computation using our metric is tractable

THE IEEE High Level Design Validation and Test (HLDVT) Workshop was originally established to act as a focal point for research in verification and simulation. This special section on simulation-based design validation is based on the best papers in the Proceedings of HLDVT 2005. The focus areas of HLDVT are well represented by the papers in this special section, including formal verification and simulation-based functional verification research. Several papers in this special section focus on functional test generation, the process of generating test sequences for use during simulation. Boolean satisfiability is a fundamental solving technique which is applied to a wide range of design automation problems, including automatic test generation and model checking. An extension to SAT solving theory is presented in “B-Cubing: New Possibilities for Efficient SAT-Solving” by Domagoj Babi c, Jesse Bingham, and Alan J. Hu, which generalizes the supercubing framework presented in earlier work. This paper proves the correctness of the proposed B-cubing technique using a new construct created for that purpose, the obligation-certification tree. “A New Simulation-Based Property Checking Algorithm Based on Partitioned Alternative Search Space Traversal” by Qingwei Wu and Michael S. Hsiao describes a test generation tool which is used to search for violations of safety properties. A genetic algorithm is used to generate test sequences, together with a Boolean constraint propagation engine which reduces the search space. “SimulationBased Functional Test Generation for Embedded Processors” by Charles H.-P. Wen, Li-C. Wang, and Kwang-Ting Cheng presents a functional test generation technique which uses simulation results to reduce the complexity of the search process. When test generation is performed for a component under test, the justification process must involve many additional surrounding components which are involved in fault activation and propagation. The technique presented in this paper simulates surrounding components and uses machine learning to generate a “functional mapping” which represents the inverse function of the component and is used during justification. The functional mapping is approximate, but it enables an efficient solution to the justification problem. In the paper “Harnessing Machine Learning to Improve the Success Rate of Stimuli Generation” by Shai Fine, Ari Freund, Itai Jaeger, Yishay Mansour, Yehuda Naveh, and Avi Ziv, researchers apply learning theory to establish a relationship between the start state of the search process and the success rate of test generation. The choice of start state is important because it has a strong impact on search complexity. Once a design error has been revealed by simulation, there remains the task of identifying the source cause of the error by examining the error trace. In the paper “An Optimum Algorithm for Compacting Error Traces for Efficient Design Error Debugging” by Chian-Chih Yen and Jing-Yang Jou, a technique is presented to reduce the length of an existing error trace in order to make it more easy to understand and examine. Two heuristics are presented to modify the detecting input sequence to reach the same error detecting state with fewer state machine transitions. The functional verification process often depends on the use of coverage metrics to evaluate the completeness of the verification process. Coverage metrics define a set of coverage events whose occurrence during testing is required to reveal design errors. The paper “Advanced Analysis Techniques for Cross-Product Coverage” by Hezi Azatchi, Laurent Fournier, Eitan Marcus, Shmuel Ur, Avi Ziv, and Keren Zohar evaluates the use of crossproduct coverage metrics which combine to monitor the occurrence of events which are defined by the cross-product of two or more sets of independent coverage events. The paper explores the relationship between a variety of crossproduct coverage techniques and the quality of verification. Two of the papers in the special section address the verification of essential but nontraditional aspects of design correctness. “Multilevel Design Validation in a Secure Embedded System” by Parick Schaumont, David Hwang, Shenglin Yang, and Ingrid Verbauwhede describes a simulation-based methodology to validate the security properties of an embedded system. The approach seeks to identify leaks of critical information at all multiple abstraction levels, from the software algorithms down to the microarchitecture. A cycle-accurate cosimulation platform is presented which supports the validation process. “Validating Families of Latency Insensitive Protocols” by Syed Suhaib, Deepak Mathaikutty, David Berner, and Sandeep Shukla presents both simulation-based and formal techniques to validate system correctness with respect to latency. The difficulty of transmitting a low-skew clock over an entire chip has motivated the development of latency insensitive protocols which enable asynchronous communication. This paper presents techniques to verify that a latency insensitive system is “latency equivalent” to a completely synchronous version of the system. IEEE TRANSACTIONS ON COMPUTERS, VOL. 55, NO. 11, NOVEMBER 2006 1313

Coverage metrics, which evaluate the ability of a test sequence to detect design faults, are essential to the validation process. A key source of difficulty in determining fault detection is that the control flow path traversed in the presence of a fault cannot be determined. Fault detection can only be accurately determined by exploring the set of all control flow paths, which may be traversed as a result of a fault. We present a coverage metric that determines the propagation of fault effects along all possible faulty control flow paths. The complexity of exploring multiple control flow paths is greatly alleviated by heuristically pruning infeasible control flow paths using the algorithm that we present. The proposed coverage metric provides high accuracy in designs that contain complex control flow. The results obtained are promising.

In this paper we present a flexible automatic test generation framework to detect a variety of design faults in systems with behavioral VHDL descriptions. Predefined fault models may range from the commonly used state coverage and transition coverage models to any other fault models which can be described as a set of non-linear constraints on the system's behavior. The test generation problem is formulated as a constraint logic programming problem (CLP) and an industrial CLP engine is used to solve it.

This paper describes a novel deterministic globally-asynchronous locally-synchronous (GALS) methodology called "synchro-tokens". Wrappers around the synchronous blocks keep the system globally asynchronous while ensuring that each transition, although arriving at a nondeterministic time, is sensed by the synchronous block during a deterministic cycle of the local clock. This determinism facilitates debug and test methodologies, such as the use of stored-pattern testers, which are effective only when the system behavior is predictable and repeatable. Applications of synchro-tokens to GALS systems with two or more synchronous blocks and one or more asynchronous data channels are shown. Synchro-tokens supports both pipelined and unpipelined channels and a variety of clock generation methodologies. Novel schematic level designs of the wrapper components in a 180-nm technology are used to compare the performance of several different deterministic GALS design styles.

No abstract available.

Globally asynchronous locally synchronous (GALS) clocking applied to a system-on-a-chip (SoC) results in a design in which each core is a synchronous block (SB) of logic with a locally generated clock. Inter-core communication is asynchronous and controlled by wrapper logic around the cores. The nondeterministic synchronization used by most GALS architectures makes chip-level silicon debug and functional test difficult and costly. Deterministic GALS methodologies make dataflow assumptions which are only valid for a very limited set of applications. This paper describes a novel deterministic GALS methodology called "synchro-tokens" whose parameterized wrappers are flexible enough to be useful for a wide range of applications while supporting synchronous debug and test methodologies such as 1149.1 and P1500. The validation of determinism, estimation of area overhead, and analysis of performance impact are detailed.

A new partial built-in self-test (BIST) insertion approach based on eliminating data correlation to improve pseudorandom testability is presented. Data correlation causes the circuit to be in a subset of states more or less frequently, which leads to low fault coverage in pseudorandom test. One important cause of correlation is reconvergent fanout. Incorporating BIST test flip-flops into reconvergent paths will break correlation, however, breaking all reconvergent fanout is unnecessary since some reconvergent fanout results in negligible correlation. We introduce a metric to determine the degree of correlation caused by a set of reconvergent fanout paths. We use this metric to identify problematic reconvergent fanout which must be broken through partial BIST insertion. Based on this metric, we provide an exact method and a heuristic method to measure the data correlation. We provide an algorithm to break high correlation reconvergent paths. Our algorithm provides high fault coverage while selecting fewer BIST flip-flops than required using loop-breaking techniques. Experimental results produced using our exact algorithm rank on average among the top 11.6% of all possible solutions with the same number of flip-flops.

As IC densities are increasing, cluster-based field programmable gate arrays (FPGA) architectures are becoming the architecture of choice for major FPGA manufacturers. A cluster-base architecture is one in which several logic blocks are grouped together into a coarse-grained logic block. While the high-density local interconnect often found within clusters serves to improve FPGA utilization, it also greatly complicates the FPGA interconnect testing problem. To address this issue, we have developed a hierarchical approach to define a set of FPGA configurations which enable interconnect fault detection and diagnosis. This technique enables the detection of bridging faults involving intracluster interconnect and extracluster interconnect. The hierarchical structure of a cluster-based tile is exploited to define intracluster configurations separately from extracluster configurations, thereby improving the efficiency of the configuration definition process. The cornerstone of this work is the concise expression of the detectability conditions of each fault and the distinguishability conditions of each fault pair. By guaranteeing that both intracluster and extracluster configurations have several test transparency properties, hierarchical fault detectability is ensured.

The widespread use of field programmable gate arrays (FPGAs) as components in high-performance systems has increased the significance of path delay faults in FPGAs. We present a technique for FPGA path delay fault detection which integrates test insertion with the FPGA placement and routing stages to accomplish testing with low test application time. An accurate static timing analyzer is used to identify critical paths and built-in self-test (BIST) hardware is inserted using a placement and routing tool. Initial experimental results show that testing is accomplished with low test application time for several benchmark designs.

The violation of timing constraints on signals within a complex system can create timing-induced functional errors which alter the value of output signals. These errors are not detected by traditional functional validation approaches because functional validation does not consider signal timing. Timing-induced functional errors are also not detected by traditional timing analysis approaches because the errors may affect output data values without affecting output signal timing. A timing fault model, the Mis-Timed Event (MTE) fault model, is proposed to model timing-induced functional errors. The MTE fault model formulates timing errors in terms of their effects on the lifespans of the signal values associated with the fault. We use several examples to evaluate the MTE fault model. MTE fault coverage results shows that it efficiently captures an important class of errors which are not targeted by other metrics.

As IC densities are increasing, cluster-based FPGA architectures are becoming the architecture of choice for major FPGA manufacturers. A cluster-based architecture is one in which several logic blocks are grouped together into a coarse-grained logic block. While the high density local interconnect often found within clusters serves to improve FPGA utilization, it also greatly complicates the FPGA interconnect testing problem. To address this issue, we have developed a hierarchical approach to define a set of FPGA configurations which enable interconnect faults to be detected. This technique enables the detection of bridging faults involving intra-cluster interconnect and extra-cluster interconnect. The hierarchical structure of a cluster-based tile is exploited to define intra-cluster configurations separately from extra-cluster configurations, thereby improving the efficiency of the configuration definition process. By guaranteeing that both intra-cluster and extra-cluster configurations have several test transparency properties, hierarchical fault detectability is ensured.

Fault diagnosis has particular importance in the context of field programmable gate arrays (FPGAs) because faults can be avoided by reconfiguration at almost no real cost. Cluster-based FPGA architectures, in which several logic blocks are grouped together into a coarse-grained logic block, are rapidly becoming the architecture of choice for major FPGA manufacturers. The high density interconnect found within clusters greatly complicates the problem of FPGA diagnosis. We propose a technique for the testing and diagnosis of cluster-based FPGA architectures. We present a hierarchical approach to define a set of FPGA configurations in which each fault is detectable, and each fault pair is differentiable. The cornerstone of this work is the concise expression of the distinguishing conditions of each fault pair. Experimental results demonstrate that nearly 100% fault coverage and diagnostic resolution are achieved with a low number of test configurations.

Behavioral HDL descriptions are commonly used to capture the high-level functionality of a hardware circuit for simulation and synthesis. The manual process of creating a behavioral description is error prone, so significant effort must be made to verify the correctness of behavioral descriptions. Simulation-based validation and formal verification are both techniques used to verify correctness. We investigate validation because formal verification techniques are frequently intractable for large designs. The first step toward a behavioral validation technique is the development of a validation fault coverage metric which can be used to evaluate the likelihood of design defect detection with a given test sequence. We propose a validation fault coverage metric which is based on an analysis of the control data flow description associated with the behavior. The proposed metric identifies a subset of paths through the data flow which must be traversed during testing to detect faults. The proposed metric is a tractable compromise between the statement coverage metric which requires only that each statement be executed, and the path coverage metric which requires that all data flow paths be executed. Data flow paths are identified based on the relative code locations of definitions and uses of variables which may be assigned incorrectly due to a design error. We propose an efficient method to compute all data flow paths which must be traversed, and we generate coverage results for several benchmark VHDL circuits for comparison to other approaches.

During functional validation, corner cases and boundary conditions are known to be difficult to verify. We propose a functional fault coverage metric for behavioral VHDL descriptions which evaluates the coverage of faults which cause the behavior to execute an incorrect functional domain. Although domain faults are known to be a source of design errors, they are not targeted explicitly by previous work in functional validation. We propose an efficient method to compute domain coverage in VHDL descriptions and we generate domain coverage results for several benchmark VHDL circuits for comparison to other approaches.

A new partial BIST insertion approach based on eliminating data correlation to improve pseudo-random testability is presented. Data correlation causes the circuit to be in a subset of the states more or less frequently, which leads to low fault coverage in pseudo-random test. One important cause of correlation is reconvergent fanout. Incorporating BIST test flip-flops into reconvergent paths will break correlation, however, breaking all reconvergent fanout is unnecessary since some reconvergent fanout results in negligible correlation. We introduce a metric to determine the degree of correlation caused by a set of reconvergent fanout paths. We use this metric to identify problematic reconvergent fanout which must be broken through partial BIST insertion. We provide an algorithm to break high correlation reconvergent paths. Our algorithm provides high fault coverage while selecting fewer BIST flip-flops than required using loop breaking techniques. Experimental results produced using our algorithm rank on average among the top 11.6% of all possible solutions with the same number of flip-flops.

The testability of a VLSI design is strongly aected by its register-transfer level (RTL) structure.Since the high-level synthesis process determines the RTL structure, it is necessary to consider testability during high-level synthesis.A synthesis system composed of scheduling and binding components minimizes the number of hardware sharing con icts between tests in the test schedule.Novel test con ict estimates are used to direct the synthesis process.The test con ict estimation is based on examination of the in terconnect structure of the partial design state during synthesis.Test con ict estimates enable our synthesis system to select design options which increase test concurrency, thereby decreasing test time.Experimen tal results show that designs generated by this approach are testable in a highly concurrent manner.

We present a system which synthesizes, from a behavioral description, an RTL circuit which is testable with a high degree of test concurrency. The system produces a datapath containing test registers, and a BIST test plan for the testing of the chip. All design decisions are made using an estimate of test conflicts, which is based on an analysis of the reachability of each component port from I/O pins and test registers. Chip testing according to the partial-intrusion BIST methodology is assumed. Empirical results show the effect of test conflicts on the test application time, and highlight the benefit of using the proposed synthesis approach for test conflict reduction.>

The high cost of chip testing makes testability an important aspect of any chip design. Early inclusion of test considerations into the design process can reduce test time and area overhead. We have developed an algorithm which defines built-in self-testing (BIST) tests for an RTL datapath. Datapath registers are chosen to be upgraded to testable registers to execute the define tests. Test time of an individual test is reduced by considering pattern randomness and error masking transparency properties of modules involved in the test. Parallelism between different tests is increased by reducing the number of conflicts between tests. Intertwining the creation of tests with the selection of testable registers in the datapath guides the choice of testable registers to a minimal area solution. The use of these metrics provides our system with an accurate estimate of test time, and therefore facilitates the definition of tests which reduce test application time. Experimental results show that our algorithm defines tests which require low test time.>