We present a systematic and efficient ranking method to quantify the goodness of an assertion. We model dependencies among design variables as a directed graph called a variable dependency graph. We define assertion importance and assertion complexity metrics and use the dependency graph to algorithmically compute those two metrics. We repurpose an assertion coverage algorithm from the literature to form a statement-coverage-based ranking as our baseline. We compare our assertion ranking both qualitatively and quantitatively to this baseline. We demonstrate that our ranking is computationally more efficient than statement-coverage-based ranking and takes up to 4366× less computation time. We identify the potential design intents that each ranking prioritizes. We also discuss at length the effect of those prioritizations on the rank agreement and the bug detection ability of the top-ranked assertions according to the two rankings. Finally, we provide a comprehensive ranking for a set of assertions by combining our ranking and the statement-coverage-based ranking.

Assertion quality is critical to the confidence and claims in a design's verification. In current practice, there is no metric to evaluate assertions. We introduce a methodology to rank register transfer level (RTL) assertions. We define assertion importance and assertion complexity and present efficient algorithms to compute them. Our method ranks each assertion according to its importance and complexity. We demonstrate the effectiveness of our ranking for pre-silicon verification on a detailed case study. For completeness, we study the relevance of our highly ranked assertions in a post-silicon validation context, using traced and restored signal values from the design's netlist.

We analyze expert review and student performance data to evaluate the validity of the Cybersecurity Concept Inventory (CCI) for assessing student knowledge of core cybersecurity concepts after a first course on the topic. A panel of 12 experts in cybersecurity reviewed the CCI, and 142 students from six different institutions took the CCI as a pilot test. The panel reviewed each item of the CCI and the overwhelming majority rated every item as measuring appropriate cybersecurity knowledge. We administered the CCI to students taking a first cybersecurity course either online or proctored by the course instructor. We applied classical test theory to evaluate the quality of the CCI. This evaluation showed that the CCI is sufficiently reliable for measuring student knowledge of cybersecurity and that the CCI may be too difficult as a whole. We describe the results of the expert review and the pilot test and provide recommendations for the continued improvement of the CCI.