Measurement-Based Probabilistic Timing Analysis (MBPTA) produces Probabilistic Worst-Case Execution Times (pWCETs), i.e., WCET estimates associated with known low exceedance probabilities. Despite applicability and goodness-of-fit tests being used within MBPTA, any method based on the sampling of a population is subject to a degree of uncertainty. The acceptance of MBPTA in industrial engineering processes depends on obtaining enough evidence that the produced pWCETs are indeed reliable. In this paper we propose a statistical hypothesis test to check the reliability of pWCET estimates, done at a specified significance level. We assume as null hypothesis that the pWCET estimate is reliable, and as alternative hypothesis that it is optimistic. Both Type I and Type II errors are considered. The reliability test is based on a binomial experiment and it is complementary to applicability and goodness-of-fit tests. We evaluated the test using multiple synthetic and real-hardware execution time samples, and applied it on 20 pWCET estimates generated for each of them. The combined use of the proposed reliability test with applicability and goodness-of-fit tests could detect most of the knowingly unreliable estimates on synthetic samples. Similar behaviour was observed for real-hardware samples, evidencing the test's usefulness for selecting pWCET estimates with increased confidence.

In order to support the computational demand of Real-Time Systems' (RTSs`) applications, the use of complex hardware will be fundamental. This fact makes analysis of software to derive Worst-Case Execution Time (WCET) an increasingly harder challenge. State-of-the-art WCET analysis techniques are hampered by the ever-growing cost and complexity of obtaining accurate knowledge of the internal operation of advanced processors, and by difficulties in reliably associating variable execution times with tasks' worst-case behaviour through measurement-based techniques. In this work, we performed an empirical assessment of the adequacy of Measurement-Based Probabilistic Timing Analysis (MBPTA) to estimate the pWCET - through Extreme Value Theory (EVT) - of a task executed on a complex computer architecture with Linux. The proposed evaluation has the objective of detecting whether EVT can be deemed suitable for determining pWCET estimates for tasks executed on complex computers.

As computer architectures evolve, guaranteeing that Real-Time Systems’ (RTSs’) timing requirements are met through Worst Case Execution Time (WCET) upper bounds becomes increasingly difficult. Techniques such as Measurement-Based Probabilistic Timing Analysis (MBPTA) have emerged that estimate WCET bounds exceeded only with arbitrarily low probabilities (i.e., pWCETs) through Extreme Value Theory (EVT). The Peaks Over Threshold (POT) approach for applying EVT involves adjusting a tail-shaped distribution, e.g., Generalized Pareto (GP) or Exponential, to the values that exceed a carefully selected high threshold. Several works suggest that GP should be used within POT for best representing different tail shapes, while others consider the Exponential model more adequate for providing upper bounds with increased reliability. This work presents empirical reliability and tightness evaluations of the pWCET estimates yielded by the GP and Exponential models while applying MBPTA through the POT approach. It mainly provides counter-evidence to the GP model reliability and evidence of the Exponential model adequacy in this context.

The technique known as Measurement-Based Probabilistic Timing Analysis (MBPTA) promises producing Worst-Case Execution Time (WCET) bounds for real-time systems' tasks based on the analysis of execution time measurements through Extreme Value Theory (EVT), a statistical framework designed to estimate the probability of extreme events. For that MBPTA requires the analysed tasks' maximum observed execution times to adhere to an extreme value distribution, such as Gumbel or Generalized Extreme Value (GEV), and allows determining execution time values expected to be exceeded only with arbitrarily small probabilities. Several works on the area assume that the Gumbel model should be employed in such analysis, while others consider GEV, which generalizes Weibull, Gumbel and Fréchet models, would be more adequate. In this work we perform an empirical assessment on the reliability and tightness of the WCET bounds determined through the GEV and Gumbel models. We do so by comparing the yielded estimates and their associated confidence intervals against the maximum values observed on large samples (e.g. of size 100 million), of both real and synthetic nature, as the modelling sample size is increased.

This paper presents a method to determine the latency of memory instructions, aimed for use in WCET tools. We use a technique known as value analysis. Considering a processor with different data memory latencies, value analysis is used to determine the possible values of processor registers statically, allowing the recognition of memory addresses (main memory or ScratchPad Memory) in order to obtain tighter WCET upper bounds.