Automatic Differentiation (AD) is a technique that computes the derivatives of numerical programs by systematically applying the chain rule, playing a critical role in domains such as machine learning, simulation, and control systems. However, parallelizing differentiated programs remains a significant challenge due to the conflict between tapes (a data structure for intermediate variable storage) and summations: the differentiation process inherently introduces inter-thread summation patterns, which require prohibitively expensive atomic operations; and traditional tape designs tightly couple data retrieval with the program’s control flow, preventing code restructuring needed to eliminate these costly dependencies.

Tensor compilers play a critical role in optimizing deep neural networks (DNNs), with memory performance emerging as a key bottleneck in code generation for DNN models. Existing tensor compilers are constrained by inefficient auto-tuning algorithms. They either must deploy coarse-grained descriptions, thus miss potential optimization, or struggle with vast search spaces, rendering auto-tuning inapplicable. Tensor compilers require a more holistic optimization of memory performance to overcome these constraints. To address this issue, we focus our optimization objective on memory performance, which allows us to design monotonic optimization methods, significantly enhancing the efficiency of auto-tuning and thus enabling auto-tuning on a fine-granularity description. Based on these observations, we propose IntelliGen, a tensor compiler with instruction-level auto-tuning and monotonic memory optimization. We design an instruction-level graph description, and a monotonic optimization method for optimization on . Benefiting from auto-tuning techniques with fine-grained description, IntelliGen demonstrates significant speedup of up to 3.13×, 3.55×, and 16.9× (averaging 1.46×, 1.85×, and 2.30×, respectively) on NVIDIA GPUs, AMD GPUs, and Cambricon MLUs over the most efficient existing frameworks.

This paper analytically derives the system reliability estimate, and the associated variance estimate for multi-state systems respectively using reliability, and variance estimates of binary-capacitated components. The derivation utilizes the universal generating function method to formulate a state table and a product expectation table when replacing two components with an equivalent virtual component. Closed-form expressions of the system reliability estimate and the associated variance estimate are formulated through an iterative derivation process. The derivation can be applied to multi-state systems with series-parallel configurations. Three example systems in the literature are used to illustrate the effectiveness and accuracy of the proposed analytical estimation approach. The confidence interval for the system reliability estimate is developed based on the derived results. The developed interval is then compared with another interval from the literature that approximated the variance estimate using a pseudo binomial distribution. Comparisons through Monte Carlo simulations on the example systems indicate that the coverage probabilities have been significantly improved by the interval constructed based on the proposed derivation.