Integrating AI hardware accelerators into safety-critical real-time systems to speed up the inference execution of safety-critical AI applications demands rigorous assurance to prevent potentially catastrophic outcomes, especially in environments where timely and accurate results are crucial. Even in cases where AI models are potentially designed and constructed correctly using AI frameworks, the systems safety will also rely on the real-time behavior of the AI hardware accelerator. While AI hardware accelerators can achieve the necessary throughput, conventional accelerators such as the Versatile Tensor Accelerator (VTA) encounter significant challenges in predictability and reliability. These challenges stem from the variability in event-driven inference execution and insufficient timing control, posing considerable risks in safety-critical scenarios where delays in providing inference results can have severe consequences. To address this challenge, previous work introduced the Time-Triggered Versatile Tensor Accelerator (TT-VTA) to ensure timely execution of tensor operations. Nonetheless, the TTVTA exhibited a slightly longer average inference time of 53ms compared to the conventional VTAs 51ms, underscoring the ongoing need for optimization in this crucial domain to speed up the inference execution, while sustaining the deterministic and predictable behavior of the TT-VTA. This paper proposes a novel Superscalar Time-Triggered VTA (STT-VTA) architecture specifically designed to address the deficiencies of conventional VTAs and TT-VTAs. The STT-VTA architecture employs pattern-based timing schedules generated by an extended software simulator and an architecture configuration manager to analyze tensor operations within a given AI model and determine the required number of additional VTA modules for faster inference than a single (TT-)VTA setup. It integrates DRAMSim2 for memory instructions and a cycle-accurate simulator for non-memory instructions. Evaluation using various models demonstrates that the STT-VTA achieves identical classification accuracy as the conventional VTA and TT-VTA, while improving performance and reducing inference time by 20-41%. Moreover, it ensures deterministic temporal use of shared resources such as memories and memory-buses and precise timing control to avoid interference. These results contribute towards safety and reliability of AI systems deployed in a safety-critical environment

This paper introduces an innovative simulation-based analysis and optimizing technique for managing memory access in neural network deployment, specifically focusing on the Versatile Tensor Accelerator (VTA). Our work addresses critical issues and challenges in memory access involving inefficient resource utilization, resource contention, memory collisions, unpredictable memory access, and erratic behavior, which are detrimental to safety-critical applications. These challenges underscore the need to not only eliminate memory contention but also improve memory throughput and efficiently utilize memory access idle phases while ensuring deterministic execution. Our proposed techniques aim to enhance the safety and predictability of neural networks by offering a schedule guaranteeing timing while significantly improving memory access efficiency. Our approach strategically leverages idle zones for prefetching data, achieving 5% overall time savings in memory execution. Evaluated on the simulation testbench, this advancement paves the way for further hardware integration with a time-triggered controller, optimizing memory access across diverse neural network architectures. This advancement represents progress towards a safer, more predictable, and more efficient neural network memory access architecture for applications in safety-critical domains.

Safety-critical AI systems demand the utmost reliability to prevent catastrophic outcomes. However, conventional AI hardware accelerators, such as the Versatile Tensor Accelerator (VTA), suffer from limitations in predictability and reliability due to inherent variability in event-driven task execution and the lack of precise timing control. These limitations make them unsuitable for safety-critical applications where erroneous or untimely operations could lead to catastrophic consequences. This paper proposes a novel time-triggered VTA (TT-VTA) architecture specifically designed to address the shortcomings of conventional VTAs and enhance safety and reliability in safety-critical AI systems. The TT-VTA architecture utilizes pattern-based timing schedules which are generated by a software simulator that integrates DRAMSim2 to simulate memory-related instructions as well as by a cycle-accurate simulator to provide accurate cycle counts for non-memory-related instructions. A comparative evaluation using a ResNet18 classification model demonstrates that the TT-VTA achieves identical classification accuracy while maintaining deterministic resource utilization and enabling precise timing control. Our proposed TT-VTA architecture demonstrates significant promise for enhancing the safety and reliability of AI systems in safety-critical applications.

Embedded systems powered by artificial intelligence (AI) are widely employed in diverse domains. However, the lack of inherent predictability in existing AI accelerators poses significant challenges, especially in safety-critical applications where deterministic safety specifications are essential. Moreover, temporal unpredictability caused by memory access contention further hinders the suitability of current platforms for safety-critical tasks. To address these issues, we propose a time-triggered memory access approach for the versatile tensor accelerator (VTA) that provides temporal predictability guarantees for load data. Our research focuses on investigating the temporal predictability of Neural Network load input, weight, and operations. We introduce a time-triggered memory access mechanism that pre-fetches data from DDR memory ahead of the VTA load module request. Through extensive experimentation and analytical evaluations in the VTA runtime environment, we demonstrate the effectiveness of our time-triggered memory access concept. The results reveal an 8% performance improvement and reduced execution time while upholding strict safety specifications and predictability. These findings establish the feasibility of employing a time-triggered approach for runtime neural network prediction.