Uncertainty estimation in Neural Networks (NNs) is vital in improving reliability and confidence in predictions, particularly in safety-critical applications. Bayesian Neural Networks (BayNNs) with Dropout as an approximation offer a systematic approach to quantifying uncertainty, but they inherently suffer from high hardware overhead in terms of power, memory, and computation. Thus, the applicability of BayNNs to edge devices with limited resources or to high-performance applications is challenging. Some of the inherent costs of BayNNs can be reduced by accelerating them in hardware on a Computation-In-Memory (CIM) architecture with spintronic memories and binarizing their parameters. However, numerous stochastic units are required to implement conventional Dropout-based BayNN. In this paper, we propose the Scale Dropout, a novel regularization technique for Binary Neural Networks (BNNs), and Monte Carlo-Scale Dropout (MC-Scale Dropout)-based BayNNs for efficient uncertainty estimation. Our approach requires only one stochastic unit for the entire model, irrespective of the model size, leading to a highly scalable Bayesian NN. Furthermore, we introduce a novel Spintronic memory-based CIM architecture for the proposed BayNN that achieves more than 100× energy savings compared to the state-of-the-art. We validated our method to show up to 1% improvement in predictive performance and superior uncertainty estimates compared to related works.

Resistive Random Access Memory (RRAM)-based analog in-memory computing (IMC) AI accelerators offer significant advantages over digital accelerators, including lower power consumption, reduced data movement, and higher computational efficiency. However, their deployment in safety-critical and edge applications is challenging due to their hardware non-idealities, such as programming error, conductance drift, and read noise, which degrade the inferencing accuracy of the implemented neural networks (NNs). Existing methods, including noise injection during training and activation function modifications, provide limited fault-tolerance in realistic scenarios with non-idealities. We propose a fault-tolerant activation function with architectural optimization that enhances robustness against hardware-induced variations with minimal hardware and NN architectural changes. During training, the proposed activation function features a stochastic negative region, which inherently injects noise into the negative region of the activation. During inferencing, the proposed activation function operates deterministically, ensuring compatibility with existing hardware while maintaining computational efficiency. Extensive evaluations with benchmark datasets demonstrate that the proposed approach significantly improves inferencing accuracy by up to 60% under varying noise levels, outperforming conventional activation functions as well as existing fault-tolerant activation functions. By enhancing fault-tolerance to hardware-induced errors, the proposed method enables reliable and energy-efficient RRAM-based analog IMC.

Achieving optimal test coverage and quality with fewer test patterns is a persistent challenge for DFT teams. This endeavor demands expert user involvement and lengthy iterative processes to fine-tune various tool parameters specific to each design core. Additionally, optimizing the test pattern count and reducing test data volume require refining the test configuration and compression logic during design implementation. This presentation explores how AI-driven DFT solutions can optimize the number of structural test (Scan/ATPG) patterns to meet target test coverage. By integrating advanced AI techniques in ATPG pattern generation and test configuration during DFT implementation, we demonstrate significant improvements in the engineering efficiency and test cost reduction.

Bayesian Neural Networks (BayNNs) naturally provide uncertainty in their predictions, making them a suitable choice in safety-critical applications. Additionally, their realization using memristor-based in-memory computing (IMC) architectures enables them for resource-constrained edge applications. In addition to predictive uncertainty, however, the ability to be inherently robust to noise in computation is also essential to ensure functional safety. In particular, memristor-based IMCs are susceptible to various sources of non-idealities such as manufacturing and runtime variations, drift, and failure, which can significantly reduce inference accuracy. In this paper, we propose a method to inherently enhance the robustness and inference accuracy of BayNNs deployed in IMC architectures. To achieve this, we introduce a novel normalization layer combined with stochastic affine transformations. Empirical results in various benchmark datasets show a graceful degradation in inference accuracy, with an improvement of up to 58.11%.

Internet of Things (IoT) and smart wearable devices for personalized healthcare will require storing and computing ever-increasing amounts of data. The key requirements for these devices are ultra-low-power, high-processing capabilities, autonomy at low cost, as well as reliability and accuracy to enable Green AI at the edge. Artificial Intelligence (AI) models, especially Bayesian Neural Networks (BayNNs) are resource-intensive and face challenges with traditional computing architectures due to the memory wall problem. Computing-in-Memory (CIM) with emerging resistive memories offers a solution by combining memory blocks and computing units for higher efficiency and lower power consumption. However, implementing BayNNs on CIM hardware, particularly with spintronic technologies, presents technical challenges due to variability and manufacturing defects. The NeuSPIN project aims to address these challenges through full-stack hardware and software co-design, developing novel algorithmic and circuit design approaches to enhance the performance, energy-efficiency and robustness of BayNNs on sprintronic-based CIM platforms.

Bayesian Neural Networks (BayNNs) can inherently estimate predictive uncertainty, facilitating informed decision-making. Dropout-based BayNNs are increasingly implemented in Spintronics-based computation-in-memory architectures for resource-constrained yet high-performance safety-critical applications. Although uncertainty estimation is important, the reliability of Dropout generation and BayNN computation is equally important for target applications but is overlooked in existing works. However, testing BayNNs is significantly more challenging compared to conventional NNs, due to their stochastic nature. In this paper, we present for the first time the model of the non-idealities of the Spintronics-based Dropout module and analyze their impact on uncertainty estimates and accuracy. Furthermore, we propose a testing framework based on repeatability ranking for Dropout-based BayNN with up to 100% fault coverage while using only 0.2% of training data as test vectors.

The applications of artificial intelligence (AI) are rapidly evolving, and they are also commonly used in safety-critical domains, such as autonomous driving and medical diagnosis, where functional safety is paramount. In AI-driven systems, uncertainty estimation allows the user to avoid overconfidence predictions and achieve functional safety. Therefore, the robustness and reliability of model predictions can be improved. However, conventional uncertainty estimation methods, such as the deep ensemble method, impose high computation and accordingly hardware (latency and energy) overhead because they require the storage and processing of multiple models. Alternatively, Monte Carlo dropout (MC-dropout) methods, although having low memory overhead, necessitate numerous (~ 100) forward passes, leading to high computational overhead and latency. Thus, these approaches are not suitable for battery-powered edge devices with limited computing and memory resources. In this paper, we propose the Tiny-Deep Ensemble approach, a low-cost approach for uncertainty estimation on edge devices. In our approach, only normalization layers are ensembled M times, with all ensemble members sharing common weights and biases, leading to a significant decrease in storage requirements and latency. Moreover, our approach requires only one forward pass in a hardware architecture that allows batch processing for inference and uncertainty estimation. Furthermore, it has approximately the same memory overhead compared to a single model. Therefore, latency and memory overhead are reduced by a factor of up to ~ M ×. Nevertheless, our method does not compromise accuracy, with an increase in inference accuracy of up to ~ 1% and a reduction in RMSE of 17.17% in various benchmark datasets, tasks, and state-of-the-art architectures.

Neural networks (NNs) have shown outstanding performance in various domains, leading to widespread deployment on various hardware devices. They require large memories to store the NN weight parameters, which are susceptible to numerous permanent and transient faults. Therefore, error detection and correction mechanisms with certain guarantees should be provided to ensure reliable NN operation, especially in safety-critical applications. Error Correction Codes (ECCs) are a common approach to protecting memories against these failures, but they impose significant memory overheads. This work proposes NN-ECC, a multi-task learning objective that integrates linear block ECC into the NN parameters during training. The proposed NN-ECC does not increase the total number of NN parameters and eliminates all storage requirements for parity bits of different ECCs. Unlike existing methods, which selectively eliminate the storage overhead with limited error correction and necessitate specific weight distributions to utilize redundant weight bits for ECC parity, our approach is versatile and can accommodate various ECC schemes with different correction capabilities without imposing constraints on weight distributions. Moreover, the proposed NN-ECC does not deteriorate the baseline accuracy due to ECC encoding.

Neural Networks (NN) can be efficiently accelerated in a neuromorphic fabric based on emerging resistive non-volatile memories (NVM), such as Spin Transfer Torque Magnetic RAM (STT-MRAM). Compared to other NVM technologies, STT-MRAM offers many benefits, such as fast switching, high endurance, and CMOS process compatibility. However, due to its low ON/OFF-ratio, process variations and runtime temperature fluctuations can lead to miss-quantizing the sensed current and, in turn, degradation of inference accuracy. In this article, we analyze the impact of the sensed accumulated current variation on the inference accuracy in Binary NNs and propose a design-time reference current generation method to improve the robustness of the implemented NN under different temperature and process variation scenarios (up to 125 °C). Our proposed method is robust to both process and temperature variations. The proposed method improves the accuracy of NN inference by up to 20.51% on the MNIST, Fashion-MNIST, and CIFAR-10 benchmark datasets in the presence of process and temperature variations without additional runtime hardware overhead compared to existing solutions.

Neural networks (NNs) are capable of learning complex patterns and relationships in data to make predictions with high accuracy, making them useful for various tasks. However, NNs are both computation-intensive and memory-intensive methods, making them challenging for edge applications. Hence, hardware AI accelerators architectures based on computation-in-memory (CiM) architectures with non-volatile memristive crossbars are gaining attention. Although memristive devices offer benefits such as power efficiency, parallelism, and nonvolatility, they suffer from non-idealities, leading to various faults and parameter variations, both during manufacturing and lifetime operations. This can lead to faulty computations and, in turn, degradation of post-mapping inference accuracy, which is unacceptable for many applications, including safety-critical applications. Therefore, proper testing of NN hardware accelerators is of utmost importance for functional safety. In this paper, we propose a one-shot testing approach that can test NNs accelerated on memristive crossbars with only one test vector, making it very suitable for online testing applications. Our approach can consistently achieve 100% fault coverage across several large topologies with up to 201 layers and challenging tasks like semantic segmentation. Nevertheless, compared to existing methods, the fault coverage is improved by up to 24%, the memory overhead is only 0.0123 MB, a reduction of up to 19980× and the number of test vectors is reduced by 10000×.

Typical neural networks are incapable of effectively estimating prediction uncertainty, leading to overconfident predictions. Estimating uncertainty is crucial for safety-critical tasks such as autonomous vehicle driving and medical diagnosis and treatment. Bayesian Neural Networks (BayNNs), which combine the capabilities of neural networks and Bayesian inference, are an effective approach for uncertainty estimation. However, BayNNs are computationally demanding and necessitate substantial memory resources. Computation-in-memory (CiM) architectures uti-lizing emerging resistive non-volatile memories such as Spin- Orbit Torque (SOT) have been proposed to increase the resource efficiency of traditional neural networks. However, training scalable and efficient BayNNs and implementing them in the CiM architecture presents its own challenges. In this paper, we propose a scalable Bayesian NN framework via Subset-Parameter inference and its Spintronic-based CiM implementation. Our method is evaluated on large datasets and topologies to show that it can achieve comparable accuracy while still being able to estimate uncertainty efficiently at up to 70 × lower power consumption and 158.7× lower storage memory requirements.

Neural networks (NNs) are a widely-used problem-solving tool, but their high computational and power consumption makes them expensive. Computation-in-Memory (CiM) architecture, which uses resistive non-volatile memories, is a promising solution due to its high energy efficiency. However, manufacturing defects and in-field faults can reduce the reliability and inference accuracy of CiM-implemented neural networks. Existing sophisticated fault detection and tolerance techniques require long downtime for testing and repair. In certain applications, e.g., "always on" NN applications, such downtime may not be acceptable. Thus, in this paper, a low-cost online fault tolerance technique based on local approximations is proposed to ensure continuous neural network operation with acceptable accuracy. Our approach reduces hardware overhead by up to 99.37% compared to conventional redundancy-based approaches while still achieving accuracy within 2% of the trained NNs.

The matrix-vector multiplication (MVM) is one of the most frequent operations performed in deep learning hardware accelerators. The crossbar array structure with memristive devices as a building block has an inherent capability to perform energy-efficient MVM. However, the memristive devices suffer from various non-idealities as well as limited number of stable levels. Therefore, the reliability and in turn inference accuracy of the deep learning application is negatively impacted. Thus, this paper presents a low overhead checksum-based error correction method for memristive crossbars for MVM computation. The proposed methodology alleviates the problem of storing the checksum value into multiple columns of the crossbar due to the limited number of stable levels of memristive devices. The number of extra columns required for storing the checksum value is reduced, resulting in a significant reduction in the memory overhead by up to 75%. The proposed method scales the checksum value of trained neural networks (NNs) and then performs checksum-aware retraining, and results show negligible impact (∼3%) on the inference accuracy of the NNs on MNIST, Fashion-MNIST, CIFAR-10, and Veg-15 datasets.

Neuromorphic computation-in-memory fabric based on emerging nonvolatile memories considered an attractive option to accelerate neural networks (NNs) in hardware as they provide high-performance, low-power, and reduced data movement. Although nonvolatile resistive memories (NVMs) offer many benefits, they are susceptible to data retention faults, where previously stored data are not retained after a certain amount of time due to external influence. These faults are more likely to happen unidirectional and severely impact the inference accuracy of the hardware implementation of NNs since the synaptic weights stored in the NVMs are subject to retention faults. In this work, we propose an approximate scrubbing technique for NVM-based neuromorphic fabric to mitigate unidirectional retention faults with virtually zero storage overhead depending on the definition of scrub area for multilayer perceptron (MLP) and convolutional NNs (CNNs). The training of the NNs is adjusted accordingly to meet the requirements of the proposed approximate scrubbing scheme. On different benchmarks, the proposed scrubbing approach can improve the inference accuracy up to 85.51% for MLP and 87.76% for CNN over the expected device operational time with negligible storage overhead.

Recent development in neural networks (NNs) has led to their widespread use in critical and automated decision-making systems, where uncertainty estimation is essential for trustworthiness. Although conventional NNs can solve many problems accurately, they do not capture the uncertainty of the data or the model during optimization. In contrast, Bayesian neural networks (BNNs), which learn probabilistic distributions for their parameters, offer a sound theoretical framework for estimating uncertainty. However, traditional hardware implementations of BNNs are expensive in terms of computational and memory resources, as they (i) are realized with inefficient von Neumann architectures, (ii) use a significantly large number of random number generators (RNGs) to implement the distributions of BNNs, and (iii) have a substantially greater number of parameters than conventional NNs. Computing-in-memory (CiM) architectures with emerging resistive non-volatile memories (NVMs) are promising candidates for accelerating classical NNs. In particular, spintronic technology, which is distinguished by its low latency and high endurance, aligns very well with these requirements. In the specific context of Bayesian neural networks (BNNs), spintronics technologies are very valuable, thanks to their inherent potential to act as stochastic or as deterministic devices. Consequently, BNNs mapped on spintronic-based CiM architectures could be a highly efficient implementation strategy. However, the direct implementation on CiM hardware of the learned probabilistic distributions of BNN may not be feasible and can incur high overhead. In this work, we propose a new Bayesian neural network topology, named SpinBayes , that is able to perform efficient sampling during the Bayesian inference process. Moreover, a Bayesian approximation method, called in-memory approximation , is proposed that approximates the original probabilistic distributions of BNN with a distribution that can be efficiently mapped to spintronic-based CiM architectures. Compared to state-of-the-art methods, the memory overhead is reduced by 8× and the energy consumption by 80×. Our method has been evaluated on several classification and semantic segmentation tasks and can detect up to 100% of various types of out-of-distribution data, highlighting the robustness of our approach, without any performance sacrifice.

Analog Computation-in-Memory (CiM) with emerging non-volatile memories leads to significant performance and energy efficiency. Spin-Transfer Torque Magnetic Memory (STT-MRAM) is one of the promising technologies for CiM architectures. Although STT-MRAM has various benefits, it does not have the potential to be used directly in analog multi-value CiM operations due to its limited levels of cell resistance states. In this paper, we propose a novel flexible multi-value design for STT-MRAM (MVSTT) with the potential to be used for multi-value CiM. In the multi-value CiM, we are able to have various 2sresistive state combinations from$s$selected MTJs, which is not possible in the normal STT-MRAM CiM. The size of the MVSTT can be adjusted at run-time depending on the application's requirements. The benefits of the proposed scheme are quantified in representative applications such as multi-value matrix multiplications, which is the basic computation of Neural Networks applications. For the multi-value matrix multiplication, the energy, and delay gain is up to 9.7 × and 13.3 ×, respectively, to non-CiM matrix-vector-multiplication. Also, for the neural network, the proposed design allows up to a 32 × reduction in the STT-MRAM cells per crossbar to achieve a similar inference accuracy as the binarized neural network.

Neural Networks (NN) can be efficiently accelerated using emerging resistive non-volatile memories (eNVM), such as Spin Transfer Torque Magnetic RAM(STT-MRAM). However, process variations and runtime temperature fluctuations can lead to miss-quantizing the sensed state and in turn, degradation of inference accuracy. We propose a design-time reference current generation method to improve the robustness of the implemented NN under different thermal and process variation scenarios with no additional runtime hardware overhead compared to existing solutions.

Deep learning has been applied in many fields such as autonomous driving, medical imaging, and sensor data processing to solve complex computational problems. Hardware implementation of deep learning applications can be accelerated with Memristive devices such as Resistive Random-Access Memory (ReRAM), Phase Change Memories (PCM), and Spin Transfer Torque-Magnetic RAM (STT-MRAM) via utilizing compute-in-memory (CiM) architectures. They offer many benefits compared to conventional charge-based memories including non-volatility, zero leakage power, and low power consumption. However, memristors suffer from various manufacturing and runtime defects and variations due to their immature fabrication process, inherent device properties, and external environmental factors such as temperature. Those faults can negatively impact the performance and reliability of the implemented deep learning applications. Therefore, detection of such faults before they catastrophically affect the accuracy of deep learning applications is very important. In this paper, we proposed a simple and non-invasive functional test generation method with a black-box approach that samples a small subset of the training data as the test vectors based on their approximate gradient ranking. The proposed method requires only up to 0.128% of training data stored in hardware as test vectors and achieves test coverage up to 100% on the Fashion-MNIST, CIFAR-10, and CIFAR-100 benchmark datasets.

Neuromorphic fabric based on emerging resistive non-volatile memories (NVM), such as Resistive Random Access Memory (ReRAM), Phase Change Memories (PCM) and Spin Transfer Torque (STT) is a promising approach for efficient hardware implementation of Neural Networks (NNs) due to their low power consumption and latency. However, NVMs suffers from manufacturing process variations and manufacturing defects resulting in a shift in the distribution of NN activations and can lead to degradation of inference accuracy. The shifted distribution can be tracked and re-calibrated by re-calculating the statistics of the batch normalization layer of NN. However, such re-calibration and re-calculation have high overhead in terms of memory, power, and latency. In this paper, we proposed a low overhead post-manufacturing calibration of NVM-based neuromorphic fabric by approximating batch normalization to reduce re-calibration overhead. The proposed method requires only 0.2% of training data as re-calibration input and can regain inference accuracy by up to 72.32% on the MNIST, Fashion-MNIST, and CIFAR-10 benchmark datasets.

Neuromorphic computation-in-memory fabric based on emerging non-volatile memories (NVM) is considered an attractive option to accelerate neural networks (NNs) in hardware as they provide high-performance, low-power, and reduced data movement. Although NVMs offer many benefits, they are susceptible to data retention faults, where previously stored data is not retained. This severely impacts the inference accuracy of mapped NNs. Traditionally, memory scrubbing with error-correcting codes (ECC) is employed to mitigate retention faults in conventional CMOS memories. This is not feasible in NVM-based neuromorphic fabric due to high overhead and inability to represent encoding or decoding in analog computing. In this work, we propose an approximate scrubbing technique for NVM-based neuromorphic fabric to mitigate uni-directional retention faults with minimal storage overhead. The training of the NNs adjusted accordingly to meet the requirements of the scrubbing scheme. On different benchmarks, the proposed scrubbing approach can improve the inference accuracy up to 85.51% over the lifetime with virtually zero storage overhead.