Printed electronics (PE) enable lightweight, flexible, and low-cost devices for the Internet of Things (IoT) and wearable applications. Compared to conventional silicon-based electronics, PE trades peak performance for advantages in cost efficiency, mechanical flexibility, and large-area fabrication. However, its manufacturing processes remain unreliable and are prone to structural defects and variation due to inherent limited control in additive manufacturing. Printed neuromorphic circuits (pNCs) leverage the benefits of PE for on-demand analog edge computation in target applications but remain vulnerable to such defects. Diagnostic testing is therefore essential not only for detection but also for localizing faults to specific subcircuits and regions in the layout, a step critical for guiding yield improvement and reducing the cost of downstream inspection. We propose a diagnostic test pattern generation (DTPG) framework for fault localization in pNCs under black-box access. While ATPG is typically formulated as an optimization problem for fault detection, our approach extends this formulation by explicitly optimizing for fault distinguishability. On ten UCI datasets, the framework achieves up to 20.7% higher diagnostic coverage with a reduction of up to 3.6 times the number of undetectable subcircuits than detection-only test sets, while constraining the number of patterns to reduce storage overhead. These results demonstrate effective fault localization and establish a foundation for finer-grained, component-level diagnosis in future work.

Uncertainty estimation in Neural Networks (NNs) is vital in improving reliability and confidence in predictions, particularly in safety-critical applications. Bayesian Neural Networks (BayNNs) with Dropout as an approximation offer a systematic approach to quantifying uncertainty, but they inherently suffer from high hardware overhead in terms of power, memory, and computation. Thus, the applicability of BayNNs to edge devices with limited resources or to high-performance applications is challenging. Some of the inherent costs of BayNNs can be reduced by accelerating them in hardware on a Computation-In-Memory (CIM) architecture with spintronic memories and binarizing their parameters. However, numerous stochastic units are required to implement conventional Dropout-based BayNN. In this paper, we propose the Scale Dropout, a novel regularization technique for Binary Neural Networks (BNNs), and Monte Carlo-Scale Dropout (MC-Scale Dropout)-based BayNNs for efficient uncertainty estimation. Our approach requires only one stochastic unit for the entire model, irrespective of the model size, leading to a highly scalable Bayesian NN. Furthermore, we introduce a novel Spintronic memory-based CIM architecture for the proposed BayNN that achieves more than 100× energy savings compared to the state-of-the-art. We validated our method to show up to 1% improvement in predictive performance and superior uncertainty estimates compared to related works.

For AI hardware, functional safety is crucial, especially for neural network (NN) accelerators used in safety-critical systems. A key requirement for maintaining this safety is the precise detection of out-of-distribution (OOD) instances, which are inputs significantly distinct from the training data. Neglecting to integrate robust OOD detection may result in possible safety hazards, diminished performance, and inaccurate decision-making within NN applications. Existing methods for OOD detection have been explored for full-precision models. However, the evaluation of methods on quantized neural network (QNN), which are often deployed on hardware accelerators such as FPGAs, and on-device hardware realization of concurrent OOD detection (COD) is missing in literature. In this paper, we provide a novel approach to OOD detection for NN FPGA accelerators using power measurements. Utilizing the power side-channel through digital voltage sensors allows on-device OOD detection in a non-intrusive and concurrent manner, without relying on explicit labels or modifications to the underlying NN. Furthermore, our method allows OOD detection before the inference finishes. Additionally to the evaluation, we provide an efficient hardware implementation of COD on an actual FPGA.

With the rising demand for ultra-low-cost and flexible electronics in applications like smart packaging and wearable health monitoring, printed electronics provide an affordable, adaptable, and customizable alternative to conventional silicon. However, these systems often rely on printed batteries or energy harvesters with limited power capacity, making strict power budgets critical. Printed neuromorphic circuits (pNCs) are promising for their analog signal processing, reduced circuit complexity, and energy efficiency in low-power environments. Nonetheless, maintaining robust performance under strict power constraints remains challenging, necessitating advanced optimization techniques. In this work, we propose an augmented Lagrangian approach to enforce task-specific power constraints in pNCs, validated across 13 benchmark datasets. Our method preserves accuracy within strict power budgets while achieving Pareto-optimal power-accuracy trade-offs in a single training run. In contrast, the penalty-based method, which serves as the baseline, requires up to 150 runs per dataset to generate the Pareto front. For low-power scenarios ($\approx 20 \%$ of the original power), our method demonstrates a $52 \times$ improvement in accuracy-to-power ratio over the baseline. At higher power budgets $(\approx 80 \%$ of the original power), it achieves a $59 \times$ improvement, maintaining competitive performance. Experimental results demonstrate that our approach achieves $\mathbf{8 1. 8 2 \%}$ accuracy with p-tanh activation function (AF) at high power budgets and excels with p-Clipped_ReLU AF under low power constraints. This highlights the computational efficiency and effectiveness of our approach for power-constrained circuit design.

The rise of the Internet of Things demands flexible, biocompatible, and cost-effective devices. Printed electronics provide a solution through low-cost and on-demand additive manufacturing on flexible substrates, making them ideal for IoT applications. However, variations in additive manufacturing processes pose challenges for reliable circuit fabrication. Adapting neuromorphic computing to printed electronics could address these issues. Printed neuromorphic circuits offer robust computational capabilities for near-sensor processing in IoT. One limitation of existing printed neuromorphic circuits is their inability to process temporal sensory inputs. To address this, integrating temporal components in printed neuromorphic circuit architectures enables the effective processing of time-series sensory data. Printed neuromorphic circuits face challenges from manufacturing variations such as ink dispersion, sensor noise, and temporal fluctuations, especially when processing temporal data and using time-dependent components like capacitors. To mitigate these challenges, we propose robustness-aware temporal processing neuromorphic circuits with low-pass second-order learnable filters (SO-LF). This approach integrates variation awareness by considering the variation potential of component values during training and using data augmentation to enhance adaptability against physical and sensor data variations. Simulations on 15 benchmark time-series datasets show our circuit effectively handles noisy temporal information under 10% process variations, achieving an average accuracy and power improvement of ≈24.7% and ≈91% respectively compared to models lacking variation with ≈1.9×more devices.

Biologically-inspired Spiking Neural Networks (SNNs) have emerged as a promising avenue toward energy-efficient neuromorphic computing, particularly in edge applications such as soft robotics, wearable health monitors, and IoT devices. Printed Electronics (PE), offering advantages of ultra-low cost fabrication and mechanical flexibility, present a viable platform to realize such neuromorphic systems at scale. However, designing adaptable and efficient spiking circuits that meet the unique constraints of PE applications remains a challenge. To address this, we propose a novel analog spiking neuromorphic circuit with a learnable spike generator (LSG). Unlike fixed-threshold models, our generator adapts spike timing dynamics during training, enabling better task-specific performance. To optimize for ultra-low power consumption on resource-constrained platforms, we further introduce a robustness-aware training framework that further minimizes the energy consumption adaptively. Simulation results across 13 benchmarks demonstrate an average 57.6% power reduction for the LSG while improving the average classification accuracy by 8%, area and energy reduction by 89% and 28.7% respectively compared to the state-of-the-art printed analog spiking neural networks (P-SNNs).

As artificial intelligence becomes integral in domains like healthcare and autonomous systems, dedicated hardware accelerators are becoming increasingly essential. These are structurally tested at manufacturing, independent of the AI model executed. However, in-field reliability demands model-based functional testing using the Deep Neural Networks (DNNs) deployed during inference. Faults in DNNs can degrade performance, making in-field testing critical under memory and time constraints. We propose a framework to generate a set of test patterns according to memory constraints while ensuring effective fault coverage of the fault distribution through joint optimization of patterns. Results show 100% coverage, outperforming random and adversarial inputs.

The successful training of deep neural networks requires addressing challenges such as overfitting, numerical instabilities leading to divergence, and increasing variance in the residual stream. A common solution is to apply regularization and normalization techniques that usually require tuning additional hyperparameters. An alternative is to force all parameters and representations to lie on a hypersphere. This removes the need for regularization and increases convergence speed, but comes with additional costs. In this work, we propose a more holistic, approximate normalization via simple scalar multiplications motivated by the tight concentration of the norms of high-dimensional random vectors. Additionally, instead of applying strict normalization for the parameters, we constrain their norms. These modifications remove the need for weight decay and learning rate warm-up as well, but do not increase the total number of normalization layers. Our experiments with transformer architectures show up to 40% faster convergence compared to GPT models with QK normalization, with only 3% additional runtime cost. When deriving scaling laws, we found that our method enables training with larger batch sizes while preserving the favorable scaling characteristics of classic GPT architectures.

Deep neural networks (DNNs) have emerged as an effective approach in many artificial intelligence tasks. Several specialized accelerators are often used to enhance DNN's performance and lower their energy costs. However, the presence of faults can drastically impair the performance and accuracy of these accelerators. Usually, many test patterns are required for certain types of faults to reach a target fault coverage, which in turn hence increases the testing overhead and storage cost, particularly for in-field testing. For this reason, compression is typically done after test generation step to reduce the storage cost for the generated test patterns. However, compression is more efficient when considered in an earlier stage. This paper generates the test pattern in a compressed form to require less storage. This is done by generating all test patterns as a linear combination of a set of jointly used test patterns (basis), for which only the coefficients need to be stored. The fault coverage achieved by the generated test patterns is compared to that of the adversarial and randomly generated test images. The experimental results showed that our proposed test pattern outperformed and achieved high fault coverage (up to 99.99%) and a high compression ratio (up to 307.2$\times$).

Printed electronics (PEs) is an additive fabrication technology which not only allows for a highly flexible printing of circuit patterns, but also produce soft, nontoxic, and degradable electronics at an extremely low cost. These properties make PE an enabler of new application domains, e.g., fast moving consumer goods and disposable healthcare devices. A particularly promising class of circuits in this technology is the printed analog neuromorphic circuits, offering efficient and highly tailored computational functionalities. In this work, we leverage the highly flexible fabrication process of PE to address the bottleneck of PE, i.e., the large feature sizes and low device counts. This issue is crucial, as it impairs the integration of printed circuits into target applications with limited footprint, such as smart band-aids. We propose an evolutionary algorithm (EA) to improve the circuit compactness through circuit architecture optimization. As baseline, we compare the proposed EA method with a state-of-the-art pruning method and a modified area-aware pruning method. All of them are able to optimize circuit architecture. Experimental simulation reveals that the proposed EA approach can effectively achieve compact circuits and outperform the pruning method by$3.1\times $lower area with no loss of accuracy. As a byproduct, the power is reduced by$3.0\times $, paving the way to energy-harvested printed systems.

The demand for next-generation flexible electronics in applications like smart packaging and smart bandages has driven the need for cost-effective solutions. Traditional silicon-based electronics struggle with high costs and rigidity, making them unsuitable for these emerging markets. In this regard, additive printed electronics (PE) offer a viable alternative with their flexibility and ultra-low-cost manufacturing. printed analog neuromorphic circuits (pNCs) are well-suited for these target applications, especially for classification tasks, as their low device count can efficiently meet the needs of the technology. However, low-cost additive manufacturing comes with higher defect rates, such as misprints, broken connections, and defective components, posing significant challenges to the reliability of printed circuits. This article presents a novel co-design of training algorithm and hardware for fault-tolerant pNCs using fault-aware training (FAT). The proposed method introduces a fault-tolerant version of printed nonlinear transformation circuits, combined with a bespoke training process that selects different types of printed activation functions (AFs) for different neurons to optimize both fault endurance and hardware costs. Experiments on benchmark datasets demonstrate an improvement in the accuracy of fault-tolerant (FT) pNCs from 62.1% to 79.4% under a 10% fault rate. Moreover, combining both normal and fault-tolerant versions of activation functions (AFs) using gumble-softmax distribution shows an acceptable accuracy drop with an average reduction in power and area of 54.5% and 6.54%, respectively, while reducing the training time significantly by 56.2%, compared to only using FT-AFs.

Printed Electronics (PE) stands out as a promising technology for widespread computing due to its distinct attributes, such as low costs and flexible manufacturing. Unlike traditional silicon-based technologies, PE enables stretchable, conformal, and non-toxic hardware. However, PE are constrained by larger feature sizes, making it challenging to implement complex circuits such as machine learning (ML) classifiers. Approximate computing has been proven to reduce the hardware cost of ML circuits such as Multilayer Perceptrons (MLPs). In this paper, we maximize the benefits of approximate computing by integrating hardware approximation into the MLP training process. Due to the discrete nature of hardware approximation, we propose and implement a genetic-based, approximate, hardware-aware training approach specifically designed for printed MLPs. For a 5% accuracy loss, our MLPs achieve over 5 × area and power reduction compared to the baseline while outperforming state-of-the-art approximate and stochastic printed MLPs.

Biologically-inspired Spiking Neural Networks have emerged as a promising avenue for energy-efficient, high-performance neuromorphic computing. With the demand for highly-customized and cost-effective solutions in emerging application domains like soft robotics, wearables, or IoT-devices, Printed Electronics has emerged as an alternative to traditional silicon technologies leveraging soft materials and flexible substrates. In this paper, we propose an energy-efficient analog printed spiking neuromorphic circuit and a corresponding learning algorithm. Simulations on 13 benchmark datasets show an average of 3.86 x power improvement with similar classification accuracy compared to previous works.

Bayesian Neural Networks (BayNNs) can inherently estimate predictive uncertainty, facilitating informed decision-making. Dropout-based BayNNs are increasingly implemented in Spintronics-based computation-in-memory architectures for resource-constrained yet high-performance safety-critical applications. Although uncertainty estimation is important, the reliability of Dropout generation and BayNN computation is equally important for target applications but is overlooked in existing works. However, testing BayNNs is significantly more challenging compared to conventional NNs, due to their stochastic nature. In this paper, we present for the first time the model of the non-idealities of the Spintronics-based Dropout module and analyze their impact on uncertainty estimates and accuracy. Furthermore, we propose a testing framework based on repeatability ranking for Dropout-based BayNN with up to 100% fault coverage while using only 0.2% of training data as test vectors.

Printed Electronics (PE) is an emerging technology with flexible substrates and ultra-low-cost manufacturing, providing an appealing alternative to traditional wafer-scale silicon fabrication. With the increasing integration of various printed neural network (NN) architectures in diverse applications, the reliability of printed circuits has become a critical concern. This work provides a comprehensive analysis of the fault sensitivity on a variety of classification tasks for various digital and analog realizations of printed multilayer perceptrons (MLPs). We further evaluate different digital architectures, i.e., generic, bespoke, and approximate, to provide a comprehensive fault analysis on different benchmark datasets.

The applications of artificial intelligence (AI) are rapidly evolving, and they are also commonly used in safety-critical domains, such as autonomous driving and medical diagnosis, where functional safety is paramount. In AI-driven systems, uncertainty estimation allows the user to avoid overconfidence predictions and achieve functional safety. Therefore, the robustness and reliability of model predictions can be improved. However, conventional uncertainty estimation methods, such as the deep ensemble method, impose high computation and accordingly hardware (latency and energy) overhead because they require the storage and processing of multiple models. Alternatively, Monte Carlo dropout (MC-dropout) methods, although having low memory overhead, necessitate numerous (~ 100) forward passes, leading to high computational overhead and latency. Thus, these approaches are not suitable for battery-powered edge devices with limited computing and memory resources. In this paper, we propose the Tiny-Deep Ensemble approach, a low-cost approach for uncertainty estimation on edge devices. In our approach, only normalization layers are ensembled M times, with all ensemble members sharing common weights and biases, leading to a significant decrease in storage requirements and latency. Moreover, our approach requires only one forward pass in a hardware architecture that allows batch processing for inference and uncertainty estimation. Furthermore, it has approximately the same memory overhead compared to a single model. Therefore, latency and memory overhead are reduced by a factor of up to ~ M ×. Nevertheless, our method does not compromise accuracy, with an increase in inference accuracy of up to ~ 1% and a reduction in RMSE of 17.17% in various benchmark datasets, tasks, and state-of-the-art architectures.

The market demand for next-generation flexible electronics is experiencing a significant upsurge, particularly in cost-sensitive consumer applications like smart packaging and smart bandages. These products are beyond the reach of traditional silicon-based electronics due to their high production cost and rigid form factor. Printed electronics (PE), with its adaptable and ultra-low-cost solutions, essentially meet the unique needs of these emerging application areas. This work presents a novel approach using an evolutionary algorithm (EA) to design highly bespoke printed analog neuromorphic circuits (pNCs) offering robustness against variability inherent in the printing process. By leveraging this algorithm and designing robust activation circuits, not only the resistances (weights) in the crossbar and parameters in the activation circuits, but also the types of nonlinear circuits (i.e., functional forms of activation functions) as well as the circuit topologies (neural architecture) can be learned to enhance the circuit robustness against printing variations. Experiments on 13 benchmark datasets demonstrate that, compared to the baseline, the proposed methodology can further outperform the normalized classification error rate by ≈ 55.38% and ≈ 25.11% under high-precision (±5%) and low-precision (±10%) printing scenarios, respectively. Moreover, the algorithm suggests the ReLU as the most robust activation function (AF) circuit family with only ≈ 21% susceptible to low precision (±10%) printing variation.

Pruning is a pivotal approach in network compression. It not only encourages lightweight deep neural networks, but also helps to mitigate overfitting. Generally, regularization is used to guide more parameters towards zero and thus reduce the overall model complexity. Unfortunately, there are two issues remaining unsolved in regularization based pruning. One is that the optimal trade-off between regularization and loss minimization, often expressed via a scaling hyperparameter, needs to be found through extensive experimentation. The other one is the importance criteria that can reflect the true relative importance. The most widely used criterion is the magnitude-based, which has been argued to be inaccurate. To these two issues, in this paper, we propose a progressive regularization scheme, in which the factor scaling the regularization term is gradually increased during training, until the target sparsity for filter pruning is reached. Compared to the previous approach, the scaling factor is no longer a hyperparameter that needs to be tuned, but is replaced with a sparsity-aware parameter that increases progressively. In this way, the value of the scaling factor can be automatically aligned with the target sparsity, avoiding the drawbacks in its tuning. Furthermore, only parameters below a minimal and learnable soft threshold are pruned, therefore, informative parameters (even with small magnitudes) are optimally preserved. Consequently, the performance loss due to pruning is mitigated. Experiments with various models and benchmark datasets prove the effectiveness of the proposed method.

Regularization is a critical component in deep learning. The most commonly used approach, weight decay, applies a constant penalty coefficient uniformly across all parameters. This may be overly restrictive for some parameters, while insufficient for others. To address this, we present Constrained Parameter Regularization (CPR) as an alternative to traditional weight decay. Unlike the uniform application of a single penalty, CPR enforces an upper bound on a statistical measure, such as the L$_2$-norm, of individual parameter matrices. Consequently, learning becomes a constraint optimization problem, which we tackle using an adaptation of the augmented Lagrangian method. CPR introduces only a minor runtime overhead and only requires setting an upper bound. We propose simple yet efficient mechanisms for initializing this bound, making CPR rely on no hyperparameter or one, akin to weight decay. Our empirical studies on computer vision and language modeling tasks demonstrate CPR's effectiveness. The results show that CPR can outperform traditional weight decay and increase performance in pre-training and fine-tuning.

Neural Networks (NN) can be efficiently accelerated in a neuromorphic fabric based on emerging resistive non-volatile memories (NVM), such as Spin Transfer Torque Magnetic RAM (STT-MRAM). Compared to other NVM technologies, STT-MRAM offers many benefits, such as fast switching, high endurance, and CMOS process compatibility. However, due to its low ON/OFF-ratio, process variations and runtime temperature fluctuations can lead to miss-quantizing the sensed current and, in turn, degradation of inference accuracy. In this article, we analyze the impact of the sensed accumulated current variation on the inference accuracy in Binary NNs and propose a design-time reference current generation method to improve the robustness of the implemented NN under different temperature and process variation scenarios (up to 125 °C). Our proposed method is robust to both process and temperature variations. The proposed method improves the accuracy of NN inference by up to 20.51% on the MNIST, Fashion-MNIST, and CIFAR-10 benchmark datasets in the presence of process and temperature variations without additional runtime hardware overhead compared to existing solutions.

Deep Neural Networks (DNNs) have gained considerable attention lately due to their excellent performance on a wide range of recognition and classification tasks. Accordingly, fault detection in DNNs and their implementations plays a crucial role in the quality of DNN implementations to ensure that their post-mapping and infield accuracy matches with model accuracy. This paper proposes a functional-level automatic test pattern generation approach for DNNs. This is done by generating inputs which causes misclassification of the output class label in the presence of single or multiple faults. Furthermore, to obtain a smaller set of test patterns with full coverage, a heuristic algorithm as well as a test pattern clustering method using K-means were implemented. The experimental results showed that the proposed test patterns achieved the highest label misclassification and a high output deviation compared to state-of-the-art approaches.

Typical neural networks are incapable of effectively estimating prediction uncertainty, leading to overconfident predictions. Estimating uncertainty is crucial for safety-critical tasks such as autonomous vehicle driving and medical diagnosis and treatment. Bayesian Neural Networks (BayNNs), which combine the capabilities of neural networks and Bayesian inference, are an effective approach for uncertainty estimation. However, BayNNs are computationally demanding and necessitate substantial memory resources. Computation-in-memory (CiM) architectures uti-lizing emerging resistive non-volatile memories such as Spin- Orbit Torque (SOT) have been proposed to increase the resource efficiency of traditional neural networks. However, training scalable and efficient BayNNs and implementing them in the CiM architecture presents its own challenges. In this paper, we propose a scalable Bayesian NN framework via Subset-Parameter inference and its Spintronic-based CiM implementation. Our method is evaluated on large datasets and topologies to show that it can achieve comparable accuracy while still being able to estimate uncertainty efficiently at up to 70 × lower power consumption and 158.7× lower storage memory requirements.

Deep neural networks (DNNs) have achieved record-breaking performance in various applications. To reduce the energy footprint and increase performance, DNNs are often implemented on specific hardware accelerators, such as Tensor Processing Units (TPU) or emerging Memristive technologies. Unfortunately, the presence of various hardware faults can threaten these accelerators' performance and degrade the inference accuracy. This necessitates the development of efficient testing methodologies to unveil hardware faults in DNN accelerators. In this work, we propose a test pattern generation approach to detect fault patterns in DNNs for a common type of hardware fault, namely, faulty weight value representations on the bit level. Contrary to most related works which reveal faults via output deviations, our test patterns are constructed to reveal faults via misclassification which is more realistic for black-box testing.

Printed and flexible electronics promises smart devices for application domains, such as smart fast moving consumer goods and medical wearables, which are generally untouchable by conventional rigid silicon technologies. This is due to their remarkable properties such as flexibility, non-toxic materials, and having low-cost per area. Combined with neuromorphic computing, printed neuromorphic circuits pose an attractive solution for these application domains. Particularly, the additive printing technologies can reduce large amount of fabrication complexities and costs. On the one hand, high-throughput additive printing processes, such as roll-to-roll printing, can reduce the per-device fabrication time and cost. On the other hand, jet-printing can provide point-of-use customization at the expense of lower fabrication throughput. In this work, we propose a machine learning based design framework, that respects the objective and physical constraints of split additive manufacturing for printed neuromorphic circuits. With the proposed framework, multiple printed neural networks are trained jointly with the aim to sensibly combine multiple fabrication techniques (e.g., roll-to-roll and jet-printing). This should lead to a cost-effective fabrication of multiple different printed neuromorphic circuits and achieve high fabrication throughput, lower cost, and point-of-use customization.

With the rapid growth of the Internet of Things, smart fast-moving consumer products, and wearable devices, requirements such as flexibility, non-toxicity, and low cost are desperately required. However, these requirements are usually beyond the reach of conventional rigid silicon technologies. In this regard, printed electronics offers a promising alternative. Combined with neuromorphic computing, printed neuromorphic circuits offer not only the aforementioned properties, but also compensate for some of the weaknesses of printed electronics, such as manufacturing variations, low device count, and high latency. Generally, (printed) neuromorphic circuits express their functionality through printed resistor crossbars to emulate matrix multiplication, and nonlinear circuitry to express activation functions. The values of the former are usually learned, while the latter is designed beforehand and considered fixed in training for all tasks. The additive manufacturing feature of printed electronics allows the design of highly-bespoke designs. In the case of printed neuromorphic circuits, the circuit is optimized to a particular dataset. Moreover, we explore an approach to learn not only the values of the crossbar resistances, but also the parameterization of the nonlinear components for a bespoke implementation. While providing additional flexibility of the functionality to be expressed, this will also allow an increased robustness against printing variation. The experiments show that the accuracy and robustness of printed neuromorphic circuits can be improved by 26% and 75% respectively under 10% variation of circuit components.

Machine learning acceleration has become increasingly popular in recent years, with machine learning-as-a-service (MLaaS) scenarios offering convenient and efficient ways to access pre-trained neural network models on devices such as cloud FPGAs. However, the ease of access and use also raises concerns over model theft or misuse through model manipulation. To address these concerns, this paper proposes a method for identifying neural network models in MLaaS scenarios by their unique power consumption. Current fingerprinting methods for neural networks rely on input/output pairs or characteristic of the decision boundary, which might not always be accessible in more complex systems. Our proposed method utilizes unique power characteristics of the black-box neural network accelerator to extract a fingerprint by measuring the voltage fluctuations of the device when querying specially crafted inputs. We take advantage of the fact that the power consumption of the accelerator varies depending on the input being processed. For evaluation of our method we conduct 200 fingerprint extraction and matching experiments and the results confirm that the proposed method can distinguish between correct and incorrect models in 100% of the cases. Furthermore, we show that the fingerprint is robust to environmental and chip-to-chip variations.

There is an increasing demand for next-generation flexible electronics in emerging low-cost applications such as smart packaging and smart bandages, where conventional silicon electronics cannot enter due to cost and form factor. In these domains, ultra-low-cost, high flexibility, and customizability are required. In this regard, printed electronics emerge as a complementary solution offering the aforementioned properties. To respect the constraints in those application scenarios and equip printed devices with the fundamental capability to process information, analog printed neuromorphic circuits offer multiple advantages, including strong expressiveness, streamlined circuit primitives, and a highly efficient machine learning-based design process. In this work, we focus on designing low-power printed neuromorphic circuits at the algorithmic level. By developing accurate power models for the circuit primitives, the power consumption can be considered into the design process. Subsequently, Pareto analysis is employed to examine the relationship between accuracy and power consumption. Experimental results reveal that, with the proposed approach, 2 x reduction of the power consumption can be realized while maintaining 95 % of classification accuracy. This approach has significant implications for the future development of energy-efficient printed neuromorphic circuits and their potential applications in IoT and AI intersections.

To this day, various approaches for providing local explanation of black box machine learning models have been introduced. Despite these efforts, existing methods suffer from deficiencies such as being difficult to comprehend, only considering one feature at a time and disregarding inter-feature dependencies, lacking meaningful values for each feature, or only highlighting features that support the model's decision. To overcome these drawbacks, this study presents a new approach to explain the predictions of any black box classifier, called Monte Carlo tree search for eXplainable Artificial Intelligence (McXai). It employs a reinforcement learning strategy and models the explanation generation as two distinct games. In the first game, the objective is to identify feature sets that support the model's decision, while in the second game, the aim is to find feature sets that lead to alternative decisions. The output is a human-friendly representation in the form of a tree structure, where each node represents a set of features to be examined, with less specific interpretations at the top of the tree. Our experiments demonstrate that the features identified by McXai are more insightful with regard to the classifications compared to traditional algorithm like LIME and Gram-cam. Furthermore, the ability to identify misleading features provides guidance towards improved robustness of the black box classifier.

Neuromorphic computation-in-memory fabric based on emerging nonvolatile memories considered an attractive option to accelerate neural networks (NNs) in hardware as they provide high-performance, low-power, and reduced data movement. Although nonvolatile resistive memories (NVMs) offer many benefits, they are susceptible to data retention faults, where previously stored data are not retained after a certain amount of time due to external influence. These faults are more likely to happen unidirectional and severely impact the inference accuracy of the hardware implementation of NNs since the synaptic weights stored in the NVMs are subject to retention faults. In this work, we propose an approximate scrubbing technique for NVM-based neuromorphic fabric to mitigate unidirectional retention faults with virtually zero storage overhead depending on the definition of scrub area for multilayer perceptron (MLP) and convolutional NNs (CNNs). The training of the NNs is adjusted accordingly to meet the requirements of the proposed approximate scrubbing scheme. On different benchmarks, the proposed scrubbing approach can improve the inference accuracy up to 85.51% for MLP and 87.76% for CNN over the expected device operational time with negligible storage overhead.

Recent development in neural networks (NNs) has led to their widespread use in critical and automated decision-making systems, where uncertainty estimation is essential for trustworthiness. Although conventional NNs can solve many problems accurately, they do not capture the uncertainty of the data or the model during optimization. In contrast, Bayesian neural networks (BNNs), which learn probabilistic distributions for their parameters, offer a sound theoretical framework for estimating uncertainty. However, traditional hardware implementations of BNNs are expensive in terms of computational and memory resources, as they (i) are realized with inefficient von Neumann architectures, (ii) use a significantly large number of random number generators (RNGs) to implement the distributions of BNNs, and (iii) have a substantially greater number of parameters than conventional NNs. Computing-in-memory (CiM) architectures with emerging resistive non-volatile memories (NVMs) are promising candidates for accelerating classical NNs. In particular, spintronic technology, which is distinguished by its low latency and high endurance, aligns very well with these requirements. In the specific context of Bayesian neural networks (BNNs), spintronics technologies are very valuable, thanks to their inherent potential to act as stochastic or as deterministic devices. Consequently, BNNs mapped on spintronic-based CiM architectures could be a highly efficient implementation strategy. However, the direct implementation on CiM hardware of the learned probabilistic distributions of BNN may not be feasible and can incur high overhead. In this work, we propose a new Bayesian neural network topology, named SpinBayes , that is able to perform efficient sampling during the Bayesian inference process. Moreover, a Bayesian approximation method, called in-memory approximation , is proposed that approximates the original probabilistic distributions of BNN with a distribution that can be efficiently mapped to spintronic-based CiM architectures. Compared to state-of-the-art methods, the memory overhead is reduced by 8× and the energy consumption by 80×. Our method has been evaluated on several classification and semantic segmentation tasks and can detect up to 100% of various types of out-of-distribution data, highlighting the robustness of our approach, without any performance sacrifice.

Printed electronic (PE) can meet the requirements of many application domains with requirements on cost, conformity, and non-toxicity which silicon-based computing systems cannot achieve. A typical computational task to be performed in many of such applications is classification. Therefore, printed Neural Networks (pNNs) have been proposed to meet these requirements. However, PE suffers from high process variations due to low resolution printing in low-cost additive manufacturing. This can severely impact the inference accuracy of pNNs. In this work, we show how a unique feature of PE, namely additive printing can be leveraged to perform in-situ tuning of pNNs to compensate accuracy losses induced by device variations. The experiments show that, even under 30 % variation of the conductances, up to 90 % of the initial accuracy can be recovered.

Neural Networks (NN) can be efficiently accelerated using emerging resistive non-volatile memories (eNVM), such as Spin Transfer Torque Magnetic RAM(STT-MRAM). However, process variations and runtime temperature fluctuations can lead to miss-quantizing the sensed state and in turn, degradation of inference accuracy. We propose a design-time reference current generation method to improve the robustness of the implemented NN under different thermal and process variation scenarios with no additional runtime hardware overhead compared to existing solutions.

Printed electronics allow for ultra-low-cost circuit fabrication with unique properties such as flexibility, non-toxicity, and stretchability. Because of these advanced properties, there is a growing interest in adapting printed electronics for emerging areas such as fast-moving consumer goods and wearable technologies. In such domains, analog signal processing in or near the sensor is favorable. Printed neuromorphic circuits have been recently proposed as a solution to perform such analog processing natively. Additionally, their learning-based design process allows high efficiency of their optimization and enables them to mitigate the high process variations associated with low-cost printed processes. In this work, we address the aging of the printed components. This effect can significantly degrade the accuracy of printed neuromorphic circuits over time. For this, we develop a stochastic aging-model to describe the behavior of aged printed resistors and modify the training objective by considering the expected loss over the lifetime of the device. This approach ensures to provide acceptable accuracy over the device lifetime. Our experiments show that an overall 35.8% improvement in terms of expected accuracy over the device lifetime can be achieved using the proposed learning approach.

With ever-increasing transistor density in nanoscale-integrated circuits, the impact of process variations on circuit performance and chip yield becomes dominant. To prevent failures in the field, simulation-based circuit optimization tools are performed during design time as a countermeasure. However, the efficiency of these tools requires accurate modeling of failure probabilities. Especially, for high-sigma problems such as yield estimation of memory cells, which require very high production yield, the failure rate assessment must be highly accurate. Importance sampling (IS) methods are deployed in this context to uncover very rare failure events, which cannot be revealed by standard Monte Carlo methods. Besides a highly accurate yield prediction model, the limited time budget of the simulation-based analysis tools has to be taken into account. Especially in conjunction with yield optimization techniques, the required number of circuit simulations for the failure rate estimation has to be substantially reduced. In this article, we propose a yield optimization method, which is based on a Bayesian optimization (BO) failure rate estimation technique for high-sigma yield extraction. The BO-based IS method is combined with a kernel density estimator for finding the most probable failure events, which have significant contribution to the chip yield. By integration into a global optimization framework, we show how the proposed yield optimization method can be applied to high-sigma yield optimization problems, such as memory cells. The experimental results indicate that the proposed method consumes only 5% circuit simulations to achieve the same optimization effect as the state-of-the-art yield optimizer techniques.

Printed electronics (PE) offers flexible, extremely low-cost, and on-demand hardware due to its additive manufacturing process, enabling emerging ultra-low-cost applications, including machine learning applications. However, large feature sizes in PE limit the complexity of a machine learning classifier (e.g., a neural network (NN)) in PE. Stochastic computing Neural Networks (SC-NNs) can reduce area in silicon technologies, but still require complex designs due to unique implementation tradeoffs in PE. In this paper, we propose a printed mixed-signal system, which substitutes complex and power-hungry conventional stochastic computing (SC) components by printed analog designs. The printed mixed-signal SC consumes only 35% of power consumption and requires only 25% of area compared to a conventional 4-bit NN implementation. We also show that the proposed mixed-signal SC-NN provides good accuracy for popular neural network classification problems. We consider this work as an important step towards the realization of printed SC-NN hardware for near-sensor-processing.

Neuromorphic computation-in-memory fabric based on emerging non-volatile memories (NVM) is considered an attractive option to accelerate neural networks (NNs) in hardware as they provide high-performance, low-power, and reduced data movement. Although NVMs offer many benefits, they are susceptible to data retention faults, where previously stored data is not retained. This severely impacts the inference accuracy of mapped NNs. Traditionally, memory scrubbing with error-correcting codes (ECC) is employed to mitigate retention faults in conventional CMOS memories. This is not feasible in NVM-based neuromorphic fabric due to high overhead and inability to represent encoding or decoding in analog computing. In this work, we propose an approximate scrubbing technique for NVM-based neuromorphic fabric to mitigate uni-directional retention faults with minimal storage overhead. The training of the NNs adjusted accordingly to meet the requirements of the scrubbing scheme. On different benchmarks, the proposed scrubbing approach can improve the inference accuracy up to 85.51% over the lifetime with virtually zero storage overhead.

Neuromorphic computing systems have demonstrated many advantages for popular classification problems with significantly less computational resources. We present in this paper the design, fabrication and training of a programmable neuromorphic circuit, which is based on printed electrolytegated field-effect transistor (EGFET). Based on printable neuron architecture involving several resistors and one transistor, the proposed circuit can realize multiply-add and activation functions. The functionality of the circuit, i.e. the weights of the neural network, can be set during a post-fabrication step in form of printing resistors to the crossbar. Besides the fabrication of a programmable neuron, we also provide a learning algorithm, tailored to the requirements of the technology and the proposed programmable neuron design, which is verified through simulations. The proposed neuromorphic circuit operates at 5V and occupies 385mm2of area.

Due to the aggressive technology downscaling, process variations are becoming pre-dominent, causing performance fluctuations and impacting the chip yield. Therefore, individual circuit components have to be designed with very small failure rates to guarantee functional correctness and robust operation. The assessment of high-sigma failure rates however cannot be achieved with conventional Monte Carlo (MC) methods due to the huge amount of required time-consuming circuit simulations. To this end, Importance Sampling (IS) methods were proposed to solve the otherwise intractable failure rate estimation problem by focusing on high-probable failure regions. However, the failure rate could largely be underestimated while the computational effort for deriving them is high. In this paper, we propose an eXtended Bayesian Optimized IS (XBOIS) method, which addresses the aforementioned shortcomings by deployment of an accurate surrogate model (e.g. delay) of the circuit around the failure region. The number of costly circuit simulations is therefore minimized and estimation accuracy is substantially improved by efficient exploration of the variation space. As especially memory elements occupy a large amount of on-chip resources, we evaluate our approach on SRAM cell failure rate estimation. Results show a speedup of about 16x as well as a two orders of magnitude higher failure rate estimation accuracy compared to the best state-of-the-art techniques.

Printed Electronics technology is a key-enabler for smart sensors, soft robotics, and wearables. The inkjet printed electrolyte-gated field effect transistor (EGFET) technology is a promising candidate for such applications due to its low-power operation, high field-effect mobility, and on-demand fabrication. Unlike conventional silicon-based technologies, inkjet printed electronics technology is an additive manufacturing process where multiple layers are printed on top of each other to realize functional devices such as transistors and their interconnections. Due to the additive manufacturing process, the technology has limited routing layers. For routing of complex circuits, insulating crossovers are printed at the intersection of routing paths to isolate them. The crossover can alter the electrical properties of a circuit based on specific location on a routing path. In this work, we propose a crossover-aware placement and routing (COPnR) methodology for inkjet-printed circuits by integrating the crossover constraints in our design framework. Our proposed placement methodology is based on a state-of-the-art evolutionary algorithm while the routing optimization is done using a genetic algorithm. The proposed methodology is compared with the industrial standard placement and routing (PnR) tools. On average, the proposed methodology has 38% fewer crossovers and 94% fewer failing paths compared to the industrial PnR tools applied to printed circuit designs.

In many application domains, in particular automotives, guaranteeing a very low failure rate is crucial to meet functional and safety standards. Especially, reliable operation of memory components such as SRAM cells is of essential importance. Due to aggressive technology downscaling, process and runtime variations significantly impact manufacturing yield as well as functionality. For this reason, a thorough memory failure rate assessment is imperative for correct circuit operation and yield improvement. In this regard, Monte Carlo (MC) simulations have been used as the conventional method to estimate the variability induced failure rate of memory components. However, MC methods become infeasible when estimating rare events such as high-sigma failure rates. To this end, importance sampling (IS) methods have been proposed which reduce the number of required simulations substantially. However, existing methods still suffer from inaccuracies and high computational efforts, in particular for high-sigma problems. In this article, we fill this gap by presenting an efficient mixture IS approach based on Bayesian optimization, which deploys a surface model of the objective function to find the most probable failure points. Its advantages include constant complexity independent of the dimensions of design space, the potential to find the global extrema, and the higher trustworthiness of the estimated failure rate by accurately exploring the design space. The approach is evaluated on a 6T-SRAM cell as well as a master-slave latch based on a 28-nm FDSOI process. The results show an improvement in accuracy, resulting in up to 63× better accuracy in estimating failure rates compared to the best state-of-the-art solutions on a 28-nm technology node.

Printed electronics (PE) circuits have several advantages over silicon counterparts for the applications where mechanical flexibility, extremely low-cost, large area, and custom fabrication are required. The custom (personalized) fabrication is a key feature of this technology, enabling customization per application, even in small quantities due to low-cost printing compared with lithography. However, the personalized and on-demand fabrication, the non-standard circuit design, and the limited number of printing layers with larger geometries compared with traditional silicon chip manufacturing open doors for new and unique reverse engineering (RE) schemes for this technology. In this paper, we present a robust RE methodology based on supervised machine learning, starting from image acquisition all the way to netlist extraction. The results show that the proposed RE methodology can reverse engineer the PE circuits with very limited manual effort and is robust against non-standard circuit design, customized layouts, and high variations resulting from the inherent properties of PE manufacturing processes.

Printed Electronics is perceived to have a major impact in the fields of smart sensors, Internet of Things and wearables. Especially low power printed technologies such as electrolyte gated field effect transistors (EGFETs) using solution-processed inorganic materials and inkjet printing are very promising in such application domains. In this paper, we discuss a modeling approach to describe the variations of printed devices. Incorporating these models and design flows into our previously developed printed design system allows for robust circuit design. Additionally, we propose a reliability-aware routing solution for printed electronics technology based on the technology constraints in printing crossovers. The proposed methodology was validated on multiple benchmark circuits and can be easily integrated with the design automation tools-set.

Due to aggressive technology downscaling, process and runtime variations have a strong impact on the correct functionality in the field as well as manufacturing yield. The assessment of the yield and failure rate is extremely crucial for design optimization. The common practice is to use Monte Carlo simulations in order to account for device variations and estimate failure rate. However, Monte Carlo methods are infeasible for estimating rare events such as high sigma failure rates, and hence, various importance sampling methods have been proposed. In this paper, we present an efficient importance sampling approach based on Bayesian optimization. Its advantages include constant complexity independent of the dimensions of design space, the potential to find the global extrema, and higher trustworthiness of the estimated failure rate. We evaluated the approach on a 6T SRAM cell based on a 28nm FDSOI process. The results show significant speedup and more than two orders of magnitude better accuracy in failure rate estimation, compared to the best state-of-the-art technique.