Hyperdimensional Computing (HDC) is an emerging paradigm that stands as a compelling alternative to conventional Deep Learning algorithms. HDC holds four key promises. First, the ability to learn from little data. Second, to be robust against noise in this data. HDC also promises to be resilient against errors in the underlying hardware. This includes the memory on which the model is stored and errors in the computations of the operations, which is attributed to the encoding of information across an expansive dimensional space. Fourth, HDC can be implemented efficiently in hardware due to its lightweight and embarrassingly parallel computations. In this work, those four key promises are evaluated in a holistic way. A fixed-point and a binary HDC implementation are compared against neural network implementations. The models are executed on a RISC-V processor to ensure a fair comparison. While the results confirm the ability to learn from little data and the resiliency against errors, the higher inference accuracy of neural networks favors them in most experiments. Based on these insights, we formulate challenges and opportunities for HDC. Our implementations for QNN, binary and fixed-point HDC are available online: https://github.com/TUM-AIPro/HDC-vs-QNN

Graph Neural Networks (GNNs) are one of the best-performing models for processing graph data. They are known to have considerable computational complexity, despite the smaller number of parameters compared to traditional Deep Neural Networks (DNNs). Operations-to-parameters ratio for GNNs can be tens and hundreds of times higher than for DNNs, depending on the input graph size. This complexity indicates the importance of arithmetic operation optimization within GNNs through model quantization and approximation. In this work, for the first time, we combine both approaches and implementquantization-andapproximation-aware trainingfor GNNs to sustain their accuracy under the errors induced by inexact multiplications. We employ matrix multiplication CUDA kernel to speed up the simulation of approximate multiplication within GNNs. Further, we demonstrate the execution speed, accuracy, and energy efficiency of GNNs with approximate multipliers in comparison with quantized low-bit GNNs. We evaluate the performance of state-of-the-art GNN architectures (i.e., GIN, SAGE, GCN, and GAT) on various datasets and tasks (i.e., Reddit-Binary, Collab for graph classification, Cora and PubMed for node classification) with a wide range of approximate multipliers. Our framework is available online:https://github.com/TUM-AIPro/AxC-GNN.

This work investigates the joint impact of device variability and transistor aging on the data integrity of SRAM cells implemented using 22 FDSOI. Our analysis is based on well-calibrated TCAD simulations that reproduce measurements from a commercial 22nm FDSOI technology node. The calibrations are done against measurement data for both I-V characteristics and variability data. We perform error analysis for SRAMs during hold and read operations under three different scenarios: (i) Fresh: time-zero variation (PV) alone caused by manufacturing variability, (ii) Aged: combined impact of PV and aging-induced increase in the transistor threshold voltage ($V_{TH}$) at the room temperature, (iii) Aged@85°C: combined impact of PV and transistor aging but at an elevated temperature of 85°C. Further, we explore how SRAM errors are exacerbated when the voltage is scaled down due to the reductions in noise margins. All error analyses were accurately performed in TCAD mixed-mode simulations for a complete 6-T SRAM cell. Finally, to investigate further how such errors impact the system level, we explore the corresponding induced accuracy drop in Deep Neural Networks (DNNs). Different quantized NNs are studied, and their sensitivity to errors in weights and activations is also explored. We demonstrate that short-term aging (i.e., when aging effects are combined with voltage scaling) results in a noticeable accuracy drop when ResNet20 and ResNet18 DNN models are examined on the CIFAR100 and Imagenet datasets, respectively.

To overcome the memory wall in neural network (NN) inference systems, recent studies have proposed to use approximate memory, in which the supply voltage and access latency parameters are tuned, for lower energy consumption and faster access at the cost of reliability. To tolerate the occuring bit errors, the state-of-the-art approaches apply bit flip injections to the NNs during training, which require high overheads and do not scale well for large NNs and high bit error rates. In this work, we focus on binarized NNs (BNNs), whose simpler structure allows better exploration of bit error tolerance metrics based on margins. We provide formal proofs to quantify the maximum number of bit flips that can be tolerated. With the proposed margin-based metrics and the well-known hinge loss for maximum margin classification in support vector machines (SVMs), we are able to construct a modified hinge loss (MHL) to train BNNs for bit error tolerance without any bit flip injections. Our experimental results indicate that the MHL enables the possibility for BNNs to tolerate higher bit error rates than with bit flip training and, therefore, allows to further lower the requirements on approximate memories used for BNNs.