The Internet of Things (IoT) has led to the emergence of big data. Processing this data, specially in learning algorithms, poses a challenge for current embedded computing systems. Brain-inspired hyperdimensional (HD) computing reduces several complex learning operations to simpler bitwise and arithmetic operations. However, it requires the use of large dimensional vectors, hypervectors, further increasing the amount of data to be processed. Processing in-memory (PIM) enables in-place computation which reduces data movement, a major latency bottleneck in conventional systems. In this article, we propose Tri-HD, an in-memory HD computing architecture that performs HD classification in memory. To the best of authors’ knowledge, Tri-HD is the first ReRAM PIM architecture to implement the complete HD computing-based classification pipeline, including encoding, training, retraining, and inference for nonbinary data. We also propose a novel distance metric that is PIM-friendly and provides similar application accuracy as the more complex baseline metric. Our proposed architecture is enabled in PIM by fast and energy-efficient in-memory logic operations. We exploit the voltage threshold-based memristors to enable single cycle operations. We also increase the amount of in-memory parallelism in our design by segmenting bitlines using switches. Our evaluation shows that for all applications tested using HD, Tri-HD provides on average$434\times $($2170\times $) speedup and consumes$4114\times $($26019\times $) less energy as compared to the CPU while running end-to-end HD training (inference). Tri-HD also achieves at least 2.2% higher-classification accuracy than the existing PIM-based HD designs.

Hyperdimensional Computing (HDC) has emerged as a lightweight learning paradigm garnering considerable attention in the IoT domain. Despite its appeal, HDC has lagged behind more intricate Machine Learning (ML) algorithms in accuracy, prompting prior research to propose sophisticated encoding and training techniques at the expense of efficiency. In this study, we present a novel approach for selecting projection vectors, used to encode input data into high-dimensional spaces, to enable HDC to attain high accuracy with significantly reduced vector sizes. We adopt a neural network-based mechanism to learn the projection vectors, and demonstrate their efficacy when integrated into a conventional HDC system. Furthermore, we introduce a novel sparsity technique to enhance hardware efficiency by compressing projection vectors and reducing computational operations with minimal impact on accuracy. Our experimental results reveal that at larger vector dimensions (e.g., 10k), our method (PIONEER), leveraging INT4 or binary vectors, outperforms the state-of-the-art high-precision nonlinear encoding in terms of accuracy, while preserving noteworthy accuracy even at extremely lower dimensions of 50–100. Additionally, by applying our proposed sparsification technique, PIONEER achieves significant performance and energy efficiency compared to previous work.

Hyperdimensional Computing (HDC) represents an emerging paradigm within the domain of cognitive computing, inspired by the information processing mechanisms observed in the human brain. Despite the research efforts devoted to improving and extending HDC algorithms and hardware, the efficacy and privacy of HDC remains challenging in handling image data. In this study, we highlight the accuracy, efficiency and privacy concerns of existing HDC-based methods on image data. We propose a novel vector-free encoding that shrinks the energy consumption and obviates the need for vector storage. We repurpose the released resources to augment the proposed encoding with a well crafted and privacy-aware feature extractor. Experimental results indicate that our proposed design, designated as VisionHD, gains a significant accuracy improvement (> 22%) while its energy consumption remains within the confines of the baseline HDC. To evaluate the privacy of VisionHD, we introduce a more effective and generic reversing technique, which reveals that VisionHD successfully obfuscates the information and improves the privacy metric by 16.9X.

Convolutional Neural Networks (CNNs) have been explored to detect forced oscillations in windfarm systems in the past. However, these CNNs require a significant amount of data samples between inference queries and a significant amount of computational power and time. This leads to systems that have a large delay between a forced oscillation occurring and detecting the forced oscillation. This paper presents a novel approach applying Hyperdimensional Computing (HDC) as an effective solution for the first time in forced oscillation detection to overcome the problems of CNNs. HDC is able to reduce the time to detect forced oscillations in two ways: First, by reducing the time needed to collect data to create a new inference sample by reducing the number of data points required. Second, by providing a significantly smaller, more energy efficient, and faster model for detection than current state-of-the-art. Our results show that HDC, with an FPGA implementation, is able to achieve $ 55\times$ faster detection of forced oscillations in windfarms while achieving the same accuracy as the best current CNN models using software solutions.

In this paper, we propose EdgeHD, a hierarchy-aware learning solution that performs online training and inference in a highly distributed, cost-effective way. We use brain-inspired hyperdimensional (HD) computing as the key enabler. HD computing performs the computation tasks on a high-dimensional space to emulate functionalities of the human memory, such as inter-data relationship reasoning and information aggregation. EdgeHD exploits HD computing to effectively learn the classification models on individual devices and combine the models through the hierarchical IoT nodes without high communication costs. We also propose a hardware design that accelerates EdgeHD on low-power FPGA platforms. We evaluated EdgeHD for a wide range of real-world classification applications. The evaluation shows that EdgeHD provides highly efficient computation with reduced communication. For example, EdgeHD achieves on average$3.4\times$and$11.7\times (1.9\times$and$7.8\times$) speedup and energy efficiency improvement during the training (inference) as compared to the centralized learning approach. It reduces the communication costs by 85% for the training and 78% for the inference.

Today’s machine learning (ML) systems, running workloads, such as deep neural networks, which require billions of parameters and many hours to train a model, consume a significant amount of energy. Due to the complexity of computation and topology, even the quantized models are hard to deploy on edge devices under energy constraints. To combat this, researchers have been focusing on new emerging neuromorphic computing models. Two of those models are hyperdimensional computing (HDC) and spiking neural networks (SNNs), both with their own benefits. HDC has various desirable properties that other ML algorithms lack, such as robustness to noise, simple operations, and high parallelism. SNNs are able to process event-based signal data in an efficient manner. This work develops$\mathsf {HyperSpike}$, which utilizes a single, randomly initialized, and untrained SNN layer as a feature extractor connected to a trained HDC classifier. HDC is used to enable more efficient classification as well as provide robustness to errors. We experimentally show that$\mathsf {HyperSpike}$is on average$31.5\times $more robust to errors than traditional SNNs. On Intel’s Loihi (Davies et al., 2018),$\mathsf {HyperSpike}$is$10\times $faster and$2.6\times $more energy efficient over traditional SNN networks. We further develop$\mathsf {HyperSpikeASIC}$, a customized accelerator for$\mathsf {HyperSpike}$. By decoupling the neuron and synapses,$\mathsf {HyperSpikeASIC}$skips the inactive neurons and limits the neuron state updating to once per time step at most.$\mathsf {HyperSpikeASIC}$is$601\times $faster and$3467\times $more energy efficient than$\mathsf {HyperSpike}$running on Intel’s Loihi for SNN acceleration, and$12.2\times $faster and$211\times $more energy efficient than the state-of-the-art SNN ASIC implementation (Wang et al., 2022).

Hyperdimensional Computing (HDC) mimics the brain's basic principles in performing cognitive tasks by encoding the data to high-dimensional vectors and employing non-complex learning techniques. Conventional processing platforms such as CPUs and GPUs are incapable of taking full advantage of the highly-parallel bit-level operations of HDC. On the other hand, existing HDC encoding techniques do not cover a broad range of applications to make a custom design plausible. In this paper, we first propose a novel encoding that achieves high accuracy for diverse applications. Thereafter, we leverage the proposed encoding and design a highly efficient and flexible ASIC accelerator, dubbed GENERIC, suited for the edge domain. GENERIC supports both classification (train and inference) and clustering for unsupervised learning on edge. Our design is flexible in the input size (hence it can run various applications) and hypervectors dimensionality, allowing it to trade off the accuracy and energy/performance on-demand. We augment GENERIC with application-opportunistic power-gating and voltage over-scaling (thanks to the notable error resiliency of HDC) for further energy reduction. GENERIC encoding improves the prediction accuracy over previous HDC and ML techniques by 3.5% and 6.5%, respectively. At 14 nm technology node, GENERIC occupies an area of 0.30 mm2, and consumes 0.09 mW static and 1.97 mW active power. Compared to the previous inference-only accelerator, GENERIC reduces the energy consumption by 4.1×.

Today's Machine Learning(ML) systems, especially those running in server farms running workloads such as Deep Neural Networks, which require billions of parameters and many hours to train a model, consume a significant amount of energy. To combat this, researchers have been focusing on new emerging neuromorphic computing models. Two of those models are Hyperdimensional Computing (HDC) and Spiking Neural Networks (SNNs), both with their own benefits. HDC has various desirable properties that other Machine Learning (ML) algorithms lack such as: robustness to noise in the system, simple operations, and high parallelism. SNNs are able to process event based signal data in an efficient manner. In this paper, we create HyperSpike, which utilizes a single, randomly initialized and untrained SNN layer as feature extractor connected to a trained HDC classifier. HDC is used to enable more efficient classification as well as provide robustness to errors. We experimentally show that HyperSpike is on average 31.5× more robust to errors than traditional SNNs. We also implement HyperSpike in hardware, and show that it is 10x faster and 2.6× more energy efficient over traditional SNN networks run on Intel's Loihi [1].

Brain-inspired hyper-dimensional (HD) computing is a new computing paradigm emulating the neuron’s activity in high-dimensional space. The first step in HD computing is to map each data point into high-dimensional space (e.g., 10 000), which requires the computation of thousands of operations for each element of data in the original domain. Encoding alone takes about 80% of the execution time of training. In this article, we propose, ReHD, an entire rework of encoding, training, and inference in HD computing for a more hardware friendly implementation. ReHD includes a full binary encoding module for HD computing for energy-efficient and high-accuracy classification. Our encoding module based on random projection with a predictable memory access pattern can be efficiently implemented in hardware. ReHD is the first HD-based approach that provides data projection with a 1:1 ratio to the original data and enables all training/inference computation to be performed using binary hypervectors. After the optimizations ReHD adds to the encoding process, retraining and inference become the energy intensive part of HD computing. To resolve this, we additionally propose model quantization. Model quantization introduces a novel method of storing class hypervectors using$n$-bits, where$n$ranges from 1 to 32, rather than at full 32-bit precision, which allows for fine-grained tuning of the tradeoff between energy efficiency and accuracy. To further improve ReHD efficiency, we developed an online dimension reduction approach that removesinsignificanthypervector dimensions during training.

Processing large amounts of data, especially in learning algorithms, poses a challenge for current embedded computing systems. Hyperdimensional (HD) computing (HDC) is a brain-inspired computing paradigm that works with high-dimensional vectors called hypervectors . HDC replaces several complex learning computations with bitwise and simpler arithmetic operations at the expense of an increased amount of data due to mapping the data into high-dimensional space. These hypervectors, more often than not, cannot be stored in memory, resulting in long data transfers from storage. In this article, we propose Store-n-Learn, an in-storage computing solution that performs HDC classification and clustering by implementing encoding, training, retraining, and inference across the flash hierarchy. To hide the latency of training and enable efficient computation, we introduce the concept of batching in HDC. We also present on-chip acceleration for HDC encoding in flash planes. This enables us to exploit the high parallelism provided by the flash hierarchy and encode multiple data points in parallel in both batched and non-batched fashion. Store-n-Learn also implements a single top-level FPGA accelerator with novel implementations for HDC classification training, retraining, inference, and clustering on the encoded data. Our evaluation over 10 popular datasets shows that Store-n-Learn is on average 222× (543×) faster than CPU and 10.6× (7.3×) faster than the state-of-the-art in-storage computing solution, INSIDER for HDC classification (clustering).

Today’s systems rely on sending all the data to the cloud and then using complex algorithms, such as Deep Neural Networks, which require billions of parameters and many hours to train a model. In contrast, the human brain can do much of this learning effortlessly. Hyperdimensional (HD) Computing aims to mimic the behavior of the human brain by utilizing high-dimensional representations. This leads to various desirable properties that other Machine Learning (ML) algorithms lack, such as robustness to noise in the system and simple, highly parallel operations. In this article, we propose 𝖧𝗒𝖣𝖱𝖤𝖠, a HyperDimensional Computing system that is Robust, Efficient, and Accurate. We propose a Processing-in-Memory (PIM) architecture that works in a federated learning environment with challenging communication scenarios that cause errors in the transmitted data. 𝖧𝗒𝖣𝖱𝖤𝖠 adaptively changes the bitwidth of the model based on the signal-to-noise ratio (SNR) of the incoming sample to maintain the accuracy of the HD model while achieving significant speedup and energy efficiency. Our PIM architecture is able to achieve a speedup of 28× and 255× better energy efficiency compared to the baseline PIM architecture for Classification and achieves 32 × speed up and 289 × higher energy efficiency than the baseline architecture for Clustering. 𝖧𝗒𝖣𝖱𝖤𝖠 is able to achieve this by relaxing hardware parameters to gain energy efficiency and speedup while introducing computational errors. We show experimentally, HD Computing is able to handle the errors without a significant drop in accuracy due to its unique robustness property. For wireless noise, we found that 𝖧𝗒𝖣𝖱𝖤𝖠 is 48 × more robust to noise than other comparable ML algorithms. Our results indicate that our proposed system loses less than 1% Classification accuracy, even in scenarios with an SNR of 6.64. We additionally test the robustness of using HD Computing for Clustering applications and found that our proposed system also looses less than 1% in the mutual information score, even in scenarios with an SNR under 7 dB, which is 57 × more robust to noise than K-means.

Recommender systems are important tools for many commercial applications such as online shopping websites. There are several issues that make the recommendation task very challenging in practice. The first is that an efficient and compact representation is needed to represent users, items and relations. The second issue is that the online markets are changing dynamically, it is thus important that the recommendation algorithm is suitable for fast updates and hardware acceleration. In this paper, we propose a new hardware-friendly recommendation algorithm based on Hyperdimensional Computing, called HyperRec. Unlike existing solutions which leverages floating-point numbers for the data representation, in HyperRec, users and items are modeled with binary vectors in a high dimension. The binary representation enables to perform the reasoning process of the proposed algorithm only using Boolean operations, which is efficient on various computing platforms and suitable for hardware acceleration. In this work, we show how to utilize GPU and FPGA to accelerate the proposed HyperRec. When compared with the state-of-the-art methods for rating prediction, the CPU-based HyperRec implementation is 13.75x faster and consumes 87% less memory, while decreasing the mean squared error (MSE) for the prediction by as much as 31.84%. Our FPGA implementation is on average 67.0x faster and has 6.9x higher energy efficient as compared to CPU. Our GPU implementation further achieves on average 3.1x speedup as compared to FPGA, while providing only 1.2x lower energy efficiency.

Hyperdimensional computing (HD) is a new brain-inspired algorithm that mimics the human brain for cognitive tasks. Despite its inherent potential, the practical efficiency of HD is tied to the underlying hardware, which throttles the efficiency of HD in conventional microprocessors. In this paper, we propose tiny-HD, a light-weight dedicated HD platform that targets low power, high energy efficiency, and low latency, while being configurable to support various applications. We leverage an enhanced HD encoding that alleviates the memory requirements and also simplifies the dataflow to make tiny-HD flexible with an efficient architecture. We further augment tiny-HD by pipelining the stages and resource sharing, as well as a data layout that enables opportunistic power reduction. We compared tiny-HD in terms of area, performance, power, and energy consumption with the state-of-the-art HD platforms. tiny-HD occupies ~0.5 mm2, consumes 1.6mW standby and 9.6mW runtime power (at 400 MHz), with a 0.016ms latency on a set of IoT benchmarks. tiny-HD consumes average per-query energy of 160 nJ, which outperforms the state-of-the-art FPGA and ASIC implementations by 95.5× and 11.2×, respectively.

Today's systems, especially in the age of federated learning, rely on sending all the data to the cloud, and then use complex algorithms, such as Deep Neural Networks, which require billions of parameters and many hours to train a model. In contrast, the human brain can do much of this learning effortlessly. Hyperdimensional (HD) Computing aims to mimic the behavior of the human brain by utilizing high dimensional representations. This leads to various desirable properties that other Machine Learning (ML) algorithms lack such as: robustness to noise in the system and simple, highly parallel operations. In this paper, we propose HyDREA, a HD computing system that is Robust, Efficient, and Accurate. To evaluate the feasibility of HyDREA in a federated learning environment with wireless communication noise, we utilize NS-3, a popular network simulator that models a real world environment with wireless communication noise. We found that HyDREA is 48× more robust to noise than other comparable ML algorithms. We additionally propose a Processing-in-Memory (PIM) architecture that adaptively changes the bitwidth of the model based on the signal to noise ratio (SNR) of the incoming sample to maintain the robustness of the HD model while achieving high accuracy and energy efficiency. Our results indicate that our proposed system loses less than 1% classification accuracy, even in scenarios with an SNR of 6.64. Our PIM architecture is also able to achieve 255× better energy efficiency and speed up execution time by 28× compared to the baseline PIM architecture.

Brain-inspired Hyperdimensional (HD) computing is a novel and efficient computing paradigm which is more hardware-friendly than the traditional machine learning algorithms, however, the latest encoding and similarity checking schemes still require thousands of operations. To further reduce the hardware cost of HD computing, we present Stochastic-HD that combines the simplicity of operations in Stochastic Computing (SC) with the complex task solving capabilities of the latest HD computing algorithms. Stochastic-HD leverages deterministic SC, which uses structured input binary bitstreams instead of the traditional randomly generated bitstreams thus avoids expensive SC components like stochastic number generators. We also propose an in-memory hardware design for Stochastic-HD that exploits its high level of parallelism and robustness to approximation. Our hardware uses in-memory bitwise operations along with associative memory-like operations to enable a fast and energy-efficient implementation. With Stochastic-HD, we were able to reach a comparable accuracy with the Baseline-HD. As compared to the best PIM design for HD [1], Stochastic-HD is also 4.4% more accurate and 43.1× more energy-efficient.

Brain-inspired Hyperdimensional (HD) computing is a novel computing paradigm emulating the neuron’s activity in high-dimensional space. The first step in HD computing is to map each data point into high-dimensional space (e.g., 10,000). This poses several problems. For instance, the size of the data can explode and all subsequent operations need to be performed in parallel in D = 10,000 dimensions. Prior work alleviated this issue with model quantization. The HVs could then be stored in less space than the original data and lower bitwidth operations can be used to save energy. However, prior work quantized all samples to the same bitwidth. We propose, AdaptBit-HD, an Adaptive Model Bitwidth Architecture for accelerating HD Computing. AdaptBit-HD operates on the bits of the quantized model one bit at a time to save energy when fewer bits can be used to find the correct class. With AdaptBit-HD, we can achieve both high accuracy by utilizing all the bits when necessary and high energy efficiency by terminating execution at lower bits when our design is confident in the output. We additionally design an endto-end FPGA accelerator for AdaptBit-HD. Compared to 16-bit models, AdaptBit-HD is 14× more energy efficient and compared to binary models, AdaptBit-HD is 1.1% more accurate, which is comparable in accuracy to 16-bit models. This demonstrates that AdaptBit-HD is able to achieve the accuracy of full precision models, with the energy efficiency of binary models.

Hyperdimensional computing (HDC) is a brain-inspired computing paradigm that works with high-dimensional vectors, hypervectors, instead of numbers. HDC replaces several complex learning computations with bitwise and simpler arithmetic operations, resulting in a faster and more energy-efficient learning algorithm. However, it comes at the cost of an increased amount of data to process due to mapping the data into high-dimensional space. While some datasets may nearly fit in the memory, the resulting hypervectors more often than not can't be stored in memory, resulting in long data transfers from storage. In this paper, we propose THRIFTY, an in-storage computing (ISC) solution that performs HDC encoding and training across the flash hierarchy. To hide the latency of training and enable efficient computation, we introduce the concept of batching in HDC. It allows us to split HDC training into sub-components and process them independently. We also present, for the first time, on-chip acceleration for HDC which uses simple low-power digital circuits to implement HDC encoding in Flash planes. This enables us to explore high internal parallelism provided by the flash hierarchy and encode multiple data points in parallel with negligible latency overhead. THRIFTY also implements a single top-level FPGA accelerator, which further processes the data obtained from the chips. We exploit the state-of-the-art INSIDER ISC infrastructure to implement the top-level accelerator and provide software support to THRIFTY. THRIFTY runs HDC training completely in storage while almost entirely hiding the latency of computation. Our evaluation over five popular classification datasets shows that THRIFTY is on average 1612× faster than a CPU-server and 14.4× faster than the state-of-the-art ISC solution, INSIDER for HDC encoding and training.

Many classification problems in practice map each sample to more than one label - this is known as multi-label classification. In this work, we present Multi-label HD, an in 3D storage multi-label classification system that uses Hyperdimensional Computing (HD). Multi-label HD is the first HD system to support multi-label classification. We propose two different mappings of HD to Multi-label HD. The first, Power Set HD, transforms the multi-label problem into single-label classification by creating a new class for each label combination. The second, Multi-Model HD, creates a binary classification model for each possible label. Our evaluation shows that Multi-Model HD achieves, on average,$47.8\times$higher energy efficiency and$47.1\times$faster execution time while achieving 5% higher classification accuracy as state-of-the-art light-weight multi-label classifiers. Power Set HD achieves 13% higher accuracy than Multi-Model HD, but is$2\times$slower. Our 3D-flash acceleration further improves the energy efficiency of Multi-label HD training by$228\times$and reduces the latency by$610\times$vs training on a CPU.

Brain-inspired Hyperdimensional (HD) computing is a new computing paradigm emulating the neuron's activity in high-dimensional space. The first step in HD computing is to map each data point into high-dimensional space (e.g., 10,000), which requires the computation of thousands of operations for each element of data in the original domain. Encoding alone takes about 80% of the execution time of training. In this paper, we propose BRIC, a fully binary Brain-Inspired Classifier based on HD computing for energy-efficient and high-accuracy classification. BRIC introduces a novel encoding module based on random projection with a predictable memory access pattern which can efficiently be implemented in hardware. BRIC is the first HD-based approach which provides data projection with a 1:1 ratio to the original data and enables all training/inference computation to be performed using binary hypervectors. To further improve BRIC efficiency, we develop an online dimension reduction approach which removes insignificant hypervector dimensions during training. Additionally, we designed a fully pipelined FPGA implementation which accelerates BRIC in both training and inference phases. Our evaluation of BRIC a wide range of classification applications show that BRIC can achieve 64.1× and 9.8× (43.8× and 6.1×) energy efficiency and speed up as compared to baseline HD computing during training (inference) while providing the same classification accuracy.

Hyperdimensional (HD) computing is a mathematical framework, inspired by neuroscience, which can be used to represent many machine learning (ML) problems. Data is first encoded into high dimensional space (on the order of 103or 104dimensions) to create hypervectors. HD computing combines these hypervectors to create a model used for inference. However, due to the high dimensionality of the hypervectors, inference in HD is very expensive, especially when it runs on embedded devices with limited resources. One naive approach to improve the efficiency of HD computing is to simply lower the dimensionality of hypervectors, which comes with a corresponding loss in accuracy. However, if the data is compressed intelligently, we can reduce the dimensionality of an HD model without sacrificing accuracy. To that end, we propose CompHD, a novel approach for compressing HD models while maintaining the accuracy of the original model. CompHD utilizes the mathematics of high-dimensional spaces to compress hypervectors into shorter vectors while maintaining the information of full length hypervectors. We evaluated the efficiency of CompHD on a variety of applications. Our results show that CompHD can reduce model size by an average of 69.7%, resulting in a execution time speed up of 4.1 × and improving energy efficiency by 74% while maintaining the accuracy of the original model. This enables more low powered IoT devices to utilize HD computing for ML problems.