The proliferation of Internet of Things (IoT) and edge computing devices has become an essential aspect of our daily routines. Particularly, the rise of wearable technology like smartwatches, health trackers, and smart glasses has contributed significantly to their popularity. These gadgets are equipped with diverse sensors that enable researchers and manufacturers to collect user data. Subsequently, this data undergoes processing through on-device Machine Learning (ML) algorithms, enhancing user interactions. However, implementing ML algorithms on these compact IoTs and edge devices consumes substantial power and energy. It’s crucial to recognize that these devices operate within strict energy and power constraints. Thus, optimizing battery usage is paramount for prolonging a device’s lifespan. Therefore, we propose a Processing-In-Memory (PIM) architecture utilizing Look-up-Table (LUT) based processing for improved performance and energy efficiency. To further enhance energy efficiency in this work we introduce a framework that efficiently utilizes kinetic energy harvesting to intermittently support ML computations/tasks, thereby alleviating the load on the device’s built-in battery. By offloading ML computations to the PIM architecture, the framework reduces the reliance on the device’s internal battery power, optimizing the use of harvested kinetic energy and extending battery life. Furthermore, PIM architecture facilitates seamless integration of harvested kinetic energy, ensuring efficient ML computations with minimal energy consumption. This integrated approach presents a compelling solution for energy management in IoT and edge-based applications, as evidenced by experiments and analysis showing significant reductions in overall energy usage. We evaluated the proposed Energy Harvesting-assisted PIM architecture on various CNN architectures, such as LeNet, AlexNet, ResNet -18, -34, -50.

Convolutional neural networks (CNNs) have achieved significant success in various applications. Numerous hardware accelerators are introduced to accelerate CNN execution with improved energy efficiency compared to traditional software implementations. Despite the achieved success, deploying traditional hardware accelerators for bulky CNNs on current and emerging smart devices is impeded by limited resources, including memory, power, area, and computational capabilities. Recent works introduced processing-in-memory (PIM), a non-Von-Neumann architecture, which is a promising approach to tackle the problem of data movement between logic and memory blocks. However, as observed from the literature, the existing PIM architectures cannot congregate all the computational operations due to limited programmability and flexibility. Furthermore, the capabilities of the PIM are challenged by the limited available on-chip memory. To enable faster computations and address the limited on-chip memory constraints, this work introduces a novel reconfigurable approximate computing-based PIM, termed ReApprox-PIM. The proposed ReApprox-PIM is capable of addressing the two challenges mentioned above in the following manner: (i) it utilizes a programmable look-up-table (LUT)-based processing architecture that can support different approximate computing techniques via programmability, and (ii) followed by resource-efficient, fast CNN computing via the implementation of highly-optimized approximate computing techniques. This results in improved computing footprint, operational parallelism, and reduced computational latency and power consumption compared to prior PIMs relying on exact computations for CNN inference acceleration at a minimal sacrifice of accuracy. We have evaluated the proposed ReApprox-PIM on various CNN architectures, for inference applications including standard LeNet, AlexNet, ResNet-18, -34, and -50. Our experimental results show that the ReApprox-PIM achieves a speedup of 1.63× with 1.66 × lower area for the processing components compared to the existing PIM architectures. Furthermore, the proposed ReApprox-PIM achieves 2.5× higher energy efficiency and 1.3× higher throughput compared to the state-of-the-art LUT-based PIM architectures.

Processing-in-Memory (PIM) has shown great potential for a wide range of data-driven applications, especially Deep Learning and AI. However, it is a challenge to facilitate the computational sophistication of a standard processor (i.e. CPU or GPU) within the limited scope of a memory chip without contributing significant circuit overheads. To address the challenge, we propose a programmable LUT-based area-efficient PIM architecture capable of performing various low-precision floating point (FP) computations using a novel LUT-oriented operand-decomposition technique. We incorporate such compact computational units within the memory banks in a large count to achieve impressive parallel processing capabilities, up to 4x higher than state-of-the-art FP-capable PIM. Additionally, we adopt a highly-optimized low-precision FP format that maximizes computational performance at a minimal compromise of computational precision, especially for Deep Learning Applications. The overall result is a 17% higher throughput and an impressive 8-20x higher compute Bandwidth/bank compared to the state-of-the-art of in-memory acceleration.

Reconfigurable nanotechnologies such as Silicon Nanowire Field Effect Transistors (FETs) serve as a promising technology that not only facilitates lower power consumption but also supports multi-functionality through reconfigurability. It enables reconfigurability and supports multiple functionalities per computational unit. These features motivate us to design a novel state-of-the-art energy-efficient hardware accelerator for implementing memory-intensive applications including convolutional neural networks (CNNs) and deep neural networks (DNNs). To accelerate the computations, we design Multiply and Accumulate (MAC) units to perform the computations. For the design of MACs, we employ Silicon nanowire reconfigurable FETs (RFETs). The use of RFETs leads to nearly 70% power reduction compared to the traditional CMOS implementation and also reduced latency in performing the computations. Further to optimize the overheads and improve memory efficiency, we introduce a novel approximation technique for RFETs. The RFET-based approximate adders lead to reduced power, area, and delay while having a minimal impact on the accuracy of the DNN/CNN. In addition, we carry out a detailed study of varied combinations of architectures involving CMOS, RFETs, accurate adders, and approximate adders to demonstrate the benefits of the proposed RFET-based approximate acclerator. The proposed RFET-based accelerator achieves an accuracy of 94% on MNIST datasets with 93% and 73%reduction in the area, power and delay metrics respectively compared to the state-of-the-art hardware accelerator architectures.

Improving the performance of real-time Traffic Sign Recognition (TSR) applications using Deep Learning (DL) algorithms such as Convolutional Neural Networks (CNN) on software platforms is challenging due to the sheer computational complexity of these algorithms. In this work, we adopt a hardware-software combined approach to address this issue. We introduce a data-centric Processing-in-Memory (PIM) architecture that leverages Look-up-Table (LUT)-based processing for minimal data movement and superior performance and efficiency. Despite the superior performance, the limited available memory in PIM makes it complex to deploy deep CNNs. We propose merging CNN layers in this work to meet the limited resource constraints. One specific challenge in the TSR is the continuous change in the deployed environment, which makes a CNN model train over static data, leading to performance degradation over time. To address these challenges, we introduce a lightweight, performance-aware Generative Adversarial Network (GAN)-based on-device learning on PIM architecture. This compact CNN on PIM architecture attains data-level parallelism and reduces pipelining delays and makes it easier for on-device training and inference. Evaluation is performed on multiple state-of-the-art DL networks such as LeNet, AlexNet, ResNet using the German Traffic Sign Recognition Benchmark (GTSRB) Dataset, and the Belgium Traffic Sign Dataset (BTSD). With the proposed learning technique, it is observed to achieve maximum accuracy of 92.8% and 89.27% on GTSRB, and BTSD datasets. Also, it is observed the proposed mechanism maintains an average accuracy to be above 85% despite changes in the environment on all the CNNs deployed on the PIM accelerator.

Recent research has demonstrated that machine learning algorithms are vulnerable to adversarial attacks, in which small but carefully crafted input perturbations can lead to algorithm failure. It has been demonstrated that certain adversarial attack algorithms are capable of producing these types of perturbations. These attack methods are inapplicable when the attack must be generated in near real time. The use of a hardware accelerator, such as a Process-in-Memory (PIM) archi-tecture, is a potential method for addressing this issue. The PIM architecture is regarded as a superior option for data-intensive applications such as solving optimization problems and Deep Neural Networks (DNN) due to its capacity for ultra-low-latency parallel processing. However, implementing an adversarial attack algorithm directly on the PIM platform is inefficient due to the PIM architecture's complexity and overhead costs. To address this issue, we utilize a novel adversarial attack scheme based on the PIM that leverages Look-up-Table (LUT)-based processing. The proposed LUT-based PIM architecture is capable of being dynamically programmed to execute the operations necessary for an adversarial attack algorithm. Our simulations reveal that the proposed method is capable of achieving an ultra-low operating delay and energy-efficiency performance.

Processing in memory (PIM) architecture, with its ability to perform ultra-low-latency parallel processing, is regarded as a more suitable alternative to von Neumann computing architectures for implementing data-intensive applications such as Deep Neural Networks (DNN) and Convolutional Neural Networks (CNN). In this article, we present a Look-up Table (LUT) based PIM architecture aimed at CNN/DNN acceleration that replaces logic-based processing with pre-calculated results stored inside the LUTs in order to perform complex computations on the DRAM memory platform. Our LUT-based DRAM-PIM architecture offers superior performance at a significantly higher energy-efficiency compared to the more conventional bit-wise parallel PIM architectures, while at the same time avoids fabrication challenges associated with the in-memory implementation of logic circuits. Alongside, the processing elements can be programmed and re-programmed to perform virtually any operation, including operations of Convolutional, Fully Connected, Pooling, and Activating Layers of CNN/DNN. Furthermore, it is capable of operating on several combinations of bit-widths of the operand data and thereby offers a wider range of flexibility across performance, precision, and efficiency. Transmission Gate (TG) realization of the circuitry ensures minimal footprint from the PIM architecture. Our simulations demonstrate that the proposed architecture can perform AlexNet inference at a nearly 13× faster rate and 125× more efficiency compared to state-of-the-art GPU and also provides 1.35× higher throughput at 2.5× higher energy-efficiency than another recent DRAM-implemented LUT-based PIM architecture in its baseline operation mode. Moreover, it offers 12× higher frame-rate at 9× more efficiency per frame for the lowest operand precision setting, with respect to its own baseline operation mode.

to meet the extensive computational load presented by the rapidly growing Machine Learning (ML) and Artificial Intelligence (AI) algorithms such as Deep Neural Networks (DNNs) and Convolutional Neural Networks (CNNs). In order to obtain hardware solutions to meet the low-latency and high-throughput computational demands from these algorithms, Non-Von Neumann computing architectures such as In-memory Computing (IMC)/ Processing-in-memory (PIM) are being extensively researched and experimented with. In this survey paper, we analyze and review pioneer IMC/PIM works designed to accelerate ML algorithms such as DNNs and CNNs. We investigate different architectural aspects and dimensions of these works and provide our comparative evaluations. Furthermore, we discuss challenges and limitations in IMC research and also present feasible directions based on our observations and insight.