Numerous quantum algorithms assume that classical data has already been converted into quantum states, a process known as Quantum State Preparation (QSP). However, achieving precise QSP requires a circuit depth that scales exponentially with the number of qubits, posing a significant challenge to realizing quantum advantage. Recent research explores Parameterized Quantum Circuits (PQCs) as an approximate alternative, offering improved scalability with reduced circuit depth. However, the iterative, state-by-state optimization required by this approach creates substantial runtime overhead, which severely limits its practicality.To improve the efficiency of approximate QSP, we introduce a novel two-stage framework that can potentially generate QSP circuits for arbitrary quantum states. In theoffline training stage, our model learns a direct mapping from target states to circuit parameters, thereby bypassing the need foronline, state-by-state optimizationduring theinference stage. Extensive evaluations show that our approach significantly reduces runtime overhead by up to 132×, making a steady step towards efficient neural approximate QSP.

The Number Theoretic Transform (NTT) significantly impacts the execution time of Fully Homomorphic Encryption (FHE) in practical applications, driving research into accelerated NTT methods. Computing-in-Memory (CIM) offers a promising solution to handle NTT's memory bottlenecks, yet efficiently implementing a CIM-based NTT engine remains challenging due to unique operations and large data sizes. We propose HP-CIM, a high-parallelism digital SRAM-based CIM NTT engine designed for large-scale NTT. HP-CIM integrates MVM-based NTT with a hierarchical SRAM architecture and novel even-odd data mapping, achieving nearly 3.08× faster execution and 4.96× energy savings compared to prior CIM-based designs.

Computing-in-memory (CIM) architectures demonstrate superior performance over traditional architectures. To unleash the potential of CIM accelerators, many compilation methods have been proposed, focusing on application scheduling optimization specific to CIM. However, existing compilation methods often overlook CIM's capability to switch dynamically between compute and memory modes, which is crucial for accommodating the diverse memory and computational needs of real-world deep neural network architectures, especially the emerging large language models. To fill this gap, we introduce CMSwitch, a novel compiler to optimize resource allocation for CIM accelerators with adaptive mode-switching capabilities, thereby enhancing the performance of DNN applications. Specifically, our approach integrates the compute-memory mode switch into the CIM compilation optimization space by introducing a new hardware abstraction attribute. Then, we propose a novel compilation optimization pass that identifies the optimal network segment and the corresponding mode resource allocations using dynamic programming and mixed-integer programming. CMSwitch uses the tailored meta-operator to express the compilation result in a generalized manner. Evaluation results demonstrate that CMSwitch achieves an average speedup of 1.31x compared to existing SOTA CIM compilation works, highlighting CMSwitch's effectiveness in fully exploiting the potential of CIM processors for a wide range of real-world DNN applications.

The billion-scale Large Language Models (LLMs) necessitate deployment on expensive server-grade GPUs with large-storage HBMs and abundant computation capability. As LLM-assisted services become popular, achieving cost-effective LLM inference on budget-friendly hardware becomes the current trend. This has sparked extensive research into relocating LLM parameters from expensive GPUs to external host memory. However, the restricted bandwidth between the host and GPU memory limits the inference performance of existing solutions. This work introduces Hermes, a budget-friendly system that leverages the near-data processing units (NDP) within commodity DRAM DIMMs to enhance the performance of a single consumer-grade GPU, achieving efficient LLM inference. We recognize that the inherent activation sparsity in LLMs naturally divides weight parameters into two categories, termed “hot” and “cold” neurons, respectively. Hot neurons, which consist of only approximately 20% of all weight parameters, account for 80% of the total computational load. In contrast, cold neurons make up the other 80% of parameters but are responsible for just 20% of the computational workload. Leveraging this observation, we propose a heterogeneous computing strategy: mapping hot neurons to a single computation-efficient GPU without large-capacity HBMs, while offloading cold neurons to NDP-DIMMs, which offer large memory size but limited computation capabilities. In addition, the dynamic nature of activation sparsity necessitates a real-time partition of hot and cold neurons and adaptive remapping of cold neurons across multiple NDP-DIMM modules. To tackle these issues, we introduce a lightweight predictor that ensures optimal real-time neuron partition and adjustment between GPU and NDP-DIMMs. Furthermore, we utilize a window-based online scheduling mechanism to maintain load balance among multiple NDP-DIMM modules. In summary, Hermes facilitates the deployment of LLaMA2-70B on consumer-grade hardware at a rate of 13.75 tokens/s and realizes an average 75.24 × speedup over the state-of-the-art offloading-based inference system on popular LLMs.

Near DRAM Processing (NDP) architectures have emerged to be a promising solution for commercializing in-memory computing and addressing the “memory wall” problem, especially for the memory-intensive machine learning (ML) workloads. In NDP architectures, the Processing Units (PUs) are distributed next to different memory units to exploit the high internal bandwidth. Therefore, in order to fully utilize the bandwidth advantage of NDP architectures for ML applications, meticulous evaluations and optimizations of data placement in DRAM and workload scheduling among different PUs are required. However, existing simulation and compilation tools face two insuperable obstacles to achieving these targets. On the one hand, tools for traditional von Neumann architectures only focus on the data access behaviors between the host and DRAM and treat DRAM as a whole part, which cannot support NDP architectures with multiple independent processing and memory units working simultaneously. On the other hand, existing NDP simulators and compilers are designed for specific DRAM technology and NDP architecture, lacking compatibility for various NDP architectures. In order to overcome these challenges and optimize data mapping and workload scheduling for different NDP architectures, we propose UniNDP, a unified NDP compilation and simulation tool for ML applications. Firstly, we propose a unified tree-based NDP hardware abstraction and the corresponding instruction set, enabling the support for various NDP architectures based on different DRAM technologies. Secondly, we design a cycle-accurate and instruction-driven NDP simulator to evaluate hardware performance by accurately tracking the working status of memory elements and PUs. The accurate simulation can provide effective guidance for compilation. Thirdly, we design an NDP compiler that optimizes data partition, mapping, and workload scheduling in different DRAM hierarchies. Furthermore, to enhance the compilation efficiency, we propose a hardware status-guided search space pruning strategy and a fast performance predictor using DRAM timing parameters. Extensive experimental results show that, compared to existing mapping and compilation methods, UniNDP can achieve 1.05-3.43 $\times$ speedup across multiple NDP architectures and different ML workloads. Furthermore, based on the results of UniNDP, we provide insights for the future NDP architecture design and deployment in ML applications.

Layout Synthesis for Quantum Computing (LSQC) is a critical component of quantum design tools. Traditional LSQC studies primarily focus on optimizing for reduced circuit depth by adopting a device-centric design methodology. However, these approaches overlook the impact of classical processing and communication time, thereby being insufficient for Dynamic Quantum Circuits (DQC).To address this, we introduce CLASS, a controller-centric layout synthesizer designed to reduce inter-controller communication latency in a distributed control system. It consists of a two-stage framework featuring a hypergraph-based modeling and a heuristic-based graph partitioning algorithm. Evaluations demonstrate that CLASS effectively reduces communication latency by up to 100% with only a 2.10% average increase in the number of additional operations.

Unstructured pruning introduces significant sparsity in deep neural networks (DNNs), enhancing accelerator hardware efficiency. However, three critical challenges constrain performance gains: 1) complex fetching logic for nonzero (NZ) data pairs; 2) load imbalance across processing elements (PEs); and 3) PE stalls from write-back contention. This brief proposes an energy-efficient accelerator addressing these inefficiencies through three innovations. First, we propose a Cartesian-product output-row-stationary (CPORS) dataflow that inherently matches NZ data pairs by sequentially fetching compressed data. Second, a multilevel partial sum reduction (MLPR) strategy minimizes write-back traffic and converts random PE stalls into manageable load imbalance. Third, a kernel sorting and load scheduling (KSLS) mechanism resolves PE idle/stall and achieves PE array-level load balancing, attaining 76.6% average PE utilization across all sparsity levels. Implemented in 22-nmCMOS, the accelerator delivers$1.85\times $speedup and$1.4\times $energy efficiency over baseline and achieves 25.8 TOPS/W peak energy efficiency at 90% sparsity.

In recent years, various computing-in-memory (CIM) processors have been presented, showing superior performance over traditional architectures. To unleash the potential of various CIM architectures, such as device precision, crossbar size, and crossbar number, it is necessary to develop compilation tools that are fully aware of the CIM architectural details and implementation diversity. However, due to the lack of architectural support in current popular open-source compiling stacks such as TVM, existing CIM designs either manually deploy networks or build their own compilers, which is time-consuming and labor-intensive. Although some works expose the specific CIM device programming interfaces to compilers, they are often bound to a fixed CIM architecture, lacking the flexibility to support the CIM architectures with different computing granularity. On the other hand, existing compilation works usually consider the scheduling of limited operation types (such as crossbar-bound matrix-vector multiplication). Unlike conventional processors, CIM accelerators are featured by their diverse architecture, circuit, and device, which cannot be simply abstracted by a single level if we seek to fully explore the advantages brought by CIM.

Fully Homomorphic Encryption (FHE) is one of the most promising privacy-preserving techniques that has drawn increasing attention from academia and industry due to its ideal security. Chiplet-based designs integrate multiple dies into the package delivering high performance and thereby are embraced by the resources-hungry FHE. Despite the chiplet-based system with various specialized accelerators, it falls short in supporting FHE with the novel polynomial operations. For a chiplet-based system that is not tailored for FHE, one common approach to support FHE is designing a new dedicated accelerator, However, this full design-and-build approach overlooks the existing abundant resources of accelerators in the system and incurs repeated customization and resource waste.

Dynamic neural network is an emerging research topic in deep learning. Dynamic networks selectively skip redundant computations conditioned on the input during inference (i.e., dynamic inference). And they have demonstrated superior trade-offs between accuracy and inference efficiency. However, memory I/O turns irregular and dominant because of the fine-grained computation skip in dynamic networks. Processing-In-Memory (PIM) can perform Matrix-Vector Multiplications inside the memory, eliminating the data movement of network parameters. So, it is promising to address the memory I/O challenge. However, deploying dynamic networks on PIM architectures faces severe performance degradation caused by (1) Pipeline stall when deciding on computation to be skipped. (2) Mismatch between fine-grained algorithm computation skip and coarse-grained hardware computing granularity. (3) Improper proxy of hardware performance during training. To tackle these problems, we propose DyPIM, the dynamic inference-enabled PIM accelerator with software-hardware co-optimizations. At the algorithm level, a PIM-friendly dynamic network with a standalone mask generation network and a throughput-optimal training technique is proposed. At the hardware level, a PIM architecture supporting dynamic networks is proposed, with a pipeline controller to process the dynamic dataflow. Peripheral circuits are also designed in processing units to enable non-contiguous activating of non-zero wordlines to better utilize the computation skip. Experiments show that DyPIM can achieve 1.52x to 2.74x speedup and 2.05x to 3.95x throughput improvement over the existing PIM architectures for Res Net networks.

ReRAM-based Computing-in-Memory (CiM) architecture has been considered a promising solution to high-efficiency neural network accelerator, by conducting in-situ matrix multiplications and eliminating the movement of neural parameters from off-chip memory to computing units. However, we observed specific features of Graph Convolutional Network (GCN) tasks pose design challenges to implement a high-efficiency ReRAM GCN accelerator. The ultra-large input feature data in some GCN tasks incur massive data movements, the extremely sparse adjacency matrix and input feature data involve the valid computation, and the super-large adjacency matrix that exceeds available ReRAM capacity causes frequent expensive write operations. To address the above challenges, we propose TARe, a Task-Adaptive CiM architecture, which consists of a hybrid in-situ computing mode to support the input feature in crossbar computing, a compact mapping scheme for efficient sparse matrix computing, and a write-free mapping to eliminate write activities in the computations with the super-large adjacency matrix. Additionally, TARe is facilitated with a task adaptive selection algorithm to generate optimized design schemes for graph neural network tasks that have various operand sizes and data sparsity. We evaluate TARe on 11 diverse graph neural network tasks and compare it with different design counterparts, and the results show that achieves 168.06× speedup and 10.95× energy consumption reduction on average over the baseline in common graph convolutional network workloads.

Graph-based approximate nearest neighbor search (ANNS) algorithms achieve the best accuracy for fast high-recall searches on billion-scale datasets. Because of the irregular and large-volume data access, existing CPU-based systems suffer from heavy data movements when dealing with graph-based ANNS algorithms. Near-memory-computing (NMC) architectures have demonstrated great potential in boosting the performance of big-data processing. However, existing NMC architectures face two serious problems when processing graph-based ANNS algorithms: (1) the memory capacity of main memory level NMC (e.g., 64GB) cannot meet the storage requirement of ANNS on billion-scale datasets (e.g., 800GB), resulting in heavy data transfers between main memory and storage; (2) the contradiction between the irregular and fine-grained graph access and the page-level read granularity hinder the throughput of storage level NMC.This paper proposes Pyramid, the processing-in-hierarchical-memory architecture for graph-based ANNS on billion-scale datasets. Pyramid combines the internal bandwidth benefits of main memory level NMC with the capacity benefits of storage level NMC. A hierarchical graph-cluster-based ANNS is also proposed for Pyramid. It transforms the irregular data access on large-scale graphs into the irregular access on small-scale graphs at the main memory level and regular sequential in-cluster access at the storage level. Experimental results show that with the same recall of 0.9, Pyramid improves the throughput by 21.1~72.8× and 26.0~50.7× compared with existing CPU/GPU-based ANNS systems on million-scale and billion-scale datasets, respectively.

Deep convolutional neural networks (CNNs) are widely used in Artificial Intelligence of Things (AIoT) systems. Limited by power and area, conventional edge devices are insufficient to handle the cost of CNN computation. The idea of SRAM based Processing-In-Memory (SRAM-PIM) has been advocated to implement CNN on edge devices because of its high area and power efficiency. To further excavate the potential of SRAM-PIM on edge inferences, this paper proposes an SRAM-PIM design with Kullback-Leibler (KL) divergence-based dynamic precision quantization. The proposed quantization method decouples the effect of different CNN layers on accuracy and introduces the SRAM-PIM hardware performance in quantization, realizing SRAM-PIM-aware layer-wise precision adjustment. The proposed SRAM-PIM design has been applied in image classification tasks on edge devices. Our evaluation shows that the implemented design achieves up to 2.03x energy efficiency improvement and 2.54% accuracy improvement compared with existing dynamic precision PIM design. Compared with existing reinforcement-learning-based dynamic quantization method that requires several hours quantization time, the proposed dynamic precision quantization method takes only 26.28us to get the optimal quantization results.

Quantum circuit simulation (QCS) is essential in the noisy intermediate scale quantum (NISQ) era when real quantum computers are scarce. However, fully tracking the states of a quantum system in QCS is highly challenging due to the exponential memory growth that significantly limits the computational reach of classical systems for QCS. Though it is straightforward to leverage secondary storage to extend the scale of QCS, excessive data movement between memory and storage dominates the simulation time, making this solution unrealistic. To tackle this challenge, we identify an intrinsic property of QCS and implement an open-source framework to effectively reduce data movement by >116x. We evaluate the framework on various benchmarks and demonstrate 4x memory reduction with only <20% overhead. On a memory constrained system, we show that it extends the scale of QCS to 32 qubits (64 GB memory requirement) while existing simulators are bounded to 28 qubits (4 GB memory requirement). Our implementation can be accessed via https://github.com/Zhaoyilunnn/qdao.

The RRAM-based neuromorphic computing system (NCS) has amassed explosive interests for its superior data processing capability and energy efficiency than traditional architectures, and thus being widely used in many data-centric applications. The reliability and security issues of the NCS, therefore, become an essential problem. In this article, we systematically investigated the adversarial threats to the RRAM-based NCS and observed that the RRAM hardware feature can be leveraged to strengthen the attack effect, which has not been granted sufficient attention by previous algorithmic attack methods. Thus, we proposed two types of hardware-aware attack methods with respect to different attack scenarios and objectives. The first is an adversarial attack, VADER, which perturbs the input samples to mislead the prediction of neural networks. The second is fault injection attack, EFI, which perturbs the network parameter space such that a specified sample will be classified to a target label, while maintaining the prediction accuracy on other samples. Both attack methods leverage the RRAM properties to improve the performance compared with the conventional attack methods. Experimental results show that our hardware-aware attack methods can achieve nearly 100% attack success rate with extremely low operational cost, while maintaining the attack stealthiness.

Network sparsity or pruning is a pivotal technology for edge intelligence. Resistive random access memory (RRAM)-based accelerators, featuring dense storage and processing in memory capability, have demonstrated the superior computing performance and energy efficiency over the traditional CMOS-based accelerators for neural network applications. Unfortunately, RRAM-based accelerators suffer the performance or energy degradation when deploying pruned models, impairing their competition in the edge intelligence scenarios. We observed the essential reason is the pruning technology and the mapping strategy in prior RRAM-based accelerator and are optimized individually. As a result, the random zeros in the pruned deep neural network are irregularly distributed in the crossbars, rendering the degradation of computation parallelism of the crossbar without crossbar demand reduction. In this work, we propose a coordinated model pruning and mapping framework to jointly optimize of model accuracy and efficiency of RRAM-based accelerators. As for the mapping, we first decouple weight matrices in the bit-wise way and map the bit matrices to different crossbars, where the signed weights are represented with the two’s complement so as that save half desired crossbars. As for the pruning, we prune weight bits at the crossbar granularity so that free the crossbars holding the pruned bits. Furthermore, we employ an reinforcement learning (RL) approach to automatically select the optimal crossbar-aware bit-pruning strategy for any given neural network without laborious human efforts. We conducted the experiments on a set of representative neural networks and compared our framework with the state-of-the-art (SOTA) bit-sparsity works. The results show that automatic structured bit-pruning saves up to 89.64% energy reduction and 84.12% area overhead compared to existing PRIME-like architecture. Besides, our framework outperforms the SOTA bit-sparsity design by$1.5\times $in terms of the energy reduction on the RRAM-based accelerator.

ReRAM-based accelerators have shown great potential in neural network acceleration via in-memory analog computing. However, high-precision analog-to-digital converters (ADCs), which are required by the ReRAM crossbars to achieve high-accuracy network model inference, play an essential role in the energy-efficiency of the accelerators. Based on the discovery that the ADC precision requirements of crossbars are different, we propose the model-aware crossbarwise ADC precision assignment and the accompanied information-lossless low-bit ADCs to reduce energy overhead without sacrificing model accuracy. In experiments, the proposed information-lossless ReRAM accelerator, InfoX, only consumes 8.97% ADC energy of the SOTA baseline with no accuracy degradation at all.

While a tremendous amount of effort has been devoted to localization, the orientation of a device, especially in 3D space, is seldom explored. Although many sensor-based methods utilizing gyro-scope, accelerometer, and magnetometer have been proposed to measure 3D orientation, these methods generally suffer from high cumulative errors and performance degradation when the device is moving. In this paper, we present MOM, the first microphone-based system that estimates the 3D orientation of a device. The key idea of MOM is to employ free sound sources in our surrounding environment as anchors. The prior knowledge of these sound sources, including the signal waveform and the locations of the sound sources, is not required to be known. In particular, we propose an angle-of-arrival (AoA) extraction algorithm that compares fine-grained time delays over microphones at a low computational cost. We implement our system on three platforms including a 6-microphone array Seeed Studio ReSpeaker, a commodity earphone Sennheiser AMBEO smart headset and a commodity smartphone Google Pixel 4. Extensive experiments show that MOM can achieve significantly higher accuracy compared with status quo approaches and is robust against cumulative errors. We apply MOM to two real-life applications, i.e., head tracking and 3D reconstruction, to demonstrate the applicability and generality of MOM in practice.

With the advancement of deep neural networks (DNNs), the applications driven by DNNs have been spread from the cloud to the edge. However, the intensive computations and data movements in CNNs impede the adoption of DNNs in resource-constraint edge devices.Quantization, a common model compression method, has attracted a lot of attention as it enables efficient inference by lowering the data bit-width of CNN parameters. Due to the features of massive storage and computing-in-memory array, resistive memory (RRAM) has established the energy efficiency and small area processing-in-memory (PIM) for the acceleration of DNNs at the edge end. However, when deploying the network onto resistive-memory-based PIM (RRAM-based PIM), there will be tremendous unused cells due to the mismatch between the structure of the neural network layer and memory array, resulting in the resource under-utilization and low computation efficiency. In this work, we observed prior quantization approaches fail to improve hardware resource utilization as they ignored the hardware structure information in RRAM. Thus, combining the information of the neural network model and hardware information is essential for a high-utilization RRAM-based PIM design. Considering the vast model parameters and heterogeneous RRAM crossbar structure, we develop a novel quantization framework by leveraging the AutoML technique, i.e., RaQu, which automatically generates a fine-grained quantization strategy for any model that fully utilizes the resource of RRAM-based PIM. The experimental results show that RaQu achieves at most 29.2%–37.4% and 1.8%–3.3% improvement in resource utilization and model accuracy, respectively, compared to prior coarse-grained quantization methods.

Adversarial attacks have been viewed as the primary threat to the security of neural networks. Hence, extensive adversarial defense techniques have been proposed to protect the neural networks from adversarial attacks, allowing for the application of neural networks to the security-sensitive tasks. Recently, the emerging devices, e.g., Resistive RAM (RRAM), attracted extensive attention for establishing the hardware platform for neural networks to tackle the inadequate computing capability of the traditional computing platform. Though the emerging devices exhibit the instinct instability issues due to the advanced manufacture technology, including hardware variations and defects, the error-resilience capability of neural networks enables the wide deployment of neural networks on the emerging devices. In this work, we find that the natural instability in emerging devices impairs the security of neural networks. Specifically, we design an enhanced adversarial attack, Variation-oriented ADvERsarial (VADER) attack which leverages the inherent hardware variations in RRAM chips to penetrate the protection of adversarial defenses and mislead the prediction of neural networks. We evaluated the effectiveness of VADER across various protected neural network models and the result shows that VADER achieves higher success attack rate over other adversarial attacks.

Network sparsity or pruning is an extensively studied method to optimize the computation efficiency of deep neural networks (DNNs) for CMOS-based accelerators, such as FPGAs and GPUs. Though the RRAM-based accelerator has demonstrated superior performance and energy efficiency for DNN tasks, deploying the sparse neural networks desires dedicated consideration to save resource consumption without introducing the expensive index overhead and sophisticated control. To exploit the potential of sparse neural network design on the RRAM-based accelerator, we propose an automatic structured bit-pruning design, ASBP, to harmonize the optimization objective of DNN sparsity with efficient RRAM deployment. Specifically, ASBP prunes the bits of weight which are split into different crossbars and thus, free the zero-value crossbar when mapping the neural network into RRAM-based accelerators without extra hardware modification. Meanwhile, ASBP employs the reinforcement learning (RL) approach to automatically select the best crossbar-aware bit-sparsity strategy for any given neural network without laborious human efforts. According to our experiments on a set of representative neural networks, ASBP saves up to 79.01% energy consumption and 54.79% area overhead compared to the baseline that deploys the original DNN on the RRAM-based accelerator. Besides, ASBP outperforms the state-of-the-art bit-sparsity design by 1.4x in terms of the energy reduction on the RRAM-based accelerator.

As neural networks (NNs) are being widely deployed in many cloud-oriented systems for safety-critical tasks, the privacy and security of NNs become significant concerns to users in the cloud platform that shares the computation infrastructure such as memory resource. In this work, we observed that the memory timing channel in the shared memory of cloud multi-core architecture poses the risk of network model information leakage. Based on the observation, we propose a learning-based method to steal the model architecture of the NNs by exploiting the memory timing channel without any high-level privilege or physical access. We first trained an end-to-end measurement network offline to learn the relation between memory timing information and NNs model architecture. Then, we performed an online attack and reconstructed the target model using the prediction from the measurement network. We evaluated the proposed attack method on a multi-core architecture simulator. The experimental results show that our learning-based attack method can reconstruct the target model with high accuracy and improve the adversarial attack success rate by 42.4%.

Multi-task learning systems are commonly adopted in many real-world AI applications such as intelligent robots and self-driving vehicles. Instead of improving single-network performance, this work proposes a specialized Multi-Task Deep Learning Accelerator architecture, MT-DLA, to improve the performance of concurrent networks by exploiting the shared feature and parameters across these models. It is shown in our evaluation with realistic multi-task workloads, MT-DLA dramatically eliminates the memory and computation overhead caused by the shared parameters, activations and computation result. In the experiments with real-world multi-task learning workloads, MT-DLA brings about 1.4x-7.0x energy efficiency boost when compared to the baseline neural network accelerator without multi-task support.

The era of 5G extends the available spectrum from the microwave band to the millimeter-wave band. The thriving Internet of Things (IoT) also enriches the user equipment (UEs) we used in our daily life, such as smart glasses, smart watches, and drones. With such a larger spectrum and massive UEs, existing dynamic spectrum access (DSA) suffers both low spectrum utilization efficiency and unfair spectrum allocation. Thus, a more sophisticated dynamic spectrum access (DSA) system is required in the 5G context. In this paper, we propose a federated learning based system, FedSwap, the first decentralized DSA system that improves both efficiency and fairness simultaneously. In FedSwap, we deploy an improved multi-agent reinforcement learning (iMARL) algorithm on each UE, enabling UEs to share the spectrum coordinately with fewer collisions. Furthermore, we also propose a novel swapping mechanism for aggregating UEs' models periodically so that UEs can fairly share the spectrum resources. Meanwhile, the sensory data of UEs are not transmitted and hence privacy is protected. We evaluate FedSwap's performance in 5G simulations with various settings. Compared to the state-of-the-art decentralized DSA methods, FedSwap can significantly improve the efficiency and fairness of spectrum utilization.

Convolutional neural networks (CNNs) have become the state-of-the-art technique in many classification tasks in IoT system. However, the low-power and area-constraint edge devices are unable to afford the expensive cost of CNNs. Resistive random access memory (RRAM) is attractive for establishing the CNN accelerator at the edge end due to the features of scalability, low-power and in-situ dot-product. However, mapping a random network architecture onto a general-purpose RRAM accelerator suffers a severe issue of resource underutilization. The neural network quantization offers an opportunity to rescue the degraded resource utilization. Selecting the bit-width for the vast parameters is impractically completed by human labor. This paper proposes an AutoML-based array-aware quantization and mapping framework that generates the fine-grained mixed-precision neural networks to optimize resource utilization in RRAM. In this framework, we design a two-stage learning and array-aware grouping strategy to quickly explore the huge searching space. The experimental results show that the proposed framework achieves 18.2%~36.1% improvement in resource utilization and 0.9%~3.3% increase in model accuracy over prior coarse-grained quantization methods.

With the rapid progress of artificial intelligence (AI) algorithms, multi-modal deep neural networks (DNNs) have been applied to some challenging tasks, e.g., image and video description to process multi-modal information from vision and language. Resistive-memory-based processing-in-memory (ReRAM-based PIM) has been extensively studied to accelerate either convolutional neural network (CNN) or recurrent neural network (RNN). According to the requirements of their core layers, i.e. convolutional layers and linear layers, the existing ReRAM-based PIMs adopt different optimization schemes for them. Directly deploying multi-modal DNNs on the existing ReRAM-based PIMs, however, is inefficient because multi-modal DNNs have combined CNN and RNN where the primary layers differ depending on the specific tasks. Therefore, a high-efficiency ReRAM-based PIM design for multi-modal DNNs necessitates an adaptive optimization to the given network. In this work, we propose HitM, a high-throughput ReRAM-based PIM for multi-modal DNNs with a two-stage workflow, which consists of a static analysis and an adaptive optimization. The static analysis generates the layer-wise resource and computation information with the input multi-modal DNN description and the adaptive optimization produces a high-throughput ReRAM-based PIM design through the dynamic algorithm based on hardware resources and the information from the static analysis. We evaluated HitM using several popular multi-modal DNNs with different parameters and structures and compared it with a naïve ReRAM-based PIM design and an optimal-throughput ReRAM-based PIM design that assumes no hardware resource limitations. The experimental results show that HitM averagely achieves 78.01% of the optimal throughput while consumes 64.52% of the total hardware resources.

Deep Reinforcement Learning (DRL) is substantially resource-consuming, and it requires large-scale distributed computing-nodes to learn complicated tasks, like videogame and Go play. This work attempts to down-scale a distributed DRL system into a specialized many-core chip and achieve energy-efficient on-chip DRL. With the customized Network-on-Chip that handles the communication of on-chip data and control-signals, we proposed a Synchronous Asynchronous RL Architecture (SARLA) and the according many-core chip that completely avoids the unnecessary data duplication and synchronization activities in multi-node RL systems. In evaluation, the SARLA system achieves considerable energy-efficiency boost over the GPU-based implementations for typical DRL workloads built with OpenAI-gym.

Deep neural network (DNN) models are being expanded to a broader range of applications. The computational capability of traditional hardware platforms cannot accommodate the growth of model complexity. Among recent technologies to accelerate DNN, resistive memory (ReRAM)-based processing-in-memory (PIM) emerged as a promising solution for DNN inference due to its high efficiency for matrix-based computation. We face two major technical challenges in extending the use of ReRAM-based accelerators for training: (1) full-precision data is essential in back-propagation; (2) the need to support both feed-forward and back-propagation aggravates the data-movement burden. We propose a heterogeneous architecture named as 3D-ReG, which leverages full-precision GPU to ensure training accuracy and low-overhead 3D integration to provide low-cost data movements. Moreover, we introduce conservative and aggressive task-mapping schemes, which partition the computation phases in different ways to balance execution efficiency and training accuracy. We evaluate 3D-ReG implemented with two 3D integration technologies, through-silicon vias (TSVs) and monolithic inter-tier vias (MIVs), and compare them with GPU-only and PIM-only counterparts. Various GPU-only platforms using two main-memory technologies (DRAM, ReRAM) and three interconnect technologies (2D, TSV, MIV) are evaluated as well. Experimental results show that 3D-ReG can achieve on average 5.64× training speedup and 3.56× higher energy efficiency compared with the GPU with DRAM as main memory, at the cost of 0.05%–3.39% accuracy drop. We define a new metric, gain-loss ratio (GLR), which quantitatively evaluates the capability of a DNN training hardware in terms of the model accuracy and hardware efficiency. The results of our comparison show that the aggressive task-mapping scheme on MIV-based 3D-ReG outperforms the other methods.

Deconvolution is a key component in contemporary neural networks, especially, generative adversarial networks (GANs) and fully convolutional networks (FCNs). Due to extra operations of deconvolution compared to convolution, considerable degradation of performance, as well as energy efficiency is incurred when implementing deconvolution on the existing resistive random access memory (ReRAM)-based processing-in-memory (PIM) accelerators. In this article, we propose an ReRAM-based accelerator design, RED, for providing high-performance and low-energy deconvolution. We analyze the deconvolution execution on the existing ReRAM-based PIMs and utilize its interior computation pattern for design optimization. RED includes two major contributions: 1) pixel-wise mapping scheme and 2) zero-skipping data flow. Pixel-wise mapping scheme removes the zero insertion and performs convolutions over several ReRAM arrays and thus enables parallel computations with nonzero inputs. Zero-skipping data flow, assisted with customized input buffers design, enhances the computation parallelism and input data reuse. In evaluation, we compare RED against the existing ReRAM-based PIMs and CMOS-based counterpart with a variety of GAN and FCN models, each of which contains multiple deconvolution layers. The experimental results show that RED achieves a$4.0\times $–$56.16\times $speedup and a$1.05\times $–$18.17\times $energy efficiency improvement over previous related accelerator designs.

Neuromorphic computing is a revolutionary approach of computation, which attempts to mimic the human brain's mechanism for extremely high implementation efficiency and intelligence. Latest research studies showed that the memristor technology has a great potential for realizing power- and area-efficient neuromorphic computing systems (NCS). On the other hand, the memristor device processing is still under development. Unreliable devices can severely degrade system performance, which arises as one of the major challenges in developing memristor-based NCS. In this paper, we first review the impacts of the limited reliability of memristor devices and summarize the recent research progress in building reliable and efficient memristor-based NCS. In the end, we discuss the main difficulties and the trend in memristor-based NCS development.

Deconvolution has been widespread in neural networks. For example, it is essential for performing unsupervised learning in generative adversarial networks or constructing fully convolutional networks for semantic segmentation. Resistive RAM (ReRAM)-based processing-in-memory architecture has been widely explored in accelerating convolutional computation and demonstrates good performance. Performing deconvolution on existing ReRAM-based accelerator designs, however, suffers from long latency and high energy consumption because deconvolutional computation includes not only convolution but also extra add-on operations. To realize the more efficient execution for deconvolution, we analyze its computation requirement and propose a ReRAM-based accelerator design, namely, RED. More specific, RED integrates two orthogonal methods, the pixel-wise mapping scheme for reducing redundancy caused by zero-inserting operations and the zero-skipping data flow for increasing the computation parallelism and therefore improving performance. Experimental evaluations show that compared to the state-of- the-art ReRAM-based accelerator, RED can speed up operation 3.69~31.15× and reduce 8%~88.36% energy consumption.

The growing popularity of Convolutional Neural Networks (CNNs) has led to the search for efficient computational platforms to enable these algorithms. Resistive random-access memory (ReRAM)-based architectures offer a promising alternative to commonly used GPU-based platforms for CNN training. However, backpropagation in CNNs is susceptible to the limited precision of ReRAMs. As a result, training CNNs on ReRAMs affects the final accuracy of learned model. In this work, we propose REGENT, a heterogeneous architecture that combines ReRAM arrays with GPU cores, and exploits the benefits provided by 3D integration along with a high-throughput yet energy efficient Network-on-Chip (NoC) for training CNNs. We also propose a bin-packing based framework that maps CNN layers and then optimize the placement of computing elements to meet the targeted design objectives. Experimental evaluations indicate that REGENT improves full-system EDP by 55.7% on average compared to conventional GPU-only platforms for training CNNs.

The conventional von Neumann architecture has been revealed as a major performance and energy bottleneck for rising data-intensive applications. The decade-old idea of leveraging in-memory processing to eliminate substantial data movements has returned and led extensive research activities. The effectiveness of in-memory processing heavily relies on memory scalability, which cannot be satisfied by traditional memory technologies. Emerging non-volatile memories (eNVMs) that pose appealing qualities such as excellent scaling and low energy consumption, on the other hand, have been heavily investigated and explored for realizing in-memory processing architecture. In this paper, we summarize the recent research progress in eNVM-based in-memory processing from various aspects, including the adopted memory technologies, locations of the in-memory processing in the system, supported arithmetics, as well as applied applications.

Time Dependent Variability (TDV) is one of the major concerns in implementing a Neuromorphic Computing System (NCS) with Resistive Random Access Memory (ReRAM). In this work, we propose a variation-distribution aware training algorithm to enhance the robustness of NCS to TDV without incurring extra hardware overhead by leveraging algorithm-level regularization and hardware-level 2R synapse structure. Simulation results on image recognition tasks show that our method improves the system accuracy by up to ∼4% and ∼10% under the worst-case TDV condition for MNIST and CIFAR-10, respectively. Detailed analysis also shows that our method allows the NCS to use synapses with higher resistance than conventional design for the same accuracy requirement, introducing potential energy saving.

Massive multi-threading in GPU imposes tremendous pressure on memory subsystems. Due to rapid growth in thread-level parallelism of GPU and slowly improved peak memory bandwidth, memory becomes a bottleneck of GPU’s performance and energy efficiency. In this article, we propose an integrated architectural scheme to optimize the memory accesses and therefore boost the performance and energy efficiency of GPU. First, we propose a thread batch enabled memory partitioning (TEMP) to improve GPU memory access parallelism. In particular, TEMP groups multiple thread blocks that share the same set of pages into a thread batch and applies a page coloring mechanism to bound each stream multiprocessor (SM) to the dedicated memory banks. After that, TEMP dispatches the thread batch to an SM to ensure high-parallel memory-access streaming from the different thread blocks. Second, a thread batch-aware scheduling (TBAS) scheme is introduced to improve the GPU memory access locality and to reduce the contention on memory controllers and interconnection networks. Experimental results show that the integration of TEMP and TBAS can achieve up to 10.3% performance improvement and 11.3% DRAM energy reduction across diverse GPU applications. We also evaluate the performance interference of the mixed CPU+GPU workloads when they are run on a heterogeneous system that employs our proposed schemes. Our results show that a simple solution can effectively ensure the efficient execution of both GPU and CPU applications.

Deep Neural Networks (DNNs) are pervasively applied in many artificial intelligence (AI) applications. The high performance of DNNs comes at the cost of larger size and higher compute complexity. Recent studies show that DNNs have much redundancy, such as the zero-value parameters and excessive numerical precision. To reduce computing complexity, many redundancy reduction techniques have been proposed, including pruning and data quantization. In this paper, we demonstrate our co-optimization of the DNN algorithm and hardware which exploits the model redundancy to accelerate DNNs.

Big data computing applications such as deep learning and graph analytic usually incur a large amount of data movements. Deploying such applications on conventional von Neumann architecture that separates the processing units and memory components likely leads to performance bottleneck due to the limited memory bandwidth. A common approach is to develop architecture and memory co-design methodologies to overcome the challenge. Our research follows the same strategy by leveraging resistive memory (ReRAM) to further enhance the performance and energy efficiency. Specifically, we employ the general principles behind processing-in-memory to design efficient ReRAM based accelerators that support both testing and training operations. Related circuit and architecture optimization will be discussed too.

Benefiting from its ultra-high storage density, high energy efficiency, and non-volatility, racetrack memory demonstrates great potential in replacing conventional SRAM as large on-chip memory. Integrating the tape-like racetrack memory, however, faces unique design challenges from cell structure to architecture design. This paper reviews some cross-layer design methodologies for racetrack memory as on-chip cache hierarchy. Research studies show that with proper architectural design and data management, racetrack memory can achieve significant area reduction, system performance enhancement, and energy saving compared to state-of-the-art memory technologies.

Phase change memory (PCM) is a promising alternative to Dynamic Random Access Memory (DRAM) as main memory due to its merits of high density and low leakage power. Multi-level Cell (MLC) PCM is more attractive than Single-level Cell (SLC) PCM, because it can store multiple bits per cell to achieve higher density and lower per-bit cost. With the iterative program-verify write technique, MLC PCM writes demand at much higher power than DRAM writes, while the power supply system of MLC memory system is similar to that of DRAM, and the power capability is limited. The incompatibility of high write power and limited power budget results in the degradation of the write throughput and performance in MLC PCM. In this work, we investigate both write scheduling policy and power management to improve the MLC power utility and alleviate the negative impacts induced by high write power. We identify the power-utility-driven write scheduling as an online bin-packing problem and then derive a power-utility-driven scheduling (PUDS) policy from the First Fit algorithm to improve the write power usage. Based on the ramp-down characteristic of the SET pulse (the pulse changes the PCM to high resistance), we propose the SET Power Amortization (SPA) policy, which proactively reclaims the power tokens at the intra-SET level to promote the power utilization. Our experimental results demonstrate that the PUDS and SPA respectively achieve 24% and 27% performance improvement over the state-of-the-art power management technique, and the PUDS8SPA has an overall 31% improvement of the power utility and 50% increase of performance compared to the baseline system.

Phase change memory (PCM) is a promising nonvolatile memory technology developed as a possible DRAM replacement. Although it offers the read latency close to that of DRAM, PCM generally suffers from the long write latency. Long write request may block the read requests on the critical path of cache/memory access, incurring adverse impact on the system performance. Besides, the write performance of PCM is very asymmetric, i.e, the SET operation (writing `1') is much slower than that of the RESET operation (writing `0'). In this work, we re-examine the resistance transform process during the SET operation of PCM and propose a novel Partial-SET scheme to alleviate the long write latency issue of PCM. During a write access to a memory line, a short Partial-SET pulse is applied first to program the PCM cells to a pre-stable state, achieving the same write latency as RESET. The partially-SET cells are then fully programmed within the retention window to preserve the data integrity. Experimental results show that our Partial-SET scheme can improve the memory access performance of PCM by more than 45% averagely with very marginal storage overhead.

Phase change memory (PCM) has emerged as a mostly promising non-volatile memory. Multi-level Cell (MLC) PCM that stores multiple bits in a single cell, has the benefits of increasing capacity and lower cost-per-bit. However, as feature size scales down, prior work reports that low frequency noise and random telegraph noise would greatly jeopardize the reliability of MLC PCM. In this paper, we firstly analyze the multi-bit error rate induced by noise and then propose a multi-bit ECC (Error Correction Code) to alleviate the deleterious noise effects in MLC PCM. As far as we know, this is the first paper to utilize of error correction method to mitigate the impact of noise at architectural level. However, a strong multi-bit ECC requires additional storage and latency. Thus, we propose a 6EC-7ED BCH scheme which achieves a tradeoff between correction capability and overhead. Compared to conventional DRAM ECC, this scheme effectively improves the reliability of MLC PCM system, while has the comparable storage overhead. Moreover, the experimental results show this scheme incurs negligible latency cost with merely 1% performance degradation.