Hybrid computing-in-memory (CIM) has recently gained significant attention due to its ability to leverage the strengths of both digital CIM (DCIM) and analog CIM (ACIM). The multibit fusion (MF) scheme enhances energy efficiency by fusing low-bit results, which typically require multiple read-out cycles, into a single-cycle read out. However, the relationship between hybrid INT-CIM circuit design and network performance based on the MF scheme has not yet been systematically explored. In addition, we investigate how different MF configurations affect the performance of various neural networks. To address this gap, we first propose a less-significant group quantization (LSGQ) model, which defines and explores the design space of hybrid INT-CIM. Second, we develop a FastFuse-Search (FFS) algorithm, which optimizes configurations for different networks to strike a better balance between model accuracy and energy efficiency. Based on the experimental results, some key considerations on hybrid CIM design are derived. FFS yields a$1.72\times $energy-efficiency boost with negligible accuracy loss. Finally, we fabricate a 28-nm hybrid INT-CIM test chip, achieving 59.74 TOPS/W and 0.96 TOPS/mm2, with performance metrics of 23.21 perplexity for GPT-2, 68.69% accuracy for ResNet18, and 80.53% accuracy for ViT.

Floating-point compute-in-memory (FP-CIM) is regarded as an attractive approach to enhancing the energy efficiency of complex neural networks. Digital domain compute mechanism has been widely utilized in CIM designs owing to its high robustness to PVT variations. However, the energy consumption of digital CIM is significantly influenced by the toggle rate of the compute-tree. This work proposes a toggle-rate-immune floating-point digital CIM (TRIFP-DCIM) design with 34.03% compute energy reduction on average. Combined with the TRIFP-DCIM design, a toggle-rate gathering method is employed in the neural network training/inference process with almost no accuracy loss. Experiment results show that the TRIFP-DCIM can achieve 14.51--36.83 TFLOPS/W @BF16 in 28nm technology process.

SRAM Processing-in-Memory (PIM) has emerged as the most promising implementation for high-performance PIM, delivering superior computing density, energy efficiency, and computational precision.However, the pursuit of higher performance necessitates more complex circuit designs and increased operating frequencies, which exacerbate IR-drop issues.Severe IR-drop can significantly degrade chip performance and even threaten reliability.Conventional circuit-level IR-drop mitigation methods, such as back-end optimizations, are resource-intensive and often compromise power, performance, and area (PPA).To address these challenges, we propose AIM, comprehensive software and hardware co-design for architecture-level IR-drop mitigation in high-performance PIM.Initially, leveraging the bit-serial and in-situ dataflow processing properties of PIM, we introduce R tog and HR, which establish a direct correlation between PIM workloads and IR-drop.Building on this foundation, we propose LHR and WDS, enabling extensive exploration of architecture-level IR-drop mitigation while maintaining computational accuracy through software optimization.Subsequently, we develop IR-Booster, a dynamic adjustment mechanism that integrates software-level HR information with hardwarebased IR-drop monitoring to adapt the V-f pairs of the PIM macro, achieving enhanced energy efficiency and performance.Finally, we propose the HR-aware task mapping method, bridging software and hardware designs to achieve optimal improvement.Post-layout simulation results on a 7nm 256-TOPS PIM chip demonstrate that AIM achieves up to 69.2% IR-drop mitigation, resulting in 2.29× energy efficiency improvement and 1.152× speedup.

Hybrid computing-in-memory (CIM) has gained growing interest in recent times due to its ability to combine the strengths of both digital CIM (DCIM) and analog CIM (ACIM). The Multi-Bit Fusion (MF) scheme enhances energy efficiency by fusing low-bit results, which would typically require multiple readout cycles, into a single-cycle readout. However, the relationship between hybrid INT-CIM circuit design and network performance based on the MF scheme has yet to be systematically explored. To fill this gap, a less-significant group quantization (LSGQ) model is proposed, defining and exploring the hybrid INT-CIM design space. Experimental results demonstrate up to 1.83x improvement in energy efficiency with minimal impact on performance. A 28nm hybrid INT-CIM test chip is fabricated, achieving 52.16 TOPS/W, 0.96 TOPS/mm2, with respective performance metrics of 48.58 perplexity for GPT-2, 75.51% accuracy for ResNet18, and 81.5% accuracy for ViT.

DNNs (Deep Neural Networks) have demonstrated great success in numerous applications such as image classification, speech recognition, video analysis, etc. However, DNNs are much more computation-intensive and memory-intensive than previous shallow models. Thus, it is challenging to deploy DNNs in both large-scale data centers and real-time embedded systems. Considering performance, flexibility, and energy efficiency, FPGA-based accelerator for DNNs is a promising solution. Unfortunately, conventional accelerator design flows make it difficult for FPGA developers to keep up with the fast pace of innovations in DNNs. To overcome this problem, we propose FP-DNN (Field Programmable DNN), an end-to-end framework that takes TensorFlow-described DNNs as input, and automatically generates the hardware implementations on FPGA boards with RTL-HLS hybrid templates. FP-DNN performs model inference of DNNs with our high-performance computation engine and carefully-designed communication optimization strategies. We implement CNNs, LSTM-RNNs, and Residual Nets with FPDNN, and experimental results show the great performance and flexibility provided by our proposed FP-DNN framework.

We introduce FxpNet, a framework to train deep convolutional neural networks with low bit-width arithmetics in both forward pass and backward pass. During training FxpNet further reduces the bit-width of stored parameters (also known as primal parameters) by adaptively updating their fixed-point formats. These primal parameters are usually represented in the full resolution of floating-point values in previous binarized and quantized neural networks. In FxpNet, during forward pass fixed-point primal weights and activations are first binarized before computation, while in backward pass all gradients are represented as low resolution fixed-point values and then accumulated to corresponding fixed-point primal parameters. To have highly efficient implementations in FPGAs, ASICs and other dedicated devices, FxpNet introduces Integer Batch Normalization (IBN) and Fixed-point ADAM (FxpADAM) methods to further reduce the required floating-point operations, which will save considerable power and chip area. The evaluation on CIFAR-10 dataset indicates the effectiveness that FxpNet with 12-bit primal parameters and 12-bit gradients achieves comparable prediction accuracy with state-of-the-art binarized and quantized neural networks.