Low bit-precisions and their bit-slice sparsity have recently been studied to accelerate general matrix-multiplications (GEMM) during large-scale deep neural network (DNN) inferences. While the conventional symmetric quantization facilitates low-resolution processing with bit-slice sparsity for both weight and activation, its accuracy loss caused by the activation’s asymmetric distributions cannot be acceptable, especially for largescale DNNs. In efforts to mitigate this accuracy loss, recent studies have actively utilized asymmetric quantization for activations without requiring additional operations. However, the cuttingedge asymmetric quantization produces numerous nonzero slices that cannot be compressed and skipped by recent bit-slice GEMM accelerators, naturally consuming more processing energy to handle the quantized DNN models.To simultaneously achieve high accuracy and hardware efficiency for large-scale DNN inferences, this paper proposes an Asymmetrically-Quantized bit-Slice GEMM (AQS-GEMM) for the first time. In contrast to the previous bit-slice computing, which only skips operations of zero slices, the AQS-GEMM compresses frequent nonzero slices, generated by asymmetric quantization, and skips their operations. To increase the slicelevel sparsity of activations, we also introduce two algorithm-hardware co-optimization methods: a zero-point manipulation and a distribution-based bit-slicing. To support the proposed AQS-GEMM and optimizations at the hardware-level, we newly introduce a DNN accelerator, Panacea, which efficiently handles sparse/dense workloads of the tiled AQS-GEMM to increase data reuse and utilization. Panacea supports a specialized dataflow and run-length encoding to maximize data reuse and minimize external memory accesses, significantly improving its hardware efficiency. Numerous benchmark evaluations show that Panacea outperforms existing DNN accelerators, e.g., $1.97 \times$ and $3.26 \times$ higher energy efficiency, and $1.88 \times$ and $2.41 \times$ higher throughput than the recent bit-slice accelerator Sibia and the SIMD design, respectively, on OPT-2.7B, while providing better algorithm performance with asymmetric quantization.

Despite the energy efficiency of memory-centric deep neural network (DNN) computations, the nonlinearities inherent in existing processing-in-memory (PIM) architectures cause severe accuracy drops. These imperfections necessitate additional methods to correct inaccurate vector-matrix multiplication (VMM) results. To address this issue without modifying DNN weights, we first propose an input sparsity-based error compensation method. This approach dynamically corrects accumulated errors along the column direction of the non-volatile memory (NVM) array using pre-collected errors and input characteristics. We then present a new PIM architecture along with the proposed compensation scheme by slightly modifying the existing analog-to-digital converter (ADC) or adding a few extra rows to the NVM array. Experimental results show that the proposed work mitigates the nonlinear effects of various emerging memory cells, achieving near-ideal DNN accuracy with negligible hardware overheads.