Self-attention mechanisms, the key enabler of transformers' remarkable performance, account for a significant portion of the overall transformer computation. Despite its effectiveness, self-attention inherently contains considerable redundancies, making sparse attention an attractive approach. In this paper, we propose SpARC, a sparse attention transformer accelerator that enhances throughput and energy efficiency by reducing the computational complexity of the self-attention mechanism. Our approach exploits inherent row-level redundancies in transformer attention maps to reduce the overall self-attention computation. By employing row-wise clustering, attention scores are calculated only once per cluster to achieve approximate attention without seriously compromising accuracy. To leverage the high parallelism of the proposed clustering approximate attention, we develop a fully pipelined accelerator with a dedicated memory hierarchy. Experimental results demonstrate that SpARC achieves attention map sparsity levels of 85-90% with negligible accuracy loss. SpARC achieves up to 4× core attention speedup and 6× energy efficiency improvement compared to prior sparse attention transformer accelerators.

While recent cutting-edge deep neural network (DNN) models, such as large language models (LLMs), demonstrate remarkable capabilities, their inherent dense data characteristics limit the performance and energy gains achievable through sparse acceleration. In this paper, we introduce the iSPADE architecture, which sparsifies end-to-end execution of dense DNNs to directly adapt the advantages of sparse acceleration without applying accuracy-sensitive techniques such as pruning. First, we propose inverted-bit representation to eliminate repetitive sign bits in 2's complement representation. Leveraging the inverted-bit representation that generates a significant number of zero bits, we propose data packing and computation skipping techniques to reduce both redundant data movement and computation. Finally, we present an iSPADE bit-slice hardware architecture that efficiently supports and accelerates the proposed sparse dataflow. In the evaluation results, we assess performance across general DNN workloads using 8 popular DNNs. iSPADE achieves 4.1X and 4.5X improvements in energy efficiency and speedup, respectively, over the previous state-of-the-art bit-slice accelerators, and it realizes a 1.7X reduction in memory footprint.