Operator fusion, as a key performance optimization technique in the deployment of AI models, significantly improves execution efficiency and has been widely adopted in modern AI compilers. However, for cascaded reduction operations involving multiple loops with inter-loop data dependencies, such as the safe softmax followed by GEMM within attention mechanisms, existing compilers lack effective automated fusion and kernel generation capabilities. Although some works have addressed specific instances through hand-crafted fusion strategies, their solutions are limited in generality and difficult to extend to other similar structures. Given the prevalence of such computational patterns in deep learning models, there remains significant untapped potential in achieving general and automated fusion optimization. In this paper, we present a formal theoretical methodology for analyzing cascaded reductions which can fuse them into a single loop and introduce an incremental computation form. Based on this methodology, we design Red uction Fuser (RedFuser), a framework that automatically identifies supported cascaded reduction patterns and generates optimized fused kernels. Experiments show that RedFuser successfully fuses diverse workloads, achieving up to 2× to 5× speedup over state-of-the-art AI compilers and matching the performance of highly optimized hand-written kernels.

Attention-based models provide significant accuracy improvement to Natural Language Processing (NLP) and computer vision (CV) fields at the cost of heavy computational and memory demands. Previous works seek to alleviate the performance bottleneck by removing useless relations for each position. However, their attempts only focus on intra-sentence optimization and overlook the opportunity in the temporal domain. In this paper, we accelerate attention by leveraging the tempo-spatial similarity across successive sentences, given the observation that successive sentences tend to bear high similarity. This is rational owing to many semantic similar words (namely tokens) in the attention-based models. We first propose an online-offline prediction algorithm to identify similar tokens/heads. We then design a recovery algorithm so that we can skip the computation on similar tokens/heads in succeeding sentences and recover their results by copying other tokens/heads features in preceding sentences to reserve accuracy. From the hardware aspect, we propose a specialized architecture TSAcc that includes a prediction engine and recovery engine to translate the computational saving in the algorithm to real speedup. Experiments show that TSAcc can achieve 8.5X, 2.7X, 14.1X, and 64.9X speedup compared to SpAtten, Sanger, 1080TI GPU, and Xeon CPU, with negligible accuracy loss.

Spiking neural networks (SNNs) have been promising applications in the image recognition domain, and their key component is the spiking neuron. SNN s mainly contain integration and firing processes, which are essentially weight accumulation and threshold comparison, respectively. However, spike trains of the neurons exhibit high sparsity and irregularity in both temporal and spatial domains, leading to inefficient memory access and computation. Therefore, designing an efficient accelerator for SNNs is urgent. This paper presents an elaborate accelerator Early in a software-hardware co-design way. At the software level: (i) Noticing the importance of weights, where larger weights disproportionately affect the membrane potential, we devise a weight importance-aware early firing solution for the firing neurons. It prioritizes the accumulation of these large weights, thereby accelerating the membrane potential's rise to surpass the threshold sooner. (ii) Meanwhile, given the observation that a large proportion of neurons do not eventually be fired even after experiencing a long delay of weight accumulation, we propose a weight importance-aware early exit mechanism. It preferentially accumulates large weights and compares the membrane potential with the predetermined threshold, which early halts the accumulation of neurons that are unlikely to be fired, enhancing efficiency. At the hardware level, we design a specialized processing element (PE) featuring the reorder engine for spikes and weights, tailored to realize the aforementioned strategies. Experimental results show that Early averagely achieves 20.3 x, 6.5 x, and 2.4 x speedup compared to the state-of-the-art accelerators Spinalflow, PTB, and SATO. Meanwhile, it averagely achieves 25.2x, 7.4x, and 3.2x energy savings with respect to the three accelerators.

Scaling bottlenecks the making of digital quantum computers, posing challenges from both the quantum and the classical components. We present a classical architecture to cope with a comprehensive list of the latter challenges all at once , and implement it fully in an end-to-end system by integrating a multi-core RISC-V CPU with our in-house control electronics. Our architecture enables scalable, high-precision control of large quantum processors and accommodates evolving requirements of quantum hardware. A central feature is a microarchitecture executing quantum operations in parallel on arbitrary predefined qubit groups. Another key feature is a reconfigurable quantum instruction set that supports easy qubit re-grouping and instructions extensions. As a demonstration, we implement the widely-studied surface code quantum computing workflow, which is instructive for being demanding on both the controllers and the integrated classical computation. Our design, for the first time, reduces instruction issuing and transmission costs to constants, which do not scale with the number of qubits, without adding any overheads in decoding or dispatching. Our system uses a dedicated general-purpose CPU for both qubit control and classical computation, including syndrome decoding. Implementing recent theoretical proposals as decoding firmware that parallelizes general inner decoders, we can achieve unprecedented decoding capabilities of up to distances 47 and 67 with the currently available systems-on-chips for physical error rate p = 0.001 and p = 0.0001, respectively, all in just 1 μs.