Transformer-based models have shown huge success in natural language processing (NLP) with increasing model size and attention mechanism. However, this makes Von Neumann architecture based accelerators memory-bound such that the accelerators cannot leverage all the advantages of Transformer-based models. Although computing-in-memory (CIM) processors have emerged to tackle this problem through in-situ computing, the mismatch of computing patterns and low computing precision of CIM make it still challenging to accelerate Transformers. In this paper, we propose C2OPU, a hybrid dual-core processor to accelerate Transformers with hardware and software co-optimization. The dual-core architecture uses CIM arrays to accelerate weight-stationary vector-matrix multiplications, which accounts for the main computation complexity of Transformers. Meanwhile, a coarse-grained reconfigurable architecture (CGRA) is used to address the issues of computing pattern mismatch and low precision of the CIM. In addition, we propose an accuracy-bound workload allocation strategy, which considers non-ideal characteristics in analog computing, to balance throughput and accuracy. Furthermore, C2OPU provides a compiler to automatically determine optimal system configurations when Transformer model changes. Experimental results show that C2OPU achieves an average speedup of 145.41×, 4.73×, 4.70x and 3.85×, and 1.37x compared to CPU, GPU, Science23, Nature23, and VLSI24, respectively, on ten different Transformer models.

Model order reduction (MOR) has long been a mainstream strategy to accelerate large scale transient circuit simulation. Exponential integrator (EI) based on Krylov subspace approximation methods, on the other hand, are more recently developed for a similar goal. This article aims to examine in-depth the underlying relationship between model order reduction (MOR) and exponential integrator (EI) that are commonly seen as two separate methods. The main finding is that EI can be viewed as a moment-matching MOR in the time-domain. Specifically, EI, under certain conditions, is equivalent to performing moment-matching MOR based on rational Krylov subspace projection at each time step with a single input vector and a selected expansion point, then advancing the reduced system one step in the time-domain. The equivalence is mathematically proved under different settings and numerically verified in the experiments. Their differences in the transient circuit analysis context are also elaborated from various perspectives. It is hoped that these new insights would benefit the future development of this classical EDA topic.