While 3D-stacked SRAM architectures have demonstrated prominent performance speedup for tensor computing by exploiting higher bandwidth, larger buffer and reduced latency, they suffer from thermal challenges owing to vertical stacking nature of chips. In this paper, we identify that tensor dataflow may further exacerbate the thermal issues, so we propose T3D, the first thermal-aware tensor framework for 3D-stacked SRAM architectures, leveraging tensor dataflow characteristics to significantly enhance thermal performance. Specifically, we first perform a quantitative formulation to identify the most energy-efficient tensor dataflow with given 3D constraints, effectively reducing heat generation without performance loss. Then, we develop a thermal-aware 3D architectural floorplan to improve heat spreading by optimizing the spatial arrangement of multiple SRAM macros with varying power overheads, which is caused by mismatched data access rates of tensor computing. Experimental results show that, our proposed T3D can reduce the peak chip temperature by 12.9°C on certain LLM and DNN workloads over the state-of-the-art 3D solutions.

Although design space exploration (DSE) is good at finding dataflow for optimal memory access in tensor accelerators, it is very timing-consuming and lacks architecture insight. In this study, we for the first time propose several principles for dataflow optimization that provides lower bound of memory communication for tensor operators such as matrix multiplication. Through these principles we can calculate the best tiling, scheduling and mapping for both intra- and inter-operator dataflow. In addition, we can identify all the tensor-wise opertor fusion that are profitable in memory communication, so we propose FuseCU, a new architecture that supports these profitable fusion which can be applied to existing spatial architectures for data movement saving. Experimental results show that FuseCU delivers 63.6%, 62.4% and 38.7% data movement saving and $1.33 \times, 1.25 \times$ and $1.14 \times$ speedup compared to the TPUv4i, Gemmini and Planaria designs without increasing buffer size or bandwidth. Additionally, FuseCU is open-sourced.

The diversity of activation functions has gradually increased to accommodate different tasks in modern deep neural networks (DNNs). However, these novel activation functions involve more nonlinear operations relative to traditional activation functions, which increases the computational complexity. To address these issues, a piecewise linear (PWL) approximation algorithm called Equi-Error Partitioning Algorithm is proposed in this paper. The algorithm aims at balancing the errors between segments and solves the problem of excessive precision that exists in other PWL approximation methods and achieves on average 30.07× better mean squared error compared to the previous works. Based on this algorithm, we propose an activation function unit (AFU) which enables the addressing scheme of non-uniform segments and provides reconfigurability for all common activation functions by reloading parameters. End-to-end evaluation with several DNNs shows the accuracy loss is all less than 0.06% with 64 segments.