Tucker decomposition approximates the original tensor using a set of factor matrices and a core tensor, thereby reducing the storage and computational cost. Existing incremental Tucker decomposition techniques have limitations in efficiency and are not applicable to diverse dynamic incremental tensors. This paper proposes a dynamic incremental Tucker decomposition method based on heterogeneous computing, which efficiently handles complex dynamic tensors while preserving computational efficiency. Specifically, the target optimization function of Tucker decomposition is reconstructed to decouple the intrinsic correlation between factor matrix rows and fixed tensor dimensions, and the optimal solutions for the factor matrix rows are derived. This enables the algorithm to adapt to arbitrary forms of incoming incremental data, including time-series tensors, without recalculating existing data, thus enhancing data utilization efficiency. Then, a lightweight precision optimization strategy is introduced, which iteratively updates factor matrix rows using the most element-rich dimensions of each tensor order based on recorded observable data volumes. This strategy improves precision while significantly reducing computational costs. Furthermore, a task-characteristic-based heterogeneous computing scheduling strategy is proposed. By extracting highly parallelizable intermediate operators and leveraging the high throughput of GPUs, the algorithm is accelerated, leading to a 10 to 40 times speedup in incremental tensor decomposition compared to state-of-the-art methods, while maintaining accuracy.

Large language models in deep learning have numerous parameters, requiring significant storage space and computational resources. Compression techniques are highly effective in addressing these challenges. With the development of hardware like Graphics Processing Unit (GPU), Tensor Core can accelerate low-precision matrix multiplication but achieve acceleration for sparse matrices is challenging. Due to its sparsity, the utilization of Tensor Cores is relatively low. To address this, we propose the based onTensorCoreCompressedSparseRow format (TC-CSR), which facilitates data loading on GPUs and matrix operations on Tensor Cores. Based on this format, we designed block Sparse Matrix-Matrix Multiplication (SpMM) and Sampled Dense-Dense Matrix Multiplication (SDDMM) kernels, which are common operations in deep learning. Utilizing these designs, we achieved a$\mathbf {1.41\times }$speedup on Sputnik in scenarios of moderate sparsity and a$\mathbf {1.38\times }$speedup with large-scale highly sparse matrices. Benefit from our design, we achieved a$\mathbf {1.75\times }$speedup in end-to-end inference with sparse Transformers and save memory.