Distributed training improves the accuracy of large models by scaling parameter sizes. However, this scalability introduces communication bottlenecks between nodes. Gradient compression can alleviate these bottlenecks, but most existing methods are lossy, potentially compromising convergence and accuracy. This paper proposes a novel lossless compression algorithm for the floating-point gradient data, significantly improving compression efficiency compared to traditional lossless float compression algorithms while overcoming the shortcomings of lossy compression. First, new distance-run-length encoding and exponent prediction encoding using dynamic Huffman coding (D.Huff.) are designed based on the distinct statistical characteristics of zero and non-zero gradient data, effectively representing data values and zero locations. Second, an innovative hybrid Huffman coding (H.Huff.) is introduced, combining the advantages of dynamic and static Huffman coding. It enhances compression efficiency via local data adaptability, and simultaneously improves computational efficiency and hardware implementation friendliness with multiple fixed Huffman code tables. We also present a K-means-based probability vector clustering algorithm to effectively construct multiple prebuilt Huffman code tables. To evaluate the effectiveness of the proposed compression method, we selected two different deep neural network (DNN) learning tasks: (1) image classification, using ResNet50 and VGG16 for CIFAR-100 training, and ViT for ImageNet; and (2) language modeling, employing BERT on the IMDB dataset. The results are presented in Table 1. Furthermore, we tested mixed-precision training compression using the BF16 format. Experimental results demonstrate that our method achieves a lossless compression ratio of 0.4931 on typical gradient data, representing a significant 16.61% improvement over the state-of-the-art lossless float data compression approach, ndzip.

Asymmetric Numeral Systems (ANS) play a significant role in learned image codecs (LIC) because of their high coding efficiency. However, it constitutes a substantial portion of inference time, making it the main bottleneck in real-time LIC due to its high computational demands, complex control logic, and serial execution flow. To address these challenges, this paper introduces a hardware-oriented ANS algorithm hANS that reduces complex calculations for state encoding and state-symbol decoding. Furthermore, hANS employs fixed-latency calculation to eliminate control logic, which often causes inconsistent delays. To further enhance throughput, we propose a hardware architecture of ANS for LIC (ANS-LIC), introducing a novel hardware parallelism scheme that incorporates pipeline execution and multi-bin parallelism for encoding, along with multi-stream parallelism for decoding. Additionally, by optimizing the execution order, we achieve a reduction in hardware resource utilization during the decoding process. The proposed ANS-LIC hardware is implemented in RTL and synthesized using TSMC 65nm technology and the Alveo U250 Data Center Accelerator Card. We evaluate ANS-LIC on the Kodak and DIV2K LIC datasets, achieving a 1.17% compression ratio improvement over the SOTA method, Recoil. The implementation results and comparison with other works are presented in Table 1. The synthesis indicates that ANS-LIC requires only 385.5/393.0k gates for encoding and decoding, without SRAM. ANS-LIC achieves throughput improvements of 13.29×/1.47× for encoding and decoding over Recoil. In summary, the proposed ANS-LIC demonstrates substantial advantages.

Graph Attention Network (GAT) has gained widespread adoption thanks to its exceptional performance in processing non-Euclidean graphs. The critical components of a GAT model involve aggregation and attention, which cause numerous main-memory access, occupying significant inference time. Recently, much research has proposed near-memory processing (NMP) architectures to accelerate aggregation. However, graph attention requires additional operations distinct from aggregation, making previous NMP architectures less suitable for supporting GAT, as they typically target aggregation-only workloads. In this paper, we propose GATe, a practical and efficientGATaccelerator with NMP architecture. To the best of our knowledge, this is the first time that accelerates both attention and aggregation computation on DIMM. We unify feature vector access to eliminate the two repetitive memory accesses to source nodes caused by the sequential phase-by-phase execution of attention and aggregation. Next, we refine the computation flow to reduce data dependencies in concatenation and softmax, which lowers on-chip memory usage and communication overhead. Additionally, we introduce a novel sharding method that enhances data reusability of high-degree nodes. Experiments show that GATe achieves substantial speedup of GAT attention and aggregation phases up to 6.77× and 2.46×, with average to 3.69× and 2.24×, respectively, compared to state-of-the-art NMP works GNNear and GraNDe.

Personalized recommendation systems are massively deployed in production data centers. The memory-intensive embedding layers of recommendation systems are the crucial performance bottleneck, with operations manifesting as sparse memory lookups and simple reduction computations. Recent studies propose near-memory processing (NMP) architectures to speed up embedding operations by utilizing high internal memory bandwidth. However, these solutions typically employ a fixed vector partitioning strategy that fail to adapt to changes in data center deployment scenarios and lack practicality. We propose Flips, aflexiblepartitioningstrategy NMP architecture that accelerates embedding layers. Flips supports more than ten partitioning strategies through hardware-software co-design. Novel hardware architectures and address mapping schemes are designed for the memory-side and host-side. We provide two approaches to determine the optimal partitioning strategy for each embedding table, enabling the architecture to accommodate changes in deployment scenarios. Importantly, Flips is decoupled from the NMP level and can utilize rank-level, bank-group-level and bank-level parallelism. In peer-level NMP evaluations, Flips outperforms state-of-the-art NMP solutions, RecNMP, TRiM, and ReCross by up to 4.0×, 4.1×, and 3.5×, respectively.

The promotion of the HEVC standard has significantly alleviated the burden of network transmission and video storage. However, its inherent complexity and data dependencies pose a significant challenge in achieving high compression efficiency hardware encoder. To tackle this challenge, we propose several hardware-oriented algorithms and achieve a hardware encoder supporting both intra and inter coding. In terms of algorithms, our optimizations focus on intra mode decision, motion estimation (ME), rate estimation, and merge mode estimation. These optimizations reduce the computational complexity and address the data dependencies within and between encoder modules while maintaining an acceptable compression efficiency. As for hardware, we propose an encoder architecture that supports not only 35 intra prediction modes but also ME with an extensive search range of [±64, ±64]. The uniform$4\times 4$engine, 2-D data reuse, and timing schedule for intra and inter coding are presented in this architecture to optimize the hardware resource consumption and throughput. Compared with HM 15.0, the proposed hardware-oriented algorithms lead to a 1.88% and 14.57% increase in BD-Rate under the configurations of all intra and low delay P, respectively. Notably, the BD-Rate outperforms all existing hardware encoders supporting 4K resolution. In a GF 28nm fabrication process, the hardware design achieves a clock frequency of 550MHz, supporting 4K@30fps throughput with a hardware gate count of 3154K and memory usage of 1.02MB, and the proposed architecture demonstrates substantial advantages in terms of area, throughput, and power compared to other studies.