Sparse attention can effectively alleviate the significant demands on memory when large language models (LLMs) process long contexts. Existing methods typically apply the same sparse pattern across different attention heads and inputs. However, this uniform approach fails to capture the inherent diversity of attention patterns within LLMs — the intrinsic attention clustering. To address this, we propose ClusterAttn, a training-free sparse attention method that provides an efficient prompt cache compression scheme under intrinsic attention clustering for efficient LLM inference.Our findings show that attention heads consistently focus on specific clusters of the prompt during decoding, a pattern detectable from an observation window at the prompt’s end. ClusterAttn adaptively fits these clusters utilizing a density-based attention clustering algorithm, thus compressing the KV cache of the prompt. Evaluations on different models across various benchmarks demonstrate ClusterAttn’s superior compression rates and efficiency. By utilizing only 1024 tokens, it can reduce memory usage by 10%–65%, resulting in a latency reduction of 12%–23% and a throughput increase of 2.6–4.8 times, all with nearly no accuracy loss. Additionally, ClusterAttn can handle up to 128k context on a single A100-80GB GPU, outperforming existing methods.

Web recommender systems play a crucial role in enhancing user experience and engagement by providing personalized content suggestions. These systems often leverage Recommendation Models (RMs) that integrate dense Deep Neural Networks (DNNs) with traditional sparse components to improve accuracy. The inference process for RMs in web recommender systems is typically executed in a CPU-GPU hybrid environment to handle the computational complexity. However, existing approaches have struggled to fully optimize RM inference and achieve real-time recommendations due to the challenges posed by the intricate RM operator graphs and heterogeneous hardware architecture. This paper presents RACE, an operator choreography framework designed to accelerate RM inference specifically for web recommender systems. RACE introduces three key components: an Operator Graph Placement (OGP) module for strategic operator placement, an Inference Embedding Table (IET) for efficient GPU lookups, and an Automatic Subgraph Fusion (ASF) module to optimize GPU utilization. Evaluated in production settings, RACE achieves a 3.2× speedup in RM inference, significantly enhancing the performance and responsiveness of web recommender systems.

Cybercrime poses a severe threat to the entire Internet ecosystem. Various cyber assets, such as domain name, IP address, and security certificate, are staple infrastructures of cybercrime. A cyber asset graph (CAG) is a collection of closely related cyber assets held by a cybercrime gang to support online criminal activities. Analyzing CAGs provides rich data insights for cybercrime investigation and governance. This paper introduces an open dataset of CAGs comprised of 2.37 million nodes with eight types of cyber assets and 3.28 million edges with eleven types of relations. This paper introduces the dataset construction process, applied areas, and the experience of using the dataset in the ChinaVis Data Challenge 2022. This dataset contains numerous CAGs of cybercrime gangs in the real world, which is the first open dataset of CAGs for cybercrime research. This dataset can also support the development of other application-oriented areas, such as cyber asset management and cyber-physical-social system, and various graph-related research areas, such as graph theory, graph mining, and graph visualization.

Deep Multi-agent Reinforcement Learning (MARL) relies on neural networks with numerous parameters in multi-agent scenarios, often incurring substantial computational overhead. Consequently, there is an urgent need to expedite training and enable model compression in MARL. This paper proposes the utilization of dynamic sparse training (DST), a technique proven effective in deep supervised learning tasks, to alleviate the computational burdens in MARL training. However, a direct adoption of DST fails to yield satisfactory MARL agents, leading to breakdowns in value learning within deep sparse value-based MARL models. Motivated by this challenge, we introduce an innovative Multi-Agent Sparse Training (MAST) framework aimed at simultaneously enhancing the reliability of learning targets and the rationality of sample distribution to improve value learning in sparse models. Specifically, MAST incorporates the Soft Mellowmax Operator with a hybrid TD-($\lambda$) schema to establish dependable learning targets. Additionally, it employs a dual replay buffer mechanism to enhance the distribution of training samples. Building upon these aspects, MAST utilizes gradient-based topology evolution to exclusively train multiple MARL agents using sparse networks. Our comprehensive experimental investigation across various value-based MARL algorithms on multiple benchmarks demonstrates, for the first time, significant reductions in redundancy of up to $20\times$ in Floating Point Operations (FLOPs) for both training and inference, with less than 3% performance degradation.

Machine learning-based services offer intelligent solutions with powerful models. To enhance model robustness, Federated Learning (FL) emerges as a promising collaborative learning paradigm, which iteratively trains a global model through parameter exchange among multiple clients based on their local data. Generally, the local data are heterogeneous, which slows down convergence. Knowledge distillation is an effective technique against data heterogeneity while existing works distill the ensemble knowledge from local models, ignoring the natural global knowledge from the aggregated model. This places limitations on their algorithms, such as the need for proxy data or the necessary exposure of local models to the server, which is prohibited in most privacy-preserving FL with a clueless server. In this work, we propose FedDGT, a novel knowledge distillation method for industrial server-clueless FL. FedDGT regards the aggregated model as the only one teacher to impart its global knowledge into a generator and then regularizes the drifted local models through the generator, overcoming previous limitations and providing better privacy and scalability support. Extensive experiments demonstrate that FedDGT can achieve highly-competitive model performance while greatly reducing the communication rounds in a server-clueless scenario.