The KV cache in self-attention has emerged as a major bottleneck in long-context and large-batch inference for LLMs. Existing approaches often treat sparsity prediction and compression as separate modules—relying on auxiliary index structures to select relevant tokens, and on complex quantization schemes to reduce memory usage. This fragmented design introduces redundant overhead and limits scalability. In this paper, we propose a novel paradigm: treating the compressed key representation not merely as storage, but as a self-indexing structure that directly enables efficient sparse attention. By designing a sign-based 1-bit vector quantization (VQ) scheme, our method unifies compression and retrieval in a single, hardware-friendly format. This approach eliminates the need for external indices or learning-based predictors, offering a lightweight yet robust solution for memory-constrained inference. All components are designed to be hardware-efficient and easy to implement. By implementing custom CUDA kernels, our method integrates seamlessly with FlashAttention, minimizing additional runtime and memory overhead. Experimental results demonstrate that our approach delivers both effectiveness and efficiency.

Research in the area of Distributed Denial of Service (DDoS) attack detection is of paramount importance in the field of network security. Many existing studies employ static evaluation methods that fail to account for the reduction in accuracy due to the impact of inference latency on the timeliness of classification results. Furthermore, these studies frequently rely on simulated datasets for experimentation, which often lack the complexity and challenge of real-world attacks. These limitations significantly hinder the applicability of such research in practical scenarios. To overcome these challenges, we propose an evaluation methodology for real-time DDoS attack detection incorporating inference latency considerations. Additionally, we have developed a challenging DDoS dataset named THU-DDoS2024 and conducted experiments across four classification algorithms. This novel evaluation method and the newly generated dataset are integrated into an evaluation framework named E-DDoS. Leveraging E-DDoS, the “Intelligent Classification of High-Speed Network Traffic (ICNT)”, Grand Challenge was initiated. This event aims to motivate both academic and industrial sectors to delve into high-speed traffic classification tasks, thereby enhancing the applicability of research outputs to real-world applications.

The rapid development in technology has spurred the widespread use of unmanned aerial vehicles (UAVs) in various fields, including military, civilian, and scientific research. In the process of an UAV performing a mission, accurate tracking of its trajectory is crucial, so studying the trajectory tracking control of quadrotors has important practical application value. In view of the characteristics of quadrotors such as nonlinearity and strong coupling, this paper designs an active disturbance rejection control (ADRC) method that combines phase compensator and improved expanded state observer (ESO). This method adds a phase compensator with filtering function behind the tracking differentiator, which solves the problem of phase lag of the output signal. And it uses the inverse hyperbolic sine function to replace the Fal function in the traditional ESO, which reduces the chattering of the system. Finally, simulation experiments verified the proposed method's effectiveness.

Abstract Aiming at the problems of low detection accuracy, high computational complexity and long‐time consumption of visual perception model in a complex mining environment, this research designs a visual information perception system of coal mine comprehensive excavation working face for an edge computing terminal. Firstly, the C3‐Fast feature extraction module, spatial pyramid pooling with cross‐stage partial connection (SPPCSPC) pooling module, bi‐directional feature pyramid network and lightweight decoupled detection head are used to optimize the YOLOv5s model, so as to construct the FSBD‐YOLOv5s multi‐object detection model. Secondly, the pruning and distillation algorithm is used to lighten the FSBD‐YOLOv5s model, and the model complexity is greatly reduced while maintaining the model detection accuracy. Further, the lightweight FSBD‐YOLOv5s model is migrated and deployed to the edge computing terminal platform and the TensorRT engine is used to accelerate model inference. Finally, experiments are carried out based on the data set of the coal mine comprehensive excavation working face. The experimental results show that on the edge computing terminal platform, the parameters and computational volume of the lightweight FSBD‐YOLOv5s model are reduced by 50.8% and 34.0%, while its detection accuracy and speed reach 94.0% and 43.7 fps, which can fully satisfy the requirements of the accuracy and real‐time for the coal mine engineering applications.

This paper solves the problem of learning dense visual correspondences between different object instances of the same category with only sparse annotations. We decompose this pixel-level semantic matching problem into two easier ones: (i) First, local feature descriptors of source and target images need to be mapped into shared semantic spaces to get coarse matching flows. (ii) Second, matching flows in low resolution should be refined to generate accurate point-to-point matching results. We propose asymmetric feature learning and matching flow super-resolution based on vision transformers to solve the above problems. The asymmetric feature learning module exploits a biased cross-attention mechanism to encode token features of source images with their target counterparts. Then matching flow in low resolutions is enhanced by a super-resolution network to get accurate correspondences. Our pipeline is built upon vision transformers and can be trained in an end-to-end manner. Extensive experimental results on several popular benchmarks, such as PF-PASCAL, PF-WILLOW, and SPair-71 K, demonstrate that the proposed method can catch subtle semantic differences in pixels efficiently. Code is available on https://github.com/YXSUNMADMAX/ACTR.

Knowledge graph embedding aims to transform the entities and relations of triplets into the low-dimensional vectors. Previous methods are oriented towards the static knowledge graphs, in which all entities and relations are assumed to be known and only some unknown triplets need to be predicted. However, the real-world knowledge graphs can grow dynamically, and some new knowledge are often added. To embed the new knowledge into the space of original knowledge graph, the classic models have to perform the entire re-embedding with including the new and original knowledge. This causes heavy computational burden for embedding. To address this problem, this study proposes a new model of anchors-based incremental embedding (ABIE) to implement the dynamical embedding for the growing knowledge graph. According to ABIE, every knowledge graph has some key entities, called anchors, which can fix the embedding space of knowledge graph. When some new knowledge is added into the graph, only a few updated entities and relations are embedded into the embedding space with the help of anchors, and the entire re-embedding on the whole graph is not necessary. By this way, the computational burden of embedding caused by the growth of knowledge graph is reduced significantly.

This paper presents new hierarchically cascaded transformers that can improve data efficiency through attribute surrogates learning and spectral tokens pooling. Vision transformers have recently been thought of as a promising alternative to convolutional neural networks for visual recognition. But when there is no sufficient data, it gets stuck in overfitting and shows inferior performance. To improve data efficiency, we propose hierarchically cascaded transformers that exploit intrinsic image structures through spectral tokens pooling and optimize the learnable parameters through latent attribute surrogates. The intrinsic image structure is utilized to reduce the ambiguity between foreground content and background noise by spectral tokens pooling. And the attribute surrogate learning scheme is designed to benefit from the rich visual information in image-label pairs instead of simple visual concepts assigned by their labels. Our Hierarchically Cascaded Transformers, called HCTransformers, is built upon a self-supervised learning framework DINO and is tested on several popular few-shot learning benchmarks. In the inductive setting, HCTransformers surpass the DINO baseline by a large margin of 9.7% 5-way 1-shot accuracy and 9.17% 5-way 5-shot accuracy on miniImageNet, which demonstrates HCTransformers are efficient to extract discriminative features. Also, HCTransformers show clear advantages over SOTA few-shot classification methods in both 5-way 1-shot and 5-way 5-shot settings on four popular benchmark datasets, including miniImageNet, tieredImageNet, FC100, and CIFAR-FS. The trained weights and codes are available at https://github.com/StomachCold/HCTransformers.

Deep features have been proven powerful in building accurate dense semantic correspondences in various previous works. However, the multi-scale and pyramidal hierarchy of convolutional neural networks has not been well studied to learn discriminative pixel-level features for semantic correspondence. In this paper, we propose a multi-scale matching network that is sensitive to tiny semantic differences between neighboring pixels. We follow the coarse-to-fine matching strategy and build a top-down feature and matching enhancement scheme that is coupled with the multi-scale hierarchy of deep convolutional neural networks. During feature enhancement, intra-scale enhancement fuses same-resolution feature maps from multiple layers together via local self-attention and cross-scale enhancement hallucinates higher-resolution feature maps along the top-down pathway. Besides, we learn complementary matching details at different scales thus the overall matching score is refined by features of different semantic levels gradually. Our multi-scale matching network can be trained end-to-end easily with few additional learnable parameters. Experimental results demonstrate that the proposed method achieves state-of-the-art performance on three popular benchmarks with high computational efficiency. The code has been released at https://github.com/wintersun661/MMNet.

We present a bimanual text input method on a miniature fingertip keyboard, that invisibly resides on the first segment of a user's index finger on both hands. Text entry can be carried out using the thumb-tip to tap the tip of the index finger. The design of our keyboard layout followed an iterative process, where we first conducted a study to understand the natural expectation of the handedness of the keys in a QWERTY layout for users. Among a choice of 67,108,864 design variations, we identified 1295 candidates offering a good satisfaction for user expectations. Based on these results, we computed an optimized bimanual keyboard layout, while considering the joint optimization problems of word ambiguity and movement time. Our user evaluation revealed that participants achieved an average text entry speed of 23.4 WPM.

As huge commercial value of the recommender system, there has been growing interest to improve its performance in recent years. The majority of existing methods have achieved great improvement on the metric of click, but perform poorly on the metric of conversion possibly due to its extremely sparse feedback signal. To track this challenge, we design a novel deep hierarchical reinforcement learning based recommendation framework to model consumers' hierarchical purchase interest. Specifically, the high-level agent catches long-term sparse conversion interest, and automatically sets abstract goals for low-level agent, while the low-level agent follows the abstract goals and catches short-term click interest via interacting with real-time environment. To solve the inherent problem in hierarchical reinforcement learning, we propose a novel multi-goals abstraction based deep hierarchical reinforcement learning algorithm (MaHRL). Our proposed algorithm contains three contributions: 1) the high-level agent generates multiple goals to guide the low-level agent in different sub-periods, which reduces the difficulty of approaching high-level goals; 2) different goals share the same state encoder structure and its parameters, which increases the update frequency of the high-level agent and thus accelerates the convergence of our proposed algorithm; 3) an appreciated reward assignment mechanism is designed to allocate rewards in each goal so as to coordinate different goals in a consistent direction. We evaluate our proposed algorithm based on a real-world e-commerce dataset and validate its effectiveness.

Target's spectral emissivity changes variously, and how to obtain target's continuous spectral emissivity is a difficult problem to be well solved nowadays. In this letter, an activation-function-tunable neural network is established, and a multistep searching method which can be used to train the model is proposed. The proposed method can effectively calculate the object's continuous spectral emissivity from the multispectral radiation information. It is a universal method, which can be used to realize on-line emissivity demarcation.